AIP-4117

External Account Credentials (Workload Identity Federation)

Using workload identity federation, your application can access Google Cloud resources from Amazon Web Services (AWS), Microsoft Azure or any identity provider that supports OpenID Connect (OIDC) or SAML 2.0.

Traditionally, applications running outside Google Cloud have used service account keys to access Google Cloud resources. Using identity federation, you can allow your workload to impersonate a service account. This lets you access Google Cloud resources directly, eliminating the maintenance and security burden associated with service account keys.

Note: Because this AIP describes guidance and requirements in a language-neutral way, it uses generic terminology which may be imprecise or inappropriate in certain languages or environments.

Guidance

This section describes the general guidance of supporting non-Google external credentials (AWS, Azure, OIDC and SAML IdPs, etc) as a means of authentication.

Prerequisite

In order to use workload identity federation to access Google cloud resources from non-Google cloud platforms, the following steps are needed to configure workload identity pools, providers, service account impersonation and generate the JSON configuration file to be used by the auth libraries.

Configuration File Generation and Usage

After workload identity federation is configured, the JSON configuration file should be generated. This sample shows how an AWS configuration file is generated:

$ gcloud iam workload-identity-pools create-cred-config \
    projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
    --service-account=$SERVICE_ACCOUNT_EMAIL \
    --aws \
    --output-file=$FILEPATH.json

The following values would need to be replaced:

  • PROJECT_NUMBER: Project number of the project that contains the workload identity pool.
  • POOL_ID: ID of the workload identity pool.
  • PROVIDER_ID: ID of the workload identity pool provider.
  • SERVICE_ACCOUNT_EMAIL: Email address of the service account to impersonate.
  • FILEPATH: File to save configuration to.

If you are using AWS IMDSv2, an additional flag --enable-imdsv2 should be added to the gcloud iam workload-identity-pools create-cred-config command:

$ gcloud iam workload-identity-pools create-cred-config \
    projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
    --service-account=$SERVICE_ACCOUNT_EMAIL \
    --aws \
    --enable-imdsv2 \
    --output-file=$FILEPATH.json

If you wish to configure the service account access token lifetime, an additional flag --service-account-token-lifetime-seconds should be added to the gcloud iam workload-identity-pools create-cred-config command (this example uses an AWS configuration, but the token lifetime can be configured for all workload identity federation providers):

$ gcloud iam workload-identity-pools create-cred-config \
    projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
    --service-account=$SERVICE_ACCOUNT_EMAIL \
    --aws \
    --service-account-token-lifetime-seconds=$TOKEN_LIFETIME \
    --output-file=$FILEPATH.json

The service-account-token-lifetime-seconds flag is optional. If not provided, this defaults to one hour. The minimum allowed value is 600 (10 minutes) and the maximum allowed value is 43200 (12 hours). If a lifetime greater than one hour is required, the service account must be added as an allowed value in an Organization Policy that enforces the constraints/iam.allowServiceAccountCredentialLifetimeExtension constraint.

The external identities configuration file can be used with Application Default Credentials. In order to use external identities with Application Default Credentials, the full path to this file should be stored in the GOOGLE_APPLICATION_CREDENTIALS environment variable.

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/config.json

The library can now automatically choose the right type of client and initialize credentials from the context provided in the configuration file:

import google.auth

credentials, project = google.auth.default()

External account credentials can also be initialized explicitly using the generated configuration file.

# Sample for Azure or OIDC/SAML providers.
import json

from google.auth import identity_pool

json_config_info = json.loads(function_to_get_json_config())
credentials = identity_pool.Credentials.from_info(json_config_info)
scoped_credentials = credentials.with_scopes(
    ['https://www.googleapis.com/auth/cloud-platform'])

# Sample for AWS.
import json

from google.auth import aws

json_config_info = json.loads(function_to_get_json_config())
credentials = aws.Credentials.from_info(json_config_info)
scoped_credentials = credentials.with_scopes(
    ['https://www.googleapis.com/auth/cloud-platform'])

Expected Behavior

The auth libraries should use the information in the JSON configuration file to retrieve the external credentials and exchange them for Google access tokens using the GCP Security Token Service (via the token exchange endpoint https://sts.googleapis.com/v1/token) and then impersonating a service account by calling the IamCredentials generateAccessToken API to access GCP resources.

All external account JSON files must share the following fields:

Field Name Required Description
type Yes This identifies the new type of credential object. This must be "external_account"
audience Yes This is the STS audience which contains the resource name for the workload identity pool and the provider identifier in that pool.
subject_token_type Yes This is the STS subject token type based on the OAuth 2.0 token exchange spec.
service_account_impersonation_url No This is the URL for the service account impersonation request. If this is not available, the STS returned access token should be directly used without impersonation.
service_account_impersonation.* No This object defines additional service account impersonation options. Only one field is currently supported: “token_lifetime_seconds": This is the requested access token lifetime, e.g. 2800.
token_url Yes This is the STS token exchange endpoint.
credential_source.* Yes This object defines the mechanism used to retrieve the external credential from the local environment so that it can be exchanged for a GCP access token via the STS endpoint.

The auth libraries and applications must follow the steps below for all types of external account credentials:

  • Check credential_source to determine the necessary logic to retrieve the external credential which should be used to construct the subject token to pass to the STS endpoint. This is covered in detail for every credential configuration below.
  • Construct the STS request, based on rfc8693:
    • STS audience should be constructed using the audience field.
    • grant_type must be urn:ietf:params:oauth:grant-type:token-exchange
    • requested_token_type must be urn:ietf:params:oauth:token-type:access_token
    • subject_token_type is the subject_token_type field as described in the RFC.
    • subject_token is the retrieved external credentials. Check the subsequent sections on how this is retrieved in various environments.
    • scope: the list of space-delimited, case-insensitive OAuth scopes that specify the desired scopes of the requested security token in the context of the service or resource where the token should be used. If service account impersonation is used, the cloud platform or IAM scope should be passed to STS and then the customer provided scopes should be passed in the IamCredentials call to generateAccessToken.
    • The STS token exchange URL should be the token_url (e.g. https://sts.googleapis.com/v1/token).
  • Send the STS token exchange request to get the Google access token and its expiration.
  • If the service_account_impersonation_url is available, trigger service account impersonation flow by POSTing to that endpoint with the previously returned Google access token.
    • If this is not available, end the flow and just use the STS access token for authorization.
    • The list of scopes also need to be provided for this endpoint. The customer provided scopes should be used for this endpoint.
    • In order to access this API, the (Cloud platform https://www.googleapis.com/auth/cloud-platform or IAM scope https://www.googleapis.com/auth/iam) are required in the underlying access token.
    • The service account access token lifetime also needs to be provided for this endpoint. The value in service_account_impersonation.token_lifetime_seconds will be used if it was provided, otherwise it will default to 1 hour.

Determining the subject token in AWS

External account configuration JSON files should contain the following information in the credential_source object to facilitate retrieval of AWS credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

Field Name Required Description
environment_id Yes This is the environment identifier, of format aws${version}. A version should be specified to indicate to the auth library whether breaking changes were introduced to the underlying AWS implementation. So if aws1 is supported in the current version of the library but a credential file with aws2 is provided, an error should be thrown instructing the developer to upgrade to a newer version of the library.
region_url Yes This URL should be used to determine the current AWS region needed for the signed request construction.
url No This AWS metadata server URL should be used to retrieve the access key, secret key and security token needed to sign the GetCallerIdentity request. The $ROLE_NAME should be retrieved from calling this endpoint without any parameter and then calling again with the returned role name appended to this URL: http://169.254.169.254/latest/meta-data/iam/security-credentials/$ROLE_NAME
regional_cred_verification_url Yes This defines the regional AWS GetCallerIdentity action URL. This URL should be used to determine the AWS account ID and its roles. This should not actually be called by the Auth libraries. It should be called on the STS token server. The region should be substituted by SDK, e.g. sts.eu-west-1.amazonaws.com.
imdsv2_session_token_url No Presence of this URL enforces the auth libraries to fetch a Session Token from AWS. This field is required for EC2 instances using IMDSv2. This Session Token would later be used while making calls to the metadata enpoint.

The JSON file for AWS configuration files should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
    "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
    "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
    "imdsv2_session_token_url": "http://169.254.169.254/latest/api/token"
  }
}

The auth libraries and applications must follow the steps below:

  • Check credential_source for environment ID. If the environment ID is aws${version}, this should be an AWS native credential.
  • Inspect the version in the environment ID. If this is a newer unexpected error, trigger an error that the auth library needs to be updated to handle this type of credentials.
  • Validate the host for the url, regional_url and imdsv2_session_token_url fields if they are provided. The host should either be 169.254.169.254 or fd00:ec2::254.
  • If imdsv2_session_token_url is available, then fetch session token from imdsv2_session_token_url.
  • Check the environment variables in the following order (AWS_REGION and then the AWS_DEFAULT_REGION) to determine the AWS region. If found, skip using the AWS metadata server to determine this value.
  • If the region environment variable is not provided, use the region_url to determine the current AWS region. The API returns the zone name, e.g. us-east-1d. The region should be determined by stripping the last character, e.g. us-east-1.
  • Check the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and the optional AWS_SESSION_TOKEN for the AWS security credentials. If found, skip using the AWS metadata server to determine these values.
  • If url is available and the security credentials environment variables are not provided:
    • Call url to retrieve the attached AWS IAM role name to the current instance.
    • Call url/$ROLE_NAME to get the access key, secret key and security token needed to sign the GetCallerIdentity request.

  • Construct the AWS signed request (AWS Signature Version 4) using the GetCallerIdentity regional_cred_verification_url (with the region substituted). This should be serialized by formatting it as a url-encoded JSON and passed as the subject_token to STS endpoint. Here is a sample of the JSON format used:

    {
  "url": "https://sts.us-east-1.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
  "headers": [
    {
      "value": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
      "key": "x-goog-cloud-target-resource"
    },
    {
      "value": "20200228T225005Z",
      "key": "x-amz-date"
    },
    {
      "value": "AWS4-HMAC-SHA256 Credential=AKIASOZTBDV4D7ABCDEDF/20200228/us-east-1/sts/aws4_request, SignedHeaders=host;x-amz-date,Signature=abcedefdfedfd",
      "key": "Authorization"
    }, 
    {
      "value": "sts.us-east-1.amazonaws.com",
      "key": "host"
    },
    {
      "value": "IQoJb3JpZ2luX2VjEIz//////////wEaCXVzLWVh...",
      "key": "x-amz-security-token"
    }
  ], 
  "method": "POST",
  "body": ""
}
    For the AWS token, STS requires a special header x-goog-cloud-endpoint to recognize that the token is for a specific workload identity provider.

Determining the subject token in Microsoft Azure and URL-sourced credentials

External account configuration JSON files should contain the following information in the credential_source object to facilitate retrieval of Azure and other URL-sourced credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

Field Name Required Description
url Yes This defines the local metadata server to retrieve the external credentials from. For Azure, this should be the Azure Instance Metadata Service (IMDS) URL used to retrieve the Azure AD access token.
headers No This defines the headers to append to the GET request to credential_source.url.
format.type No This indicates the format of the URL response. This can be either "text" or "json". The default should be "text".
format.subject_token_field_name No Required for JSON URL responses. This indicates the JSON field name where the subject_token should be stored.

The JSON file for URL-sourced configuration files (OIDC / SAML) should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "url": "http://localhost:5000/token",
    "format": {
      "type": "json",
      "subject_token_field_name": "id_token"
    }
  }
}

Azure configuration files are a type of OIDC URL-sourced credentials.

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "headers": {
      "Metadata": "True"
    },
    "url": "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
    "format": {
      "type": "json",
      "subject_token_field_name": "access_token"
    }
  }
}

The auth libraries and applications must follow the steps below:

  • Check credential_source has a url field and no environment_id, otherwise skip the rest of the steps.
  • An HTTP GET request should be sent to this local url while injecting the headers key/values (if provided in the configuration file) in the request header. The request should respond with the external credentials subject token to be passed to STS token endpoint.
  • Before parsing the token, check the format field.
  • If the format is not available, assume the external credential returned by the URL response is provided in plain text format.
  • If available, check if the type is json
    • If json, check the subject_token_field_name. For Azure, this is set to access_token.
    • Parse the file as JSON and then retrieve the external credential from the field name based on the value of subject_token_field_name.

Determining the subject token in file-sourced credentials

External account configuration JSON files contain the following information in the credential_source object to facilitate retrieval of file-sourced credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

Field Name Required Description
file Yes This is the source of the credential. This should be used for a credential locally available. This should take precedence over url when both are provided.
format.type No This indicates the format of the file where the token is stored. This can be either "text" or "json". The default should be "text".
format.subject_token_field_name No Required for JSON file formats. This indicates the JSON field name where the subject_token should be stored.

The JSON file for file-sourced configuration files (OIDC / SAML) should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:saml2",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "file": "/var/run/saml/assertion/token"
  }
}

The auth libraries and applications must follow the steps below:

  • Check credential_source has a file field and no environment_id. If not, this is not a file-sourced credential and the proceeding steps do not apply.
  • Get the external credential from the file location specified by the credential_source.file field.
  • Before parsing the token, check the format field.
  • If the format is not available, assume the external credential is provided in plain text format.
  • If available, check if the type is json
    • If json, check the subject_token_field_name.
    • Parse the file as JSON and then retrieve the external credential from the field name based on the value of subject_token_field_name.

Determining the subject token in executable-sourced credentials

External account configuration JSON files contain the following information in the credential_source object to facilitate retrieval of executable-sourced credentials to be passed as subject tokens to the GCP STS token exchange endpoint.

Field Name Required Description
executable Yes Holds the information necessary to run the executable.
executable.command Yes Specifies the full command to run to retrieve the subject token. This can include arguments. Must be an absolute path for the program.
executable.timeout_millis No Specifies the timeout duration, in milliseconds. Defaults to 30 seconds when not provided.
executable.output_file No Specifies the absolute path to the output file where the executable will cache the response. By specifying this path, the auth libraries will first check this location before running the executable. The format of the file should match the JSON format expected by the auth libraries defined below.

The JSON file for executable-sourced configuration files (OIDC / SAML) should have the following form:

{
  "type": "external_account",
  "audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
  "subject_token_type": "urn:ietf:params:oauth:token-type:saml2",
  "token_url": "https://sts.googleapis.com/v1/token",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL@project.iam.gserviceaccount.com:generateAccessToken",
  "credential_source": {
    "executable": {
      "command": "/path/to/executable --arg1=value1 --arg2=value2",
      "timeout_millis": 5000,
      "output_file": "/path/to/cached/credentials"
    }
  }
}

To use executable-sourced credentials, the GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES environment variable must be set to 1.

Additionally, the executable must adhere to the following response format:

Successful responses:

Field Name Type Description
version number The version of the JSON output. Currently only version 1 is supported.
success boolean The status of the response. True in this case.
token_type string The 3rd party subject token type. Must be urn:ietf:params:oauth:token-type:jwt, urn:ietf:params:oauth:token-type:id_token, or urn:ietf:params:oauth:token-type:saml2.
id_token OR saml_response string The 3rd party OIDC token or SAML response.
expiration_time number The optional 3rd party subject token expiration time in seconds (unix epoch time). Only required in the response when an output file is specified in the credential configuration.

A sample successful executable OIDC response: 

{
  "version": 1,
  "success": true,
  "token_type": "urn:ietf:params:oauth:token-type:id_token",
  "id_token": "...",
  "expiration_time": 1620499962
}

A sample successful executable SAML response: 

{
  "version": 1,
  "success": true,
  "token_type": "urn:ietf:params:oauth:token-type:saml2",
  "saml_response": "...",
  "expiration_time": 1620499962
}

Error responses:

Field Name Type Description
version number The version of the JSON output. Currently only version 1 is supported.
success boolean The status of the response. False in this case.
code string The error code.
message string The error message.

A sample executable error response: 

{
  "version": 1,
  "success": false,
  "code": "401",
  "message": "Caller not authorized."
}

The auth libraries and applications must follow the steps below:

  • Check credential_source has an executable field and no environment_id. If not, this is not a executable-sourced credential and the proceeding steps do not apply.
  • Retrieve the external credential's executable information from the credential_source.executable field.
  • Check that the GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES environment variable is set to 1. If not, error out.
  • Before the next step, check if credential_source.executable.output_file was specified in the credential configuration.
    • If present, check if there is an executable response at that location.
    • If the response is valid and unexpired, or there is no response at that location, continue execution.
    • If the response is malformed or invalid, error out.
  • Ensure the following environment variables will be available to the executable:
    • GOOGLE_EXTERNAL_ACCOUNT_AUDIENCE: The audience field from the credential configuration. Must always be present.
    • GOOGLE_EXTERNAL_ACCOUNT_TOKEN_TYPE: The subject token type. Must always be present.
    • GOOGLE_EXTERNAL_ACCOUNT_IMPERSONATED_EMAIL: The service account email. Only present when service account impersonation is used.
    • GOOGLE_EXTERNAL_ACCOUNT_OUTPUT_FILE: The output file location from the credential configuration. Only present when specified in the credential configuration.
  • Run the command specified at credential_source.executable.command.
    • Fail in the following scenarios:
      • The executable failed to complete in the timeout duration specified.
      • The executable's response is invalid, was unsuccessful or expired.
      • The executable finished with a non-zero exit code.
  • Parse the executable response as JSON and then retrieve the external credential from the field name based on the value of token_type.
    • The token_type value must be urn:ietf:params:oauth:token-type:jwt, urn:ietf:params:oauth:token-type:id_token, or urn:ietf:params:oauth:token-type:saml2.
    • If the token_type is urn:ietf:params:oauth:token-type:saml2, the subject token will be parsed from the saml_response field.
    • Otherwise it will be parsed from the id_token field.

Changelog

  • 2021-12-10: Add AIP for External Account Credentials (AIP 4117).
  • 2022-05-18: Document executable-sourced credentials (AIP 4117).
  • 2022-08-31: Document configurable token lifetime (AIP 4117).