A couple of months ago, we received the following email from AWS:
Hello,
You are receiving this email because AWS Lambda is making a change to the IAM policy evaluation when accessing Lambda function APIs, and we identified that your account will be impacted by this change.
It provided in-depth details about the new requirement and options for remediation. The long and short of it was that we needed to fix the problematic IAM policies right now or risk permissions problems that would render our Lambdas inoperable.
Not being overfond of production outages, we knew we had some work to do. The JupiterOne stack is almost entirely serverless. We have more than a few Lambda functions, ECS task definitions, many of which invoke other Lambdas. This often requires granting IAM policies to invoke permissions.
The JupiterOne platform has hundreds of code repos where the infrastructure for these services is defined. Finding all the places where unqualified ARNs were utilized was a bit of a chore. Enter JupiterOne, a powerful cyber asset management and governance solution. Because we ingest information about our AWS infrastructure into the JupiterOne graph via the AWS integration, locating the services that required updating turned into a fairly simple query.
FIND
(aws_lambda_function|aws_ecs_task_definition) as offendingCompute
THAT ASSIGNED aws_iam_role as offendingRole
THAT ASSIGNED >> aws_iam_policy as offendingPolicy
THAT ALLOWS AS r aws_lambda_function as invokedFunction
WHERE
r.permission = "EXECUTE" AND
(r.resources !~= "*" OR r.resources !~= ":")
RETURN offendingCompute.displayName as offendingComputeName, offendingCompute._type as offendingComputeType, offendingRole.displayName as offendingRoleName, count(offendingPolicy)
ORDER BY offendingComputeName
At first glance, it may appear complex, but the above query simply follows an assignment chain from one Lambda function to another where an InvokeFunction permission is granted, like in the IAM statement below.
{
"Action": "lambda:InvokeFunction",
"Effect": "Allow",
"Resource": [
"arn:aws:lambda:us-east-1:account_id:function:a-lambda-function",
],
"Sid": ""
}
Our AWS integration ingests that statement and connects the two Lambdas via aws_iam_role and aws_iam_policy entities. And because the relationship between aws_iam_policy and aws_lambda_function contains details like the permission and resources, we can use our string comparison operators to find resources that contain unqualified ARNs (those lacking a ':' or '*').
To make it easier to locate and update your affected resources, please refer to your direct email from AWS (full text below), which provides further context on the AWS Lambda policy change.
Asset management is so much more than security, and we hope this will inspire customers to unlock the immense value of gaining full cloud infrastructure visibility via the JupiterOne platform.
Full text of AWS Lambda ARN policy change notification:
Hello,
You are receiving this email because AWS Lambda is making a change to the IAM policy evaluation when accessing Lambda function APIs, and we identified that your account will be impacted by this change.
We would like to offer some background about usage of Lambda function APIs. A Lambda function may include versions [1] and aliases [2] which are sub-resources of a function. You can operate on versions and aliases using the Lambda function APIs in one of two ways:
(1) You can either append the version number or alias name as a suffix to the function ARN. We will refer to this as a "qualified" Amazon Resource Name (ARN); or
(2) You can specify only the function ARN (we refer to this as an "unqualified" ARN), and add a separate "qualifier" parameter in the API command. For example, a function "helloworld" with version "42" can be passed into API request using the function ARN as arn:aws:lambda:aws-region:acct-id:function:helloworld:42 or arn:aws:lambda:aws-region:acct-id:function:helloworld, Qualifier=42. These call types are equivalent.
Currently, you can restrict the scope of a user's permissions to the Lambda function APIs by using an unqualified ARN in the Resource element of your IAM or VPC endpoint policy. Previously, when used in this way, Lambda interpreted such permissions in one of two ways depending on the API usage. Either by granting permission to requests to the function only, or granting permissions to requests to the function and all of its sub-resources. For example, when using IAM or VPC endpoint policy with resource element as unqualified ARN, arn:aws:lambda:aws-region:acct-id:function:helloworld, API Request(s) with qualified ARN as arn:aws:lambda:aws-region:acct-id:function:helloworld:42 is denied and the request using arn:aws:lambda:aws-region:acct-id:function:helloworld, Qualifier=42 is allowed. We received customer feedback that this dual interpretation was not always intuitive. To address this feedback and increase the consistency of the Lambda function APIs authorization, starting January 24, 2022, Lambda requires IAM or VPC endpoint policies to specify the fully qualified ARN in the Resource element when authorizing API calls that use an unqualified ARN with a qualifier parameter.
We identified that your AWS account calls AWS Lambda APIs using the unqualified ARN with a qualifier parameter while the associated permissions for the user who makes these API calls use the unqualified ARN in the policy's Resource element. To continue making these API calls successfully, you need to append the version number, alias name to the unqualified function ARN in your policy's Resource element. To allow access to all the sub-resources of a function, append ":" and "*" to the unqualified function ARN and to allow access to both function (unqualified ARN) and all sub-resources of a function, append "*" to the unqualified function ARN. Additionally, you need to ensure the updated policies are used by the IAM users/roles that make the Lambda API calls. Please find the following instructions on how to make this change.
This change will begin on April 30, 2022, to give you ample time to update your IAM policies or permissions attached to the users and roles that call Lambda APIs before that. We have also provided a list of your affected resources in the US-EAST-2 Region at the end of this message to make it easier to locate and update them. If you do not take this action by April 30, 2022, Lambda API calls will fail with permission errors.
