Fix CloudWatch log resource policy failed: LimitExceededException

The LimitExceededException error can occur when running a Terraform script. It indicates that while the Terraform configuration is valid, there’s an issue with CloudWatch. If you run the same Terraform script in a different infrastructure, it will pass without errors. In this article, we’ll discuss how to address the CloudWatch log resource policy failure in detail.

There are two primary reasons for encountering the failed: LimitExceededException error:

• Exceeding ten resource policies per region, per account
• Exceeding the maximum policy document size of 5120 characters

Unfortunately, both errors stem from fixed Amazon CloudWatch limitations that cannot be adjusted. Since AWS sets these quotas, there is no way to increase them. The only solution is to modify either the number of resource policies or the size of the policy documents, depending on the error root cause.

How to fix exceeding ten resource policies

Amazon allows a maximum of ten resource policies per account, per region. If you attempt to create more than ten, Terraform will throw a LimitExceededException error.

To resolve this, you’ll need to delete any unused policies. Follow these steps to list and manage your resource policies:

1. List the existing policies:

$ aws logs describe-resource-policies --output json | jq .

2. Review the policies and delete any that are no longer needed:

$ aws logs delete-resource-policy --policy-name [POLICY_NAME]

3. Re-run your Terraform script. It should now execute successfully.

How to fix exceeding a resource policy’s policy document size of 5120

CloudWatch enforces a maximum policy document size of 5120 characters. If a policy document exceeds this limit, you’ll encounter an error like this:

Error: creating Transfer Server: operation error Transfer: CreateServer, https response error StatusCode: 400, RequestID: 669c95d1-e655-4d1c-bb74-59d2933acb7e, InvalidRequestException: Unable to enable logging. Policy document length breaking Cloudwatch Logs Constraints at ‘policyDocument’ failed to satisfy constraint: Member must have length less than or equal to 5120

To fix this issue, follow these steps:

1. Retrieve the problematic policy document details:

$ aws logs describe-resource-policies --query 'resourcePolicies[?policyName==`AWSLogDeliveryWrite20150319`].policyDocument' --output text

The output will look something like this:

{
   "resourcePolicies":[
      {
         "policyName":"AWSLogDeliveryWrite20150319",
         "policyDocument":"{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"AWSLogDeliveryWrite\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"delivery.logs.amazonaws.com\"},\"Action\":[\"logs:CreateLogStream\",\"logs:PutLogEvents\"],\"Resource\":[\"arn:aws:logs:eu-central-1:411742385997:log-group:\/aws\/transfer\/s-bfb6e9027ae344e8a:log-stream:*\",\"arn:aws:logs:eu-central-1:411742385997:log-group:test-sftp-server20241010101313329400000001:log-stream:*\"],\"Condition\":{\"StringEquals\":{\"aws:SourceAccount\":\"512897385689\"},\"ArnLike\":{\"aws:SourceArn\":\"arn:aws:logs:eu-central-1:512897385689:*\"}}}]}",
         "lastUpdatedTime":1729090468461
      }
   ]
}

2. Save the policy document to a file (updated_policy.json) and reduce its size by removing unnecessary statements or entries until it is under 5120 characters.

3. Apply the updated policy:

$ aws logs put-resource-policy --policy-name AWSLogDeliveryWrite20150319 --policy-document file://updated_policy.json

Conclusion

AWS CloudWatch imposes two hard limits: (1) a maximum of ten resource policies per region, per account, and (2) a policy document size limit of 5120 characters. If you exceed these limits, you will encounter the LimitExceededException error. Unfortunately, there is no way to increase these quotas. The only solution is to adjust the number of policies or reduce the size of the policy document.

In this tutorial, we covered two approaches to resolve the CloudWatch log resource policy failed: LimitExceededException error.

Inline/featured images credits

• Featured image generated by ChatGPT