I’m currently working on a feature for runbooks.app which allows users to upload images for their runbooks. I’m using the Python boto3 library to make a PutObject API requests. Simply provide the bytes, the target bucket, and object key, and you should be all set. However, to my considerable frustration, I spent most of the morning trying to figure out why I was getting this error:

botocore.exceptions.ClientError:
An error occurred (AccessDenied)
when calling the PutObject operation:
Access Denied

Not super helpful, as there are, of course, a whole host of things which could be wrong.

Access Keys

I wanted my bucket to only be available to a specific IAM user I set up for my application code. This user should only be given permissions for the specific API operations I want my code to perform: and nothing else. When you set up the user, you’re given an Access Key and a Secret Access Key. The former is a jumble of letter which identifies the account, and the latter is a shared secret so AWS can be sure the request comes from a trusted source.

I’m using Heroku, so I went to my application’s settings page to verify that my Config Vars contained the correct values. They did. So, that wasn’t the problem.

User Policy

Each new user you create in IAM has a section which specifically lists what things that user is permitted to do. There are many ways to assign permissions. You can create a separate policy and associate it with that user directly. You could make such a policy, associate it with a group, and then add the user to the group. You could assign the policy to a role, and then have that user don that role temporarily to do its work. You can even create an inline policy which is merely attached directly to that one user.

It turns out, you don’t need any of that if the user is specifically called out in the access policy for the object being accessed. So, in the end, I could simply remove all policies, groups, and roles from the application’s user so long as the S3 bucket call it out specifically.

Bucket Policy

Each S3 bucket can have its own security policy which specifically lists what each user (group, role, etc.) is permitted to do. As I before, I wanted to limit this user’s access to just those functions I knew my code was going to try to perform. So, I created a bucket policy which looked like this:

{
    "Version": "2012-10-17",
    "Id": "policy-123",
    "Statement": [
        {
            "Sid": "statement-id-123
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<user-id>:<user-name>"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::<bucket-name>/*"
        }
    ]
}

I knew I was only ever calling the PutObject API, so I didn’t want to grant any more permissions than that. However, no matter what I did, I kept getting the error message at the top of this post stating that I didn’t have permissions to do exactly the action listed!

After much fiddling about and reading of StackOverflow, I found the solution. I needed to grant more permissions!

{
    "Version": "2012-10-17",
    "Id": "policy-123",
    "Statement": [
        {
            "Sid": "statement-id-123
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<user-id>:<user-name>"
            },
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:PutObjectTagging"
            ]
            "Resource": "arn:aws:s3:::<bucket-name>/*"
        }
    ]
}

Even though I was only calling PutObject myself, the implementation of that API endpoint was both trying to set up the Access Control List (ACL) for the newly uploaded object, and, since I included some tags in my PutObject request, it was trying to set those as well. It was failing at both because I hadn’t set up those permissions.

The thing which threw me off the scent was the error message. It was merely saying that it couldn’t perform the PutObject API request because some permission was lacking. If it had clearly stated which permission was actually lacking, I would have been past this whole thing in a second. Instead, I erroneously assumed that the “PutObject” part of the message was referring to what I need to know in order to solve the problem (i.e., the name of the missing permission), not what I already knew (i.e., the name of API endpoint)! Since they have the same exact name, I thought telling me that it was lacking the one permission it actually did have!

So, a few lessons learned:

• User policies aren’t actually needed if the user is specifically mentioned in the bucket policy.
• If you get an access-denied message from AWS, the error will only mention the API which it couldn’t perform, not the actual permission it is lacking.
• API calls may require several permissions beyond simply the one which shares the name of the API call. Some of these are conditional, depending upon which additional parameters you provide in your call.
• AWS continues to have the most byzantine APIs and documentation in the business.