How to generate presigned URLs for S3 Private Bucket files via a Python Lambda Function

How to generate presigned URLs for S3 Private Bucket files via a Python Lambda Function

We are going to see how to generate presigned file URLs of the object stored in a private AWS S3 Bucket via a Lambda Function.

In some cases, we need temporary access to an object stored in a private S3 Bucket. If you want to access private files for a certain period of time, generating presigned URLs becomes handy to achieve that.

Before we go ahead we can try to access an S3 Bucket using HTTP URL copied from an S3 management console.

img

If the Bucket is private, and try to access that Object HTTP URL in the browser, you'll get access denied error.

img

Access denied error

A generated presigned URL is an HTTP S3 Bucket URL with access keys and expiration time attached to it. Example:

<https://my-Bucket-name.s3.amazonaws.com/path/to/the/object.jpg?AWSAccessKeyId=ASIA6JSMILHVNKWVP5OV&Signature=2MfQCtfKGK3%2BTrS3N24Fagj6V10%3D&x-amz-security-token=AgoJb3JpZ2luX2VjEMT%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMSJGMEQCIBkGj7wCSkEzlFI95Fs6GZvA8o8A1tB45%2BoDN6xmX1UrAiAcm%2FfFqduO7QMXsOSEfM%2F%2FZk4IAY4mYi5CLMumQxjVkCqMAgjs%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAAaDDk4MjYzMzY5Nzc3MCIMnouxqXWe2YUusUMVKuABJOwRrPpKUppY14XcMH8l4Ur7Kqk%2Bopvfv3L7EDiR67VDM9G5DJQGhFRanL2XR0%2BzOQ9%2FmmFgLBGmTxJShln7aBtiuJ4cozmf7rdHHdicjzVtNtz3rPRTvk25JFgQUaMgAbhVVX%2BElFcUZXDBMQULs1GX95NNTMITvdmEpxHgAPTlMABfe7APnkCAsjX1ErAjsqIpaewkB70buzCigf5JRahENFW2z9nvOjabJT9nVD%2Bc5125UVPT5qgwrCDDftg8%2FDaEmsrLQx737wCIHXYG6VnMaZM3FfG%2BnjbIXFGcdvYwnoaO6AU6tQGCwMnoDWdbb8OIIaBh1UGROAPW17shDB59tmIGU3js1KgBSh5zCeAEpT3WKIioZt0fZEazH4KZCku0%2FI3h8l9ONtOozbRSUlSGeOKE3ihsskEz6LTJr98qQtU5spRK8c%2BWtL4R7fTcUILbMib5S5BO5qRYvDhDWGnE7I5FKx56XYgNmorCcNM3O6odju2lzjpUKGOKWTw6zfcwV4smYNeEFgyL6A%2Ff56UwvKrkjbTSxKvKug4I&Expires=1560536518>

Lambda Function

We'll use a python Lambda Function which will return an object HTTP URL. There are numerous ways you can create a Lambda Function. In our script, we'll use AWS Boto which is the Amazon Web Services (AWS) SDK for Python. It is used to interact with different AWS services, such as S3 and Kinesis.

Here is the Lambda Function:

# AWS SDK
import boto3

# Lambda handler method
def handler(event, context):
  s3_client = boto3.client('s3')
  # AWS Bucket
  s3_bucket = 'images' 
  # Generate presigned product image urls:
  s3_image_path = 'images/picture.png' # S3 Object key
  presigned_image_url = s3_client.generate_presigned_url('get_object',
                      Params={'Bucket': s3_bucket,
                          'Key': s3_image_path},
                      ExpiresIn=60)
  return {
    'image': presigned_image_url
  }

The method to generate presigned URL in the above script is generate_presigned_url, it is taking three arguments:

Operation type: get_object

Params: a dictionary which specifies the S3 Bucket and the Object path

ExpiresIn: this specifies the period of time in seconds in which the URL is valid from the time of it is created.

You can read more about the method here

The above Lambda Function will generate HTTP URL link which will be valid for one minute.

If you try to access the generated link, you may get an error due to insufficient permission on the Bucket of the execution role of the Lambda Function. In that case you need to grant the execution role of the Lambda Function access to the S3 Bucket.

Grant access the Lambda Function Execution Role

You can get the Lambda Function execution role name from its page in Lambda Function Management Console. If not you can run the Lambda function using IAM role which has access to the AWS bucket you're using.

After obtaining the name, you can go to in IAM Management console → Roles → lambda_execution_role_name

Then you can add the inline policy to grant Lambda Execution Role access to the S3 Bucket

For example

{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": [
                    "s3:GetObject"
                ],
                "Resource": "arn:aws:s3:::images/*"
            }
        ]
    }

In the above policy summary, we're doing two main things:

  • We are granting read access only to the Lambda Function execution role:

    "Action": [ "s3:GetObject" ],

  • We are also specifying the S3 Bucket:

    "Resource": "arn:aws:s3:::images/*"

Presigned URL expires

When the period of time is over, the generated presigned URL will not be accessible again. When you try, you will get the following error.

img

Thanks for reading!](Link)