How to Secure S3 Objects Using a Presigned URL

Photo by Glenn Carstens-Peters on Unsplash

Suppose we need to expose an API where the client can upload an image file to S3 and the backend server needs to send back the response with the S3 object URLs. How can we achieve this securely without exposing our S3 bucket URLs to the public? I am writing this article to explain how we can leverage the power of AWS Presigned URLs to achieve the above task.

If our mind is full of problems, we will evolve faster by solving them faster.

Prerequisites

1. Basic Knowledge of AWS S3
2. Any Programming Language (using Python here)
3. Can’t think of anything else :)

Why need Presigned URLs?

Most companies widely use the Amazon AWS S3 (Simple Storage Service) to store objects, files, or more data that is persistent and easily accessible. We can easily integrate AWS S3 buckets with almost any modern infrastructure such as mobile applications, web applications, etc. If we use S3 in our infrastructure, we need to return the secure URL to the S3 objects so that if anyone tried to steal that URL and access the S3 object then it itself gets expired after some time. And to make this secured URL, S3 provides the so-called Presigned URLs.

Let's say we host some files on an S3 bucket and we need to expose these to a user but we don’t want to set up the bucket as open (public), also we want to keep some control over the access to these files, for example by limiting the time-frame where the files can be accessed by the user. We can expose our files present in the S3 bucket to the outside world without making the bucket public but by sharing the resource using Presigned URLs for the specific time period. Here is how a Presigned URL looks like.

https://bucket.s3.region.amazonaws.com/myfile.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=random-aws-credential-to-identify-the-signer&X-Amz-Date=timestamp-of-generation-of-url&X-Amz-Expires=validity-from-generation-timestamp&X-Amz-Signature=6ffca338-f5b2-48ad-89ec-4ae462cb46dc&X-Amz-SignedHeaders=host

Formatted Presigned URL

https://bucket.s3.region.amazonaws.com/myfile.jpg ?
X-Amz-Algorithm     =   AWS4-HMAC-SHA256 &
X-Amz-Credential    =   random-aws-credential-to-identify-the-signature &
X-Amz-Date          =   timestamp-of-generation-of-url &
X-Amz-Expires       =   validity-from-generation-timestamp &
X-Amz-Signature     =   6ffca338f5b248ad89ec4ae462cb46dc &
X-Amz-SignedHeaders =   host

We can generate the signed headers and credentials of the Presigned URL using AWS SDK functionalities. When someone tries to access the S3 files using the Presigned URL then, S3 will try to compute the same signature for the specified credentials, including into its calculation the optional SignedHeaders parameter and checking if the signature is valid and if the link is not expired yet.

Generate Presigned URL for S3 Bucket using Python

First, let's install the AWS SDK for Python ie, Boto3. Boto3 makes it easy to integrate our python application, library, or script with AWS services which include S3, EC2, Amazon Dynamo DB, etc.

pip install boto3

Now we can generate Presigned URL for our S3 bucket using the below code snippets.

import boto3
AWS_S3_REGION = 'ap-south-1'
AWS_S3_BUCKET_NAME = "my_s3_bucket"
AWS_S3_FILE_NAME = "my-file.jpg"
PRESIGNED_URL_EXPIRY = 100 # in seconds
s3_client = boto3.client('s3', aws_access_key_id=AWS_ACCESS_KEY_ID, region_name=AWS_S3_REGION, aws_secret_access_key=AWS_SECRET_ACCESS_KEY,)
presigned_url = s3_client.generate_presigned_url('get_object',    Params={"Bucket": AWS_S3_BUCKET_NAME, "Key": AWS_S3_FILE_NAME}, ExpiresIn=PRESIGNED_URL_EXPIRY)

How can we use Presigned URLs to upload resources on the AWS S3 bucket?

We can simply make a PUT request to the Presigned URL created with action “put_object” and the file will be uploaded to S3 securely.

Block diagram for uploading the file to S3 using Presigned URL

Let's first create the Presigned URL which can upload files to S3 using the below code snippet.

import boto3
s3_client = boto3.client('s3', aws_access_key_id=AWS_ACCESS_KEY_ID, region_name=AWS_S3_REGION,aws_secret_access_key=AWS_SECRET_ACCESS_KEY, config=Config(signature_version='s3v4'))
presigned_url = s3_client.generate_presigned_url('put_object',
Params={"Bucket": AWS_S3_BUCKET_NAME, "Key": AWS_S3_FILE_NAME},
ExpiresIn=PRESIGNED_URL_EXPIRY)
print presigned_url

Now we can use the above Presigned URL and upload the file using the below code snippet.

import requests

response = requests.put(presigned_url)
print "File uploaded successfully!"

Greetings! 🙂

We have successfully created the Presigned URL for our S3 bucket and now we can securely share this URL with our client. The client will be able to access the resources up to the expiry time of the Presigned URL. Also, the client is able to upload the files to S3 using the Presigned URL up to expiry time.

• We can use Presigned URLs to generate a URL that can be used to access our S3 buckets.
• When we create a Presigned URL and associate it with a specific action.
• When we share the Presigned URL with anyone then that user can perform the action embedded in the URL as if they were the original signing user.
• The URL will expire and no longer be able to access the S3 files when it reached its expiration time.
• The capabilities of the Presgined URL are limited to the permissions of the user who is actually creating the Presigned URL.

Summary

For the sake of completeness of this article, let’s have a quick recap of what we learn till now.

• Learned it's not safe to create the S3 bucket public every time.
• Learned it's not the backend server that can only upload files to S3, but by using Presigned URLs clients can also upload files to S3.
• We learned how can we securely share the S3 files with our client using Presigned URLs.
• We learned how we can give responsibility to clients for uploading any files to S3.

If you enjoyed this article, don’t forget to give it a clap!

Please feel free to ping me on Linkedin and stay tuned for the next one!