AWS Cloud — Secure access to S3 buckets from EC2 using IAM Roles
S3 (Simple Storage Service) — Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as data lakes, websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of applications for companies all around the world.
S3 Buckets are a great option to store the files over the cloud and forget about the management and scalability as their flexible nature provides a reliable storage option for any type of application.
As S3 buckets are backed up by the object storage it's important to download the files first before making any modifications to them. There are many ways to access data stored on S3 buckets but the more important use case is to download them on EC2 machines and process the data.
Amazon EC2 (Elastic Compute Cloud) : Amazon Elastic Compute Cloud (Amazon EC2) offers the broadest and deepest compute platform, with over 500 instances and a choice of the latest processor, storage, networking, operating system, and purchase model to help you best match the needs of your workload. We are the first major cloud provider that supports Intel, AMD, and Arm processors, the only cloud with on-demand EC2 Mac instances, and the only cloud with 400 Gbps Ethernet networking.
Let’s learn how we can establish trust between EC2 instance and S3 bucket using AWS IAM Roles so that EC2 machine can access the data stored on S3 bucket and also upload the data back with modifications.
Create your S3 Bucket and EC2 Instance using the AWS Management console.
Setup IAM Role
Go to IAM service using your AWS management console.
Select Roles menu item then click on Create Role
Click on Next and Add Permissions to access your S3 Bucket.
You can define your own policy or select from the existing policies, AWS provides predefined Policy names S3 Full Access, you can choose it to provide read and write permissions on your S3 Bucket.
Note : You can also customize the policy in json editor and provide the ARN of the Bucket if you want to add permissions for specific buckets only.
You can combine multiple access permissions based on your requirement, after selecting the permission, Click on Next.
Now provide the Role Name e.g. EC2_S3_IAM_Role and review the settings, you can also add Tags.
if you are fine with the setting click on Create Role button.
You have successfully created IAM Role. Now you need to attach this role to your EC2 machine so that it can perform operations on S3 Bucket.
Go to your EC2 Console and select the EC2 instance on which you want to attach this role,
Go to Actions, Security, Modify IAM Role
Now you can select the IAM Role that you have created in the above steps and click on Update IAM Role.
It's Done, you can now perform operations from your EC2 machine to your S3 Bucket either using the command line or using the various SDKs provided by AWS.
e.g. aws s3 ls (to list all the available buckets)