Integrating Amazon S3 with Drupal for File Storage and Website Backup

Customize Amazon Web Services S3 for maximum security with user-specific access privileges.

Amazon Web Services Simple Storage Service (AWS S3) is an ideal space for backing up your Drupal websites, and it's an inexpensive place to store images and files.

There are multiple ways to gain access to your AWS account from a website or app. The simplest is to generate a root access key, plug it into your Drupal module, and it works. But this method is not recommended by Amazon, since it allows access to everything in your account. If you have multiple places accessing your AWS account, it's better security practice to set up individual users and let each one have access to only the service it needs. This means configuring IAM Users, a policy for each user, and an S3 Bucket Policy for each bucket.

This tutorial assumes you have an AWS account and have set up your first S3 bucket. The steps below will provide S3 access for Drupal's Backup and Migrate module, and also for using S3 as Drupal file storage via the  AmazonS3 module . (And possibly the S3 File System module but I haven't tested that yet.) It should also work for any other system, including Wordpress. I also use it to backup accounts from WHM/cPanel to S3.

First, set up an S3 bucket. Then we'll walk through these 5 steps:

  1. Set up a User in AWS
  2. Create a Bucket Policy that allows a user to access a specific bucket
  3. Attach the Bucket Policy to the S3 bucket
  4. Create/Attach a Policy to the User that designates what part(s) of your AWS account it has access to
  5. Add the User Credentials where needed in Drupal

Step 1. Set up the user

  • Log into your AWS account
  • Go to your account (upper right) > Security Credentials
  • Click Users in the left column
  • Click Create New Users
  • Add a name for the new user
  • Check Generate an access key for each user
  • Click Create button
  • See the Credentials by clicking Show User Security Credentials
  • Click Download Credentials

Learn more about User Credentials here

Step 2. Create a bucket policy for the user

Now that you have an S3 bucket, a user, and the user credentials, we'll create a bucket policy. 
In this example, the username is myclient, and its bucket name is myclient-bucket .
(You can give the user and the bucket the same name - I used 2 different names here because it makes this example easier to understand.) 

This is my Bucket Policy structure, and it contains 2 Statements: "List_client_policy" and  "ReadWrite_client_policy" - each one lets the user do specific things with a specific bucket. You'll need both. 

{
  "Statement": [
     {
      "Sid": "List_client_policy",
       "Action": [
         "s3:ListBucket",
         "s3:ListBucketVersions"
       ],
       "Effect": "Allow",
       "Resource": "arn:aws:s3:::myclient-bucket",
       "Principal":{"AWS":"arn:aws:iam::0123456789:user/myclient"}
     },
     {
       "Sid": "ReadWrite_client_policy",
       "Action": [
         "s3:GetObject",
         "s3:PutObject",
         "s3:DeleteObject"
       ],
       "Effect": "Allow",
       "Resource": "arn:aws:s3:::myclient-bucket/*",
       "Principal":{"AWS":"arn:aws:iam::0123456789:user/myclient"}
     }
  ]
} 

Here's a quick explanation of the parts, and what you'll need to change.

  • SID: Just a label that indentifies each Statement. Change this to whatever name you want, or leave as is.
  • Action: What action are you trying to achieve? Leave as is.
  • Effect: Are you Allowing or Denying the Action? Leave as is.
  • Resource: This identifies the bucket.
    • Change "myclient-bucket" to your bucket name. 
    • NOTE that in the "ReadWrite_client_policy", the bucket name must be followed by "/*" or you may get the error "Action does not apply to any resource(s) in statement". 
  • Principal: This indicates the user who can access the bucket.
    • Change "arn:aws:iam::0123456789:user/myclient" to your user's ARN.
    • To find it, go to your user list at https://console.aws.amazon.com/iam/home?#users, click on the user, and the "User ARN" is right at the top. Your number and username will be unique to your account.

Bucket policies can be customized in many ways - you can add users to groups, and assign permissions to those groups; add mulitple users to one bucket; allow access to only certain folders within a bucket, and much more. See this example page and this walkthrough if you need more information about bucket policies.

Step 3. Add bucket policy to each S3 bucket

Now that you've created a custom bucket policy, you'll add it to the bucket. 

  • Click Services in Upper left corner, choose S3
  • In the list of buckets, click icon to the left of the bucket you want to use
  • On the right panel that appears, click Permissions to open that section
  • Click Add bucket policy
  • Add the customized policy you created above - for each bucket, use the complete code; make sure to capture the entire structure, including the opening and closing brackets
  • Click Save

Step 4. Add an IAM User Policy

Your user also needs its own policy specifying what parts of your AWS system it is allowed to access.

Note: If you're only using AWS for the Backup & Migrate module, this step doesn't appear to be necessary - it works without it. But image storage using the S3 module won't work without it - it throws an error that starts with "Warning: filesize() [function.filesize]: stat failed for s3:...." 

  • From the User Console, click the User's name.
  • Click Attach User Policy.
  • You can add one 3 ways: start with a Template, use Amazon's policy generator, or write your own policy from scratch.
  • I used and modified the template for Amazon S3 Full Access, modifying it to include only the bucket that I want that user to access:
{ 
  "Version": "2012-10-17", 
  "Statement": [ 
    { 
    "Effect": "Allow", 
    "Action": "s3:*", 
    "Resource": "arn:aws:s3:::myclient-bucket/*"
    } 
  ] 
}

If you compare this to the default Amazon S3 Full Access template, the only difference is that I changed the "Resource" from all access ("*") to access only the specific bucket. You'll need to change "myclient-bucket" to match yours.

Step 5. Add the credentials to your Drupal website

Now on your Drupal website, you can enter the security credentials you generated in Step 1 at the configuration page for the module that requests it. In Drupal 7: For Backup & MIgrate, you'll add it as a Destination at admin/config/system/backup_migrate/settings/destination. For storing images and files, you'll add it to the AWS SDK (required by the AmazonS3 module) configuration page at admin/config/media/awssdk.