Rails and Amazon S3

I set out to write this post after struggling to build a Ruby on Rails application that uses Amazon S3. S3 is a service for storing and retrieving files over the Internet in a scalable, reliable and fast manner. Since Rails and S3 are popular tools, I assumed there would be no difficulty integrating them. That was not the case. The information I found online was inconsistent and convoluted. Hopefully, your experience integrating Rails and S3 more pleasurable than mine, thanks to this guide.

There are a bunch of different Ruby gems for handling file storage: The official AWS gems, AWS SDK for Ruby and AWS SDK for Rails, were too complicated. I found some third-party alternatives, including something called Paperclip, a file attachment library for ActiveRecord. The more I searched, the more I got overwhelmed by the options. I eventually settled on two gems: Fog, a Ruby cloud services library, and CarrierWave, a “classier” solution for file uploads. I picked them because they had good documentation and large communities.

Sign up for Amazon S3. Don’t worry – you won’t be charged for anything until you burn through the free tier. Go to the AWS Management Console, and select S3 from the Services menu.

Create a bucket, which is Amazon terminology for storage space. There’s a method to naming your buckets: Amazon S3 has a global namespace, so no two buckets can have the same name. Bucket names also have to comply with DNS, as they are used to form URLs. Once you have your bucket set up, you can add and remove “objects” (files) to the bucket through the Management Console. However, we won’t need this functionality, since we are planning to upload and view files through our Rails application.

To enable this functionality, you need to configure one more bucket setting. First of all, we don’t want just anyone making requests to our storage space. We need to selectively allow web applications running on other domains to access our bucket using CORS. Cross-origin resource sharing enables a server from one domain and a client from another to determine whether it’s safe to interact with each other.

Hit the Properties button, the Permissions tab, and Edit CORS Configuration. Paste the following code snippet in there, and save. We’re going to allow basic REST calls from the default Rails rails port for now, but remember to change this later on, or your bucket will remain exposed.

<CORSConfiguration>
  <CORSRule>
    <AllowedOrigin>http://localhost:3000</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
  </CORSRule>
</CORSConfiguration>

Create a new Rails application. I’m going to assume you have Rails installed already. Run:

$ rails new s3-demo

Switch into the s3-demo directory, and open the entire directory with your preferred text editor. First thing’s first: Add Fog and CarrierWave to the Gemfile:

gem 'carrierwave', '0.10.0'
gem 'fog'

Download the new gems with:

$ bundle install

Now, set up your AWS credentials. Since we don’t want to commit our credentials to version control, we should read them from a .env file. Create one with:

$ touch .env

Open the .env file, and paste your credentials in so that the file looks like this:

AWS_SECRET_ACCESS_KEY = YOUR_SECRET_ACCESS_KEY_HERE
AWS_ACCESS_KEY_ID = YOUR_ACCESS_KEY_ID_HERE
S3_BUCKET_NAME = YOUR_BUCKET_NAME_HERE

(Note: All-caps indicates a placeholder value.) Now, we need to inject these values into your secrets.yml file:

development:
  secret_key_base: LONG_STRING_OF_CHARACTERS
  amazon_access_key_id: ENV['AWS_ACCESS_KEY_ID']
  amazon_secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']

test:
  secret_key_base: LONG_STRING_OF_CHARACTERS

production:
  secret_key_base: ENV['SECRET_KEY_BASE']

I’ll be honest, I don’t know what the secret_key_base stuff is. I never looked.

Create an initializer file for CarrierWave. Run:

$ touch /config/initializers/s3.rb

In s3.rb, we need to inject our credentials again. Paste the following, and make the appropriate changes:

CarrierWave.configure do |config|

  config.fog_credentials = {
    provider: 'AWS',
    aws_access_key_id: ENV['AWS_ACCESS_KEY_ID'],
    aws_secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
    region: 'us-west-2',
  }

  config.fog_directory = ENV["S3_BUCKET_NAME"]

The region listed above should match your bucket’s region, which you can view in the Properties tab. I believe the corresponding region for Oregon is ‘us-west-2’. You could read this value from your .env also, but it’s not exactly a security flaw.

Okay, the CarrierWave initializer is making things confusing. What are we doing here? On a high level, we are configuring our file upload tool to use Fog for storing these files. Without this configuration, CarrierWave defaults to using the local filesystem (I think).

So we’ve gotten started with Amazon S3, created our Rails app, and configured everything. Now, it’s time to create a model that references the uploaded files. For example, a User model might reference a profile picture. These “references” are in fact string attributes, as you will see.

Start by generating a model:

$ rails generate model user

In the generated migration, we need to add the reference attribute:

class CreateUsers < ActiveRecord::Migration[5.0]
  def change
    create_table :users do |t|
      t.string :email
      t.string :password

      # Reference for mounting the CarrierWave Uploader
      t.string :picture

      t.timestamps
    end
  end
end

Next, we need to “mount” a CarrierWave Uploader to the reference attribute. Your User model should look like this:

class User < ApplicationRecord
  mount_uploader :picture, PictureUploader
end

What’s this PictureUploader class? It’s a child of the CarrierWave Uploader class, which handles the storing and retrieving of files. We need to create it:

$ touch app/uploaders/picture_uploader.rb

Inside:

class PictureUploader < CarrierWave::Uploader::Base
  storage :fog
end

We also need a controller for handling the routes:

$ rails generate controller users

Inside:

class UsersController < ApplicationController
  before_action :find_user, only: [:show, :update, :destroy]

  def index
    @users = User.all
  end

  def show
  end

  def new
    @user = User.new
  end

  def create
    @user = User.new(user_params)

    # PictureUploader stores file on save
    if @user.save
      redirect_to @user
    else
      render :new
    end
  end

  def destroy
    @user.destroy
    redirect_to users_url
  end

  private

    def find_user
      @user = User.find(params[:id])
    end

    # White list parameters
    def user_params
      params.require(:user).permit(:email, :password, :picture)
    end

end

Add these routes to your routes.rb file:

Rails.application.routes.draw do
  resources :users
  root 'users#index'
end

The only thing that’s left is to create our views. Run:

$ touch app/views/users/index.html.erb
$ touch app/views/users/show.html.erb
$ touch app/views/users/new.html.erb

Index.html.erb:

<h2>Index</h2>
<% @users.each do |user| %>

  <%= link_to user do %>
    <%= user.email %>
    <img src="<%= user.picture.url %>">
  <% end %>

<% end %>
<%= link_to 'New', new_user_path %>

The CarrierWave Uploader, since it’s mounted on the User picture attribute, gives access to a url subfield that contains the URL of the object in your S3 bucket.

Show.html.erb:

<h2>Show</h2>
<%= @user.email %>
<img src="<%= @user.picture.url %>">
<%= link_to 'Destroy', @user,  method: :delete, data: { confirm: 'Are you sure?' } %>

New.html.erb:

<h2>New</h2>
<%= form_for(@user) do |f| %>
<div>
    <%= f.label :email %>
<%= f.text_field :email %>
<%= f.label :password %>
<%= f.text_field :password %>
<%= f.label :picture %>
<%= f.file_field :picture %></div>
<div>
    <%= f.submit 'Upload' %></div>
<% end %>
<%= link_to 'Back', users_path %>

Anyone who has experience with user profiles in Rails will point out that this is an extremely poor implementation. Keep in mind that I’m just using User as an example of a model that might have a file associated with it – in this case, a profile picture.

That’s everything. Now you should start up the server and try creating a new User. If the file upload is successful, it should appear in your S3 bucket shortly. You should also be able to view the picture through the index and show routes on your application.

You have done it! You have built a Rails app that uses Amazon S3. Phew.

There are a lot of places where this demo could be improved. For example, you can currently upload any type of file (not just pictures) when creating a new User. But I’ll leave that for future posts.