Keeping binary files in database using mongoDB's GridFS

2011-06-29

In my side project, I tried to keep all binary files related to it in the database using mongoDB's GridFS. There are numerous posts about that, but what could stop me from writing another one? Main difference from them is the usage of rack-gridfs gem for GETting files.

If you haven't do anything yet you need to create new rails project skipping activerecord:

$ rails new rails-gridfs-demo -O

I decided to use carrierwave as a file uploader and mongoid as mongoDB ODM because they are much better than their analogs for a couple of reasons.

So we need to add them into the project:

Gemfile

gem 'mongoid'
gem 'bson_ext'   , '~> 1.3'
gem 'carrierwave', :git => 'https://github.com/jnicklas/carrierwave.git'
gem 'rack-gridfs', :git => 'https://github.com/skinandbones/rack-gridfs.git' \
                 , :require => 'rack/gridfs'

Next step is to run bundle command, generate mongoid config and configure it as default orm:

$ bundle
$ rails g mongoid:config

config/application.rb

config.generators.orm = :mongoid

Then we are ready to scaffold resource, for example, Image:

$ rails g scaffold Image

To use carrierwave as file uploader it's needed to mount it in the model but before that, we must create and configure it to use gridFS as file storage:

$ rails g uploader Image

app/uploaders/image_uploader.rb

storage :grid_fs

config/initializers/carrierwave.rb

CarrierWave.configure do |config|
  config.grid_fs_database = "rails_gridfs_demo_#{Rails.env}"
  config.grid_fs_host = 'localhost'
  config.grid_fs_access_url = "/uploads"
end

app/models/post.rb

class Image
  include Mongoid::Document

  mount_uploader :image, ImageUploader
end

Then add file field and pass :multipart => true parameter to our form:

app/views/images/_form.html.erb

<%= form_for @image, :html => { :multipart => true } do |f| %>
<..>
<%= f.label :image %>
<%= f.file_field :image %>

And show uploaded image:

app/views/images/show.html.erb

<%= image_tag(@image.image_url) %>

So it seems that everything is configured and ready to run, but if you try to upload an image, it will be uploaded successfully, but won't show up. It happens because carrierwave cares only about how to put your files in GridFS, but not how to get them, so this is why we need rack-gridfs

config/application.rb

config.middleware.insert_after Rack::Runtime, Rack::GridFS,
  :prefix => 'uploads',
  :lookup => :path,
  :database => "rails_gridfs_demo_#{Rails.env}"

From this moment everything should work as expected. Hurray! You can get this demo project from github.

Here is the benchmark using rainbows:

Server Software:
Server Hostname:        0.0.0.0
Server Port:            3000

Document Path:          /uploads/uploads/image/image/4e0a567569a99757d4000001/220px-MVC_schem.png
Document Length:        183 bytes

Concurrency Level:      10
Time taken for tests:   24.966 seconds
Complete requests:      15000
Failed requests:        0
Write errors:           0
Non-2xx responses:      15000
Total transferred:      5325000 bytes
HTML transferred:       2745000 bytes
Requests per second:    600.82 [#/sec] (mean)
Time per request:       16.644 [ms] (mean)
Time per request:       1.664 [ms] (mean, across all concurrent requests)
Transfer rate:          208.29 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0      12
Processing:     2   17   9.9     13      85
Waiting:        2   16   9.9     13      85
Total:          4   17   9.9     13      85

Percentage of the requests served within a certain time (ms)
  50%     13
  66%     13
  75%     14
  80%     14
  90%     40
  95%     41
  98%     44
  99%     50
 100%     85 (longest request)