Jamie van Dyke

One of my last projects required me to spend a ridiculous amount of time categorising images into groups of colour. What a pain in the gluteus maximus. So bollocks to that, let’s see if we can’t automate it a little to cut down on the flak.

Prepare

So, the tools for today’s script are:

1. Imagemagick (Mac OS X: port install imagemagick)
2. Color-Tools (gem install color-tools)
3. Optional: Ruby on Rails (I’m doing this in a Rails app)
4. Optional: ActiveRecord
5. Optional: Paperclip (For file uploads)
6. Optional: Marmite on Toast (Om nom nom)

Once you have these tools, then let’s get a rails application together and a model to handle our images.

23:57 abloke@hercules:~/a/demo % rails classify
      ...
      create  log/test.log
23:57 abloke@hercules:~/a/demo % ./script/plugin install git://github.com/thoughtbot/paperclip.git
From git://github.com/thoughtbot/paperclip
 * branch            HEAD       -> FETCH_HEAD
23:57 abloke@hercules:~/a/demo % ./script/generate model Fabric

Adjust the migration to add the paperclip fields (what the heck, add a design field too).

# app/models/fabric.rb
class CreateFabrics < ActiveRecord::Migration
  def self.up
    create_table :fabrics do |t|
      t.string   :design
      t.string   :nearest_color
      t.string   :file_file_name, :string
      t.string   :file_content_type, :string
      t.integer  :file_file_size
      t.datetime :file_updated_at

      t.timestamps
    end
  end

  def self.down
    drop_table :fabrics
  end
end

Add the Fabric model with our Paperclip method call has_attached_file.

  class Fabric < ActiveRecord::Base
    has_attached_file :file, 
                      :styles => { :large => "340x340#", :medium => "135x135#", :thumb => "70x70#" }
  end

Now, let’s not jump right in to using a form in our Rails application, nobody likes to be premature. We’re going to play around on the console, and then you can decide to implement this in a rake task, a script, or a form. We’re going to need to require our color-tools.

# config/initializers/color-tools.rb
require 'color/palette/monocontrast'

Now let’s put a few helpers on our model so that we know everything’s working okay.

class Fabric < ActiveRecord::Base
  has_attached_file :file, 
                    :styles => { :large => "340x340#", :medium => "135x135#", :thumb => "70x70#" }

  attr_reader :red, :green, :blue
  
  def to_rgb
    command = "convert #{file.to_file.path} -scale 1x1\! -format '%[pixel:u]' info:-"
    color = %x[#{command}]
    
    if color && $?.exitstatus != 0
      raise StandardError, "There was an error determining the color!"
    end
    @red, @green, @blue = color[/rgb\((.*)\)/, 1].split(",").collect(&:to_i)
  end
end

This could be a lot cleverer? Silence minion! We’re just hacking away. Let’s try this out on our trusty console. Grab a lollipop (I have red or green or blue for your picking) and tally ho!

00:20 abloke@hercules:~/a/demo/classify % rake db:migrate
(in /Users/abloke/a/demo/classify)
==  CreateFabrics: migrating ==================================================
-- create_table(:fabrics)
   -> 0.0100s
==  CreateFabrics: migrated (0.0102s) =========================================

00:20 abloke@hercules:~/a/demo/classify % ./script/console --sandbox
Loading development environment in sandbox (Rails 2.2.2)
Any modifications you make will be rolled back on exit
>> f = Fabric.new :design => 'red'
=> #<Fabric id: nil, design: "red", file_file_name: nil, string: nil, file_content_type: nil, file_file_size: nil, file_updated_at: nil, created_at: nil, updated_at: nil>
>> f.file = File.open('test/fixtures/images/red.jpg')
=> #<File:test/fixtures/images/red.jpg>
>> f.to_rgb
=> [247, 2, 1]

Pow! Now we have RGB values for any files we upload, that’s pretty schweet. Let me explain.

command = "convert #{file.to_file.path} -scale 1x1\! -format '%[pixel:u]' info:-"
color = %x[#{command}]

Here we build up a command that we’re going to fire off on the command line. The command is ‘convert’, which is an ImageMagick command that has lots of juicy options. One of them is --scale 1x1\! which will reduce an image (!) to whatever dimensions you give it. We give an FX Expression of --format '%[pixel:u]', this lets ImageMagick know we want the first image out of our sequence (yes we only have one, but it’s necessary). Finally, we pass info:-, which changes the output from a file to information only.

So really it’s self explanatory. Take our image, squash it to 1 pixel (giving us the predominant colour), then output RGB information on it. We take that information and parse it. So what now, buttercup? Well, surely we don’t want to categorise by just red green or blue, we need something else.

class Fabric < ActiveRecord::Base
  COLORS = {
    :blacks   => { :readable => 'Blacks & Aubergines',  :rgb => [ 30, 0, 0 ] },
    :browns   => { :readable => 'Beiges & Browns',      :rgb => [ 150, 75, 0 ] }, 
    :greens   => { :readable => 'Greens',               :rgb => [ 18, 60, 53 ] }, 
    :neutrals => { :readable => 'Neutrals',             :rgb => [ 210, 188, 151 ] }, 
    :reds     => { :readable => 'Reds & Pinks',         :rgb => [ 133, 51, 53 ] },
    :blues    => { :readable => 'Blues',                :rgb => [ 131, 155, 181 ] }
  }
  
  def self.readable_color_for( color )
    COLORS[color.to_sym][:readable]
  rescue NoMethodError
    raise Fabric::UnknownColorException
  end
end

Here’s a simple set of colours I want to group my images in. The RGB needs tweaking, but it will do for our examples. I’ve made a hash of the RGB value and the readable version, I dislike this implementation immensely, but this was only hacked together for an initial import so there’s no need to be anal. At the bottom I tagged on a quick helper for iterating over in my views.

Okay, now on to categorizing the colours by group. To start off with, I’ll override Paperclip’s file= method, because I want access an uploaded file immediately.

def file=( file_object )
  categorize(file_object.to_tempfile.path) unless file_object.is_a? String
  attachment_for(:file).assign(file_object)
end

def to_rgb( filename=nil )
  filename ||= file.to_file.nil? ? nil : file.to_file.path
  raise("There is no image to determine the RGB values on") if filename.nil?
  command = "convert #{filename} -scale 1x1\! -format '%[pixel:u]' info:-"
  color = %x[#{command}]
  
  if color && $?.exitstatus != 0 && @whiny_thumbnails
    raise StandardError, "There was an error determining the color!"
  end
  @red, @green, @blue = color[/rgb\((.*)\)/, 1].split(",").collect(&:to_i)
end

def to_hex
  to_rgb if @red.nil? or @green.nil? or @blue.nil?
  "#%2x%2x%2x" % [ @red, @green, @blue ]
end

Once I’ve categorised the file I want Paperclip to continue on with it’s business, so I use attachment_for. Notice I’ve shown the to_rgb method again, purely because I changed it. When file= is called we don’t have a file object yet, it’s not been moved from the Tempfile that uploading creates, so we pass over the filename directly. I’ve also shown my rudimentary to_hex method (with a recursive error problem that could potentially lock our app up, note to user), which was useful for colouring in text field backgrounds with visual indicators of the colour group, for debugging.

Next, the actual categorising…grip on to something, this is both ugly and confusing!

def categorize( filename=nil )
  filename ||= file.to_file.nil? ? nil : file.to_file.path
  # Create a new contrast object
  mono = Color::Palette::MonoContrast.new(Color::RGB.new(0,0,0))
  # Grab the RGB values of our current image
  my_color = self.to_rgb(filename)
  compare_to = Color::RGB.new(my_color[0], my_color[1], my_color[2])
  
  # Which category does it have the least contrast against?
  self.nearest_color = COLORS.min do |a, b| 
    color1 = Color::RGB.new(a[1][:rgb][0],a[1][:rgb][1],a[1][:rgb][2])
    color2 = Color::RGB.new(b[1][:rgb][0],b[1][:rgb][1],b[1][:rgb][2])
    mono.color_diff(color1, compare_to) <=> mono.color_diff(color2, compare_to)
  end[0].to_s
end

Oh my gosh. I am ashamed of this hackery, but it did the trick in a short amount of time. The key to what we’re doing here is using the MonoContrast class to tell us the difference in contrast between two colours. Which will work for the most part, but I wouldn’t enter it into any competitions!

In essence its really simple, just badly written. The first part is commented, as for the second part, we merely loop over each colour group creating an RGB object for each. Once we have that we can get a value for the amount of contrast difference. By doing this in a min loop on the Hash, we get the winner.

Summary

Clearly this isn’t the most accurate colour categorising in the world. In some cases it’s plain stupid. However, with the limited amount of knowledge I have on ImageMagick, and technical info on colours, this worked for my simple little task. It took a total of 30 minutes (ignoring the 30 minutes I tweaked it after I’d used it for the import), and if it helps anyone then I’m happy to have helped. It was certainly an interesting problem and no doubt I’ll revisit it later as there are numerous applications. I mean, what if a user would like to upload a photo of their kitchen and be shown matching items that would go with it? Interesting possibility.

Farewell, my blog reading chums.