For some background, Resque is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later. It uses redis-namespace, a 187 sloc gem that adds a Redis::Namespace class which can be used to namespace Redis keys.

Yesterday, a discussion took place in #redis regarding redis-namespace compatibility with redis-rb 2.0.0. There even was a suggestion to incorporate redis-namespace into redis-rb, so all libraries could profit from that solution. Even if I opposed that move (I consider redis-namespace the wrong solution to a simple problem), the underlying issue was not with redis-namespace, but with Resque itself. None of the participants in that discussion were direct redis-namespace users: they only wanted Resque to work with redis-rb 2.0.0.

Not being a Resque user myself, I had not bothered looking at its code before. Turns out I was able to make Resque 100% compatible with redis-rb 2.0.0 by replacing redis-namespace with… nothing. It wasn’t needed at all, and the namespacing functionality is now handled by an attr_accessor called namespace.

This code lives at my without-redis-namespace branch. I sent a pull request yesterday, I hope it will be incorporated soon to the main repository.

Explicit > Implicit

The previous version of the client relied on method_missing to execute commands against Redis. While this was great for quickly putting together a simple client, it proved insufficient for more advanced commands like pub/sub, blocking pop, etc. Plus, relying on this technique lead to longer and more branched methods as new features were added to Redis.

The new version explicitly defines each command as a method in the Redis class. We strongly believe that explicit beats implicit. This way it’s much easier to follow what is going on for each command. We did leave method_missing in there because it’s a nice way of being able to run simple commands that are not yet known to the client, but we’ll keep adding new commands as needed.

A nice consequence of this change is that the complexity of the code was reduced drastically. Many branches were removed in favor of separate, smaller methods (this is something that could only be done with an explicit method for each command).

New protocol

This version uses the new protocol only. It means people running a Redis version older than 1.2 will be stuck with the 1.0 branch. This multi-bulk protocol is binary safe and will be the only one available once Redis 2.0 is released, for which a release candidate is expected soon.

Faster!

One of our concerns when refactoring was the impact on performance this decoupling would bring. We agreed that a good goal would be to have the new version performing at least as fast as the old one. To our surprise –and probably because of the removal of branches, hash look-ups and by avoiding method_missing– the new version performed even better.

Before:

$ ruby -Ilib benchmarking/speed.rb
7.88 Kops

After:

$ ruby -Ilib benchmarking/speed.rb
8.79 Kops

The higher the number, the better. It means that the new version executes almost a thousand operations more per second.

Redis::Distributed

The old client shipped with DistRedis, which included a hashring algorithm and supported a small subset of operations. As it wasn’t tested or documented, we started by defining all the test cases and proceeded to implement them. Some operations are impossible in a distributed scenario, but we would cover a lot of interesting cases and it is now a very reliable tool. Check the testing suite to see the range of commands that are now possible.

Support/Bugs/Help

Jump to IRC if you need instant help. We are most of the time available at #redis-rb on Freenode. If nobody can help you there, try #redis. You can also create an issue on GitHub, or fork the project and send a patch directly.

By using hashes, Ohm is able to squeeze all the object attributes in a single key instead of using one key per attribute, a common pattern in key/value datastores (for some history, check the article that inspired Ohm almost one year ago: case study for a Twitter clone).

This release candidate uses Redis-rb 2.0.0.rc3, the new client that is faster, binary safe and no longer relies on method_missing for building the commands.

Upgrading your database

You will need to migrate your database if you want to use Ohm 0.1.0. It comes with a helper just for that, follow the example code to upgrade.

You will also need Redis 1.3.10 or newer, grab it from http://github.com/antirez/redis.

Within a day of the request, Citrusbyte had the new frame up and running on the Official Sony Electronics Facebook page.  Take a look for yourself:

It also includes tasks for starting and stopping Redis, and overall should provide more hints about what to configure and where.

To try it, just run monk init myapp, then cd myapp and monk to check the available options.

If you never installed Monk, run sudo gem install monk. If you already have it, there’s no need to update the gem.

So, how would you declare a reference to a DataMapper model or a set of ActiveRecord instances within Ohm?

class User < ActiveRecord::Base
  def posts
    Post.find(user: self)
  end
end

class Comment
  include DataMapper::Resource

  property :id,          Serial
  property :body,        String
end

class Post < Ohm::Model
  attribute :body
  reference :user, lambda { |id| User.find(id) }
  list      :comments, lambda { |id| Comment.get(id) }
end

Each time Ohm needs to refer to User, it will call the lambda the way it calls other Ohm models.

>> user = User.create :name => "foo"
>> post = Post.create :user => user, :body => "bar"

>> post.user.name
=> "foo"

>> comment = Comment.new
>> comment.attributes = { :body => "baz" }
>> comment.save
=> true

>> post.comments << comment
>> post.comments.first.body
>> "baz"

And what about Sequel? Because Sequel behaves exactly like Ohm, there’s no need for a lambda. You can mix them directly.

In a recent Forbes article Brian Reynolds, chief designer of Zynga, said his company was “platform agnostic,” stating Zynga “just [wants] to be on whatever platforms enable good experiences. We’re also on MySpace, and we’ve done iPhone stuff. It’s all about which platform allows it to be most social.”

Today, Citrusbyte announces tools to help your Facebook game be just a bit more platform agnostic with only a few minor code tweaks.

Commissioned by MySpace, Citrusbyte developed a reference toolkit for porting any Facebook app to the MySpace platform. The document goes through Facebook’s Authentication and RESTful APIs, noting their MySpace equivilent or providing code snippets whenever necessary. The document concludes with a step-by-step example of how to port one Facebook app to the MySpace platform, including before and after code.

Following this guide, Facebook game developers can quickly and easily bring their apps to the MySpace platform, accessing over 100 million additional active users.

In ActiveRecord it can be implemented like this:

validates_uniqueness_of :name

You may now add an index to your database and consider your task done, but it’s also true that you can improve the user experience by using a better error message:

validates_uniqueness_of :name, :message => "The name {{value}} is taken. <a href="/login">Log in</a> instead?"

Did you see the use of {{value}} in there? That’s because ActiveRecord’s DSL for validations operates on a class level, and that means that when you are declaring the validation, you don’t have access to the instance you will be dealing with. Note also that there’s HTML in the error message. It’s nice to point the user in the right direction, and here it only makes sense to send the user to the login page. But HTML in the models is wrong in many levels, not to mention that the URL helpers are not available there.

It would make much more sense to declare the validations at the instance level, something ActiveRecord allows with the validate method. If we had validates_uniqueness_of as an instance method, this would be possible:

def validate
  validates_uniqueness_of :name, :message => "The name #{name} is taken. <a href="/login">Log in</a> instead?"
end

A detail here is that we can build the message with Ruby interpolations instead of the {{value}} macro. The HTML is still there, but I will deal with that in another post.

Code smells in redundant DSLs

Another example of why a class level DSL for validations is a bad code smell:

validates_presence_of :card_number, :if => :paid_with_card?

def paid_with_card?
  payment_type == "card"
end

What’s wrong here? The validations DSL is reimplementing Ruby functionality, but not because it’s an improvement but because it’s operating on the wrong scope. Returning to the instance level approach, this is how it would look:

def validate
  validates_presence_of :card_number if paid_with_card?
end

There are two other examples in the Rails guides, which I reproduce here:

validates_presence_of :surname, :if => "name.nil?"

As you may know, this evaluates the string within the instance. Needless to say, it’s another case where the DSL reinvents the wheel for no good reason. The counter example:

def validate
  validates_presence_of :surname if name.nil?
end

And now the proc approach:

validates_confirmation_of :password,
  :unless => Proc.new { |a| a.password.blank? }

The counter example:

def validate
  validates_confirmation_of :password unless password.blank?
end

As you can see, these solutions are shorter and make the DSL look superfluous. It means that the DSL is unnecessary.

When you count the lines of code of your models, you should take into account how much code you are injecting. Your models could be hundreds of lines shorter if irrelevant DSLs like this one were removed.

From the documentation (updated after the conversation):

The attributes declared with attribute are only persisted after calling save. If the object is in an invalid state, no value is sent to Redis (see the section on Validations below).

Operations on attributes of type list, set and counter are possible only after the object is created (when it has an assigned id). Any operation on these kinds of attributes is performed immediately, without running the object validations. This design yields better performance than running the validations on each operation or buffering the operations and waiting for a call to save.

The idea of buffering the operations and waiting for a save is from Fictorial, who is porting Ohm to Node.js. Here is what he does:

What I did in NOM is make changes to the local/client-side object while at the same time buffer (in the same order) the commands needed to realize the local changes remotely in Redis. For instance, a List object has a push method. This is JS so the local object is an Array which is updated with the corresponding .push. At the same time, I buffer a “RPUSH key length\r\nvalue\r\n”. Then, if the validation passes, I send the buffered commands.

I know from my experience with Ohm that I’m comfortable with the current approach, and if I need to check for validity before operating on lists, sets or counters, I can use the pattern described in the docs.

Thoughts?

Ok we’re a few days late, but http://www.sharespost.com has just launched. Citrusbyte helped the company get to an internal beta in only 6 weeks with a lean team of two. We then ramped in their new internal developer who pushed the site to live.

Check out more about the company here: http://www.crunchbase.com/company/sharespost

And the press: http://www.sharespost.com/pages/press

SharesPost makes private equity liquid by efficiently matching buyers, sellers of private company stock and giving them the information, tools and process to make transactions easy and safe. At SharesPost you can download research reports and corporate documents for hundred’s of private companies, including Facebook, Twitter and LinkedIn. Plus, see prices from previous transactions. When you’re ready, connect directly with buyers and sellers of private company shares without brokers or their commissions. SharesPost provides you with automated contracts and integrated e-signature and escrow services to handle transfer restrictions like company rights of first refusal and help process your transaction.