May 12, 2008

Ruby Symbols in 1.9—End of an Era (and a good thing, too)

:rip.to_i

Ruby symbols have always been immediate objects. That means that, inside the interpreter, they were represented as small integers which reference the corresponding symbol text in a lookup table. This made them fast, but it also left code open to denial of service attacks (particular in the context of web applications)—malicious clients could force server code to create arbitrary numbers of symbol table entries, and these were never garbage collected.

Some recent changes in Ruby 1.9 point to the transition away from symbols being immediate objects. In particular, they lose their integer representation, and hence the methods Fixnum.id2name, Fixnum.to_sym, and Symbol.to_i have been removed. I'm expecting to see symbols migrate to the heap as 1.9 continues to evolve.

Just shipped tons of books (literally)

Arr_deploy

In the last week we shipped out two much anticipated books: Advanced Rails Recipes and Deploying Rails Applications: A Step-by-Step Guide. We've finally reached the kind of shipping volumes where we get special treatment at the Post Office—I'm now allowed to make deliveries around the back, driving my truck up to the loading dock between all the 50' monsters. It's an interesting insight into their business. The vast majority of stuff sitting out there during the day is the free advertising fliers that get added to your mail—pallet upon pallet of them. If we want to save the forests, finding some way of banning those things would be a start. (But, of course, they probably subsidize the mail, so doing so would double the cost of postage. In this case, that's a price I might be willing to pay.)

Also (getting back on topic), Mike Clark has made a 12 minute video highlighting some of the more visual recipes in his book. Check out the link on the book's home page.

April 28, 2008

Shoulda used this earlier

In many ways, testing software is like going out and getting exercise. You know you should do it, and you know it does you good, but it's also pretty easy to find an excuse to skip it (I'll make it up tomorrow).

So anything that makes testing easier is good, because it cuts down on the excuses not to do it.

One thing I've never really liked about the conventional xUnit-style testing frameworks was the setup and teardown structure. In these frameworks, a test case is a class, and setup and teardown are implemented by methods in that class. Each test is also a method, so the basic flow is

  for each test method in the class
    run setup
    run the test method
    run teardown
  end

Nice and simple. Each test method got the benefit of a standard environment created by the setup method, and the teardown method got the job of tidying up after.

Except… when I'm writing tests, I typically want to set up lots of different scenarios. I'll want A and B and C, then A and B but not C, then A and not B, then A and D, and so on. I had two choices—write lots of test case classes, using subclassing to inherit common setup behavior, or write per-test method setup code (often factored out into helpers). In the end, I almost always did the latter, And that was tedious, and it made it harder to see the tests for the setup code.

I flirted with RSpec. Its spec framework seemed to have what I wanted. But I just couldn't get myself to enjoy using it. (I think it's a cat people/dog people kind of thing)

Enter shoulda

Then, a couple of weeks back, Mike Clark and Chad Fowler introduced me to shoulda. Shoulda isn't a testing framework. Instead, it extends Ruby's existing Test::Unit framework with the idea of test contexts. A context is a section of your test case where all the test methods have something in common. At it simplest, a context could be simply used as an annotation device (and, yes, this is a silly example):

context "My factorial method" do
  should "return 1 when passed 0" do
    assert_equal 1, fact(0)
  end
  should "return 1 when passed 1" do
    assert_equal 1, fact(1)
  end
  should "return 6 when passed 3" do
    assert_equal 6, fact(3)
  end
end    

The stuff in a context can share common setup code—just write a setup block.

class CartTest < Test::Unit::TestCase

  context "An empty cart" do
    setup do
      @cart = orders(:wilmas_empty_cart)
    end

    should "have no line items" do
      assert_equal 0, @cart.line_items.size
    end

    should "have a zero price" do
      assert_equal 0, @cart.price
    end
  end

  context "Some other context..." ...
  end
end

So now, within a single test case I can set up multiple contexts, and each context can have its own environment.

But, take it back to my original problem. I often want to set up hierarchies of related environments for my tests. The shaoulda code handles this wonderfully, because it lets me nest contexts. For example, I'm adding a feature to our store that gives customers some additional information if, during checkout, their credit card transaction was initially rejected because the address was wrong, and was then accepted when they fixed the address. I wanted two tests, one without the prior address error, and one with.

To set up this environment, I needed to set up a shopping cart, create a dummy response from our payment gateway, and post that response to the application. In the case of the prior address error, I also wanted to inject an entry containing that error into the transactions associated with the order prior to generating the response.

With shoulda, I simply created some nested contexts. The top level context did the shared setup, and the inner contexts then set up appropriate environments for their tests. It looked like this:

  context "Checking out"  do
    setup do
      @cart = cart_named(:freds_full_cart)
      @cart.prepare_for_store_authorize!
      @params = approved_authnet_response(@cart)
    end                  
    
    context "with no AVS errors in CC transaction history" do
      setup do
        post :post_from_authnet_authorize, @params
      end

      should_redirect_to "{:action => :receipt}"
    end 
    
    context "with AVS errors in CC transaction history" do
      setup do
        avs_error = CcTransaction.new(:response_code => 2, :response_reason_code => 27)
        @cart.cc_transactions << avs_error
        post :post_from_authnet_authorize, @params
      end

      should_redirect_to "{:action => :explain_avs_mismatch}"
    end
  end 

The outer setup gets run before the execution of each of the inner contexts. And the setup in the inner contexts gets run when running that context. And shoulda keeps track of it all, so I get very natural error messages if an assertion fails. For example, if the test in the second context above fails, I'd get

Checking out with AVS errors in CC transaction history should 
redirect to "{:action => :explain_avs_mixsmatch}". 

So, now, I can finally set up my hierarchies of test environments in a natural way. It isn't revolutionary. It's just one less excuse for not testing…

April 11, 2008

Ruby 1.9 Standard Library Changes

Here's a top-level overview of some of the changes to date in the standard library that comes with Ruby 1.9. (These are the libraries that you get preinstalled with Ruby, but that you have to require into your code.)

  • The base64 library has been removed. Use Array#pack and String#unpack instead.
  • Much of the Complex and Rational libraries are now built in to the interpreter. However, requiring the external libraries adds additional functionally. In the case of Rational, this functionality is minimal.
  • The CMath library has been added.
  • The Enumerator library is now built in.
  • Added Fiber library (adds coroutine support to fibers).
  • Removed ftools (replaced by fileutils).
  • The Generator library has been removed (use Fibers).
  • Added notes on using irb from inside applications.
  • jcode is removed in favor of built-in encoding support.
  • The json library is added.
  • The matrix library no longer requires that you include mathn.
  • The mutex library is now built in.
  • parsedate has been removed. The Date class handles most of its functionality.
  • readbytes has been removed. IO now supports the method directly.
  • require_relative added.
  • Add description of Ripper.
  • Add description of SecureRandom.
  • I've omitted the shell library, as it seems more like a curiosity than something folks would use (and it's broken under 1.9).
  • The soap library is removed.
  • I've omitted the sync library. It is broken under 1.9, and the monitor library seems to be cleaner.
  • Win32API is now deprecated in favor of using the DL library.

It's interesting to me just how much is still changing in Ruby 1.9. But, as I use it more and more, it's also gratifying to see how some of the new idioms make coding just that little sweeter.

I just pushed a new beta of the PickAxe Third Edition with all the library changes.

April 09, 2008

BabyDoc

One of the fun things about updating the PickAxe is getting to come up with examples to show the various APIs in action. Here's a very silly example of using Ripper's event-based API to extract comments that are associated with basic class definitions. It clearly has holes (it doesn't handle class A::B::C, for instance) but it's fairly easy to see how to add a proper state machine and produce something that might be interesting to play with...

require 'ripper'

# This class handles parser events, extracting
# comments and attaching them to class definitions
class BabyRDoc < Ripper::Filter
  def initialize(*)
    super
    reset_state
  end

  def on_default(event, token, output)
    reset_state
    output
  end

  def on_sp(token, output) output end
  alias on_nil on_sp

  def on_comment(comment, output)
    @comment << comment.sub(/^\s*#\s*/, "    ")
    output
  end

  def on_kw(name, output)
    @expecting_class_name = (name == 'class')
    output
  end

  def on_const(name, output)
    if @expecting_class_name
      output << "#{name}:\n"
      output <<  @comment
    end
    reset_state
    output
  end

  private

  def reset_state
    @comment = ""
    @expecting_class_name = false
  end
end

BabyRDoc.new(File.read(__FILE__)).parse(STDOUT)

Run this with Ruby 1.9 (or, I guess, 1.8 with Ripper installed), and you'll see

BabyRDoc: 
    This class handles parser events, extracting 
    comments and attaching them to class definitions

April 08, 2008

Fun with Ruby 1.9 File Encodings

Ruby 1.9 allows you to specify the character encodings of I/O streams, strings, regexps, symbols, and so on. It also lets you specify the encoding of individual source files (and a complete application can be built from many files, each with different character encodings). Expect to start seeing a rash of obscure source code, at least until the initial excitement abates and cooler thinking prevails.

In the meantime, we can get away with


# encoding: utf-8
require 'mathn'
class Numeric
   def ℃
     (self - 32) * 5/9
   end
   def ℉
     self * 9/5 + 32
   end
end
 
puts 212.℃
puts 100.℉

Or, for those who'd like a peek at the start of a road that eventually leads to madness:


alias ✎ puts 
 
✎ 212.℃
✎ 100.℉

I'm betting this post displays badly on about 50% of the machines that are used to view it. Which is reason enough to tread very lightly down this path…

April 03, 2008

Importing RSS Feeds into Mail.app

As an experiment, I wanted to try using Mail.app instead of NetNewswire to read RSS feeds (I know, I know...). To get my existing feeds across, I did the following.

  1. In NNW, export your feeds as an OPML file (say to ~Desktop/MySubscriptions.opml)
  2. In the directory holding that file, do
  3. % egrep -o 'xmlUrl="(.*?)"' MySubscriptions.opml | egrep -o 'http.*[^"]' | pbcopy
    
  4. In Mail.App, select “Add RSS Feed”, and in the text area that pops up, paste in the clipboard.

March 24, 2008

I'd like to apologize

Just before the weekend, I did something stupid and hurtful, and I'd like to make it right.

For the last few weeks, I've been getting e-mails about a book under development over at O'Reilly called Software Craftsmanship—from Apprentice to Journeyman. People kept pointing out that the main title was the same as Pete McBreen's book (for which I wrote the foreword), and that the overall structure of the title was similar to that of The Pragmatic Programmer.

After a while, this started to get under my skin. I wasn't so much concerned about the “journeyman” bit, but the duplication of the title just seemed wrong to me—I really liked Pete's book, and I didn't want to see it getting eclipsed. I complained about this to a senior editor at O'Reilly, and he said he'd bring it up with the book's editor, who worked for him. I heard nothing back.

So, at the end of a tiring week, I wrote a blog post, complaining about the title.

That was wrong of me.

It was wrong for a number of reasons.

  • I could, and probably should, have bypassed etiquette and contacted the authors directly, even though they write for a rival publisher.
  • It really wasn't any of my business.
  • But, most importantly, it took something which was a kind of intellectual annoyance and turned it into something that made the authors of the book feel bad. And for that, I apologize.

This year, I've been the target of some cruel blog posts. Most readers of these posts viewed them as fine sport. But as the recipient of the criticism, I'm here to tell you that it hurts. It doesn't matter whether it is based on truth or whether it isn't. It doesn't matter whether the person writing them knows you or is a total stranger. It hurts. Public attacks like this are virtually impossible to defend against, and that is a cruel violation. It's cruel when it is done to you, and it's cruel when you do it to others.

So, I of all people should have known better. I should have had the common sense to realize that my comments, aimed at a book, were going to be hurtful to the authors. It's kind of obvious, really.

I wasn't thinking straight, and I messed up.

So, Dave and Ade, I'm sorry for any distress I caused.

Good luck with your book.

March 19, 2008

Ruby 1.9 Built-in Library--Finished First Pass

One of the scary things about revving the PickAxe for Ruby 1.9 is updating the reference section where I describe all the built-in classes and methods. It involves working through the interpreter source, looking for all the rb_define_method calls (and their friends) and then reading the C implementation of the corresponding methods. Many methods are unchanged from 1.8. But, at the same time, many have changed. Often they take an additional parameter, or return an Enumerator where previously they required a block. Then there are the new classes (something like 6 of them) and new methods. (It looks like there are over 200 new built-in methods in the current Ruby 1.9).

All in all, I count something like 300 [1.9] flags in the new library reference. Some flag stuff is as trivial as a change of a default return type, while others flag entire new classes.

It's incredibly time-consuming work, and I'm constantly grumbling while doing it. But I come out the other end knowing a whole bunch about the library, and with a deeper respect for the folks who maintain it.

March 18, 2008

Complex and Rational are now built-in to 1.9

Just when I thought I'd finished documenting the standard library for the new PickAxe, I did one last svn up of the Ruby interpreter source and discovered that the Complex and Rational classes are now builtins—no need to require the library to get the basic functionality. The change also affects a number of other built-in classes (you can now say nil.to_c, for example). I'm not 100% sure I agree with rolling in Complex, but the addition of rational numbers is a welcome change.

Unfortunately it's back to the drawing board on my plans to release a new beta today...

Now in Beta

  • Programming Ruby, 3rd Edition
    Third Edition, Covering Ruby 1.9, now in beta
My Photo

Site Search

  • Google Search

    The web
    PragDave

Pragmatic Stuff

Photos

  • www.flickr.com
    This is a Flickr badge showing public photos from pragdave tagged with pragdave_badge. Make your own badge here.