Modifying Gem : autoloading lib directory

I am somewhat new to Rails.  I would like to add some local features to the scruffy gem.  I copied the source files into my local lib directory so that I could add my enhancements.  My example works the first time, but then when I hit the page a second time I get the following error:

NameError (uninitialized constant Scruffy::Rasterizers::RMagickRasterizer):
    /Applications/Locomotive2/Bundles/rmagickRailsMar2007_i386.locobundle/framework/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:263:in `load_missing_constant'
    /Applications/Locomotive2/Bundles/rmagickRailsMar2007_i386.locobundle/framework/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:452:in `const_missing'
    /lib/scruffy/graph.rb:108:in `initialize'

Also I have made the following changes to my environment.rb file.

  config.load_paths += %W( #{RAILS_ROOT}/lib )
  config.load_paths += %W( #{RAILS_ROOT}/lib/scruffy )

The line in question is:

      self.rasterizer = Scruffy::Rasterizers::RMagickRasterizer.new

If I restart the server it works again for the first time.  I have been reading up on the auto-loading of rails 1.2 but I must be missing something simple. 

I appreciate any advice anyone could give me. 

Thanks in advance