Will paginate for rescue

Will paginate is great plugin, used almost by every Rails application I have seen. But I’ve seen this plugin used to generate handful navigation among huge datasets. Well it was written with this in mind, but today I have found another use.

How often Your application does need export huge dataset as CSV for example? Quite often. First approach could be:

@l = Model.find :all
    str = ""
    CSV::Writer.generate(str) do |csv|
      @l.each {|l|
        csv << [ l.fields ]
      }    
    end
    send_data  str, :type => 'text/csv', :filename => 'model.csv'

This works, but… If You are operating with some memory limit (like on most shared hosting setups or on small instance of VPS) this could lead to problems. One of applications I have seen with this approach and exporting 35k objects (each with one has_one association loaded with :include) just after restart and this export done have used 215 MB (as reported by passenger-memory-stats). On VPS with 256 MB it will result in swaping out this app – and next requests will be handled with latency due to swap. And process won’t give back this memory until passenger kill this instance in normal life cycle.

Will_paginate for rescue!

Continue reading

Using non text values as arguments in functional tests

When running functional tests, You use methods like get, post, etc. a lot. They take as arguments action name to call (as a symbol) and hash with parameters. But You need to remember that this hash it is not exactly the same what params object is (from controller side).

The main difference is that params is (more or less) hash with elements being strings or other params-like hashes (results of processing HTML forms with fields named with Rails conventions object[attribute_name]). And when You call get/post from functional test, parameters hash is full blown Ruby hash and as such is passed to controller, without any changes. So it is possible to pass whole Ruby objects as values in params in test environment.

For example if You expect argument params[:time] and run Time.parse(params[:time]) in controller You will encounter error (like: NoMethodError: private method `gsub!' called for Tue Dec 23 01:06:08 +0100 2008:Time) running test if You call action with get :action, :time => Time.now. For other objects You could get other errors, but what is most important – in such case Your tests are not modeling real environment! Arguments tests are passing to controller methods have different types and can behave quite differently than when faced with real HTTP request.

It could be more cumbersome when You use respond_to to dispatch rendering on requested content type and use following sequence in test:

get :action, :id => some_id, :format => :csv
assert_response :success

Do You spot pitfall? Assuming controller’s rendering statement looks like:

  respond_to do |wants| 
     wants.html;
     wants.csv { 
        #render some CSV
     }
  end

Assertion will fail with Expected response to be a <:success>, but was <406>. HTTP 406 Not Acceptable? Yes, since this is response Rails should give when they can not provide answer with required content type. Somewhere inside of Rails Mime::Type lookup fails since it expects argument to be string ('csv') not a symbol (:csv). So, remember to specify format as :format => 'csv' (as string not a symbol).

Could Rails protect me from this kind of errors?

I don’t see any reasons not allowing to implement casting to strings all values (except hashes which should be processed recursively), but I’m not core Rails hacker – maybe there is a reason to such behavior.

Lesson from this – always use only strings and hashes as objects passed to controllers in functional tests.

It’s time to say goodbye!

Well, last 6 years I was using Windows as a main OS for my home/developer computers. Partly it was not mine decision, since my employers required me to run Windows. But, since I’ve jumped out of regular 9to5 two years ago this was not true anymore. It was convenient for me to use Windows, and during first year I was developing bigger application which was meant to be run on Windows. And it was Ruby application, so it was wise to develop on Windows. Almost year ago I have finished this assignment and Windows was still on my notebook.

Last weeks finally made me to change that. As You probably know I have converted to TDD development style. Having tests covering work already done (and already squeezed bugs) is very handy, but when running even partial tests take to much time – it decreases productivity. Waiting for test to complete (to know if they are green and I can move to another task or I have introduced some other bugs ;) and need to work on code) causes my mind do wander to other topics. You know, Internet has a lot distractions to offer. I was suffering in more or less silence – my laptop is old, I thought (and it is old indeed) and changing OS won’t help much. But then I have read this blog post about some comparisons between Ruby implementations in context of performance.

Enough! I have said to myself – I was busy last months and it was hard to find some time, and jump with replacing Windows with Linux during some work with deadlines. Quick browsing through pile of old PC parts I have accumulated through last years, and tada! Old 20 GB HDD and USB20 – IDE interface made my day (or rather all-night). After some issues with not working CDROM drive I’ve managed to install Ubuntu on this drive.

Ubuntu on my desktop have arrived

Ubuntu on my desktop have arrived. Image (c) karindalziel

Boot from USB and… I was surprised how far Linux desktop have advanced with easy of use and integration since I was last time trying. Even my backup GPRS modem was recognized and Ubuntu has configuration for my GSM operator. Cool.

Then I have installed my dev environment (of course I have not used Ruby and gems from Ubuntu’s packages, I have compiled my own) and did ultimate test.

The same notebook, HDDs of comparable speed (or rather comparable in slowness :)) ) – both 5400 RPM with small cache and the same Rails project- around 300 assertions (100 in unit tests and 200 in functionals). Time of execution of rake test: Windows XP, Ruby 1.8.6.p111 – 2:49. Ubuntu, Ruby 1.8.7-p72 own compilation – 0:29.

Bye, bye Windows…

I will stay in this setup (USB drive) for some time, and when I will be sure that something small won’t be annoying problem I’ll make final switch.

UPDATE
This post was in draft mode few days and today I can say – I’ll make switch. The only issue I had it was issue with sound. At first I thought something does not work. But it turned out I’m too old fashioned. I was using command line tools to make sure volume is set as needed, and I did something wrong since – no sound for me. I had just doubleclick speaker control in Ubuntu’s taskbar to un-mute PCM and CD and get music playing – as work in home freelancer with 3 kids working sound is a must, believe me :))