Will paginate for rescue

Will paginate is great plugin, used almost by every Rails application I have seen. But I’ve seen this plugin used to generate handful navigation among huge datasets. Well it was written with this in mind, but today I have found another use.

How often Your application does need export huge dataset as CSV for example? Quite often. First approach could be:

@l = Model.find :all
    str = ""
    CSV::Writer.generate(str) do |csv|
      @l.each {|l|
        csv << [ l.fields ]
      }    
    end
    send_data  str, :type => 'text/csv', :filename => 'model.csv'

This works, but… If You are operating with some memory limit (like on most shared hosting setups or on small instance of VPS) this could lead to problems. One of applications I have seen with this approach and exporting 35k objects (each with one has_one association loaded with :include) just after restart and this export done have used 215 MB (as reported by passenger-memory-stats). On VPS with 256 MB it will result in swaping out this app – and next requests will be handled with latency due to swap. And process won’t give back this memory until passenger kill this instance in normal life cycle.

Will_paginate for rescue!

Continue reading