GeoKit – have You slept on trigonometry?

I was just about to write that GeoKit is useful plugin but somewhat dormant (previous release was in Jan 08 if I recollect correctly), but I’ve checked project homepage and voila! It has new release! GeoKit was dived into gem (framework independent) and Rails plugin. That way You can use geokit-gem to make some distance calculations even if You didn’t like trigonometry at school :)

How often You had wished time wouldn't move so slow? Image (c) <a href='http://flickr.com/photos/fake_eyes/'>Orange42</a>” title=”Bored at lessons” width=”500″ height=”375″ class=”size-full wp-image-415″ /><figcaption id=How often You had wished time wouldn't move so slow? Image (c) Orange42

GeoKit does provide some finders for ActiveRecord like find_closest (hard to guess what it does?). Remember that GeoKit expects latitude and longitude fields in AR object to be not nil. If You have some objects with latitude or longitude equal to nil they will be always closest to any given point. Probably not what do You want.

It may be dependent on what are You trying to do, but for me setting default latitude and longitude to values I won’t spot in the wild in my dataset was good enough. But this was special case – I have set default to North Pole, and I was looking not only closest but also close – no further than single miles and I was sure this would not trigger with North Pole.

Why not set defaults to something like -200? You didn’t like trigonometry, do You? Functions used to calculate distance on the sphere are periodic, so You have chance You will hit false positive – latitude and longitude -200/-200 will me mapped into -90..90 and -180..180 ranges and could match anyway.

So when doing any searches and You can not assure Your data have always latitude and longitude You have to remember to add conditions lat is not null && lng is not null to every query (like Place.find_closest(:origin => [some_lat, some_lng], :conditions => 'lat is not null && lng is not null').

Anyway – GeoKit is nice gem/plugin and if You need to operate on latitude and longitude – use it, do not write own code. On the other hand, You could learn a bit from trigonometry :))

Will paginate for rescue

Will paginate is great plugin, used almost by every Rails application I have seen. But I’ve seen this plugin used to generate handful navigation among huge datasets. Well it was written with this in mind, but today I have found another use.

How often Your application does need export huge dataset as CSV for example? Quite often. First approach could be:

@l = Model.find :all
    str = ""
    CSV::Writer.generate(str) do |csv|
      @l.each {|l|
        csv << [ l.fields ]
      }    
    end
    send_data  str, :type => 'text/csv', :filename => 'model.csv'

This works, but… If You are operating with some memory limit (like on most shared hosting setups or on small instance of VPS) this could lead to problems. One of applications I have seen with this approach and exporting 35k objects (each with one has_one association loaded with :include) just after restart and this export done have used 215 MB (as reported by passenger-memory-stats). On VPS with 256 MB it will result in swaping out this app – and next requests will be handled with latency due to swap. And process won’t give back this memory until passenger kill this instance in normal life cycle.

Will_paginate for rescue!

Continue reading “Will paginate for rescue”

Using non text values as arguments in functional tests

When running functional tests, You use methods like get, post, etc. a lot. They take as arguments action name to call (as a symbol) and hash with parameters. But You need to remember that this hash it is not exactly the same what params object is (from controller side).

The main difference is that params is (more or less) hash with elements being strings or other params-like hashes (results of processing HTML forms with fields named with Rails conventions object[attribute_name]). And when You call get/post from functional test, parameters hash is full blown Ruby hash and as such is passed to controller, without any changes. So it is possible to pass whole Ruby objects as values in params in test environment.

For example if You expect argument params[:time] and run Time.parse(params[:time]) in controller You will encounter error (like: NoMethodError: private method `gsub!' called for Tue Dec 23 01:06:08 +0100 2008:Time) running test if You call action with get :action, :time => Time.now. For other objects You could get other errors, but what is most important – in such case Your tests are not modeling real environment! Arguments tests are passing to controller methods have different types and can behave quite differently than when faced with real HTTP request.

It could be more cumbersome when You use respond_to to dispatch rendering on requested content type and use following sequence in test:

get :action, :id => some_id, :format => :csv
assert_response :success

Do You spot pitfall? Assuming controller’s rendering statement looks like:

  respond_to do |wants| 
     wants.html;
     wants.csv { 
        #render some CSV
     }
  end

Assertion will fail with Expected response to be a <:success>, but was <406>. HTTP 406 Not Acceptable? Yes, since this is response Rails should give when they can not provide answer with required content type. Somewhere inside of Rails Mime::Type lookup fails since it expects argument to be string ('csv') not a symbol (:csv). So, remember to specify format as :format => 'csv' (as string not a symbol).

Could Rails protect me from this kind of errors?

I don’t see any reasons not allowing to implement casting to strings all values (except hashes which should be processed recursively), but I’m not core Rails hacker – maybe there is a reason to such behavior.

Lesson from this – always use only strings and hashes as objects passed to controllers in functional tests.

It’s time to say goodbye!

Well, last 6 years I was using Windows as a main OS for my home/developer computers. Partly it was not mine decision, since my employers required me to run Windows. But, since I’ve jumped out of regular 9to5 two years ago this was not true anymore. It was convenient for me to use Windows, and during first year I was developing bigger application which was meant to be run on Windows. And it was Ruby application, so it was wise to develop on Windows. Almost year ago I have finished this assignment and Windows was still on my notebook.

Last weeks finally made me to change that. As You probably know I have converted to TDD development style. Having tests covering work already done (and already squeezed bugs) is very handy, but when running even partial tests take to much time – it decreases productivity. Waiting for test to complete (to know if they are green and I can move to another task or I have introduced some other bugs ;) and need to work on code) causes my mind do wander to other topics. You know, Internet has a lot distractions to offer. I was suffering in more or less silence – my laptop is old, I thought (and it is old indeed) and changing OS won’t help much. But then I have read this blog post about some comparisons between Ruby implementations in context of performance.

Enough! I have said to myself – I was busy last months and it was hard to find some time, and jump with replacing Windows with Linux during some work with deadlines. Quick browsing through pile of old PC parts I have accumulated through last years, and tada! Old 20 GB HDD and USB20 – IDE interface made my day (or rather all-night). After some issues with not working CDROM drive I’ve managed to install Ubuntu on this drive.

Ubuntu on my desktop have arrived
Ubuntu on my desktop have arrived. Image (c) karindalziel

Boot from USB and… I was surprised how far Linux desktop have advanced with easy of use and integration since I was last time trying. Even my backup GPRS modem was recognized and Ubuntu has configuration for my GSM operator. Cool.

Then I have installed my dev environment (of course I have not used Ruby and gems from Ubuntu’s packages, I have compiled my own) and did ultimate test.

The same notebook, HDDs of comparable speed (or rather comparable in slowness :)) ) – both 5400 RPM with small cache and the same Rails project- around 300 assertions (100 in unit tests and 200 in functionals). Time of execution of rake test: Windows XP, Ruby 1.8.6.p111 – 2:49. Ubuntu, Ruby 1.8.7-p72 own compilation – 0:29.

Bye, bye Windows…

I will stay in this setup (USB drive) for some time, and when I will be sure that something small won’t be annoying problem I’ll make final switch.

UPDATE
This post was in draft mode few days and today I can say – I’ll make switch. The only issue I had it was issue with sound. At first I thought something does not work. But it turned out I’m too old fashioned. I was using command line tools to make sure volume is set as needed, and I did something wrong since – no sound for me. I had just doubleclick speaker control in Ubuntu’s taskbar to un-mute PCM and CD and get music playing – as work in home freelancer with 3 kids working sound is a must, believe me :))

NetBeans, SVN, SSH and multiple repositories

I know git is right choice for VCS, but… not all have converted yet and NetBeans does not support git (yet, since work is in progress?).

If You are still stuck in SVN world – in NetBeans FAQ You can find guide how to setup svn+ssh protocol to work with NetBeans.

On Windows (as always) are some gotchas when You setup svn+ssh.

First, connect to host with PuTTy to populate PuTTys key cache – otherwise You won’t be able use command line SVN tools.

Second – don’t dare to set default IP address in PuTTy (that is default host name for Default Settings). If You do, no matter what URL You give as SVN repo address, plink.exe will try connect to default IP/hostname instead to SVN repo. Probably this is not what You want plink to do.

All is working and You need to setup another repository?

As a freelancer I do work with different customers and if more than one uses svn+ssh URL schema, how to setup NetBeans?

Well it can be achieved with little trick. You need to create new schema named for example svn+ssh2 and adjust SVN config.

Example! Example!

Let’s assume we have:

  • first repository
    • URL: svn+ssh://example.com/repo/trunk
    • user: user1
    • password: pwd1
  • second repository
    • URL: svn+ssh://other.example.com/repo/trunk
    • user: user2
    • password: pwd2

    Now [tunnels] section of SVN config file could be:

    [tunnels]
    ssh = plink -l user1 -pw pwd1
    ssh2 = plink -l user2 -pw pwd2
    

    First URL does not change, so we can checkout code with svn+ssh://example.com/repo/trunk, and code from second repository with modified URL: svn+ssh2://other.example.com/repo/trunk (spot this little 2 in protocol name).

    It just works. Or I missed some other option and it can be done simpler way?