Why are You doomed being Ruby developer on Windows platform

Today I have lost several hours trying to figure what is going with Amazon S3 Ruby gem. A lot of stress and wasted three hours. I’m developing one site which will be storing media files on Amazon’s S3 (You didn’t see that coming? :) ). From some time I’m using NetBeans on my Windows based laptop as my development environment. NetBeans is sometimes slow (it is Java and my laptop is some almost ancient IBM T41) and has own quirks, but I like it. Mostly for its better Rails support than Eclipse (IMO). Before I settled with some IDE I was using simple text editor (JOE) and was working via SSH. It was enough for long time, but I needed some more efficient debugger.

And from debugger my todays troubles has began. To handle S3 I used AWS::S3 and attachment_fu. Unfortunately for me on my laptop I had installed RMagick. Unfortunately – since atachment_fu is using RMagic (well – requires gem) even if my uploads was application/octet-stream I had no plans to make any resizes. After attachment_fu instalation and configuration I noticed that during file upload application hangs. Literally – webrick stops processing request, not throwing any exception or giving any output to console window in NetBeans. After many tries I almost accidentally fired webrick from windows shell (cmd). When I tried to upload file – bang! I got popup! (crowd cheers Windows) SomeStrangeFunction entry point in CORE_RL_magick_.dll not found. WTF? After some googling I have theory somehow my ImageMagic version does not relate well with RMagic gem version. Uninstall ImageMagick (I’m not planing any magick) and everything seems OK. I mean – no popup.

But upload still fails. This time, when I upload file, request is processed, processed, processed I can go for coffee, and when I return got AWS::S3::RequestTimeout (Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.) Again – WTF?! But this time I don’t have debugger. Fighting with strange ‘hangs’ I did mistake and upgraded ruby-debug-ide to newer version. But this gem comes without precompiled windows version. So after gem update ruby-debug-base ruby-debug-ide I got updated first one and famous (for RubyGems users on Windows) nmake not found or something similar complaining about missing compiler or other C/C++ build tools (know somebody any Windows distribution with GNU compiler tools included by default? ;-)) ) for second one. Result – in NetBeans only available is ‘slow’ debugger – before execution stops on breakpoint in upload controller, AWS S3 timeout happens. Indeed, promises are kept – it is slow debugger.

Lost in sands of Windows development
Image from SXC

My brain was overheated already so it took me some time to find that ruby-debug-ide could be found on debug-commons package.

OK. So back to nice Timeout in AWS::S3::S3Object.store in attachment_fu. Well, I was using seasoned cargo-cult code in my controller so I was expecting it should work! To narrow problem down I wrote simple script and:

AWS::S3::S3Object.store ("dd", File.open(file), 
     bckt, :access => :public_read)

Timeout!

str = File.open(file).read
AWS::S3::S3Object.store ("dd", str, 
    bckt, :access => :public_read)

And works as a charm. I copied former version to some good ol’ BSD and voila! it works. There is bug in AWS::S3 and if You pass to S3Object.store file handle on Windows it hangs somewhat on reading this handle. Timeout exception saves Your webrick from being totally unresponsive.

Since I’m developing on Windows, but deployment will be on Linux, this ugly hack for attachment_fu is OK (reading whole file before could be expensive in memory terms):

Index: vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/backends/s3_backend.rb
===================================================================
--- vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/backends/s3_backend.rb	(version 67)
+++ vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/backends/s3_backend.rb	(version 71)
@@ -291,13 +291,32 @@
 
           def save_to_storage
             if save_attachment?
-              S3Object.store(
-                full_filename,
-                (temp_path ? File.open(temp_path) : temp_data),
-                bucket_name,
-                :content_type => content_type,
-                :access => attachment_options[:s3_access]
-              )
+              #FIXME: ugly hack to get it running on windows
+              #somehow when You give file handle it won't read it
+              if RUBY_PLATFORM =~ /(:?mswin|mingw)/
+                file_content = ""
+                if temp_path 
+                  file_content  = File.open(temp_path, "rb").read 
+                else
+                  file_content = temp_data
+                end
+
+                S3Object.store(
+                  full_filename,
+                  file_content,
+                  bucket_name,
+                  :content_type => content_type,
+                  :access => attachment_options[:s3_access]
+                )
+              else
+                S3Object.store(
+                  full_filename,
+                  (temp_path ? File.open(temp_path) : temp_data),
+                  bucket_name,
+                  :content_type => content_type,
+                  :access => attachment_options[:s3_access]
+                )
+              end
             end
 
             @old_filename = nil

Conclusion?

I’m almost ready to install some Linux on my laptop. You are doomed developing anything in Ruby on Windows for deployment on UNIX – it is mistake. However if You plan to run Your Ruby software on Windows box – You should develop it on Windows also. Or being prepared for nasty surprises after cap deploy

Join the Conversation

22 Comments

  1. Try simple vmware server or workstation install on windows and download full ruby enviroment from vmware. Than you may still edit from Netbeans but server comes from vmware virtual server. Its light, simple and probably will work even on T41. We will run right now full vmware infrastructure 3.5 for ruby slice hosting, so this is something for windows lovers, mac lovers and the other ;)

  2. I’m running in many of these same problems, and you’ve already got me over the hump of the timeout which was a big relief.

    My problem is that files aren’t getting to the S3 server correctly. When I put up a 200K jpeg and read it through the URL, I just see the file name and view source looks like encrypted junk. View info says I have a 16K jpeg file.

  3. OK, got it. It was a binary thing.
    Changed this:
    str = File.open(file).read
    to this:
    str = File.open(file, “rb”).read
    and it works like a dream.

    Thanks for writing this.

  4. @Michael
    Yeah exactly – I ran on this issue next day :) when I returned to code. But I was too busy recently to make update to this blog.
    You can open file as binary, or You can change IO stream state with binmode command.

  5. Hi. Probably you’ve encountered the bug:

    http://www.netbeans.org/issues/show_bug.cgi?id=126857

    which was already fixed in trunk. The problem here is that ruby-debug guys stops to provide pre-compiled binaries for Windows, tracked here:

    http://rubyforge.org/tracker/index.php?func=detail&aid=16774&group_id=1900&atid=7436

    Todays, in NetBeans trunk, the classic (slow) debuggers are kind of deprecated since they are really unusable.

    If you encounter any bugs feel free to contact us through NB mailing list (http://wiki.netbeans.org/RubyFeedback), you should be given quick feedback and prevent your woes ;)

  6. @Martin
    I know, that reason for debugger problems in NetBeans is located out of scope of NB developers. Trouble with being Ruby developer on Windows means You are prone to some errors (like with AWS::S3) since main deployment platform is UNIX family and You have to wait to get some gems with binaries for mswin.

  7. Nice: thanks for this!

    Wish I’d found your post 10 hours ago!

    :o)

  8. I tried your fix above but getting the same timeout error. Any Suggestions??

  9. You are probably seeing this because you are streaming a file to S3 and not rewinding the file handle’s seek pointer during retries caused by either a network failure of a 301 redirect.

  10. THANK YOU SO MUCH!
    Thanks for posting this tip. Saved me heaps of time.
    Here’s my bit of workaround code for doing things, too:

    @contents=0
    File.open(path_med, ‘rb’) {|f| @contents = f.read }
    AWS::S3::S3Object.store(‘a.jpg’, @contents, ‘mybucket’, :access => :public_read)

    I use the block around the file.open to ensure it gets closed after reading

  11. Thanks for posting the workaround for windows! It saves me tons of time.

  12. Thanks for the info! This solved my problem with storing database backups to S3.

  13. Many thanks for this, you’ve made my decision easier to dump Ruby as a development language, its for Linux nerds only, and there’s not enough stuff there to make it useful in a Windows machine at all.

    Hello Python ….

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.