Today I have lost several hours trying to figure what is going with Amazon S3 Ruby gem. A lot of stress and wasted three hours. I’m developing one site which will be storing media files on Amazon’s S3 (You didn’t see that coming? :) ). From some time I’m using NetBeans on my Windows based laptop as my development environment. NetBeans is sometimes slow (it is Java and my laptop is some almost ancient IBM T41) and has own quirks, but I like it. Mostly for its better Rails support than Eclipse (IMO). Before I settled with some IDE I was using simple text editor (JOE) and was working via SSH. It was enough for long time, but I needed some more efficient debugger.
And from debugger my todays troubles has began. To handle S3 I used
AWS::S3 and attachment_fu. Unfortunately for me on my laptop I had installed RMagick. Unfortunately – since atachment_fu is using RMagic (well – requires gem) even if my uploads was application/octet-stream I had no plans to make any resizes. After attachment_fu instalation and configuration I noticed that during file upload application hangs. Literally – webrick stops processing request, not throwing any exception or giving any output to console window in NetBeans. After many tries I almost accidentally fired webrick from windows shell (cmd). When I tried to upload file – bang! I got popup! (crowd cheers Windows)
SomeStrangeFunction entry point in CORE_RL_magick_.dll not found. WTF? After some googling I have theory somehow my ImageMagic version does not relate well with RMagic gem version. Uninstall ImageMagick (I’m not planing any magick) and everything seems OK. I mean – no popup.
But upload still fails. This time, when I upload file, request is processed, processed, processed I can go for coffee, and when I return got
AWS::S3::RequestTimeout (Your socket connection to the server was not read from or written to within the timeout period. Idle connections will be closed.) Again – WTF?! But this time I don’t have debugger. Fighting with strange ‘hangs’ I did mistake and upgraded r
uby-debug-ide to newer version. But this gem comes without precompiled windows version. So after
gem update ruby-debug-base ruby-debug-ide I got updated first one and famous (for RubyGems users on Windows)
nmake not found or something similar complaining about missing compiler or other C/C++ build tools (know somebody any Windows distribution with GNU compiler tools included by default? ;-)) ) for second one. Result – in NetBeans only available is ‘slow’ debugger – before execution stops on breakpoint in upload controller, AWS S3 timeout happens. Indeed, promises are kept – it is slow debugger.
Image from SXC
My brain was overheated already so it took me some time to find that
ruby-debug-ide could be found on debug-commons package.
OK. So back to nice Timeout in
AWS::S3::S3Object.store in attachment_fu. Well, I was using seasoned cargo-cult code in my controller so I was expecting it should work! To narrow problem down I wrote simple script and:
AWS::S3::S3Object.store ("dd", File.open(file), bckt, :access => :public_read)
str = File.open(file).read AWS::S3::S3Object.store ("dd", str, bckt, :access => :public_read)
And works as a charm. I copied former version to some good ol’ BSD and voila! it works. There is bug in AWS::S3 and if You pass to
S3Object.store file handle on Windows it hangs somewhat on reading this handle. Timeout exception saves Your webrick from being totally unresponsive.
Since I’m developing on Windows, but deployment will be on Linux, this ugly hack for attachment_fu is OK (reading whole file before could be expensive in memory terms):
Index: vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/backends/s3_backend.rb =================================================================== --- vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/backends/s3_backend.rb (version 67) +++ vendor/plugins/attachment_fu/lib/technoweenie/attachment_fu/backends/s3_backend.rb (version 71) @@ -291,13 +291,32 @@ def save_to_storage if save_attachment? - S3Object.store( - full_filename, - (temp_path ? File.open(temp_path) : temp_data), - bucket_name, - :content_type => content_type, - :access => attachment_options[:s3_access] - ) + #FIXME: ugly hack to get it running on windows + #somehow when You give file handle it won't read it + if RUBY_PLATFORM =~ /(:?mswin|mingw)/ + file_content = "" + if temp_path + file_content = File.open(temp_path, "rb").read + else + file_content = temp_data + end + + S3Object.store( + full_filename, + file_content, + bucket_name, + :content_type => content_type, + :access => attachment_options[:s3_access] + ) + else + S3Object.store( + full_filename, + (temp_path ? File.open(temp_path) : temp_data), + bucket_name, + :content_type => content_type, + :access => attachment_options[:s3_access] + ) + end end @old_filename = nil
I’m almost ready to install some Linux on my laptop. You are doomed developing anything in Ruby on Windows for deployment on UNIX – it is mistake. However if You plan to run Your Ruby software on Windows box – You should develop it on Windows also. Or being prepared for nasty surprises after