[nick@Dev1 ~]$ sudo gem install mysql
Password:
Building native extensions.  This could take a while...
ERROR:  Error installing mysql:
	ERROR: Failed to build gem native extension.

/usr/bin/ruby extconf.rb
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/usr/bin/ruby
	--with-mysql-config
	--without-mysql-config
	--with-mysql-dir
	--without-mysql-dir
	--with-mysql-include
	--without-mysql-include=${mysql-dir}/include
	--with-mysql-lib
	--without-mysql-lib=${mysql-dir}/lib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-mlib
	--without-mlib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-zlib
	--without-zlib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-socketlib
	--without-socketlib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-nsllib
	--without-nsllib
	--with-mysqlclientlib
	--without-mysqlclientlib

Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/mysql-2.7 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/mysql-2.7/gem_make.out

Yikes! Look familiar? Bummer. First step, brute force install everything I could think might be missing

sudo yum install gcc mysql-server mysql-devel ruby ruby-devel

Turns out, the gcc was having some issues:

[nick@Dev1 mysql-2.7]$ sudo yum install gcc
Loading "rhnplugin" plugin
Loading "security" plugin
intensive-rhel-i386-serve 100% |=========================| 1.2 kB    00:00
rackspace-int-rhel-i386-s 100% |=========================| 1.2 kB    00:00
Excluding Packages in global exclude list
Finished
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package gcc.i386 0:4.1.2-42.el5 set to be updated
--> Processing Dependency: libgomp = 4.1.2-42.el5 for package: gcc
--> Processing Dependency: cpp = 4.1.2-42.el5 for package: gcc
--> Processing Dependency: libgomp.so.1 for package: gcc
--> Processing Dependency: glibc-devel >= 2.2.90-12 for package: gcc
--> Running transaction check
---> Package cpp.i386 0:4.1.2-42.el5 set to be updated
---> Package glibc-devel.i386 0:2.5-24.el5_2.2 set to be updated
--> Processing Dependency: glibc-headers = 2.5-24.el5_2.2 for package: glibc-devel
--> Processing Dependency: glibc-headers for package: glibc-devel
---> Package libgomp.i386 0:4.1.2-42.el5 set to be updated
--> Running transaction check
---> Package glibc-headers.i386 0:2.5-24.el5_2.2 set to be updated
--> Processing Dependency: kernel-headers >= 2.2.1 for package: glibc-headers
--> Processing Dependency: kernel-headers for package: glibc-headers
--> Finished Dependency Resolution
Error: Missing Dependency: kernel-headers is needed by package glibc-headers
Error: Missing Dependency: kernel-headers >= 2.2.1 is needed by package glibc-headers

Googling around a little, I found this helpful page which indicated that the default RHEL yum configuration excludes updates from the kernel. That is probably good practice in general, but right now it is preventing us from doing what "needs to be done."

Looking at my /etc/yum.conf file, indeed the kernal was excepted from updates:

[nick@Dev1 ~]$ cat /etc/yum.conf
[main]
cachedir=/var/cache/yum
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1

# Note: yum-RHN-plugin doesn't honor this.
metadata_expire=1h

# Default.
# installonly_limit = 3

# PUT YOUR REPOS HERE OR IN separate files named file.repo
# in /etc/yum.repos.d
exclude=kernel*

So I commented out that line, yum installed gcc, and then off to the races. My googling also noted that passing the --with-mysql-config saves you some trouble, essentially letting it configure itself.

[nick@Dev1 ~]$ sudo gem install mysql -- --with-mysql-config=/usr/bin/mysql_config
Building native extensions.  This could take a while...
Successfully installed mysql-2.7
1 gem installed

Just in case, you might want to uncomment the kernal extension from your /etc/yum.conf file!

Digging around the web, I found this group about this exact problem.

http://groups.google.com/group/paperclip-plugin/browse_thread/thread/4a0a97a08929d3a5

I found that I had all of the patches, but it still didn't work. Most of the people on the group were complaining it that it didn't work on Windows, and maybe had something to do with file handles, etc. We were trying to get this working in JRuby on a Joyent Solaris Accelerator, each of which can make things a little different than a MRI Rails app on Linux.

Following a tip from a user on the group, simply changing the way the move happens fixes this problem. I would like to fix the root cause, but I got it working for me, and (*open source sigh*) don't want to isolate the exact problem.

After that, i made some change in storage.rb#flush_writes.
It's a short workaround for

  FileUtils.mv

Simply replace it with

  FileUtils.cp(file.path, path(style))
  FileUtils.rm(file.path)

Currently, it works for me. But please consider, it isn't checked against a well written test or any other platform.

Firstly, the ability to add arguments was added somewhere along the line, so you might as well make sure you have a recent version of Rake:

nicks-computer:~ nick$ sudo gem install rake
Password:
Successfully installed rake-0.8.3
1 gem installed
Installing ri documentation for rake-0.8.3...
Installing RDoc documentation for rake-0.8.3...

Great. Now, lets dig in to an example rake file. You can put this in a file name 'Rakefile' anywhere you like, or in RAILS_ROOT/lib/tasks/some_file.rake if you are rails-ish.

desc "An example task that uses a single argument"
task :example_with_argument, :arg1 do |t, args|
  puts "Running 'example_with_argument' with '#{args.arg1}' as the argument"
end

desc "An example task that uses multiple arguments"
task :example_with_multiple_arguments, :arg1, :arg2 do |t, args|
  puts "Running 'example_with_argument' with '#{args.arg1}' and '#{args.arg2}' as the arguments"
end

desc "An example task that uses multiple arguments, and has defaults"
task :example_with_multiple_arguments_and_defaults, :arg1, :arg2 do |t, args|
  args.with_defaults(:arg1 => 'default_arg1', :arg2 => 'default_arg2')
  puts "Running 'example_with_argument' with '#{args.arg1}' and '#{args.arg2}' as the arguments"
end

Now, for how to run them. Without specifying arguments, the defaults (or an empty string if no defaults are specified) will be used:

nicks-computer:~ nick$ rake example_with_multiple_arguments_and_defaults
(in /Users/nick)
Running 'example_with_argument' with 'default_arg1' and 'default_arg2' as the arguments

nicks-computer:~ nick$ rake example_with_multiple_arguments_and_defaults[myarg1, myarg2]
(in /Users/nick)
Running 'example_with_argument' with 'myarg1' and 'myarg2' as the arguments

TaDa!

UPDATE:

Rake a a great tool, and widely used, but there are some new tools that surpass rake in some areas. For example check out Thor and how OpsCode uses it with their Knife tool.

I was recently trying to implement an upload to Amazon S3, and it wasn't working. I couldn't tell if it was the Flex front end, or the S3 backend, or what, so I decided to Wear my ruby slippers to work, or rather my Rails slippers (so comfy and I just like them so much).

About 15 minutes and 15 lines of code later, I had a rails app running locally which accepted the upload from Flex and logged all the request parameters. I found the Simplest upload that could possibly work in the Rails Cookbook, and a Rails wiki page to that purpose. In the Rails Cookbook, note that you should use params['file'] rather than the oldschool @params['file'].

To get a simple Rails (> 2.0) app for testing the uploads, try this

rails uploader
cd uploader
script/generate controller upload

Paste in some code from the Simplest Upload Possible article above.

class UploadController < ApplicationController
  def save_file
    logger.info(params.insect)

    if request.get?
    render :text => 'Please Upoad file'
    else
      # Use "wb" mode on windows.
      File.open("#{RAILS_ROOT}/public/files/#{Time.now.to_i}.jpg", "w") do |f|
        f.write(params['file'].read)
      end
    render :text => 'Done'
    end
  end
end

Now, in Flex, have your FileReference upload to your server, and make sure to use the right field name.

[ActionScript]
var urlRequest:URLRequest = new URLRequest("http://127.0.0.1:3000/upload/save_file");
urlRequest.method = URLRequestMethod.POST;
file.upload(urlRequest, "file");
[/ActionScript]

Tails your logs, and look for the output. One step closer.

I might try to clean up a simple uploader app and thow it up (blaagh) here. Any interest?

After much googling, Wiresharking, and general debuggation, a co-worker found this Amazon S3 doc, which notes:

Some versions of the Adobe Flash Player do not properly handle HTTP responses that have an empty body. To configure POST to return a response that does not have an empty body, set success_action_status to 201. When set, Amazon S3 returns an XML document with a 201 status code. For information on the content of the XML document, see ???. For information on form fields, see HTML Form Fields.

Interesting. And small; it is a tiny message, almost a footnote, on some Amazon page. I wish Adobe could get their docs together. I particularly like "Some versions" and "???". Anyway, lesson learned, or at least we have something to try.

The S3/Flash interactions need a little massaging

It seems like the OS X Flash runtime (9.0.115.0) requires a non-empty server response from its POST. Amazon S3 conveniently has an additional parameter, success_action_status which can be set to "201" to have S3 respond to the post with a 201 CREATED response code. If you are uploading to Apache or IIS, you could have the same problem. Try to check the header response for your POST, and see it is is a) empty, or b) not a 201.

On S3, you can modify your upload policy to return the correct response. Your policy should look something like this:

{
    'expiration': '2008-12-31T12:00:00.000Z',
    'conditions': [
        {'bucket': 'my_test_bucket'},
        {'key': 'test1.jpg'},
        ['starts-with', '$Filename', ''],
        ['eq', '$success_action_status', '201']
    ]
}

Note that the success_action_status is not an object property, but an [Operator, Operand, Operand] array.

Problem Solved? Not quite: still not uploads.

S3 Upload Example

Someone at AWS posted a helpful Flex -> S3 example, or actually examples. One Flex project creates and signs a policy, and the other uploads a file using the generated policy. And whats more, they both work. Also, took a look at the code to make sure it wasn't sending my AWS key/password off to some email! Ha. I think it was also posted by an AWS employee, so that is reassuring.

#2038 FileIOError

I kept getting a #2038 FileIOError on my mac. Bummer. Taking a closer look at the working AWS example, I saw a funny comment. Not funny roflcopter, but funny smelly bug-fixy.

            /*
             * FileReference.upload likes to send cryptic IOErrors when it doesn't get a status code that it likes.
             * If we already got an error HTTP status code, don't propagate this event since the HTTPStatusEvent
             * event handler dispatches an IOErrorEvent.
             */

As it turns out, I could NOT stop these #2038 FIleIOErrors. They just kept coming. But, as the example code reminded me, if you are getting a successful server response, the file is already up there. Sure enough, the file was there.

So you can pretty much ignore the FileIOError, although you should handle it, and just ignore it if you have a successful HTTP response, something like this:

        private function onIOError(event:IOErrorEvent):void
        {
            if(!recievedSuccessfulHTTPresponse)
            {
                this.dispatchEvent(event);
            }
        }

You get the idea. Ok. Handle but ignore FileIOErrors. Done? Almost. Single uploads work, but I get an "Error #2044: Unhandled IOErrorEvent:. text=Error #2038: File I/O Error" when doing multiple uploads, via a FileReferenceList. Marvelous. This turned out to be even more of a pain. It wasn't anything to do with the server side upload script as my google searches might have me believe.

After brainstorming with a co-worker, we came to a pretty good guess: Garbage Collection. Maybe the FileReference was going out of scope before the IOError could be handled, or some such. PC/OS X differences in the runtime could have different GC implementations. Worth a shot. I changed my event listeners from "weak" to "strong" references. Shahooya. Multiple file upload city.

Other fixes

Make sure you have a publicly accessable crossdomain policy on the server you are uploading to.

Also, S3 scales well but adds extra complexity. You might try uploading to somewhere else, like a simple rails uploader.

Hope that helps. Also, please comment. In fact, today only, every commenter gets a free XBox. You know, its just a little lonely out here in the cloud sometimes.

UPDATE (1/12/10)
Since this is remarkably one of my most popular posts, I feel compelled to point you towards this Ruby Inside article that will probably help you out a lot more than this post.

As you may know, the Ruby 1.8.* Net::SMTP library does not allow for TLS (SSL) connections. This prevents you from connecting to Gmail, or other mail servers that require a secure connection. Fortunatly, there is a pretty tight bit of code to fix this problem. As far as I can tell, this code originated at a japanese site. This code is all over, if you search for 'SMTP_TLS ruby.'

Great! We have the code! This is actually one of my favorite examples of the open source community. I originally got this code from a guy in Russia, who translated the Japanese page above. Pretty cool. Anyway, I don't really want to paste this code above every script I write that sends out an email. What about a gem!

UPDATE (12/18/07): Well it turns out there already is a gem for this. TLS Mail is not exactly the same as above, but it does the trick. The code in this gem is the code in the Ruby 1.9 source. The only catch is that you have to enable the TLS protocol before you send the mail:

Net::SMTP.enable_tls(OpenSSL::SSL::VERIFY_NONE)

Grab the code with "sudo gem install tlsmail"

The RubyGems website has concise, but not overly helpful, documentation about creating gems. It is pretty easy, though. I copied the .gemspec file from the simplist gem I could find, and went from there. I don't know about copy rights or licenses for this code, and this is mostly for internal use, so the gemspec is pretty dumbed down.

Gem::Specification.new do |s|
  s.name = %q{SMTP_TLS}
  s.version = "1.0.0"
  s.date = %q{2007-12-10}
  s.summary = %q{SMTP TLS (SSL) extension for Net::SMTP}
  s.email = %q{nick@nuclearrooster.com}
  s.homepage = %q{google for smtp_tls}
  s.description = %q{This adds support for TLS (SSL) connections to SMTP mail servers}
  s.has_rdoc = false
  s.authors = ["Dunno"]
  s.files = ["lib/smtp_tls.rb"]
end

Make sure any ruby files are stuck in a lib directory.

nick-stielaus-computer:~/Desktop/smtp_tls nick$ ls
lib/                    smtp_tls.1.0.0.gemspec
nick-stielaus-computer:~/Desktop/smtp_tls nick$ gem build smtp_tls.1.0.0.gemspec
  Successfully built RubyGem
  Name: SMTP_TLS
  Version: 1.0.0
  File: SMTP_TLS-1.0.0.gem
nick-stielaus-computer:~/Desktop/smtp_tls nick$ sudo gem install SMTP_TLS
Password:
Successfully installed SMTP_TLS, version 1.0.0

And then in your scripts

  require "net/smtp"
  require "rubygems"
  require "smtp_tls"

  Net::SMTP.start('mail.server.net', 25, 'mail.from.server','usernam', 'password', :login) do |smtp|
    hdr = "From: Me <nick@nuclearrooster.com>\n"
    hdr += "To: Nick Stielau <nick@nuclearrooster.com>\n"
    hdr += "Subject: Subject\n\n"
    msg = hdr + "Email Body"
    smtp.send_message msg, 'nick@nuclearrooster.com'
  end

Now I can install this gem on my app servers, etc, and the scripts are good to go.

For quite a while, I was trying to write a ruby script to print some growl messages. I was having a hard time loading the growl code, using the "require 'ruby-growl'" syntax. I started digging around and looking at ruby's load path.

nicks-computer:~/Desktop/depot nick$ irb
irb(main):001:0> puts $:
/usr/local/lib/ruby/site_ruby/1.8
/usr/local/lib/ruby/site_ruby/1.8/i686-darwin8.9.1
/usr/local/lib/ruby/site_ruby
/usr/local/lib/ruby/1.8
/usr/local/lib/ruby/1.8/i686-darwin8.9.1
.
=> nil

After a while, I got it working and left it as

require '/usr/lib/ruby/gems/1.8/gems/ruby-growl-1.0.1/lib/ruby-growl.rb'

A couple months later I opened the script again, and laughed. Hahaha. Its as easy as

require 'rubygems'
require 'ruby-growl

It worked like a charm. At first I didn't know what YUM was, and once I found out it was a package installing I didn't exect to have it. I surely didn't want to compile that too (unless I was sure there was no other way). Turns out, Yum was already installed, and I got ruby gems going. Thanks to Lucas, a fellow defaultish wordpress theme blogger.

Specifically, I wanted to customize the SVN checkout to use sudo, have sudo default to specific user, and add a deployment message to the revisions log. This is all easy to do without editing the capistrano gem. I looked around the gem source and found the code that I wanted to override, and then pulled it out and tweaked it.

In deploy.rb

class ::Capistrano::SCM::Subversion
  alias_method :non_sudo_checkout, :checkout
  def checkout(actor)
    op = 'export'
    command = "sudo -u USER #{svn} #{op} -q --config-dir /home/USER/.subversion -r#{configuration.revision} #{configuration.repository} #{actor.release_path} &&"
   run_checkout(actor, command, &svn_stream_handler(actor))
  end
end

class ::Capistrano::Actor
  alias_method :sudo_with_optional_user, :sudo
  def sudo(command, options={}, &block)
    sudo_with_optional_user(command, {:as =>; 'SUDO_USER'}.merge(options), &block)
  end
end

class ::Capistrano::SCM::Base
  alias_method :logging_commands_without_message, :logging_commands
  def logging_commands(directory = nil)
    log = "#{configuration.deploy_to}/revisions.log"
    "(test -e #{log} || (touch #{log} && chmod 666 #{log})) && " +
    "echo `date +\"%Y-%m-%d %H:%M:%S\"` $USER #{configuration.revision} #{directory} '#{configuration.message}' >> #{log};"
  end
end

• Easy to use
• Easy to read
• Decreases the need for complexly dependant fixtures
• Speeds up tests once fixtures are removed

Not only that, but Mocha puts some constraints on your tests that force you to be a better tester. If you mock and stub instead of using fixtures, you need to build your tests up to a complete test from an empty mock, rather than whittling them down from a full Ruby object with lots of data. For example,

 def test_items_should_return_uniq_set_of_all_rentals_items

bob = Renter.new

bob.expects(:rentals).returns([stub(:items => [:item_1, :item_2, :item_3]),

stub(:items => [:item_3, :item_4, :item_5])])

assert_equal bob.items, [:item_1, :item_2, :item_3, :item_4, :item_5]

end 

Before I might have been tempted to play with the +Item+ objects, but with Mocha, they are just symbols, so the test is more transparent.

I am still grappling with some aspects of Mocha and testing in general. Like custom SQL. If you want to test your custom SQL, you need data in the database at some point. Its seems like the higher level stuff is better for mocking, and the lower level stuff gets a little trickier. Here is how I tackled testing a ActiveRecord association, and its dependency:

 require File.dirname(__FILE__) + '/../test_helper'

class RenterTest < Test::Unit::TestCase
def test_rental_association
bob = Renter.new
bob.rentals << Rental.new
assert !bob.rentals.empty?
end

def test_rental_dependency
bob = Renter.new
bob.rentals << Rental.new
bob.rentals.first.expects(:destroy)
bob.destroy
end
end

Railscasts can get you started.

technorati tags:ruby, rails, testing, mock, test, mocha

Blogged with Flock