From WhyNotWiki

This is about testing Ruby code... (mostly Test::Unit right now, but I'd also like to experiment more with RSpec). For Rails-specific testing, check out Rails / Testing

Ruby / Testing  edit   (Category  edit)

[edit] Test::Unit

Reference:

• http://stdlib.rubyonrails.org/libdoc/test/unit/rdoc/index.html
• http://stdlib.rubyonrails.org/libdoc/test/unit/rdoc/classes/Test/Unit/TestCase.html
• http://stdlib.rubyonrails.org/libdoc/test/unit/rdoc/classes/Test/Unit/Assertions.html
• http://stdlib.rubyonrails.org/libdoc/test/unit/rdoc/classes/Test/Unit/TestSuite.html

[edit] The simplest test case

t.rb

require 'test/unit'
class MyTest < Test::Unit::TestCase
  def test_foo
    assert true
  end
end

Loaded suite t
Started
.
Finished in 0.001222 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

It apparently gets the suite name from the filename, not from the name of the TestCase class. That's probably because, if there were multiple tests in the file, it would combine all of them into a single suite for the file...

[edit] The simplest test suite

(Besides the one that is created automatically for you if you don't create one yourself...)

require 'test/unit'
require 'test/unit/ui/console/testrunner'

class MyTest < Test::Unit::TestCase
  def test_foo
    assert true
  end
end

suite = Test::Unit::TestSuite.new('my suite')
suite << MyTest.suite
suite << MyTest.suite
Test::Unit::UI::Console::TestRunner.run(suite)

Loaded suite my suite
Started
..
Finished in 0.001665 seconds.

If you don't explicitly provide the runner...

Test::Unit::UI::Console::TestRunner.run(suite)

...your suite will not get run. Instead, it will still create the default suite for the file and run that instead.

[edit] Caveat: Don't forget the suite method

Intuitively I would have expected to be able to do something like this...

suite = Test::Unit::TestSuite.new('my suite')
suite << MyTest
suite << MyTest
Test::Unit::UI::Console::TestRunner.run(suite)

But that does not work! If you do it, you will get this error...

/usr/lib/ruby/1.8/test/unit/testsuite.rb:54:in `size': undefined method `size' for MyTest:Class (NoMethodError)
        from /usr/lib/ruby/1.8/test/unit/testsuite.rb:54:in `each'
        from /usr/lib/ruby/1.8/test/unit/testsuite.rb:54:in `size'
        from /usr/lib/ruby/1.8/test/unit/ui/testrunnermediator.rb:35:in `run_suite'
        from /usr/lib/ruby/1.8/test/unit/ui/console/testrunner.rb:67:in `start_mediator'
        from /usr/lib/ruby/1.8/test/unit/ui/console/testrunner.rb:41:in `start'
        from /usr/lib/ruby/1.8/test/unit/ui/testrunnerutilities.rb:29:in `run'

[edit] Another (more verbose) way to do it

The docs recommend you do something more like this...

require 'test/unit'
require 'test/unit/ui/console/testrunner'

class MyTest < Test::Unit::TestCase
  def test_foo
    assert true
  end
end
class MySuite
  def self.suite
    suite = Test::Unit::TestSuite.new('my suite')
    suite << MyTest.suite
    suite << MyTest.suite
  end
end

Test::Unit::UI::Console::TestRunner.run(MySuite)

But that seems kind of verbose. I kind of like how it's a little bit more consistent in that you're dealing with classes in both the case of the test case (MyTest) and the test suite (MySuite), rather than having a class for one and an instance for the other... I'm still not sure which style I like better though...

[edit] Assertions

assert
assert_block
assert_equal
assert_in_delta
assert_instance_of
assert_kind_of
assert_match
assert_nil
assert_no_match
assert_not_equal
assert_not_nil
assert_not_same
assert_nothing_raised
assert_nothing_thrown
assert_operator
assert_raise
assert_respond_to
assert_same
assert_send
assert_throws

[edit] How to run only a single test method

use the --name option http://www.softiesonrails.com/2007/3/7/how-to-run-a-single-test-from-the-command-line

> ruby test/unit/product_test.rb --name test_calculate_tax

Loaded suite test/unit/order_test
Started
.
Finished in 0.71622 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

[edit] [Rails (category)]-only assertions

Rails / Testing / Assertions edit

assert_dom_equal
assert_dom_not_equal
assert_generates
assert_no_tag
assert_recognizes
assert_redirected_to
assert_response
assert_routing
assert_tag
assert_template
assert_valid

[edit] How to write your own custom assertions

  def assert_contains(container, expected_contents, failure_message = nil)
    failure_message = build_message(failure_message, "Container <?> was expected to contain <?> but it didn't", container, expected_contents)
    assert_block(failure_message) do
      container.include?(expected_contents)
    end
  end

[edit] setup / teardown

"In Test::Unit, setup and teardown will be called around every single individual test, not just at the start and end of the test case. Also, the Test::Unit::TestCase is reinstantiated for every individual test." (http://woss.name/2006/05/07/notes-from-a-rails-course/)

[edit] The autorunner / auto-created test suite

http://stdlib.rubyonrails.org/libdoc/test/unit/rdoc/classes/Test/Unit.html. Retrieved on 2007-05-11 11:18.

Test Runners

So, now you have this great test class, but you still need a way to run it and view any failures that occur during the run. This is where Test::Unit::UI::Console::TestRunner (and others, such as Test::Unit::UI::GTK::TestRunner) comes into play. The console test runner is automatically invoked for you if you require ‘test/unit’ and simply run the file.

Normally it works great to have Test::Unit create the test runner for you automatically so you don't have to worry about it.

However, I have come across at least a couple times when I didn't like that automatic behavior:

• When I wanted to define my TestCase class without having it run at all (perhaps so I could do introspection on it?). (I can't actually think of a great example where this would be necessary right now.)
• When I want to actually create a test case that tests the results of another test case (to test some custom assertion or output filter you've written for Test::Unit, for instance). This would require you to run the test-under-test first (and capture its output/results) and then run the test that tests the test-under-test.
• When I want to run a test, redefine a class, and then run the same test again. (A technique to avoid duplicating a test case with only one or two lines different.) (See section below)

[edit] How does it (the auto-running) work?

At works using an at_exit callback...

/usr/lib/ruby/1.8/test/unit.rb

at_exit do
  unless $! || Test::Unit.run?
    exit Test::Unit::AutoRunner.run
  end
end

The unless $! is probably so we won't run tests if there was an error...

raise 'My error'
at_exit { puts 'Had an error:'; p $! }

The unless Test::Unit.run? is so that if manually run your suite (any suite), then it will not try to automatically run it for you...

This flag is in the scope of Test::Unit module (it should have been a class variable @@run, but for some reason they made it an instance variable of the module!), so if you do any manual running at all, it will assume that you've taken care of running all the tests that you are going to run and it will not to take care of any of that for you...

This flag is made changeable here:

# /usr/lib/ruby/1.8/test/unit.rb
  module Unit
    # If set to false Test::Unit will not automatically run at exit.
    def self.run=(flag)
      @run = flag
    end

    # Automatically run tests at exit?
    def self.run?
      @run ||= false
    end
  end

... and is set here:

# /usr/lib/ruby/1.8/test/unit/ui/testrunnermediator.rb
        # Runs the suite the TestRunnerMediator was created
        # with.
        def run_suite
          Unit.run = true
          begin_time = Time.now
          notify_listeners(RESET, @suite.size)
          ...

[edit] Manually running your test suite

(or: Using test runners...)

http://stdlib.rubyonrails.org/libdoc/test/unit/rdoc/classes/Test/Unit.html. Retrieved on 2007-05-11 11:18.

[...] The console test runner is automatically invoked for you if you require ‘test/unit’ and simply run the file. To use another runner, or to manually invoke a runner, simply call its run class method and pass in an object that responds to the suite message with a Test::Unit::TestSuite. This can be as simple as passing in your TestCase class (which has a class suite method). It might look something like this:

   require 'test/unit/ui/console/testrunner'
   Test::Unit::UI::Console::TestRunner.run(TC_MyTest)

[edit] Avoid duplication: run a test, redefine a class, and then run the same test again

This [pattern (category)] looks sort of like this:

[edit] Pattern 1: Splitting it across several files

http://svn.tylerrick.com/public/examples/ruby/testing/run_test,_redefine_class,_run_same_test_again_pattern

class_test.rb

require File.dirname(__FILE__) + '/test_helper'

# We want to test that it works with either configuration A *or* configuration B.
# But rather than create two different classes -- one with each configuration -- and one test for each, we simply reuse one class
# and one test (which refers to the same class name both times).
# We create the class using each of two different configurations and run the test for each of those cases, making sure that it
# passes each time.

class ClassUnderTest
  def self.configuration
    'configuration A'
  end
end
load_local 'class_test_included.rb'

ClassUnderTest.remove_const!
class ClassUnderTest
  def self.configuration
    'configuration B'
  end
end
load_local 'class_test_included.rb'

class_test-included.rb

# This file contains the test code that is common for all configurations of ClassUnderTest.
# You wouldn't run this independently. Instead, run class_test.rb, which will do the necessary setup and then *include* this file.

require 'qualitysmith_extensions/module/create'
test_case_class = Class.create(:MyTest, :superclass => Test::Unit::TestCase) do
  def test_whatever
    assert ClassUnderTest.configuration =~ /configuration [AB]/
  end
end

Test::Unit::UI::Console::TestRunner.run(test_case_class)

One apparent side effect of this, however, is that if you use the Rake test loader, it will apparently cause it to not load and run all of your test cases. That could be bad, so see the section on how to make your own simple test runner

I think the reason it's not running all the test cases automatically is probably because once you manually run one test case, it will turn off automatic test suite creation/running and you will have to run all of them manually (?). I could be wrong; I haven't really looked into it yet.

[edit] Story: Before I remembered that tests are not run immediately after defining/loading the test case...

test_whatever(#<Module:0xb7a026f4>::ActsAsListSimpleTests):
NameError: "#<Module:0xb7a026f4>" is not a valid constant name!
    /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/inflector.rb:247:in `constantize'
    /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/core_ext/string/inflections.rb:148:in `constantize'
    /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/core_ext/module/introspection.rb:6:in `parent'
    /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:464:in `const_missing'
    ./acts_as_list_test_included.rb:7:in `setup_without_fixtures'
    /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/fixtures.rb:576:in `setup'

require 'test/unit/ui/console/testrunner'
Test::Unit::UI::Console::TestRunner.run(ActsAsListSimpleTests)

[edit] Pattern 2: All in one file, using FreshTestEnvironmentCreator

test_extensions gem (http://code.qualitysmith.com/gemables/test_extensions)

Used, for example, here: http://svn.tylerrick.com/public/rails/examples/trees/ordered_trees/test

[edit] Miscellaneous

"You can use breakpoint in tests and it will break out to a prompt during the running of the test case. ^d resumes execution. You can inspect and modify data from there." (http://woss.name/2006/05/07/notes-from-a-rails-course/)

Agiledocs is a method for generating documentation from your unit and functional tests. Chad has a Ruby implementation on his web site. The original is TestDox [1]. (http://woss.name/2006/05/07/notes-from-a-rails-course/)

[edit] How to make your own simple test runner that runs each test case in its own process

Why would you want to do that?

• So that one test case can't interfere with another test case. For instance, by monkey-patching some base class module.

I think it's a seriously bad idea to load and run all of your test cases all in the same Ruby [session] process.

I believe the following simple rake test will make those kinds of problems impossible, by running each test file in

Rakefile:

desc 'Run tests'
task :test do
  require 'rubygems'
  require 'facets/more/filelist' unless defined?(FileList)

  files = FileList["**/*_test.rb"]
  p files.to_a
  files.each do |filename|
    system "ruby #{filename}"
  end
end

This is instead of using the usual rake test loader:

require 'rake/testtask'
Rake::TestTask.new(:test) do |task|
  task.libs << 'lib'
  task.pattern = 'test/**/*_test.rb'
  task.verbose = true
end

http://svn.tylerrick.com/public/ruby/examples/testing/test_runner provides an example of where running two tests using one test runner results in passing tests but with the other test runner results in one test failing!

 > rake test
["test/test_1_that_monkey_patches_Object_class_test.rb", "test/test2_test.rb"]
Loaded suite test/test_1_that_monkey_patches_Object_class_test
Started
.
Finished in 0.001394 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
Loaded suite test/test2_test
Started
.
Finished in 0.001411 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

but:

> rake test_using_rake_test_loader
/usr/bin/ruby -Ilib:lib "/usr/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader.rb" "test/test_1_that_monkey_patches_Object_class_test.rb" "test/test2_test.rb"
Loaded suite /usr/lib/ruby/gems/1.8/gems/rake-0.7.2/lib/rake/rake_test_loader
Started
.F
Finished in 0.025077 seconds.

  1) Failure:
test_2(Test2) [./test/test2_test.rb:8]:
<"interesting!"> expected to be =~
</#<Moo:.*>/>.

2 tests, 2 assertions, 1 failures, 0 errors

[edit] Simulating terminal/console input (stdin)

Set up $stdin in your test so that it is a StringIO object (opened for reading) containing the string representing the simulated input:

$stdin = StringIO.new('input string', 'r')

before calling your production code, which probably has something like this:

$stdin.getc.chr

[edit] Catching stdout/stderr in order to test it

[To do: I've written a capture_output method to help with that...]

[edit] Mock objects / stubbing in general

http://www.mockobjects.com/

What's the difference?

Martin Fowler. Mocks Aren't Stubs (http://www.martinfowler.com/articles/mocksArentStubs.html#RegularTests). Retrieved on 2007-02-27 14:42.

• Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.
• Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

Of these kinds of doubles, only mocks insist upon behavior verification. The other doubles can, and usually do, use state verification.

...

The key difference here is how we verify that the order did the right thing in its interaction with the warehouse. With state verification we do this by asserts against the warehouse's state [data]. Mocks use behavior verification, where we instead check to see if the order made the correct calls on the warehouse. We do this check by telling the mock what to expect during setup and asking the mock to verify itself during verification

Lori Olson 2007-01-23 on http://weblog.jamisbuck.org/2007/1/23/how-would-you-do-it

I discovered something interesting in some of my fellow programmers. They don’t use mocks because they say mocks are too hard to build. But that’s just a cover for the real problem, which is that they don’t understand the thing needing to be “mocked” well enough to mock it… which sort of explains why the unit tests suck.

Jim Morris 2007-01-23 on http://weblog.jamisbuck.org/2007/1/23/how-would-you-do-it

I agree with Lori, mocking is an acquired art, and it isn’t always obvious how to mock an object or indeed which objects to mock. Also injection of mock objects sometimes requires minor refactoring of the code under test, which I think is quite acceptable practice as good unit tests have such obvious advantages.

[edit] Are they good? Should we use them?

Error on call to template:cite web: Parameters archiveurl and archivedate must be both specified or both omitted'From inky at misnomer.us...'. Retrieved on 2007-02-27 16:29.

Francis Hwang gave a great first talk on unit testing and talked a lot about how to use mock objects to increase the speed of unit tests. He explained a bit about MockFS, including the crackrock feature that redefines Fileutils and the like. I personally think that's cool as hell but it does seem like it could be incredibly dangerous.

He mentioned the fact that <a href="http://lafcadio.rubyforge.org">Lafcadio</a> completely mocks MySQL to the point where you can test it without even having a database installed. I am absolutely sure that I need to steal this functionality for <a href="http://ruport.rubyforge.org">Ruport</a> :)

He used easyprompt as example of mock user IO. Meh. I think we handle it better(support direct StringIO) in <a href="http://highline.rubyforge.org">HighLine</a>, but that's my ego talking.

Anyway, you can check out the code examples from this first talk here: <a href="http://fhwang.net/top_to_bottom/">http://fhwang.net/top_to_bottom/</a>

DH really made it clear that he pretty much disapproves of mocking in unit tests and talked about how all the Basecamp units are gone in 60 seconds or less using a 'real' database.

[edit] Mock objects / stubbing in Ruby

Libraries available: Rspec, FlexMock, Mocha, SchMock, Mocker, Mockr, CMock, Rumock

http://sbrew.typepad.com/software/2006/11/ruby_mock_objec.html

http://metaclass.org/2006/12/22/making-a-mockery-of-activerecord

http://blog.floehopper.org/articles/2006/12/23/tell-dont-ask Tell, Don't Ask

[edit] Mocha

http://mocha.rubyforge.org/ / http://rubyforge.org/projects/mocha

Mocking/stubbing library with JMock/SchMock syntax, which allows mocking/stubbing of methods on real (non-mock) classes. Includes auto-mocking which magically provides mocks for undefined classes, facilitating unit tests with no external dependencies.

Great examples!

• http://mocha.rubyforge.org/examples/misc.html Usage Examples
• http://mocha.rubyforge.org/examples/mocha.html Star Trek Example
• http://mocha.rubyforge.org/examples/stubba.html Order Example

[edit] Flex Mock

http://onestepback.org/software/flexmock/ Flex Mock

Jim Morris 2007-01-23 on http://weblog.jamisbuck.org/2007/1/23/how-would-you-do-it

Testing network code is never easy :( and these examples you give seem especially hard.

I usually use flexmock for these kinds of problems, and mock out as high a level as I can, to focus my tests on the specific method.

For instance in the file transfer abstraction, I’d flexmock sftp, and have it return different results for testing those edge cases.

I found flexmock to be the most flexible of the mocking packages, although it takes a while to get the hang of it, but well worth it.

[edit] RSpec

[Behavior-driven development (category)]

"a testing framework that reads well"

http://rspec.rubyforge.org/

The aim of BDD is to address this shortcoming and, by using terminology focused on the behavioural aspects of the system rather than testing, attempt to help direct developers towards a focus on the real value to be found in TDD at its most successful, or BDD as we call it.

(paraphrased from behaviour-driven.org)

http://rspec.rubyforge.org/tools/ Tools Overview

[edit]

RailsConf 2007: The case of the missing BDD (http://blog.brightredglow.com/2007/5/19/railsconf-2007-the-case-of-the-missing-bdd). Retrieved on 2007-05-11 11:18.

So, my goal will be to embarrass, cajole, entice, encourage (pick your emotional poison) you into trying RSpec. Not sure if it’s right for you? Try it. Not sure if it will work well? Try it. Don’t get it yet? Try it, already!

And while you’re trying it, think about this: How can descriptions of the behavior of a system form the common language that spans the spectrum of development from clients to developers. Do you think your clients care about how many def test_should_be_super_duper methods are in your tests? Why would they?

Would your customer care about something like this?

The Home page

• has an entry field for username labelled 'User name'
• has an entry field for password labelled 'Password'
• has a submit button labeled 'Log in'
• has a link to retrieve a lost password FAILED

1) The Home page has a link to retrieve a lost password FAILED
   <insert client comprehensible failure message here>

Do you think this could form the basis of a dialogue with your clients? Would it be easier to trouble shoot this over a client saying, “The lost password link doesn’t work.” Do you see how this could make everyone’s life easier?

Is this all there is to BDD and RSpec? No, not by a long shot.

[edit] test/spec

[edit] Colorize your test output

[edit] test helper using colored

http://www.nkryptic.com/2006/10/26/rails-colorized-rake-testing -- very cool!

Depends on:

sudo gem install colored

(http://errtheblog.com/post/36)

To do: gemify test_helpers that we use at QualitySmith

[edit] red-and-green-for-ruby

http://on-ruby.blogspot.com/2006/05/red-and-green-for-ruby.html

[edit] RedGreen

[edit] Libraries

[edit] [Automated browser-based testing of web applications (category)]

[edit] WebUnit

[edit] Selenium

[edit] Watir

[edit] comparison of Watir and Selenium

• http://opensource.thoughtworks.com/papers/WatirAndSelenium.html
• http://narayanraman.blogspot.com/2006/03/sahi-vs-selenium-vs-watir.html

[edit] lazytest

"Testing and benchmarking for lazy people"

[edit] Ratchets test runner

...

[edit] ZenTest kit

http://rubyforge.org/projects/zentest

Testing, on steriods. Go red with zentest which ensures test coverage and accelerates TDD. Go green with unit_diff using Advanced Diffing Technology™ to highlight errors. Refactor with autotest, continuous integration while you code.

[edit] ZenTest

http://zentest.rubyforge.org/ZenTest/

ZenTest scans your target and unit-test code and writes your missing code based on simple naming rules, enabling XP at a much quicker pace. ZenTest only works with Ruby and Test::Unit.

...

There are two strategies intended for ZenTest: test conformance auditing and rapid XP.

For auditing, ZenTest provides an excellent means of finding methods that have slipped through the testing process. I've run it against my own software and found I missed a lot in a well tested package. Writing those tests found 4 bugs I had no idea existed.

ZenTest can also be used to evaluate generated code and execute your tests, allowing for very rapid development of both tests and implementation.

• Scans your ruby code and tests and generates missing methods for you.
• Includes a very helpful filter for Test::Unit output called unit_diff.
• Continually and intelligently test only those files you change with

autotest.

• Test against multiple Ruby versions with multiruby.

[edit] unit_diff

http://zentest.rubyforge.org/ZenTest/

unit_diff is a command-line filter to diff expected results from actual results and allow you to quickly see exactly what is wrong.

[edit] autotest

http://zentest.rubyforge.org/ZenTest/

autotest is a continous testing facility meant to be used during development. As soon as you save a file, autotest will run the corresponding dependent tests.

[edit] Test::Rails

(Rails / Testing-specific) http://www.zenspider.com/ZSS/Products/ZenTest

• Enhanced assertion library lets you ditch assert_tag and really express yourself.
• Audit your tests to ensure you have coverage on both views and controllers.
• Adds MVC to testing which gets you better feedback and lets you test partials.

[edit] ZenTest: multiruby

Polishing Ruby: multiruby is awesome (http://blog.zenspider.com/archives/2006/03/multiruby_is_aw.html). Retrieved on 2007-03-06 13:39.

Testing across multiple versions of ruby has flushed a bunch of potentially lethal bugs in ParseTree. Nothing huge in the actual logic of ParseTree, but logical testing errors that could have hidden nasty problems (or shown false negatives that were near impossible to debug). Absolutely lovely. And the ability to simply drop in a new tarball of ruby and have it automatically configured, built, installed, and used on your next run is even more lovely.

Polishing Ruby: Coming Soon: multiruby (http://blog.zenspider.com/archives/2006/03/coming_soon_mul.html). Retrieved on 2007-03-06 13:39.

Lets say you maintain a library or tool that could be sensitive to particular changes in ruby. In my case, ParseTree is very dependent on the internals of ruby staying the same. Maybe in your case you are using some methods that changed their argument semantics (they way public_methods changed in 1.8). Making sure your code worked from version to version is a bit of a pain.

[edit] [Ruby / how to create multiple levels of method overriding and prevent user from overriding your libraries version of the method (category)] MultipleSetupAndTeardown

Part of: Mocha

Lets you set up actions that should be called from the setup/teardown methods such that the user-supplied setup/teardown don't override yours!

It essentially turns Ruby's crude method overrides into a nicer stack-based method overriding system. Cool, I'd been wondering if there was a way to do that!

Wow! It looks the way this is implemented (and I can't think of any other way to do it) is to use method_added to "detect" when the user tried to add their own setup method. Thus we can prevent the user from wiping out our setup method.

/usr/lib/ruby/gems/1.8/gems/mocha-0.3.3/lib/smart_test_case/multiple_setup_and_teardown.rb

        def add_setup_method(symbol)
          setup_methods << symbol
        end

        define_method(:setup_new) do
          setup_mocha
        end

      def setup_mocha
        self.class.class_eval { setup_methods }.each { |symbol| send(symbol) }
      end
 
      ...

      def self.method_added(method)
        case method
        when :setup
          unless method_defined?(:setup_added)
            alias_method :setup_added, :setup
            define_method(:setup) do
              begin
                setup_new       # Mocha's setup
              ensure
                setup_added     # User's setup
              end
            end
          end
        ...
      end

Example usage:

/usr/lib/ruby/gems/1.8/gems/mocha-0.3.3/lib/stubba/setup_and_teardown.rb

module SetupAndTeardown

  def self.included(base)
    base.add_setup_method(:setup_stubs)
    base.add_teardown_method(:teardown_stubs)
  end

  ...
end

[edit] Mocha

mocking and stubbing (Stubba)

[edit] ↓ MockFS

I've had too many problems with this library (mostly because it's incomplete = doesn't have all the same methods as the classes it's intended to replace) to recommend it at this time. — Tyler (2007-03-22 16:47)

http://mockfs.rubyforge.org/ RDoc Documentation

MockFS is a test-obsessed library for mocking out the entire file system. It provides mock objects that clone the functionality of File, FileUtils, Dir, and other in-Ruby file-access libraries.

> sudo gem install mockfs

In your test:

require mockfs/override.rb

    MockFS.fill_path('calculator/lib/')
    File.open('calculator/lib/calculator.rb', 'w') do |file|
      file.puts "line 1"
    end
    File.open('calculator/lib/calculator.rb', 'r') do |file|
      puts file.read
    end

line 1

or

    MockFS.mock = true
    MockFS.fill_path('calculator/lib/')
    MockFS.file.open('calculator/lib/calculator.rb', 'w') do |file|
      file.puts "line 1"
    end
    MockFS.file.open('calculator/lib/calculator.rb', 'r') do |file|
      puts file.read
    end

line 1

Caveats

• IO.read(file) wasn't working (was giving an ENOENT error). (MockFS must live the IO class alone because it's not file-system-specific.) Had to change that to File.read(file).
• puts File.readlines(file) wasn't working (was just hanging there until I hit Ctrl-C). When I changed it to File.read(file), however, it worked fine.

[edit] Heckle -- mutation tester

http://seattlerb.rubyforge.org/heckle/

Heckle is a mutation tester. It modifies your code and runs your tests to make sure they fail. The idea is that if code can be changed and your tests don‘t notice, either that code isn‘t being covered or it doesn‘t do anything.

It‘s like hiring a white-hat hacker to try to break into your server and making sure you detect it. You learn the most by trying to break things and watching the outcome in an act of unit test sadism.

[edit] Test coverage

[edit] Rcov

rcov is a code coverage tool for Ruby. It is commonly used for viewing overall test coverage of target code.

[edit] Win32 GUI testing

http://rubyforge.org/projects/guitest/

Win32 GUI testing project, inspired by Perl's win32::guitest.

[edit] Ruby Web Bench

Actually should be moved to a different page...

[edit] RubyFit

http://fit.rubyforge.org/basics.html

Fixtures represent a software layer between test data as HTML tables on a side and the application to test on the other side. The FIT framework provides a means to interact with test data; so, inside the fixture a means must exist to interact within the application. A way to exercise the application under test can be provided by a private member to set up in the initialization method of the fixture.

class CalculatedDiscount < Fit::ColumnFixture
  attr_writer :amount
  def initialize
    @application = Discount.new
  end
  def discount
    @application.get_discount @amount
  end
end

The discount method in the fixture must contain the operations executed to check the test cases one at a time, using the amount field as the input data, offering actual results which will be verified on the basis of the expected results provided by the HTML table.

Supposing the CalculateDiscount table is contained in a file called TestDiscount.html, the following command runs the test against RubyFIT and generates a report in a file called ReportDiscount.html:

fit TestDiscount.html ReportDiscount.html

[edit] Faucets

[edit] Features / About

http://faucets.rubyforge.org/

The syntax is simple and only require minimal understanding of HTML. No programming knowledge is required. Watir uses an OLE object to script and query Internet Explorer.

...

We have encountered some problems using IE to run FitNesse when used for Faucets tests that include javascript popups. Our normal setup is to use Firefox to run FitNesse.

Faucets will only test your site in IE. I have not tested any version of FireWatir.

[edit] Example

http://faucets.rubyforge.org/

As an example, here is how you would test Google's spell check feature:

Faucets::Commands
open	http://www.google.com
type	name=q	javaz language
click	name=btnG
assertText	id=spellCheckedId	java language

This is a hypotetical test, since google does not put an id on their spell checked link. If this was your site and you wanted to test it, all you would have to change is add an id to the link.

[edit] Motivation/Story

http://faucets.rubyforge.org/

When a friend and coworker told me about Watir, we immediately saw the potential to automate tests for the navigation and basic functionality of our web application. We already had a large suite of unit and functional tests for the business tier, but the web tier had almost none. Several attempts in the past resulted in difficult to write and/or maintain tests. We wanted the tests to be easy to write and maintain, not require programming knowledge, and as much as possible usable in test-first development. I thought of using FitNesse to create and run the tests through a Ruby fixture. Realizing something similar might have already been implemented, I searched the web and found Selenium. It looked very promising. I installed and tested it, but soon found some real blockers. Nevertheless, I used the Selenium API as a guide for the one I was building. Faucets was written over the course of a few days and put to use. We updated and evolved it over the next few weeks, and it has been pretty stable ever since. After over 1 year of continuous use and 100's of tests written, Faucets still lives up to the task. We felt is was ready to be released as open source.

[edit] gemdev

[edit] Handoff

[edit] Example

http://handoff.rubyforge.org/

 # This example is only helpful if you're familiar with Forwardable from the
 # stdlib, which you should be if you're coding delegating behavior in Ruby.
 #
 # Foo is a simple delegating class.  We want to specify that delegation in
 # its unit test.
 class Foo
   require 'forwardable'
   extend Forwardable
   def_delegators :bar, :baz, :bat
   def_delegator :bar, :baq, :bar_baq
   def_delegator :bar, :ban, :bar_ban

   # Foo#bar gets the delegate.  This is not what Handoff tests.
   # Rather, Handoff will mock this method and ensure that the
   # delegating methods defined above use it correctly.
   def bar
     @bar ||= BarFactory.create
   end
 end

 require 'test/unit'
 require 'rubygems'

 require 'mocha' # You need to require mocha and stubba in your test.
 require 'stubba' # ditto

 require File.dirname(__FILE__) + '/../lib/handoff' # I require it this way because I don't want the installed gem.
 # require 'handoff' # This is how you'd require handoff (or do require_gem w/ a version).

 class ExampleTest < Test::Unit::TestCase

   def test_using_from_and_single_method
     assert_handoff.from(Foo.new).to(:bar).for_method(:baz)
   end

   def test_using_from_any_and_multiple_methods
     assert_handoff.from_any(Foo).to(:bar).for_methods(:baz, :bat)
   end

   def test_showing_delegating_methods_with_names_different_than_their_delegate_methods
     assert_handoff.from(Foo.new).to(:bar).for(:bar_baq => :baq, :bar_ban => :ban)
   end

   def test_specifying_args
     assert_handoff.from(Foo.new).to(:bar).for(:baz).with('arg1', 2, :three)
   end

   def test_specifying_some_arg
     assert_handoff.from(Foo.new).to(:bar).for(:baz).with_no_arguments
   end

 end