Continuous testing tools

From WhyNotWiki

Contents 1 CruiseControl.rb 1.1 Comments 1.2 Design: Basics 1.3 Usage: Basics 1.4 Build scheduling 1.5 Files and folders 1.6 Build monitoring 1.7 Configuration examples 2 DamageControl 3 Cerberus 3.1 Tesly Reporter [Externally hosted (category)] 3.2 Tinderbox 3.3 Firebrigade [Externally hosted (category)] 3.4 Growl Tests [Desktop notification (category)](Platform:[Mac (category)]) 4 Articles

[edit] CruiseControl.rb

"Continuous Integration in Ruby"

Homepage:	http://cruisecontrolrb.thoughtworks.com/
Documentation:	Online demo
Source code:	http://rubyforge.org/frs/?group_id=2918
Project/Development:	http://jira.public.thoughtworks.org/browse/CCRB
Issue tracker:	http://jira.public.thoughtworks.org/browse/CCRB
Description:	CruiseControl.rb is a continuous integration tool. Its basic purpose in life is to alert members of a software project when one of them checks something into source control that breaks the build.
License:	[[licensed under::Apache License, Version 2.0 [1]|Apache License, Version 2.0 [2]]]
Implementation language:	Ruby

“	CC.rb is easy to install, pleasant to use and simple to hack.	”
—http://cruisecontrolrb.thoughtworks.com/

[edit] Comments

• Very configurable
• Modular (allows plugins)

[edit] Design: Basics

http://cruisecontrolrb.thoughtworks.com/documentation/getting_started

CruiseControl.rb has two parts: a builder and a dashboard.

Builder is a daemon process that polls source control every once in a while for new revisions.

When someone performs a check in, the builder:

1. detects it 2. updates its own copy of the project 3. runs the build 4. notifies interested parties about the build’s outcome

Dashboard is a web application that helps to monitor the status of project builds and troubleshoot failures.

Each installation of CruiseControl.rb may have multiple projects and multiple builders (one per project). There may also be multiple installations of CruiseControl.rb per computer.

[edit] Usage: Basics

http://cruisecontrolrb.thoughtworks.com/documentation/getting_started

[Condensed]

1. From [cruise], run ./cruise add your_project --url [URL of your_project Subversion trunk].
2. From [cruise], run ./cruise start.
3. Browse to http://localhost:3333.
4. Go to [cruise]/projects/your_project/work/ and make the build pass.
   • For a regular Rails app, this involves creating a test database, editing database.yml to connect to that database, performing rake RAILS_ENV=test db:migrate, running rake test and making sure that it passes.
5. Press the “build now” button on the Dashboard to rebuild your project

[edit] Build scheduling

http://cruisecontrolrb.thoughtworks.com/documentation/manual

By default, the builder polls Subversion every 10 seconds for new revisions. This can be changed by adding the following line to cruise_config.rb:

Project.configure do |project|
  ...
  project.scheduler.polling_interval = 5.minutes
  ...
end

What if you want a scheduler with some interesting logic? Well, a default scheduler can be substituted by placing your own scheduler implementation intpo the plugins directory and writing in cruise_config.rb something like this:

Project.configure do |project|
  ...
  project.scheduler = MyCustomScheduler.new(project)
  ...
end

[edit] Files and folders

http://cruisecontrolrb.thoughtworks.com/documentation/manual

If CC.rb is unpacked into [cruise] directory , then:

• [cruise]/projects/ is the projects directory.
• [cruise]/projects/your_project/ is a directory for the project called “your_project”.
• [cruise]/projects/your_project/work/ is a local copy of your_project’s source code. Builder keeps it up to date with the source control repository and runs builds against it.
• [cruise]/projects/your_project/build-123/ contains build status file, list of changed files, and other “build artifacts” created while building revision 123.
• [cruise]/projects/your_project/cruise_config.rb is builder configuration for your_project.
• [cruise]/config/site_config.rb is the file where you can make centralized changes to the configuration of dashboard and all builders.

[edit] Build monitoring

E-mail

Project.configure do |project|
  ...
  project.email_notifier.emails = ['john@doe.com', 'jane@doe.com']
  ...
end

Jabber

http://cruisecontrolrb.thoughtworks.com/documentation/manual

CC.rb comes with a Jabber plugin.

CCTray

http://cruisecontrolrb.thoughtworks.com/documentation/manual

CCTray is a utility developed as part of CruiseControl.NET project that displays an icon in the bottom right corner of the screen. The icon changes its color to red when a build fails, and back to green when the build is fixed.

[edit] Configuration examples

Configuration: [3]

Project.configure do |project|
  case project.name
  when 'MyProject.Quick' then project.rake_task = 'test:units'
  when 'MyProject.BigBertha' then project.rake_task = 'cruise:all_tests'
  else raise "Don't know what to build for project #{project.name.inspect}" 
  end
end

Configuration.dashboard_url = 'http://your.host.name.com:3333'

[edit] DamageControl

http://dev.buildpatterns.com/trac/wiki/WhatIsDamageControl

[edit] Cerberus

http://rubyforge.org/projects/cerberus/

Cerberus Continuous Integration for Rails

Continuous Integration (CI) is a way of automating your tests when you check sourcecode into your sourcecode management system (like Subversion). Every check-in will result in all tests being run, after which developers are informed of the outcome of the build/tests. Martin Fowler wrote a nice article on continious integration.

Cerberus is a Ruby based CI system which can also be used with Ruby on Rails. What follows is a little how-to on Cerberus.

[edit] Tesly Reporter [Externally hosted (category)]

http://rubyfurnace.com/plugins/tesly_reporter

Tesly (http://www.tesly.com/) is a web service that allows developers, testers, and managers to collaborate on testing for software projects. This plugin enables automated test reporting to Tesly. When unit, functional, and integration tests are run from the Rails application directory, reports of the test run are sent to Tesly, keeping everyone on the project up to date on the latest status of the project.

Run your tests via rake as you usually do, and the results of your test cases will be sent to Tesly. A test plan will be created for each of your test files, and a test case will be generated for each test method. You can change the default names of the test plans and test cases at the Tesly site after they are created for you. As you add tests to your application, they will be added automatically to your Tesly account.

• Repository Path: http://tesly-reporter.googlecode.com/svn/trunk/rails/plugins/tesly_reporter/
• Homepage: http://www.tesly.com/

[edit] Tinderbox

http://www.mozilla.org/projects/tinderbox/

Tinderbox is the first program to allow developers and management to see at a glance what is currently going on in all aspects of the development process. The Tinderbox server prepares HTML pages which display the history of many different development variables. It shows at a glance the history of: whether the HEAD branch of the source code builds and passes all automated tests, who has checked code changes into the version control system, whether the source tree is open or closed and when the state of the tree last changed, what trouble tickets have been closed, notices and messages posted by the developers or project manager.

The new Tinderbox code is highly configurable and will allow you to work with many different types of version control or bug tracking systems. It is relatively easy to add new modules which work with other systems. It is also easy to configure Tinderbox to run without the displays that your organizations does not need or to run with duplicate modules if the need arises.

Used by: Firebrigade

[edit] Firebrigade [Externally hosted (category)]

Not really helpful to me/us, since it's not run locally and I don't think you can do anything to trigger it to run when you want it to, but interesting...

http://firebrigade.seattlerb.org/

Firebrigade tests every gem ever made on every platform under the sun and summarizes the results, enabling developers to see how their packages perform on systems they don't have access to.

How to run your own Tinderbox: http://firebrigade.seattlerb.org/home/run_a_tinderbox

Sample results: http://firebrigade.seattlerb.org/build/show/14068

[edit] Growl Tests [Desktop notification (category)](Platform:[Mac (category)])

http://agilewebdevelopment.com/plugins/growl_tests

Growl Tests gives you real time feedback of test breakage by displaying Growl notifications on your desktop. Growl is an open-source, pluggable notification framework available for Mac OS X systems.

[edit] Articles

[http://old.pupeno.com/blog/poor-man-s-continuous-unit-testing-with-ruby-on-rails/ Poor man's continuous unit testing with Ruby on Rails — Pupeno's web site] (http://old.pupeno.com/blog/poor-man-s-continuous-unit-testing-with-ruby-on-rails/). Retrieved on 2007-02-16 17:19.

I've heard the idea of unit testing is to have a continuous testing performed by one computer that would raise an alarm if something goes wrong. I always imagine a big red rotating lamp and a noisy siren. All lights would turn off in the dev-room, the lamp would starting driving the coders crazy and the siren waking those sleeping.

Continuous testing  edit   (Category  edit) .