• Fabien Potencier
• over 2 years ago
• /blog/category/
• 8

New testing framework

If you keep an eye on the timeline, you probably saw that the symfony unit tests have been completely reworked lately. This is because we switched from simpletest, which was fine but had side effects when functional tests were executed all at once, to our own testing framework, lime.

Lime is more lighweight than PHPUnit or simpletest and has several advantages. First, it launches test files in a sandbox to avoid strange effects between each test file run (one of the reasons we were unable to fix the old symfony core tests). It also introduces a new sfBrowser, sfTestBrowser and more importantly sfDomCssSelectorBrowser that allow you to write functionnal tests with ease. It is not backward compatible but is a lot more powerful than the old system. Oh, and it holds in a single file, lime.php, without any dependence.

So, you can keep your unit tests written in simpletest but functionnal tests that rely on the sfTestBrowser class have to be upgraded.

Here is a simple usage example of the new sfTestBrowser class for functional tests:

define('SF_ROOT_DIR',    realpath(dirname(__FILE__).'/..'));
define('SF_APP',         $app);
define('SF_ENVIRONMENT', 'test');
define('SF_DEBUG',       true);

require_once(SF_ROOT_DIR.DIRECTORY_SEPARATOR.'apps'.DIRECTORY_SEPARATOR.SF_APP.DIRECTORY_SEPARATOR.'config'.DIRECTORY_SEPARATOR.'config.php');

$b = new sfTestBrowser();
$b->initialize();

// default main page
$b->
   get('/')->
   isStatusCode(200)->
   isRequestParameter('module', 'default')->
   isRequestParameter('action', 'index')->
   checkResponseElement('body', '/congratulations/i') ;

As you can see, the browser now has a fluent interface. The above code is the same as:

$b->get('/');
$b->isStatusCode(200);
$b->isRequestParameter('module', 'default'); 
$b->isRequestParameter('action', 'index'); 
$b->checkResponseElement('body', '/congratulations/i');

The browser object also gives access to the response context for each request after a ->get() or a ->post(). We added shortcuts for testing request parameters or HTTP headers:

$b->isRequestParameter('module', 'action'); 
$b->isResponseHeader('content-type', 'text/html; charset=utf-8');

The new ->checkResponseElement() method is the most powerful of the new methods. It allows to test the response content by CSS selectors. Here are some examples (from sfDomCssSelectorTest unit test file):

$t->is($c->getTexts('h1'), array('Test page'), '->getTexts() takes a CSS selector as its first argument'); 
$t->is($c->getTexts('#footer'), array('footer'), '->getTexts() supports searching html elements by id'); 
$t->is($c->getTexts('.header'), array('header'), '->getTexts() supports searching html elements by class name');
$t->is($c->getTexts('div.header'), array(), '->getTexts() supports searching html elements by class name for a tag name');
$t->is($c->getTexts('ul#mylist ul li'), array('element 3', 'element 4'), '->getTexts() supports searching html elements by several selectors');
$t->is($c->getTexts('ul#list li a[class~="foo1"]'), array('link'), '->getTexts() supports checking attribute word matching'); 
$t->is($c->getTexts('ul#list li a[class^="foo1"]'), array('link'), '->getTexts() supports checking attribute starting with'); 
$t->is($c->getTexts('ul#list li a[class$="foobar1"]'), array('link'), '->getTexts() supports checking attribute ending with'); 
$t->is($c->getTexts('ul#list li a[class*="oba"]'), array('link'), '->getTexts() supports checking attribute with *'); 

To ease the reading of tests results, lime uses coloring on compatible systems.

Until we add documentation for lime, you are invited to have a look at the tests already added to the framework (in the test/ folder) and use them as an example for your own unit and functional tests.

One last word: the current alpha version is not stable enough to be used in production (hence the 'Alpha' attribute). We thank all the devs who send us feedback about remaining bugs, but we warn regular users not to upgrade their application to the 0.8 until we tag it at least beta.

Comments

• 

  #1 Marc Köhlbrugge said about 22 hours later

  Can new stuff like this be found in the (printed) book? By the way, Lime looks nice.. keep up the good work!

• 

  #2 fabien said about 24 hours later

  Yes, there is a chapter about the new testing system in the printed book.

• 

  #3 Marc Köhlbrugge said 2 days later

  Nice, looking forward to using it :) (I'll wait till I have some documentation)

• 

  #4 Troels said 3 days later

  Not trying to be a smartass here or anything, but by functional tests, do you mean procedural? If not, could you explain how simpletest got in the way for that?

• 

  #5 Marcus Baker said 10 days later

  Hi.

  It pains me how much PHP4 compatibility is holding me back :(. I wanted to add CSS selection in the DOM too as soon, as I can use the PHP5 HTML parser.

  Warning: I'll be doing something very similar over the coming year (SimpleTest2). Expect me to steal all your hard work ;).

  yours, Marcus

• 

  #6 fett said 16 days later

  What bothers me here is that tests aren't grouped in classes, which is a good way of organizing on the higher level. Examples would include: running only selected test-cases, extending reusable cases, analyzing tests and /not/ running them (via Reflection API), etc ...

  File-grouping just isn't nearly flexible enough ...

  I'm looking forward to using this anyway, for it seems fit to purpose.

  Regards

• 

  #7 Clay Loveless said about 1 month later

  Careful about this choice -- you're introducing a series of potential problems by having a unit test framework that has to test itself.

  If PHP 4 stuff within simpletest was holding you back in some way, PHPUnit would probably have been a safer bet. Plus, PHPUnit now has Selenium support (in CVS, at least).

• 

  #8 Don Hosek said 4 months later

  Some documentation on user-created unit tests is DESPERATELY NEEDED. How do I register tests for symfony test-unit?