Posts by: planet-ibuildings

[Shashikant Jagtap‘MinkExtension’ for Behat]Enjoy ‘MinkExtension’ for Behat May 9, 2012

‘MinkExtension‘ has been just released which has additional services for Behat. This extension has ‘Mink’ instance for your ‘FeatureContext’ and ‘SubContext’. UI testers would be happy now, as they don’t need to create Mink instance every time in order to use Mink API’s.

Well, in order to use ‘MinkExtension’ you need latest Behat and Mink versions (Behat 2.4+ and Mink 1.4+). Don’t worry so much about the versions, creator of Behat has taken care of all this with composer.

Ok. Let’s clone the MinkExtenion-Example repository from GitHub. See, everything is mentioned in README file.

git clone https://github.com/Behat/MinkExtension-example.git
Cloning into MinkExtension-example...
remote: Counting objects: 31, done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 31 (delta 13), reused 31 (delta 13)
Receiving objects: 100% (31/31), 4.43 KiB, done.
Resolving deltas: 100% (13/13), done.

Navigate to directory & Get the composer and Install dependencies

cd MinkExtension-example
curl http://getcomposer.org/installer | php

moonstar:MinkExtension-example sjagtap$ curl http://getcomposer.org/installer | php
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100  7438  100  7438    0     0  52386      0 --:--:-- --:--:-- --:--:--  105k
#!/usr/bin/env php
All settings correct for using Composer
Downloading...

Composer successfully installed to: /Users/sjagtap/MinkExtension-example/composer.phar
Use it: php composer.phar

Install Dependencies

php composer.phar install

oonstar:MinkExtensionTest sjagtap$ php composer.phar install
Installing dependencies
- Package symfony/yaml (dev-master)
Cloning 48fcffece8de2ade6f6dbc909f22444356b7659f

- Package symfony/translation (dev-master)
Cloning 34b64346b24024d54f42877b7c34ccc05ff1eaa8

- Package symfony/event-dispatcher (dev-master)
Cloning 0c1ae4898196f5e96b79028d8d2f35de4b584659

- Package symfony/dependency-injection (dev-master)
Cloning b01877ddbde08c1eded2502a6fdce8dbde89bd35

- Package symfony/config (dev-master)
Cloning 2f2fb45da8100bc03d2d9cbcd75e3ff36e325b89

- Package symfony/finder (2.0.x-dev)
Cloning v2.0.13

- Package symfony/css-selector (2.0.x-dev)
Cloning v2.0.12

- Package symfony/dom-crawler (2.0.x-dev)
Cloning v2.0.13

- Package symfony/browser-kit (2.0.x-dev)
Cloning v2.0.12

- Package symfony/process (dev-master)
Cloning 1bc1398d51180fd2aeded9c44f4d7b73bea817cd

- Package symfony/console (dev-master)
Cloning 8d58af16c425977018293eb0a1570c9629b37994

- Package behat/gherkin (2.2.x-dev)
Cloning ada4bda5823ffe1df8228e9dddefa38d6420f983

- Package zendframework/zend-registry (2.0.0beta3)
Downloading: 100%

- Package zendframework/zend-loader (2.0.0beta3)
Downloading: 100%

- Package zendframework/zend-stdlib (2.0.0beta3)
Downloading: 100%

- Package zendframework/zend-validator (2.0.0beta3)
Downloading: 100%

- Package zendframework/zend-http (2.0.0beta3)
Downloading: 100%

- Package zendframework/zend-uri (2.0.0beta3)
Downloading: 100%

- Package facebook/php-webdriver (master-dev)
Cloning master

- Package fabpot/goutte (master-dev)
Cloning master

- Package behat/mink-extension (dev-master)
Cloning cd8eb2c85859f7d5e88e5787d4e171eaa52002eb

- Package behat/mink (v1.4.0beta2)
Downloading: 100%

- Package behat/behat (v2.4.0beta3)
Downloading: 100%

symfony/event-dispatcher suggests installing symfony/http-kernel (dev-master)
behat/mink suggests installing behat/sahi-client (Sahi-Client is needed to use the SahiDriver)
behat/mink suggests installing alexandresalome/php-selenium (Php-Selenium is needed to use the SeleniumDriver)
behat/behat suggests installing behat/yii-extension (for integration of with Yii framework)
Writing lock file
Generating autoload files

Now Run Behat command,

./bin/behat

Look, if you still using PHP5.3.x Then you will get

moonstar:MinkExtension-example sjagtap$ ./bin/behat
PHP Parse error:  syntax error, unexpected T_USE, expecting T_FUNCTION in /Users/sjagtap/MinkExtension-example/features/bootstrap/TraitedFeatureContext.php on line 5
PHP Stack trace:
PHP   1. {main}() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/bin/behat:0
PHP   2. Symfony\Component\Console\Application-&amp;gt;run() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/bin/behat:27
PHP   3. Behat\Behat\Console\BehatApplication-&amp;gt;doRun() /Users/sjagtap/MinkExtension-example/vendor/symfony/console/Symfony/Component/Console/Application.php:106
PHP   4. Symfony\Component\Console\Application-&amp;gt;doRun() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/src/Behat/Behat/Console/BehatApplication.php:93
PHP   5. Symfony\Component\Console\Command\Command-&amp;gt;run() /Users/sjagtap/MinkExtension-example/vendor/symfony/console/Symfony/Component/Console/Application.php:193
PHP   6. Behat\Behat\Console\Command\BaseCommand-&amp;gt;initialize() /Users/sjagtap/MinkExtension-example/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:225
PHP   7. Behat\Behat\Console\Processor\AggregateProcessor-&amp;gt;process() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/src/Behat/Behat/Console/Command/BaseCommand.php:55
PHP   8. Behat\Behat\Console\Processor\LocatorProcessor-&amp;gt;process() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/src/Behat/Behat/Console/Processor/AggregateProcessor.php:57

This is time to update your PHP version. If you still want run features with same PHP version, then comment out/ remove everything from ‘TraitedFeatureContext.php’ which is for PHP5.4

Now, Start Selenium server & Run behat with ‘inheritance’ or ‘subcontext’ profile.

./bin/behat -p=inheritance

./bin/behat -p=subcontexts

You will see test running with headless as well as with browser. You will see output like this

moonstar:MinkExtension-example sjagtap$ ./bin/behat -p=subcontexts
Feature: Search
  In order to see a word definition
  As a website user
  I need to be able to search for a word

  Scenario: Searching for a page that does exist               # features/search.feature:6
    Given I am on &amp;quot;/wiki/Main_Page&amp;quot;                            # Behat\MinkExtension\Context\MinkContext::visit()
    When I fill in &amp;quot;search&amp;quot; with &amp;quot;Behavior Driven Development&amp;quot; # Behat\MinkExtension\Context\MinkContext::fillField()
    And I press &amp;quot;searchButton&amp;quot;                                 # Behat\MinkExtension\Context\MinkContext::pressButton()
    Then I should see &amp;quot;agile software development&amp;quot;             # Behat\MinkExtension\Context\MinkContext::assertPageContainsText()

  Scenario: Searching for a page that does NOT exist           # features/search.feature:12
    Given I am on &amp;quot;/wiki/Main_Page&amp;quot;                            # Behat\MinkExtension\Context\MinkContext::visit()
    When I fill in &amp;quot;search&amp;quot; with &amp;quot;Glory Driven Development&amp;quot;    # Behat\MinkExtension\Context\MinkContext::fillField()
    And I press &amp;quot;searchButton&amp;quot;                                 # Behat\MinkExtension\Context\MinkContext::pressButton()
    Then I should see &amp;quot;Search results&amp;quot;                         # Behat\MinkExtension\Context\MinkContext::assertPageContainsText()

  @javascript
  Scenario: Searching for a page with autocompletion           # features/search.feature:19
    Given I am on &amp;quot;/wiki/Main_Page&amp;quot;                            # Behat\MinkExtension\Context\MinkContext::visit()
    When I fill in &amp;quot;search&amp;quot; with &amp;quot;Behavior Driv&amp;quot;               # Behat\MinkExtension\Context\MinkContext::fillField()
    And I wait for the suggestion box to appear                # SubcontextedFeatureContext::iWaitForTheSuggestionBoxToAppear()
    Then I should see &amp;quot;Behavior Driven Development&amp;quot;            # Behat\MinkExtension\Context\MinkContext::assertPageContainsText()

3 scenarios (3 passed)
12 steps (12 passed)
0m14.59s

What’s new for Testers in MinkExtension?

You will get Mink instance in your ‘FeatureContext’ as well as ‘SubContexts’ classes.
You can extend Base Contextfile to MinkContext as shown in below. For other context you need to extend from RawMinkContext

class MyContext extends Behat\MinkExtension\Context\MinkContext

You can use MinkExtension as a SubContext in your files like this:

class SubcontextedFeatureContext extends Behat\Behat\Context\BehatContext
{
public function __construct()
{
$this-&gt;useContext('mink', new Behat\MinkExtension\Context\MinkContext);
}

This extension is a base for future extensions. We can extend Behat/Mink with any other services testing services like SauceLabs and many more. This feature will definitely useful for Web Testers using Behat and Mink for browser automation. SauceLabs extension is on it’s way.. stay tuned

Happy BDD !!!

[Shashikant Jagtap]Adding More Sauce To Behat April 28, 2012

Further to my previous post regarding ‘Behat-Sauce’ integration, ‘Adding Sauce To Behat‘, I am glad to know that there is another Behat and SauceLabs integration from Laura Beth. Thanks a lot Laura for that excellent effort.

On the occasion of Selenium Conference, I met with Noah Sussman talking about “Selenium In the Enterprise: What Went Right, What Went Wrong (So Far)” and one of his slides indicated that, team using Behat and Mink for the functional testing at Etsy. Noah then introduced me to Laura Beth who setup Behat/Mink and SauceLabs integration. Behat-Sauce configuration by Laura Beth is very easy to use.

Now, we will see this in action.

Behat Installation

Create a ‘Demo’ directory and Clone the repository from Laura Beth’s GitHub account

moonstar:~ sjagtap$ mkdir Demo
moonstar:~ sjagtap$ cd Demo
moonstar:Demo sjagtap$ git clone https://github.com/elblinkin/Behat-Sauce.git
Cloning into Behat-Sauce...
remote: Counting objects: 178, done.
remote: Compressing objects: 100% (105/105), done.
remote: Total 178 (delta 50), reused 166 (delta 38)
Receiving objects: 100% (178/178), 58.74 MiB | 503 KiB/s, done.
Resolving deltas: 100% (50/50), done.

Ok, you got all the code in ‘Behat-Sauce’ directory. Now you need to download dependencies for Behat. Navigate to Behat-Sauce and download composer.

moonstar:Behat-Sauce sjagtap$ wget  http://getcomposer.org/composer.phar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  442k  100  442k    0     0   988k      0 --:--:-- --:--:-- --:--:-- 1436k

Now, update ‘composer.phar’ file. It will download all dependencies.

moonstar:Behat-Sauce sjagtap$ php composer.phar update
Updating dependencies
  - Package symfony/yaml (dev-master)
    Cloning 52807f723e8a9743852c2105f11bd01c5af7f977

  - Package symfony/translation (dev-master)
    Cloning cac219a6dee410df41ac5dae15553313f48f1fd1

  - Package symfony/event-dispatcher (dev-master)
    Cloning b98d68d3b8513c62d35504570f09e9d3dc33d083

  - Package symfony/dependency-injection (dev-master)
    Cloning 61e664ab2eead27d83ebb45d2bf83dd55ef509a8

  - Package symfony/config (dev-master)
    Cloning 175adaf7dbed4ad5350e75a4fd3de39480df6cf3

  - Package symfony/console (dev-master)
    Cloning f354e1e5d59261ae03c0fc768bfa4043c0e6ab9c

  - Package symfony/process (dev-master)
    Cloning 3f95a0126588f00d70e745c1ef96631fb10c3a8e

  - Package symfony/css-selector (dev-master)
    Cloning dd695aadd80d8e6e726a7a050535c04325cefa02

  - Package symfony/dom-crawler (dev-master)
    Cloning 69b857977d96e4c726be5524fa00370aef5e9b20

  - Package symfony/browser-kit (dev-master)
    Cloning f03173e232da5d84eb799cad5d57354e6541b0f8

  - Package symfony/finder (dev-master)
    Cloning 5b8658e52eaf424fe943a5857e2404ce95c3b90e

  - Package behat/mink (dev-master)
    Cloning v1.3.3

  - Package behat/behat (dev-master)
    Cloning d8ff3fe0f8c44c664c5eebbb6b972bbdff70c166

  - Package alexandresalome/php-selenium (master-dev)
    Cloning master

  - Package behat/gherkin (v2.1.1)
    Downloading: 100%

symfony/translation suggests installing symfony/config (dev-master)
symfony/translation suggests installing symfony/yaml (dev-master)
symfony/event-dispatcher suggests installing symfony/dependency-injection (dev-master)
symfony/event-dispatcher suggests installing symfony/http-kernel (dev-master)
symfony/dependency-injection suggests installing symfony/yaml (dev-master)
symfony/dependency-injection suggests installing symfony/config (dev-master)
symfony/dom-crawler suggests installing symfony/css-selector (dev-master)
symfony/browser-kit suggests installing symfony/process (dev-master)
Writing lock file
Generating autoload files

You have just installed Behat and all dependencies. Now initialize project by running ‘bin/behat-sauce –init’ command.

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --init
+d features - place your *.feature files here
+d features/bootstrap - place bootstrap scripts and static files here
+f features/bootstrap/FeatureContext.php - place your feature related code here
+d config - edit you config settings here
+f /Users/sjagtap/Demo/Behat-Sauce/configbehat.yml - place your feature related code here

Once you initialized ‘Behat’. It will ‘features’ and ‘features/bootstrap’ directories and ‘features/bootstrap/FeatureContext.php’ file for you.

Feature file

You can also see ‘config’ directory with ‘behat.yml’ file for you. You need to add SauceLabs ‘Username’ and API key to the ‘behat.yml’.

Now write a simple feature file and save it to ‘features’ directory.

Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki
    Given I am on "/"
    And I fill in searchBox with "<input type="text" />"
    When I press search button
Then I should see "<output>"

 Examples:
 | input | output |
 | London | lʌndən/ |

Step Definition

You need implement step definitions using Mink API’s. You feature context file will look like this now.

<!--?php use Behat\Behat\Context\ClosuredContextInterface,     Behat\Behat\Context\TranslatedContextInterface,     Behat\Behat\Exception\PendingException; use Behat\Gherkin\Node\PyStringNode,     Behat\Gherkin\Node\TableNode; use Behat\Sauce\Context\SauceContext; /**  * Features context.  */ class FeatureContext extends SauceContext {     /**      * Initializes context.      * Every scenario gets it's own context object.      *      * @param   array   $parameters     context parameters (set them up through behat.yml)      */     public function __construct(array $parameters) {         parent::__construct($parameters);     } /**      * @Given /^I fill in searchBox with "([^"]*)"$/      */     public function iFillInSearchboxWith($input)     {         $this--->fillField("searchInput",$input);
    }

    /**
     * @When /^I press search button$/
     */
    public function iPressSearchButton()
    {
        $this->getMink()->getSession()->getDriver()->click("//*[@id='searchButton']");
         $this->getMink()->getSession()->wait("3000");
    }
}

Well, now you are ready to run your tests on SauceLabs just by running

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce

You will see test running on the SauceLabs. Once tests finished running, you will see output like this:

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce
Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7
    Given I am on "/"                       # FeatureContext::visit()
    And I fill in searchBox with "<input type="text" />"  # FeatureContext::iFillInSearchboxWith()
    When I press search button              # FeatureContext::iPressSearchButton()
Then I should see "<output>" # FeatureContext::assertPageContainsText()

 Examples:
 | input | output |
 | London | lʌndən/ |

1 scenario (1 passed)
4 steps (4 passed)
1m8.949s

You can watch video of the test here

As per Laura Beth, default arguments are:

 --browser              SauceLabs browser name.  Default is:  firefox
 --browser-version      SauceLabs browser version.  Default is:  7
 --os                   SauceLabs operating system.  Default is:  Windows 2003

Now let’s run our tests on different browser and different OS:

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --browser="firefox" --browser-version="3.6" --os="Linux"

You will see same output:

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --browser="firefox" --browser-version="3.6" --os="Linux"
Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7
    Given I am on "/"                       # FeatureContext::visit()
    And I fill in searchBox with "<input type="text" />"  # FeatureContext::iFillInSearchboxWith()
    When I press search button              # FeatureContext::iPressSearchButton()
Then I should see "<output>" # FeatureContext::assertPageContainsText()

 Examples:
 | input | output |
 | London | lʌndən/ |

1 scenario (1 passed)
4 steps (4 passed)
1m38.828s

Video of the tests can be found here

Sauce-Connect:

You can also start Sauce-Connect tunnel with the same credentials that you provided in config/behat.yml file

bin/behat-sauce --tunnel

You will see something like this:

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --tunnel
[-u, Shashikant86, -k, API Key, -d, sauce-connect.proxy, -s, 127.0.0.1, -p, 80, --ssh-port, 443, -b, --rest-url, https://saucelabs.com/rest/v1, --se-port, 4445, --squid-opts, ]
* Debug messages will be sent to sauce_connect.log
2012-04-24 17:27:32.354:INFO::jetty-7.x.y-SNAPSHOT
2012-04-24 17:27:32.439:INFO::Started SelectChannelConnector@0.0.0.0:50044
.---------------------------------------------------.
| Have questions or need help with Sauce Connect? |
| Contact us: http://support.saucelabs.com/forums |
| Terms of Service: http://saucelabs.com/tos |
-----------------------------------------------------
2012-04-24 17:27:32,509 - / Starting \
2012-04-24 17:27:32,513 - Please wait for "You may start your tests" to start your tests.
2012-04-24 17:27:32,526 - Forwarding: ['sauce-connect.proxy']:['80'] -> 127.0.0.1:['50044']
2012-04-24 17:27:32,538 - Succesfully connected to local server 127.0.0.1:50044 in 6ms
2012-04-24 17:27:33,201 - {"squid_config":[""],"use_caching_proxy":true,"fast_fail_regexps":[""],"ssh_port":443,"metadata":{"PythonVersion":"2.5.1","OwnerHost":"127.0.0.1","Release":"3.0-r18","OwnerPorts":["50044"],"Ports":["80"],"Platform":"Java-1.6.0_26-Java_HotSpot-TM-_64-Bit_Server_VM,_20.1-b02-383,_Apple_Inc.-on-Mac_OS_X-10.7.1-x86_64","Build":"26","ScriptRelease":26,"ScriptName":"sauce_connect"},"use_kgp":true,"domain_names":["sauce-connect.proxy"]}
2012-04-24 17:27:33,486 - Tunnel remote VM is provisioned (a86bd10139b8464081e185b016b6579d)
2012-04-24 17:27:33,756 - Tunnel remote VM is new ..
2012-04-24 17:27:37,068 - Tunnel remote VM is deploying ..
2012-04-24 17:27:40,440 - Tunnel remote VM is booting ..
2012-04-24 17:28:06,611 - Tunnel remote VM is running at maki10148.miso.saucelabs.com
2012-04-24 17:28:06,723 - Succesfully connected to local server 127.0.0.1:50044 in 0ms
2012-04-24 17:28:06,729 - Starting connection to tunnel host...
2012-04-24 17:28:06,732 - Connecting to tunnel host maki10148.miso.saucelabs.com as Shashikant86
2012-04-24 17:28:06,994 - Forwarding Selenium with ephemeral port 50478
2012-04-24 17:28:06.996:INFO::jetty-7.x.y-SNAPSHOT
2012-04-24 17:28:06.998:INFO::Started SelectChannelConnector@0.0.0.0:4445
2012-04-24 17:28:07,000 - Selenium HTTP proxy listening on port 4445
2012-04-24 17:28:07,855 - Successful handshake with Sauce Connect server
2012-04-24 17:28:08,109 - Tunnel host version: 0.1.0, remote endpoint ID: 2851094ae70e4e949a0e96c8ca453748
2012-04-24 17:28:08,112 - Connected! You may start your tests.

Sauce connect is used for testing internal site.

Test Locally:

Well, if you don’t want to waste sauceLabs minutes. You can run your test locally:

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --local

You will see tests running on local machine with Firefox and output is same:

moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --local
Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7
    Given I am on "/"                       # FeatureContext::visit()
    And I fill in searchBox with "<input type="text" />"  # FeatureContext::iFillInSearchboxWith()
    When I press search button              # FeatureContext::iPressSearchButton()
Then I should see "<output>" # FeatureContext::assertPageContainsText()

 Examples:
 | input | output |
 | London | lʌndən/ |

1 scenario (1 passed)
4 steps (4 passed)
0m16.109s

Contribution:

Well, I have forked repository and started implementing full scale SauceLabs grid which support all browsers & OS combination. I am also trying to plug the features on continuous integration server like Jenkins. If you also wish to contribute then fork “Behat-Sauce”

Source code:

Code for the above example can be found “Behat-Sauce-Demo” repository.

Conclusion:

Believe me, this is simply awesome. Thanks again Laura Beth.

Happy BDD !!

[Shashikant Jagtap]SeleniumConf came to London! April 23, 2012
As mentioned on the Selenium Conference 2012 website, this year conference would be across the pond and it was London. Conference venue ‘IET London‘ was amazing place situated on the bank of Thames river and great view of historical London Eye. I live and work in London so it’s very convenient for me to get down to the location of the conference. I met with with selenium users all across the world and shared thoughts and ideas about open-source testing. Selenium conference started on Monday April,16 and wrapped up on Wednesday, April,18. I spend three memorable days at Selenium conference and learned a lot about new tools, methodologies and approaches of testing applications with Selenium. I would like to take this opportunity to thank SauceLabs and all other sponsors of the Selenium Conference.

Let me see, how can I explain what happened in the conference.

WORKSHOP DAY

It was drizzling, when I came out of the Embankment tube station, and need to walk for 2-3 minutes to get to the venue. It wasn’t difficult to find venue and reception of the Selenium conference. I met with Ashley Wilson, (Customer Development Manager at SauceLabs) at registration desk. Ashley played key role in organizing and managing Selenium Conference. then I didn’t forget to pick a T-Shirt labeled with ‘Selenium Conference 2012′. The breakfast and tea was ready.

It was difficult to choose a workshop as both were interesting. I decided to go for workshop ” Contributing to Selenium” by ‘WebDiver Guy‘ (Simon Stewart) in the morning session.

“Contributing to Selenium” Workshop

The workshop aimed at contribution to selenium open-source project. I was ready with all the code downloaded from googleCode and brand new IntelliJ IDE. I have heard about cool features of the IntelliJ but never tried it before. It is really awesome. Simon already mentioned in technical requirements that ‘ Setting up project in Eclipse is not supported’.

Within the workshop, I had a look at directory structure of the selenium codebase, setup the project in the IntelliJ and also done some code modifications as suggested by Simon. At the middle of the workshop, I was lagging behind while running the JavaScript tests. I ran build with ‘build.desc’ file also ran some test with Firefox driver. Lately, I realised that I was surounded by ‘software engineer in test’ from Google. Lucky me !!

After the break, we started with running tests within IDE. Being new to the ItelliJ, I faced some issues while running tests but managed to get it worked with the help of Simon. At the end of the workshop, someone asked question about ‘How to measure coverage of the Selenium/Functional tests’ and there were interesting discussion around this topic. Being from PHP background, I couldn’t resist myself to ask about ‘PHP Webdriver bindings and current development on PHP bindings’. Simon said ” In terms of PHP, FaceBook-Webdriver bindings are recommended”. Simon then introduced me to Damien Sereni who works in Facebook. At the end of the session. Me and Demien spend some time discussing about future of the PHP webdriver bindings also shared contact details. The lunch was waiting for us then.

After lunch, again I had to make tough decision about selecting workshop. I have decided to go to “If Ikea made instructions for Selenium (or: How to fix bugs in Selenium for yourself)” by Daniel Wagner-Hall, and the Selenium Committers. I really missed the another one “Handmade Etsy Tests” by Stephen Hardisty & Michelle D’Netto. Lately realised that Etsy uses Behat and Mink for functional testing which is interesting topic for me which I missed a lot.

How to fix bugs in Selenium for yourself

In this workshop, Daniel talked about selenium codebase and different drivers. I have already checked out selenium trunk in my system and need to look into Selenium bugs to pick up. Being a tester and having limited development knowledge, I realised that I can’t fix any bug in selenium so just started doing some interesting stuff. I managed to get the tests running on my Andriod phone with the help of Andriod driver. That was enough for me it this session and it was end of first day of the selenium conference.

TUESDAY, APRIL 17

April,17 is my birthday. I reached early on that day in order to register my unconference talk on “Behat and Sauce Labs Integration”. I registered my talk at 10PM on that day. I was very excited to talk on the occasion of my birthday.

Anyway, I saw huge crowd on second day of the conference and met with people from different countries while having tea and breakfast. At 9 PM, we get into the lecture hall, which was huge and amazing. There was one tweet from Frisco Del Rosario (@frdelrosario) saying ” Auditorium at #SeConf had balconies, microphones at each chair, flawless wi-fi. http://twitpic.com/9c30la“

As per the schedule, Jason and Simon introduced sponsors and explained schedule of Track A and Track B talks.

Testing Rapidly Changing Applications With Self-Testing Object-Oriented Selenium Infrastructure

Most people turned on for Track B talk which was on “ Testing Rapidly Changing Applications With Self-Testing Object-Oriented Selenium Infrastructure” by Dan Cuellar. I also headed over to that talk. Slides of his talk are available on here Dan explained how to test rapidly changing applications with just small code change in the test infrastructure. Dan implemented object oriented approach that can handle such a rapid change. As per the Dan if you abstracted the code well then you can build robust automated test scripts. Dan is looking for someone who can help him to extend his work to support iOS. If you are interested to help him then email him at danc@zoosk.com

Web acceptance Testing with Behat

After Dan’s talk, it was my turn to give talk on ‘Web Acceptance Testing with Behat’. There was another Track A talk where Anand Ramdeo was talking about “One Step At a Time” most people turned out there. I was bit disappointed seeing only 10 people attending my talk but still I carried it forward. My talk covered BDD and ATDD with the Behat, Sauce Labs , Jenkins and Relish. Slides of my talk are available here.

So you’re automated… now let’s make it fast!

I have decided to stay at Track B where Sauce Ninja Santiago was about to talk on making test automation fast. The title of his talk was ” So you’re automated… now let’s make it fast!”. Santiago shared some tips to make selenium tests faster like avoiding static waits in the tests or running tests in parallel. At the end of the Santiago’s talk, there are couple of raised which sounds interesting to Santiago. There was a question from Mitch about database integration with selenium tests which was interesting. I asked question about flaky tests that fails in the cloud complaining “element not found” but works well in local machine. As per Santiago, it’s bit tricky to judge random failures in browsers in the cloud.

Using Selenium to Improve a Team’s Development Cycle

After the break, I headed over to lecture hall to hear Mike Devis talking about “Using Selenium to Improve a Team’s Development Cycle“. Mike shared some cool tips about improving quality of the functional tests. Mike also talked about involving developers to write tests. I really like the idea of profiling the tests. Ultimately, it was very pleasant talk with lots of good tips

Geb: Very Groovy Browser Automation

This is one of the best talk from my point of view. Thanks to Luke Daley. This talk was about Geb. Geb is a browser automation framework which combines cool features of WebDriver, jQuery and the Page Objects. Most interesting thing, I found in this talk is $() method. The dollar method can be used to interact with the web pages and accessing web elements. Another interesting thing is integration of Geb with different frameworks like JUnit, TestNG, Spock, Cucumber. At the end of the talk, I asked question on integration of Geb with new Cucumber JVM. Once Luke finished his talk, we went to reception to have a lunch.

Robots: If You Can’t Beat ‘Em, Build ‘Em

Well, This was most interesting demo to watch that how Jason “Teach Robot to play Angry Bird“. Click the link and watch the video inside the presentation, you will be amazed by seeing that how it can be achieved with Selenium. I think, only God and Jason knows how to write code for this. I just enjoyed the demo and applauded at the end of talk.

Building Quality with Foundations of Mud

After Jason’s talk, Kristian Rosenvold was talking about ‘Building Quality’. Kristian explained how to build test environment, test data and Continuous Integration practices in order to build quality products. He again explained some cool tips to handle flaky tests and errors. His slides are explanatory. Kristian made one statement who liked by selenium contributors. It was “Make your dev team live and Breathe the build” At the end of the presentation, I asked question on Tolerance rate of their functional tests. Kristian answered that they can tolerate failing integration tests but not the functional tests. Well, then it was time for quick break then.

Massively Continuous Integration

Continuous Integration is area of my interest so I didn’t want to miss talk on my favorite topic. I headed over to the Track B where Jesse Dowdle & David Tolley were talking on their huge continuous integration system. As per their slides, they have reduced cycle time of running tests from 3 days to 30 minutes. The amount of selenium tests and cycle time to run them over Grid literally amused me. They used Jenkins and amazon web for running the tests. I would rather recommend looking at the slides of the presentation. It was amazing talk.I would like to see demo how they run massive test suite within 30 mins. Thanks to speakers & Jenkins.

How to Test the Inside of Your Head

Speaker Liz Keogh is vary famous agile/BDD coach and international speaker. I was impressed by her talks on BDD at Skills Matter. No wonder her talk at Selenium Conference was one the best. Liz talked about collaboration between developers, testers, BA, project managers and customers. As per Liz “Testers are Evil so let them think before writing code“. Most of people on twitter channel were saying it was brilliant talk.

Well, how can I forget that there was amazing party sponsored by FaceBook at the GlassHouse Store, London. I met with people all around world and had great chat with them. Food and drinks were amazing. Great opportunity to celebrate birthday with unlimited drinks and food. Thanks Facebook.

WEDNESDAY, APRIL 18

Last day of the selenium conference was full of interesting talks, lighting talks and end up with closing keynote session. At the start of the day ‘ Simon Stewart’ kicked off with his talk on status of Selenium projects since last selenium conference

State of the Union

Simon didn’t forget to mention that he is on twitter (@shs96c) at the start of his talk. Simon highlighted recent enhancements of the Selenium project. He also point out an year in a browser and new selenium committers. The most interesting part of his presentation was his analysis of job trends in year, the graph amused everyone.

Simon addressed future development will be on mobile drivers as they are being valuable in recent days. He also expecting new stuff with respect to W3C standard. Let’s see how it will go on in near future. Slide of his talks are available here.

After Simon’s talk I headed over to Track B where “TFT Test Automation Framework” talk was going on. It was all about maintaining and writing automated tests just by using spreadsheets. I really missed “Introducing Selenium Builder – the Future of Test Development” talk by Adam Christian and David Stark. Hope to see videos of the talk soon.

Selenium in the Enterprise: What Went Right, What Went Wrong (So Far)

Noah Sussman was talking about how they used selenium at Etsy. This talk was more about test maintenance and test infrastructure at Etsy. Being a big fan of Behat, I came across one the slide saying “Etsy used Behat & Mink for functional testing”. I was keen to know more about that and asked question on how they used behat in their projects also had a chat after talk with Noah.

Automated Security Testing

Alan Parkinson was talking about ‘Zed Attack Proxy (ZAP) Web-Driver’ for automated security testing. Alan introduced ZAP during his talk and then integrated with build system like ANT. Alan’s demo of ZAP and Jenkins was amazing. I couldn’t resist myself to tweet about that. It’s great approach to perform automated security check on build. I will definitely give a try.

Pluggable Test Infrastructure

As far I remember, I have seen video of last selenium conference Dima talking about Selenium and Cucumber. This year Dima and Jeff comes up with Pluggable Test Infrastructure. They are using Cucumber, Capybara and webdriver at Groupon. Jeff said that using Cucumber is debatable. Jeff & Dima combined API tests, iPhone tests, Sauce tests and Selenium tests with Cucumber. It would be interesting to see demonstration of how they achieved this.

Speed up Your Selenium Tests with PhantomJS, the Headless Browser

Ivan De Marino was then talking about PhantomJS in order to make selenium tests fast. In his talk he showed how you could run selenium tests with PhantomJS and Ghostdriver. Ghostdriver is wire protocol on PhantomJS. One of the tweet was saying Ghostdriver has awesome potential but needs your contribution. If you wish to contribute then fork Github repo now.

Robot Framework

Once Ivan finished his talk on PhantomJS, I have decided to go for Track B where Philip Noetzel was talking about “Robot Framework: Semicolon free testing”. I heard a lot about Robot Framework but didn’t got chance to look into it. This talk was very beneficial for me as Philip shows some cool stuff about robot framework and also shown sample code. Philip also shared his thoughts about pros and cons of using Robot framework. At the end of the presentation we asked Philip if we can use Robot framework for PHP applications.

More Than Automation – How Good Acceptance Tests Can Make Your Team Happier

I was keen to hear Matt Wynne, author of Cucumber book talking about ‘More Than Automation‘. Matt introduced Mortgage Driven Development (MDD) where developers try to make feature completely readable and unmaintainable in order to save your job!. Matt also added concept of “Refuctoring” where developers try to add brittle steps and scenarios in order to make it messy. Well, Matt also said ” Cucumber is a threat” but then audience understood whats intention behind all this idea. As per Matt, Cucumber is cool take which allow everyone to collaborate and write tests. Matt also suggest to read Cucumber book in order to get yourself familiar with Behavior Driven Development (BDD). Matt shared cool tool for living documentation called ‘Relish’. At the end of the talk, there were couple of question around Relish. one of them was asked by me.

Lightning Talks

There were about 10 lightning talks (including mine) scheduled on the board and each one of 5 minutes. Lightning talks was fun. I liked Anand and Komal talked about how to do things wrong. It was nice talk. I am keenly waiting to see videos of Lightning talks.

Closing Keynotes

After the lighting talks there were another session called ‘Closing Keynotes’ where all the selenium committers discuss about the future enhancements and answered questions. There was one interesting question comes up in this session about Selenium 3. Conference was then wrapped up by thanking everyone who attended the conference.

SELENIUM CONFERENCE LIVE WITH #SECONF

I bet, everyone who attended conference had loads of fun and learned new things. Those were was not able to attend the conference, they got live conference updates from twitter hashtag #SeConf. I got a tweet from Michelle Flynn (@MichelleFlynn) saying “Great twitter chat about #SeConf, love how you can follow a conference even if you cannot attend“

Many people all over the world was getting updates on selenium conference every minute. Thanks to organizers for creating such a unique hashtag also thanks to all twitter users who twitted in #SeConf

Interesting things I found.
- Mortgage Driven Development
- Mobile Drivers
- Use of BDD tools with Selenium like Cucumber, SpecFlow, Behat
- Geb: Very groovy browser automation framework
- ‘Atkins’ plug-in for Jenkins
- Robot Framework for BDD and DDD
- Large scale Continuous Integration with Jenkins and Amazon cloud services
CONCLUSION

Selenium conference taught me lot of new things around Selenium also given an opportunity to meet with Selenium users all across the globe which was amazing experience. Great speakers, great learning experience and amazing location. Thanks SauceLabs, Google and all other sponsors. Looking forward next year !

[Shashikant Jagtap]PHPUnit + Behat/Mink + Page Object: The Rockstar Combination of Testing April 11, 2012

Last month, we had discussion about implementing page object pattern in Behat/Mink framework at London Behat Users meetup . Page object pattern is a cool way to make tests maintainable, reusable and readable. Everyone was interested to know more about Page Object Pattern.

In this short tutorial, we will implement Mink and PHPUnit combination for functional testing. Mink and PHPUnit combined with Pageness (Page Object) can be used for maintainable and readable tests. Mink is a web acceptance testing framework for PHP application. Mink has clean API’s which allows browsers emulation and browser controls. Mink has headless browser emulators as well as browser controls which covers all browsers interactions. PHPUnit is testing framework for PHP applications.

Installations

Lets start with some installations assuming PHP, Pear package is already installed.

Install PHPUnit

Now, upgrade pear package & install PHPUnit

$ pear update-channels
$ sudo pear channel-discover pear.phpunit.de
$ sudo pear channel-discover pear.symfony-project.com
$ pear channel-discover components.ez.no
$ sudo pear install –alldeps phpunit/PHPUnit
$ phpunit --version

Install Behat/Mink

$ pear channel-discover pear.symfony.com
$ pear channel-discover pear.behat.org
$ pear install behat/mink

Mink is ready to use just including in PHP classes.

require_once 'mink/autoload.php';

PHPUnit-Mink Framework on GitHub

PHPUnit-Mink framework designed to use combination of PHPUnit, Mink and Page Object Pattern to write functional tests with various browser emulators like Goutte, Selenium, Sahi and WebDriver. PHPUnit-Mink Framework has used Mink and PHPUnit to write tests. Driver support for Selenium, Sahi, WebDriver for browser emulation. Test Report Generation which can plugged in Continuous Integration server. Page Objects which can be used directly in tests. Abstracted common elements up in the framework in order to make it maintainable.

How to use:

Clone GitHub Repository

$git clone git@github.com:Shashi-ibuildings/PHPUnit-Mink.git
$cd PHPUnit-Mink

Start your Driver

Sahi Driver :

Download sahi zip file from SourceForge

Now launch Sahi Server using command below:

$ cd /path/to/sahi
$ cd userdata/bin
$./start_sahi.sh

Selenium Driver:

You need to download selenium server jar file and execute following command:

$ cd /path/to/selenium-server
$java -jar selenium-server-standalone-2.20.0.jar

In this tutorial, we are using sahi driver.

Now run tests using ANT

$cd /path/to/PHPUnit-Mink
$ant Mink

Directory structure

conf : YAML files can be used with Behat
core : Abstracted common elements/ methods
Page : Page objects (reusable Methods for page)
report : Generate Junit, Agile doc reports
tests : PHPUnit tests using Mink Api’s

Don’t forget to include Mink and PHPUnit in Test like this:

<?php
require_once 'mink/autoload.php';
use Behat\Mink\Mink,
Behat\Mink\PHPUnit\TestCase;

Page Objects Pattern

There are some areas within application UI where your test interacts with. These areas can be identified by their functionality, they offer e.g login, registration, search etc. Page object pattern models as a object within you test code. While writing tests, we just need to create an object of the page and access methods and variables withing that page objects. This approach is useful when UI changes, we need to make changes in page objects not in the tests. Tests become readable and maintainable using page objects.

Page objects classes generally:

Set of public methods that page offer which can be reused later.
don’t make any assertion. Assertions can be added later into tests not in the page objects
checks if user is on the same page that page object is created.
don’t represent entire page but some important functions of that particular page.

GitHub & Other sources of Page object pattern

Read more about page object on google wiki. There are couple of GitHub repositories that explains page object pattern, Saunter and pelenium

Writing Page Objects:

Common methods can be abstracted in the ‘page’ directory. You can specify driver of your choice. Now we have test with Sahi session. Example page looks like this

<?php
require_once 'mink/autoload.php';
use Behat\Mink\Mink,
Behat\Mink\PHPUnit\TestCase;
require_once 'core/core_PHPUnitMink_CommonElementFunctions.php';
class page_search extends TestCase
{
function search($input)
{
$this->getSession('sahi')->getPage()->fillField("searchInput",$input);
$this->getMink()->getSession('sahi')->getDriver()->click("//*[@id='searchButton']");
$this->getMink()->getSession('sahi')->wait("3000");
}
}
?

Using Page Objects in Tests

You can use pages just by creating new objects and accessing variables,methods in the test. Example of test looks like this

<?php
class wikiSearchTest extends core_PHPUnitMink_CommonElementFunctions
{
protected static $mink;
public function testwikiSearchsingle()
{
$this->getSession('sahi')
->visit('http://en.wikipedia.org/wiki/Main_Page');
$search = new page_search($this);
$search->search("mumbai");
assertEquals("Mumbai", $this->getMink()->getSession('sahi')->getDriver()->getText(".//*[@id='firstHeading']/span"));
}

Writing Data-Driven Tests

You can write data driven tests by using PHPUnit and Mink combination. Data can be placed in CSV or in an array. Example data-driven test looks like this

<?php
class wikiSearchTest extends core_PHPUnitMink_CommonElementFunctions
{
protected static $mink;
function searchData() {
return array(
array('london','London'),
array('newyork','New York')
);
}
/**
*
* @param &lt;type&gt; $searchfor
* @param &lt;type&gt; $expectedResult
* @dataProvider searchData
*/

public function testwikiSearchMultiple($input,$output)
{
$this->getSession('sahi')
->visit('http://en.wikipedia.org/wiki/Main_Page');
$this->getSession('sahi')->getPage()->fillField("searchInput",$input);
$this->getMink()->getSession('sahi')->getDriver()->click("//*[@id='searchButton']");
$this->getMink()->getSession('sahi')->wait("3000");
assertEquals($output, $this->getMink()->getSession('sahi')->getDriver()->getText(".//*[@id='firstHeading']/span"));
}

}

Starting Engines

Before running tests, you need to start Mink drivers, here we are using Sahi. You can update tests to run Selenium driver. Launch driver from command line like this:

Sahi : Navigate to Sahi directory and run:

moonstar:~ sjagtap$ cd sahi/userdata/bin/
moonstar:bin sjagtap$ ./start_sahi.sh
--------
SAHI_HOME: ../..
SAHI_USERDATA_DIR: ../../userdata
SAHI_EXT_CLASS_PATH:
--------
Sahi properties file = /Users/sjagtap/sahi/config/sahi.properties
Sahi user properties file = /Users/sjagtap/sahi/userdata/config/userdata.properties
Added shutdown hook.
&gt;&gt;&gt;&gt; Sahi started. Listening on port: 9999
&gt;&gt;&gt;&gt; Configure your browser to use this server and port as its proxy
&gt;&gt;&gt;&gt; Browse any page and CTRL-ALT-DblClick on the page to bring up the Sahi Controller
-----
Reading browser types from: /Users/sjagtap/sahi/userdata/config/browser_types.xml
-----

Selenium

moonstar:downloads sjagtap$ java -jar selenium-server-standalone-2.20.0.jar
Apr 7, 2012 3:34:19 PM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
1 [main] INFO org.openqa.selenium.server.SeleniumServer - Java: Apple Inc. 20.1-b02-383
1 [main] INFO org.openqa.selenium.server.SeleniumServer - OS: Mac OS X 10.7.1 x86_64
8 [main] INFO org.openqa.selenium.server.SeleniumServer - v2.20.0, with Core v2.20.0. Built from revision 16008
131 [main] INFO org.openqa.selenium.server.SeleniumServer - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
132 [main] INFO org.openqa.jetty.http.HttpServer - Version Jetty/5.1.x
132 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
133 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/selenium-server,/selenium-server]
133 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/,/]
273 [main] INFO org.openqa.jetty.util.Container - Started org.openqa.jetty.jetty.servlet.ServletHandler@62f47396
273 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/wd,/wd]
279 [main] INFO org.openqa.jetty.http.SocketListener - Started SocketListener on 0.0.0.0:4444
279 [main] INFO org.openqa.jetty.util.Container - Started org.openqa.jetty.jetty.Server@4a79717e

Running Tests with PHPUnit

Now you can run wikisearch test with PHPUnit like this :

moonstar:PHPUnit-Mink sjagtap$ phpunit tests/wikiSearchTest.php

Temporary SauceLabs fork 3.5.24 of PHPUnit by Sebastian Bergmann. Supports parallel testing. Becomes obsolete when PHPUnit 3.7.0 is released.

...

Time: 33 seconds, Memory: 11.25Mb

OK (3 tests, 3 assertions)

Running Tests with Ant

You can run tests using apache ANT. Navigate to project root directory (PHPUnit-Mink) and run an

moonstar:PHPUnit-Mink sjagtap$ ant Mink
Buildfile: /Users/sjagtap/PHPUnit-Mink/build.xml
[delete] Deleting directory /Users/sjagtap/PHPUnit-Mink/report
[mkdir] Created dir: /Users/sjagtap/PHPUnit-Mink/report

Mink:
[exec]
[exec]
[exec]
[exec]
[exec] Temporary SauceLabs fork 3.5.24 of PHPUnit by Sebastian Bergmann. Supports parallel testing. Becomes obsolete when PHPUnit 3.7.0 is released.
[exec]
[exec] ....
[exec]
[exec] Time: 45 seconds, Memory: 11.75Mb
[exec]
OK (4 tests, 4 assertions)

BUILD SUCCESSFUL
Total time: 46 seconds
moonstar:PHPUnit-Mink sjagtap$

Once all tests finished running, you will see build successful message. This build can be easily plugged into Continuous Integration server like Jenkins.

Conclusion:

Functional testing become much easier with use of Mink with PHPUnit. Page Object pattern can be used with PHPUnit-Mink to make tests reusable and readable.

You can find source code on GitHub PHPUnit-Mink repository. Happy PHP Testing !

[Devis Lucato]SHA-3 Candidate Conference - 22-23 March 2012 March 18, 2012

NIST hash function competition started in 2007 with "Announcing Request for Candidate Algorithm Nominations for a New Cryptographic Hash Algorithm (SHA–3) Family":

AGENCY: National Institute of Standards and Technology, Commerce.

ACTION: Notice and request for nominations for candidate hash algorithms.

SUMMARY: This notice solicits nominations from any interested party for candidate algorithms to be considered for SHA–3, and specifies how to submit a nomination package. It presents the nomination requirements and the minimum acceptability requirements of a "complete and proper" candidate algorithm submission. The evaluation criteria that will be used to appraise the candidate algorithms are also described.

a quest for a new hashing algorithm, less prone to collisions (with respects to MD4/MD5 and SHA-0/SHA-1), even though SHA-2 still resists to this kind of attack.

64 entries received by October 31st 2008, 51 went through to the first round and 14 to the second round in 2009.

Five finalists BLAKE, Grøstl, JH, Keccak, and Skein advanced to the third and final round and the final package was submitted last year in January (2011).

Now, a final SHA-3 Candidate Conference is scheduled for 22-23 March 2012 and will be hel at the Washington Marriott Hotel in Washington DC, where the algorithms will be discussed and public invited to provide further fedback before a winner is selected this year.

read more
[Mark van der Velden]Logging best practices March 4, 2012
Introduction

This document explains some best practices for application error/debug logging. Application logging is an important "tool" to help identifying problems. If done correctly, and if an infrastructure is in place that allows events to be put into context, it's easier to debug situations and thus makes it easier to solve problems sooner.

While out of scope for this document, I'd like to point out the following tools that can make that possible:
- Logstash - http://logstash.net
- Graylog2 - http://graylog2.org
Different logging meanings

Logging can mean different things and logging can have different purposes. The logging this document refers to is application and semantic logging.

Application logging (also called "debug" or "error" logging) is intended to keep track of the "low level" application behaviour. Application logging is used to log events, that allows for debugging a problem within an application or to monitor application health. In contrast, a different form of logging could for example be Google Analytics. Tracking (or "logging" actions of) visitors of a website, this type of logging is very different and not something this document refers too.

Why do we log

In order to understand how to log, it must first be clear what we try to accomplish with application logging.

Continue reading "Logging best practices"

[Shashikant Jagtap]Behat Meetup in a nutshell February 17, 2012

On Feb 15, we had a great evening at Ibuildings, London office. We organized the first ever Behat meetup in the United Kingdom. A group called “London Behat Users” organized an event to demonstrate Behavior Driven Development for the PHP developers and testers who wish to get started with Behat.

Behat is an amazing BDD framework for PHP developers which works exactly the same as Cucumber. Assuming that PHP developers and testers are new to BDD, we made the agenda ‘Behat from Scratch‘. There were about 15 behat users who attended this event.

As per the schedule, we had great Pizza before kick-off. While having drinks and Pizza, An Event organizer, Mitch asked everyone to introduced themselves and asked about their experience with Behat. All members were keen to know about other behat users and how long they have been using behat and BDD. After introductions, I explained a little bit about the ‘London Behat users’ group.

Live Coding (Developers View)

Jakub Zalas kicked it off with a talk about the developer’s view in BDD projects. He started the demonstration with a short introduction to BDD, how BDD is different from TDD and so on. Jakub covered following topics while coding:

1. Behat/Mink Installation

Jakub created composer.json file in the project root:

Then he downloaded composer.phar and ran install command which installed all the dependencies of Behat as well as Mink.

2. Creating first project

Once he was done with all the installations, he created a project called “Behat From Scratch” and initialized behat using:

$ behat --init
+d features - place your *.feature files here
+d features/bootstrap - place bootstrap scripts and static files here
+f features/bootstrap/FeatureContext.php - place your feature related code here

The theme of the application was very simple. There was a form with two fields called ‘Title” and ‘Body” and a button called “Preview.” When a user fills in data in the ‘Title’ & ‘Body” fields and clicked on the “Preview” button, then the user should see the text that they entered.

3. Writing first Gherkin feature

Next he wrote one Gherkin feature file with couple of scenarios:

Feature: Previewing article
  As a Content Manager
  I want to preview an article before saving it
  In order to make sure it is displayed correctly

Scenario: Article is displayed in a preview area
    Given I visit "article form"
      And I fill in "Hello Behat!" for "Title"
      And I fill in "BDD is fun" for "Body"
     When I press "Preview"
     Then I should see "Hello Behat!" in the preview area
      And I should see "BDD is fun" in the preview area

Scenario: Preview is not visible initially
    Given I visit "article form"
     Then the preview area should not be visible

After that, he implemented Step Definitions for the integrated classes based on the Silex micro-framework, uses Twig templates and Symfony‘s form component. He implemented all the step definitions until all the steps passed.

The Source code of this is available on GitHub: Behat From Scratch

Browser Automation with Mink (Testers View)

Once Jakub finished with implementation, he handed it over to me to explain how to do browser automation using Mink. I modified the ‘config/behat.yml’ file and tagged the scenarios with ‘@javascript’ tags.

I also explained how to run feature files using different Mink drivers. Behat has provided lot of driver options for testers. It is not limited to Selenium. It has following mink drivers.

Selenium Server

You need to download the selenium server jar file and execute the following command:

      $ cd /path/to/selenium-server

      $java -jar selenium-server-standalone-2.15.0.jar

Sahi Server

Download the sahi zip file from SourceForge

Now launch Sahi Server using the command below:

      $ cd /path/to/sahi

      $ cd userdata/bin

      $bash start_sahi.sh

There was a quick demonstration of the tests running in browsers using the Selenium 1, 2 (WebDriver), and Sahi servers. This was followed by a short question and answer session.

Try this on your own

You may be interested in trying all this stuff on your own. Jakub mentioned all of the instructions in his GitHub project. I will briefly explain the steps to follow:

Clone the repository “Bahat From Scratch“

      $ moonstar:~ sjagtap$ git clone git://github.com/jakzal/behat-from-scratch.git
      Cloning into behat-from-scratch...
      remote: Counting objects: 123, done.
      remote: Compressing objects: 100% (59/59), done.
      remote: Total 123 (delta 47), reused 123 (delta 47)
      Receiving objects: 100% (123/123), 63.34 KiB, done.
      Resolving deltas: 100% (47/47), done.
      moonstar:~ sjagtap$ cd behat-from-scratch/

Install Behat with Composer
Download the Composer and install Behat:

      $ moonstar:behat-from-scratch sjagtap$ wget  http://getcomposer.org/composer.phar
       % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
       100  308k  100  308k    0     0   366k      0 --:--:-- --:--:-- --:--:--  582k

      $moonstar:behat-from-scratch sjagtap$ php composer.phar install

Here, you may face some problems while installing, depending on your network and permission issues. (There may be a bug in composer.)

I managed to install it properly and it’s up and running in my localhost. You need to cross your fingers at this point.
By default the custom Silex micro-framework session is used so there’s no need for additional configuration. Behat will create a Silex application and simulate requests.

All the project’s scenarios can be run with:

  $ moonstar:behat-from-scratch sjagtap./bin/behat
  As a Content Manager
  I want to preview an article before saving it
  In order to make sure it is displayed correctly

  Scenario: Article form is accessible from the homepage # features/previewing-article.feature:7
    Given I visit "homepage"                             # FeatureContext::iVisit()
    When I follow "Add article"                          # FeatureContext::clickLink()
    Then I should see "Title" input on the article form  # FeatureContext::iShouldSeeInputOnTheArticleForm()
    And I should see "Body" textarea on the article form # FeatureContext::iShouldSeeTextareaOnTheArticleForm()

  Scenario: Article is displayed in a preview area       # features/previewing-article.feature:14
    Given I visit "article form"                         # FeatureContext::iVisit()
    And I fill in "Hello Behat!" for "Title"             # FeatureContext::fillField()
    And I fill in "BDD is fun" for "Body"                # FeatureContext::fillField()
    When I press "Preview"                               # FeatureContext::pressButton()
    Then I should see "Hello Behat!" in the preview area # FeatureContext::iShouldSeeInPreviewArea()
    And I should see "BDD is fun" in the preview area    # FeatureContext::iShouldSeeInPreviewArea()

  Scenario: Preview is not visible initially             # features/previewing-article.feature:23
    Given I visit "article form"                         # FeatureContext::iVisit()
    Then the preview area should not be visible          # FeatureContext::thePreviewAreaShouldNotBeVisible()

  Scenario: Title and body are required                  # features/previewing-article.feature:28
    Given I visit "article form"                         # FeatureContext::iVisit()
    When I press "Preview"                               # FeatureContext::pressButton()
    Then I should see "Article title is required"        # FeatureContext::assertPageContainsText()
    And I should see "Article body is required"          # FeatureContext::assertPageContainsText()
    And the preview area should not be visible           # FeatureContext::thePreviewAreaShouldNotBeVisible()

4 scenarios (4 passed)
17 steps (17 passed)
0m12.957s

You don’t need to configure the web server to run scenarios using Silex driver. But you do need to configure the web server while running tests in browser.

Configure Web Server.

Follow these steps to configure the web server.

Update ‘hosts’ file and add ‘behat.dev’ and IP as follows

     $moonstar:~ sjagtap$ cd /etc
     $moonstar:etc sjagtap$ sudo vi hosts

     # Host Database
     #
     # localhost is used to configure the loopback interface
     # when the system is booting.  Do not change this entry.
      ##
    127.0.0.1       localhost
    127.0.0.1  behat.dev

Make sure you have updated ‘/etc/apache2/httpd.conf’ to enable virtual host. Remove the comment from Virtual hosts.

     $moonstar:~ sjagtap$ cd /etc/apache2/
     $moonstar:apache2 sjagtap$ cat httpd.conf

     # Virtual hosts
     Include /private/etc/apache2/extra/httpd-vhosts.conf

Now you need to update the ‘/etc/apache2/extra/httpd-vhosts.conf ‘ file and add the following configuration to it:

     $mmoonstar:~ sjagtap$ cd /etc/apache2/extra/
     $moonstar:extra sjagtap$ sudo vi httpd-vhosts.conf
     <VirtualHost *:80>
     ServerName behat.dev

     DocumentRoot /var/www/behat.dev/web
     DirectoryIndex index.php

     <Directory /var/www/behat.dev/web>
        Options FollowSymLinks
        AllowOverride All
        Order allow,deny
        allow from all
     </Directory>
     </VirtualHost>

You are done with web server configuration, and now you can run your tests through a browser. The Paths to source code and a domain name will have to be updated. Also the base_url in ‘config/behat.yml’ needs to be changed accordingly.

Run Tests in Browser

Tag all your scenarios with ‘@javascript’ tags
Run Selenium server as mentioned above

      $ cd /path/to/selenium-server

      $java -jar selenium-server-standalone-2.15.0.jar

Update ‘config/behat.yml’

      default:
         context:
            parameters:
               default_session: goutte
               javascript_session: webdriver
               browser: firefox
               base_url: http://behat.dev/

Run ‘./bin/behat’ from a terminal and watch results in browser.

$ moonstar:behat-from-scratch sjagtap./bin/behat
Feature: Previewing article
  As a Content Manager
  I want to preview an article before saving it
  In order to make sure it is displayed correctly

  @javascript
  Scenario: Article form is accessible from the homepage # features/previewing-article.feature:7
    Given I visit "homepage"                             # FeatureContext::iVisit()
    When I follow "Add article"                          # FeatureContext::clickLink()
    Then I should see "Title" input on the article form  # FeatureContext::iShouldSeeInputOnTheArticleForm()
    And I should see "Body" textarea on the article form # FeatureContext::iShouldSeeTextareaOnTheArticleForm()

  @javascript
  Scenario: Article is displayed in a preview area       # features/previewing-article.feature:14
    Given I visit "article form"                         # FeatureContext::iVisit()
    And I fill in "Hello Behat!" for "Title"             # FeatureContext::fillField()
    And I fill in "BDD is fun" for "Body"                # FeatureContext::fillField()
    When I press "Preview"                               # FeatureContext::pressButton()
    Then I should see "Hello Behat!" in the preview area # FeatureContext::iShouldSeeInPreviewArea()
    And I should see "BDD is fun" in the preview area    # FeatureContext::iShouldSeeInPreviewArea()

  @javascript
  Scenario: Preview is not visible initially             # features/previewing-article.feature:23
    Given I visit "article form"                         # FeatureContext::iVisit()
    Then the preview area should not be visible          # FeatureContext::thePreviewAreaShouldNotBeVisible()

  @javascript
  Scenario: Title and body are required                  # features/previewing-article.feature:28
    Given I visit "article form"                         # FeatureContext::iVisit()
    When I press "Preview"                               # FeatureContext::pressButton()
    Then I should see "Article title is required"        # FeatureContext::assertPageContainsText()
    And I should see "Article body is required"          # FeatureContext::assertPageContainsText()
    And the preview area should not be visible           # FeatureContext::thePreviewAreaShouldNotBeVisible()

4 scenarios (4 passed)
17 steps (17 passed)
0m12.957s

It’s done !!

Discussion about Behat and BDD

After we finished with the demo, we started a discussion about BDD and behat. To me, this was the most interesting part of the meetup. Lot of things about BDD, tester-developer collaboration, acceptance criteria, Jenkins and other CI systems were discussed.

Some of the highlights of the discussion are as follows:

Ben and Jakub Zalas shared thoughts about test reporting with Jenkins. (using HTML reports).
Mitch raised points about making scenarios and running scenarios with different data sets and everybody contributed
Marcello talked about the role of testers and developers in BDD
Miles and Alex raised some good questions about web testing using Gherkin DSL.
John played a key role within the whole session talking about collaborative tools, challenges in BDD and some tricks about BDD.

Conclusion:

The first Behat meetup was an amazing experience for all the attendees. Everybody learned by sharing experiences and ideas. In short, all of us enjoyed our evening of knowledge sharing. We are looking forward to arrange monthly meetups to further discuss Behat and BDD. The next meetup is tentatively scheduled for Wed Mar 21 at 7:00 PM (GMT). Click here for details.

Many Thanks to the creator of Behat: ‘Konstantin Kudryashov‘

Happy BDD !!

[Shashikant Jagtap]Protected: The Agile Tester January 29, 2012

This post is password protected. You must visit the website and enter the password to continue reading.

[Shashikant Jagtap]Web Acceptance Testing Framework for PHP January 13, 2012

Web Acceptance Testing Framework for PHP

A Web acceptance testing framework for PHP is now available on GitHub. It can be used for web acceptance testing of the PHP projects. This framework is build around Behat, Mink, Sauce Labs and ANT. You can configure it with a continuous integration server like Jenkins.

Behat is a BDD framework for PHP. Behat comes up with Mink which is used for browser emulation (functional testing) where browser interaction takes place. Behat and Mink combination is used for web acceptance testing.

Key features of web acceptance testing framework

Selenium and Sahi servers are plugged into build file, user doesn’t need to download or launch Selenium or Sahi server while running features with ANT or Jenkins.
Test reports are generated in HTML and xml format in “/report” directory.
Easy switching over to another driver. You can run tests using any available drivers, just by specifying name of them.
Framework is integrated with cloud testing service called “Sauce Labs”. You can run your features in cloud with selenium driver.
Testers can get started easily without any hassles.
With little changes in config file, it is ready for your project.

Behat for Testers

Testers might have tried Cucumber & Selenium or Cucumber & capybara combination for Ruby projects. BDD for testers could be very interesting topic to discuss. Testers can now use BDD framework for PHP project as well. Thanks to Behat !

Behat has provided lot of driver options for testers. Its not limited to selenium. It has following mink drivers.

Tester can pick one of these and start implementing features. Goutte can be used for tests which doesn’t need browser interaction. Tester can always switch to another driver if feature is failing for one of the driver. There are some pro’s and con’s of each driver

	Sahi	Selenium	Webdriver

Implicit Wait	Yes	No	Yes
Hidden Link Access	Yes	Yes	No
Sauce Lab Support	No	Yes	No
Speed	Medium	Medium	Fast
Https sites testing	Hectic, need to add certificate exception	Accepts certificated automatically	Accepts certificated automatically

It is very important to choose right driver for your testing. It depends on your application and requirement. This framework makes it easy to switch over to another driver.

What you need to use framework?

Install Behat using Behat documentation which explained all these methods in simple way. Similarly install mink using Mink documentation.

Create Your Project

Initiate Behat:

$ behat --init

Clone Web Acceptance testing Framework from GitHub

Clone test framework from Github into your local machine.
Make sure you have initialized Behat in your root directory, currently its “Behat”.
Now you are ready to use the framework for adding features and running them with different drivers like Selenium, Sahi or webdriver.

How to use framework?

Change “base_url” “browser” parameter in all config files like ‘behat.yml’, ‘sahi.yml’, ‘selenium.yml’, and ‘webdriver.yml’ as per your project requirement. eg change in sahi.yml file as shown below

default:
    context:
        parameters:
            javascript_session: sahi
            base_url: http://{your url}
            browser: {your browser}
            show_cmd: open %s

Run features locally

To run feature locally, you need to start Selenium or Sahi server before executing them.

Download Selenium or Sahi and launch Selenium or Sahi server as shown below

Selenium Server

You need to download selenium server jar file and execute following command:

      $ cd /path/to/selenium-server

      $java -jar selenium-server-standalone-2.15.0.jar

Sahi Server

Download sahi zip file from SourceForge

Now launch Sahi Server using command below:

      $ cd /path/to/sahi

      $ cd userdata/bin

      $bash start_sahi.sh

Now you can run behat command to run feature locally

      $ cd /path/to/behat

      $behat --name wikiSearch (your feature)

You can see feature running on your local host.

How to run features using ANT

Requirement
You need to install Apache Ant on your localhost. You can download and install Ant from Apache Ant website.

No need install or download selenium or Sahi. It is configured inside this framework. Ant will automatically launch selenium or Sahi server still you need to update version of selenium server frequently.

Now you got three drivers options to run your features

Sahi Driver
Selenium Driver
Selenium 2 Driver (webdriver)

Run feature with selenium driver

$ cd /path/to/my/project

$ant runSelenium

Run feature with Sahi driver.

$ cd /path/to/my/project

$ant runSahi

Run feature with webdriver

$ cd /path/to/my/project

$ant runWebdriver

Run feature on Sauce Labs

$ cd /path/to/my/project

$ant sauceTests

If you are using selenium driver, then you have to implement wait in your step definitions every time there is new page is loaded like this:

 $this->getMink()->getSession()->wait("3000");

Sahi driver waits automatically for element to be appear on page or page to load. You don’t have to implement wait if you would like to use Sahi driver.

Once you run ‘ant’ command from terminal you will see your feature running in browser, you configured.You will see reports generated in “/report” directory. There will be HTML reports as well.

Configure Jenkins to run features

Download & Launch Jenkins by executing below command:

$  java -jar jenkins.war

Visit url : http://localhost:8080 to access Jenkins in browser. You can specify different port to launch Jenkins.

Create new Build a free-style software project and name it “Behat”
Configure job by specifying your repository.(Git, SVN, CSV)
You need to Invoke Ant as shown below

Jenkins Configuration

If you wish to run tests using Sahi driver then specify target as “runSahi”

You can see reports generated once you run your feature.

Conclusion

Using web acceptance test framework for PHP, you can write features and run them easily on ANT, Jenkins and Sauce labs with choice of available drivers.

Happy BDD for testers !! Any questions or comment mention me @Shashikant86 on Twitter.

[Shashikant Jagtap]Behat with Sauce Labs and Jenkins January 13, 2012

Abstract:

The key benefits of Behavior-Driven-Development (BDD) practices are communication enhancement and customer satisfaction. You can read more about that by Dan North and Gojko Adzic. Perhaps the biggest practical challenge in the way of reaping those benefits is the burden of provisioning, installation and maintenance of requisite complex and fussy infrastructure. The recent availability of latest CI servers like Jenkins & cloud based testing services like Sauce Labs carries the potential to remove that barrier. This post discusses and shows how to integrate Behat an emerging BDD framework for PHP with continuous integration server like Jenkins and cloud based testing services like Sauce Labs.

What is Behat?

Behat is a BDD framework for PHP. There are some tools available for BDD like Cucumber for Ruby, SpecFlow for .NET and Lettuce for Python. Behat is the first BDD tool for PHP applications. Developers can also use the PHPSpec framework to implement classes within the Behat projects. Behat is written in PHP by Konstantin Kudryashov. With Behat, you can write human readable stories which turns as tests to run against your application. Behat can be used for API testing, functional testing and data-driven testing. Developers will do API testing and we will carry on with functional testing (web acceptance testing) with Behat.

Functional Testing with Behat and Mink

Behat is used for acceptance testing (any tests) by executing a Gherkin scenario. Developers can implement integrated classes. Testers start thinking of more workflow level and technical level steps (actions) which turns as scenarios for features. Once tester started to think of Web Acceptance Testing (functional testing) with browser interaction then another tool called “Mink” comes into the picture.

Mink is used for browser emulation (functional testing) where browser interaction takes place. As of now, there are following Selenium drivers available for browser emulation.

Selenium 1 Driver provides bridge for Selenium RC (Selenium 1).
Selenium 2 driver provides bridge for Selenium 2. (Facebook webdriver) for PHP.

Behat In Action:

You must have pear installed in order to proceed with Behat installation. Now we will run these commands from your terminal window:

$ sudo pear channel-discover pear.behat.org
$ sudo pear channel-discover pear.symfony.com
$ sudo pear install behat/gherkin-beta
$ sudo pear install behat/behat-beta

Test your installation by running this command:

$ behat --version
Behat version 2.2.0

Now let’s install Mink, run following commands from terminal window:

$ pear channel-discover pear.symfony.com
$ pear channel-discover pear.behat.org
$ pear install behat/mink

Mink is ready to use. We have to include “mink/autoload.php” in your classes as shown below:

require_once 'mink/autoload.php';

Start Your Project

Navigate to project root directory and initialize Behat by running these commands:

$ cd /path/to/my/project

$ ls
application

$ behat --init
+d features - place your *.feature files here
+d features/bootstrap - place bootstrap scripts and static files here
+f features/bootstrap/FeatureContext.php - place your feature related code here

$ ls
application	features	

$cd features

$ ls
bootstrap

$cd bootstrap

$ls
FeatureContext.php

This will create “features” directory and “bootstrap/FeatureContext.php” for you. Now we will directly jump to the project created with a feature file.

You can use NetBeans with installed Cucumber plugin for Gherkin syntax highlighting. Project structure will look like this:

Directory Structure

Directory Structure

Behat has already created “features” directory and “features/bootstrap” directory with “FeatureContext.php” in it.

bootstrap.php

We can this create file in define some constants and some third-party libraries which needs to be included in class files. This file should look like this:

<?php
date_default_timezone_set('Europe/London');
 require_once 'mink/autoload.php';
 require_once 'PHPUnit/Autoload.php';
 require_once 'PHPUnit/Framework/Assert/Functions.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase/SauceOnDemandTestCase.php';

behat.yml

This file is a default config file that Behat uses to execute features. Example of behat.yml is shown below:

default:
    context:
        parameters:
            javascript_session: selenium
            base_url: http://en.wikipedia.org/wiki/Main_Page
            browser: firefox
            show_cmd: open %s

You can change the drivers by changing the “javascript_session” parameter. It can be “selenium” or “webdriver”.

sauce.yml

This file is used for running features on Sauce Labs. The code for this file is explained in “behat and Sauce Labs” section below.

build.xml

This file is used for running features with ANT. We can use this ANT file to plug into Jenkins. Simple ANT file should look like this:

<project name="behat" default= "behat" basedir="">
   <delete dir="${basedir}/report" />
   <mkdir dir="${basedir}/report"/>
   <target name="behat">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-f junit --out ${basedir}/report"/>
    </exec>
   </target>
   <target name="create-test-report"  
          description="Generate reports for executed JUnit tests.">  
       <junitreport todir="./report">  
          <fileset dir="${basedir}/report">  
             <include name="TEST-*.xml"/>  
          </fileset>  
          <report format="frames" todir="./report/html"/>  
       </junitreport>  
   </target>  
</project>

report

This directory is used to store *.xml reports generated by Behat’s “junit” formatter. We can use this reports to plug into Jenkins.

Start your Engine

Remember, you have to download latest version of selenium server. Now navigate to directory where you saved selenium server .jar file. You have to launch it using command shown below:

java -jar selenium-server-standalone-2.15.0.jar

Behat & Sauce Labs

Cloud Testing

Sauce Labs is a cloud testing service which allows selenium tests to run in the cloud. Sauce Labs allocates machines and browsers for your tests, capture screenshot for every step and record video of all jobs(tests). You don’t need to setup separate machines to run tests. Sauce labs helps us to write tests without complex infrastructure.

In order to integrate Behat with Sauce Labs, you need to have an account with Sauce Labs. You need “Username” and “API Key” to plug them into a config file.

Behat executes features with “behat.yml” file by default, but we can run features with any other configuration file. We can create another configuration file like “sauce.yml” to run features on Sauce Labs. Example “sauce.yml” should look like this:

default:
  context:
    parameters:
      default_session:    goutte
      javascript_session: selenium
      base_url:           http://en.wikipedia.org/wiki/Main_Page
      browser:            firefox
      selenium:
        host: ondemand.saucelabs.com
        port: 80
        browser: >
          {
            "username":         "your username",
            "access-key":       "your API key",
            "browser":          "firefox",
            "browser-version":  "7",
            "os":               "Windows 2003",
            "name":             "Testing Selenium with Behat"
          }

We will use “sauce.yml” as a config file to run features on Sauce Labs. If you wish to run all features from “features” directory on Sauce labs, you can use this command:

behat -c sauce.yml

When to implement step definitions?

If you can speak fluent Gherkin, then you don’t need to write code. Behat/Mink will understand Gherkin and run your features without suggesting step definitions.
If features written by someone else, you can take full advantage of Mink API’s in order to implement step definitions suggested by Behat/Mink.
It’s very important to write good Gherkin to write minimum code.
It’s very easy to access Mink API’s by writing simple code.

Now we will see how will you “click” particular element on page. You need to use Xpath as a locator for that element. Mink will suggest some step definitions, you need to complete it like this:

/**
* @Given /^I click Something$/
*/
public function iClickSomething()
{
$this->getMink()->getSession()->getDriver()->click("//Xpath");
}

Example : Feature wikiSearch

We will write simple feature to add product into shopping cart. Feature will look like this:

 Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

 @javascript
  Scenario Outline: Search Keywords on Google
    Given I am on "/"
    And I fill in searchBox with "<input>"
    When I press search button
    Then I should see "<output>" 

    Examples:
      | input       | output         |                                   
      | London      | lʌndən/        |
      | NewYork     | nɪu ˈjɔək      |
      | Sydney      | sɪdni/         |
      | Mumbai      | मुंबई            |
      | Bejing      | 北京           |
      | Tokyo       | 東京           |
      | Lahore      | لاہور            |
      | Paris       | paʁi           |

Feature Explained

Gherkin Language

A feature file mentioned above is a good example of data-driven testing. This feature will execute our scenario for 8 different data set mentioned in the example section. This feature will have following steps

User will navigate to Wikipedia main page.
User enter “London” in search box.
User will check if that page has city name in their local language as described in output.
This test will run for 8 different cities as shown in “examples” section of feature.

This feature is also good example of testing internationalization as it consist of test data (examples) in different languages.

Now we will run this feature using command:

behat --name wikiSearch

Remember, we are running it locally for now using default config file “behat.yml” with Selenium driver. After executing above command, we will get some step definitions suggested by Behat/Mink

Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on Wiki # features/wikiSearch.feature:8
    Given I am on "/"                         # FeatureContext::visit()
    And I fill in searchBox with "<input>"
    When I press search button
    Then I should see "<output>"              # FeatureContext::assertPageContainsText()

    Examples:
      | input   | output    |
      | London  | lʌndən/   |
        Undefined step "I fill in searchBox with "London""
        Undefined step "I press search button"
      | NewYork | nɪu ˈjɔək |
        Undefined step "I fill in searchBox with "NewYork""
        Undefined step "I press search button"
      | Sydney  | sɪdni/    |
        Undefined step "I fill in searchBox with "Sydney""
        Undefined step "I press search button"
      | Mumbai  | मुंबई     |
        Undefined step "I fill in searchBox with "Mumbai""
        Undefined step "I press search button"
      | Bejing  | 北京        |
        Undefined step "I fill in searchBox with "Bejing""
        Undefined step "I press search button"
      | Tokyo   | 東京        |
        Undefined step "I fill in searchBox with "Tokyo""
        Undefined step "I press search button"
      | Lahore  | لاہور     |
        Undefined step "I fill in searchBox with "Lahore""
        Undefined step "I press search button"
      | Paris   | paʁi      |
        Undefined step "I fill in searchBox with "Paris""
        Undefined step "I press search button"

8 scenarios (8 undefined)
32 steps (8 passed, 8 skipped, 16 undefined)
0m15.771s

You can implement step definitions for undefined steps with these snippets:

    /**
     * @Given /^I fill in searchBox with "([^"]*)"$/
     */
    public function iFillInSearchboxWith($argument1)
    {
        throw new PendingException();
    }

    /**
     * @When /^I press search button$/
     */
    public function iPressSearchButton()
    {
        throw new PendingException();
    }

You can see above, Behat/Mink has suggested some step definitions for undefined steps. We can implement these step definitions using Mink in “bootstrap/FeatureContext.php” file. We can implement first step definition “iFillInSearchboxWith($argument1)” like this:

/**
     * @Given /^I fill in searchBox with "([^"]*)"$/
     */
    /*
    public function iFillInSearchboxWith($input)
    {
        $this->fillField("searchInput",$input);
    }

Now we will run “behat”command again, you can see eight step is passed. Terminal window output should be like this:

/Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7
    Given I am on "/"                       # FeatureContext::visit()
    And I fill in searchBox with "<input>"  # FeatureContext::iFillInSearchboxWith()
    When I press search button
    Then I should see "<output>"            # FeatureContext::assertPageContainsText()

    Examples:
      | input   | output    |
      | London  | lʌndən/   |
        Undefined step "I press search button"
      | NewYork | nɪu ˈjɔək |
        Undefined step "I press search button"
      | Sydney  | sɪdni/    |
        Undefined step "I press search button"
      | Mumbai  | मुंबई     |
        Undefined step "I press search button"
      | Bejing  | 北京        |
        Undefined step "I press search button"
      | Tokyo   | 東京        |
        Undefined step "I press search button"
      | Lahore  | لاہور     |
        Undefined step "I press search button"
      | Paris   | paʁi      |
        Undefined step "I press search button"

8 scenarios (8 undefined)
32 steps (16 passed, 8 skipped, 8 undefined)
0m25.568s

You can implement step definitions for undefined steps with these snippets:

    /**
     * @When /^I press search button$/
     */
    public function iPressSearchButton()
    {
        throw new PendingException();
    }

We have to repeat this process until all steps get “passed”. We can implement these steps by adding some code in “bootstrap/FeatureContext.php” file. The code will look like this:

<?php
use Behat\Behat\Context\ClosuredContextInterface,
    Behat\Behat\Context\TranslatedContextInterface,
    Behat\Behat\Context\BehatContext,
    Behat\Behat\Exception\PendingException;
use Behat\Gherkin\Node\PyStringNode,
    Behat\Gherkin\Node\TableNode;
use Behat\Mink\Behat\Context\MinkContext;
use Behat\Mink\Session;
use Behat\Mink\Driver\DriverInterface;
require_once 'bootstrap.php';

/**
 * Features context.
 */
class FeatureContext extends MinkContext
{
   /**
     * @Given /^I fill in searchBox with "([^"]*)"$/
     */
    public function iFillInSearchboxWith($input)
    {
        $this->fillField("searchInput",$input);
    }

    /**
     * @When /^I press search button$/
     */

    public function iPressSearchButton()
    {
        $this->getMink()->getSession()->getDriver()->click("//*[@id='searchButton']");
         $this->getMink()->getSession()->wait("3000");
    }

After implementation, we have to run “behat” command again. You will see this:

 Feature: wikiSearch
  In order to search information on wiki
  As a Wiki user
  I want to get sensible results from site

  @javascript
  Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:8
    Given I am on "/"                       # FeatureContext::visit()
    And I fill in searchBox with "<input>"  # FeatureContext::iFillInSearchboxWith()
    When I press search button              # FeatureContext::iPressSearchButton()
    Then I should see "<output>"            # FeatureContext::assertPageContainsText()

    Examples:
      | input   | output    |
      | London  | lʌndən/   |
      | NewYork | nɪu ˈjɔək |
      | Sydney  | sɪdni/    |
      | Mumbai  | मुंबई       |
      | Bejing  | 北京      |
      | Tokyo   | 東京       |
      | Lahore  | لاہور       |
      | Paris   | paʁi      |

8 scenarios (8 passed)
32 steps (32 passed)
0m42.794s

We managed to get all our scenario/steps “passed”. It’s time to login into your Sauce Labs account to see this scenario running on Sauce Labs.

Now we have to use “sauce.yml” config file. We will run below mentioned command from terminal and we can see output on Sauce Labs as shown below:

 behat -c sauce.yml

Screenshots for feature running on Sauce Labs:

: Feature running on Sauce labs

Job Running on Sauce Labs

: Watch video and Setps of this job on Sauce Labs

You can see detail steps, screenshots and video of this job on Sauce Labs
See terminal output as shown below:

behat -c sauce.yml

: terminal output

Building Features with Jenkins

Continuous Integration

Continuous Integration (CI) is one of the best practice in agile projects to detect bugs early. Each integration is verified by an automated build to detect integration errors as quickly as possible. Now we will see how we can integrate Jenkins to our behat project.

Don’t forget to start your engine before building project in Jenkins. You have start Selenium/WebDriver before starting Jenkins.

There is “/report” directory to save JUnit reports in xml format generated by Behat. You can generate reports using this command:

behat -f junit --out ~/pathto/report

To continue, you need Jenkins installed on your machine. Now start Jenkins by executing *.war file from terminal on port 8080. You also need to visit “http://localhost:8080″ to see Jenkins GUI.

$  java -jar jenkins.war

Visit http://localhost:8080/ and you should see Jenkins Dashboard.
Create “New Job” for behat project.
Specify your SCM to create Jenkin’s workspace on your system.
You can specify build file as per your project structure. In here I have configured in “Acne” project by specifying build file path and test reports like this.

Save your configuration. Go back to project and click”Build Now”, you will see:

build now

You can see features running in browser.
Sit back and enjoy your features running on Jenkins till it finishes job.
Few minutes later, you will see your build is “Green”:

Build Success

Watch your test results:

Jenkins Test Result

Behat, Sauce Labs and Jenkins

You can run all your features on Sauce Labs just by updating ANT file like:

<project name="behat" default= "behat" basedir="">
   <delete dir="${basedir}/report" />
   <mkdir dir="${basedir}/report"/>
   <target name="behat">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-c sauce.yml -f junit --out ${basedir}/report"/>
    </exec>
   </target>
   <target name="create-test-report"  
          description="Generate reports for executed JUnit tests.">  
       <junitreport todir="./report">  
          <fileset dir="${basedir}/report">  
             <include name="TEST-*.xml"/>  
          </fileset>  
          <report format="frames" todir="./report/html"/>  
       </junitreport>  
   </target>  
</project>

Alternatively, you can configure Sauce On demand tunnel into your Jenkins.

Conclusion:

We can write web acceptance tests with Behat and Mink combination. We can plug them into Sauce Labs with config file (“sauce.yml”) and run them on a CI server. Now you can implement BDD practices for PHP applications by taking benefit from Bahat and Sauce Labs.

Demo

You can watch video demonstration of this blog on Vimeo and YouTube. More about me

[Shashikant Jagtap]Agile Testing with Behat, Sauce and Jenkins January 13, 2012
Abstract:

The key benefits of Behavior-Driven-Development (BDD) practices are communication enhancement and customer satisfaction. You can read more about that by Dan North and Gojko Adzic. Perhaps the biggest practical challenge in the way of reaping those benefits is the burden of provisioning, installation and maintenance of requisite complex and fussy infrastructure. The recent availability of latest CI servers like Jenkins & cloud based testing services like Sauce Labs carries the potential to remove that barrier. This post discusses and shows how to integrate Behat an emerging BDD framework for PHP with continuous integration server like Jenkins and cloud based testing services like Sauce Labs.

What is Behat?

Behat is a BDD framework for PHP. There are some tools available for BDD like Cucumber for Ruby, SpecFlow for .NET and Lettuce for Python. Behat is the first BDD tool for PHP applications. Developers can also use PHPSpec framework to implement classes underneath Behat for PHP & Zend Framework applications. Behat is written in PHP by Konstantin Kudryashov. With Behat, you can write human readable stories which turns as tests to run against your application. Behat can be used for API testing, functional testing and data-driven testing. Developers will do API testing and we will carry on with functional testing (web acceptance testing) with Behat.

Functional Testing with Behat and Mink

Behat is used for acceptance testing (any tests) by executing a Gherkin scenario. Developers can implement integrated classes. Testers start thinking of more workflow level and technical level steps (actions) which turns as scenarios for features. Once tester started to think of Web Acceptance Testing (functional testing) with browser interaction then another tool called “Mink” comes into the picture.

Mink is used for browser emulation (functional testing) where browser interaction takes place. As of now, there are following Selenium drivers available for browser emulation.
- Selenium 1 Driver provides bridge for Selenium RC (Selenium 1).
- Selenium 2 driver provides bridge for Selenium 2. (Facebook webdriver) for PHP.
Behat In Action:

You must have pear installed in order to proceed with Behat installation. Now we will run these commands from your terminal window:
```
$ sudo pear channel-discover pear.behat.org
$ sudo pear channel-discover pear.symfony.com
$ sudo pear install behat/gherkin-beta
$ sudo pear install behat/behat-beta
```
Test your installation by running this command:
```
$ behat --version
Behat version 2.2.0
```
Now let’s install Mink, run following commands from terminal window:
```
$ pear channel-discover pear.symfony.com
$ pear channel-discover pear.behat.org
$ pear install behat/mink
```
Mink is ready to use. We have to include “mink/autoload.php” in your classes as shown below:
```
require_once 'mink/autoload.php';
```
Start Your Project

Navigate to project root directory and initialize Behat by running these commands:
```
$ cd /path/to/my/project

$ ls
application

$ behat --init
+d features - place your *.feature files here
+d features/bootstrap - place bootstrap scripts and static files here
+f features/bootstrap/FeatureContext.php - place your feature related code here

$ ls
application	features	

$cd features

$ ls
bootstrap

$cd bootstrap

$ls
FeatureContext.php
```
This will create “features” directory and “bootstrap/FeatureContext.php” for you. Now we will directly jump to the project created with some feature files.

You can use NetBeans with installed Cucumber plugin for Gherkin syntax highlighting. Project structure will look like this:

Behat Directory Structure

Directory Structure Explained

Behat has already created “features” directory and “features/bootstrap” directory with “FeatureContext.php” in it.

bootstrap.php

We can this create file in define some constants and some third-party libraries which needs to be included in class files. This file should look like this:
```
<?php
date_default_timezone_set('Europe/London');
 require_once 'mink/autoload.php';
 require_once 'PHPUnit/Autoload.php';
 require_once 'PHPUnit/Framework/Assert/Functions.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
 require_once 'PHPUnit/Extensions/SeleniumTestCase/SauceOnDemandTestCase.php';
```
We have some Gherkin features written over there.

behat.yml

This file is a default config file that Behat uses for executing features. Example behat.yml should is shown below:
```
default:
    context:
        parameters:
            javascript_session: selenium
            base_url: http://uat.acnestudios.com
            browser: firefox
            show_cmd: open %s
```
You can change drivers by changing “javascript_session” variable. It can be “selenium” or “webdriver”.

sauce.yml

This file is used for running features on Sauce Labs. The code for this file is explained in “behat and Sauce Labs” section below.

build.xml

This file is used for running features with ANT. We can use this ANT file to plug into Jenkins. Simple ANT file should look like this:
```
<project name="behat" default= "behat" basedir="">
   <delete dir="${basedir}/report" />
   <mkdir dir="${basedir}/report"/>
   <target name="behat">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-f junit --out ${basedir}/report"/>
    </exec>
   </target>
   <target name="create-test-report"  
          description="Generate reports for executed JUnit tests.">  
       <junitreport todir="./report">  
          <fileset dir="${basedir}/report">  
             <include name="TEST-*.xml"/>  
          </fileset>  
          <report format="frames" todir="./report/html"/>  
       </junitreport>  
   </target>  
</project>
```
report

This directory is used to store *.xml reports generated by Behat’s “junit” formatter. We can use this reports to plug into Jenkins.

Start your Engine

Remember, you have to download latest version of selenium server. Now navigate to directory where you saved selenium server .jar file. You have to launch it using command shown below.
```
java -jar selenium-server-standalone-2.15.0.jar 
```
Behat & Sauce Labs

Sauce Labs is a cloud testing service which allows selenium tests to run in the cloud. Sauce Labs allocates machines and browsers for your tests, capture screenshot for every step and record video of all jobs(tests). You don’t need to setup separate machines to run tests. Sauce labs helps us to write tests without complex infrastructure.

In order to integrate Behat with Sauce Labs, you need to have an account with Sauce Labs. You need “Username” and “API Key” to plug them into a config file.

Behat executes features with “behat.yml” file by default, but we can run features with any other configuration file. We can create another configuration file like “sauce.yml” to run features on Sauce Labs. Example “sauce.yml” should look like this:
```
default:
  context:
    parameters:
      default_session:    goutte
      javascript_session: selenium
      base_url:           http://shop.acnestudios.com
      browser:            firefox
      selenium:
        host: ondemand.saucelabs.com
        port: 80
        browser: >
          {
            "username":         "your username",
            "access-key":       "your API key",
            "browser":          "firefox",
            "browser-version":  "7",
            "os":               "Windows 2003",
            "name":             "Testing Selenium with Behat"
          }
```
We will use “sauce.yml” as a config file to run features on Sauce Labs. If you wish to run all features from “features” directory on Sauce labs, you can use this command:
```
behat -c sauce.yml
```
When to implement step definitions?
- If you can speak fluent Gherkin, then you don’t need to write code. Behat/Mink will understand Gherkin and run your features without suggesting step definitions.
- If features written by someone else, you can take full advantage of Mink API’s in order to implement step definitions suggested by Behat/Mink.
- It’s very important to write good Gherkin to write minimum code.
- It’s very easy to access Mink API’s by writing simple code.
Now we will see how will you “click” particular element on page. You need to use Xpath as a locator for that element. Mink will suggest some step definitions, you need to complete it like this:
```
/**
* @Given /^I click first product$/
*/
public function iClickFirstProduct()
{
$this->getMink()->getSession()->getDriver()->click("//Xpath");
}
```
Example : Feature addToCart (Need implementation)

We will write simple feature to add product into shopping cart. Feature will look like this:
```
 Feature: addToCart
  In order to shop some items on Acne site
  As a customer
  I want to add items in my bag

  @javascript
  Scenario: check all category pages
    Given I am on Category page
    When I choose product
    And I configure it
    Then I should see product added to shopping cart
```
Now we will run this feature using command:
```
behat --name addToCart
```
Remember, we are running it locally for now using default config file “behat.yml” with Selenium driver. After executing above command, we will get some step definitions suggested by Behat/Mink

You can see above, Behat/Mink has suggested some step definitions for undefined steps. We can implement these step definitions using Mink in “bootstrap/FeatureContext.php” file. We can implement first step definition ( iAmOnCategoryPage) like this:
```
/**
     * @Given /^I am on Category page$/
     */
    public function iAmOnCategoryPage()
    {
        $this->visit('/shop/women/aw11/skirts.html');
    }
```
Now we will run “behat”command again, you can see one step is passed. Terminal window output should be like this:

We have to repeat this process until all steps get “passed”. We can implement these steps by adding some code in “bootstrap/FeatureContext.php” file. The code will look like this:
```
<?php
use Behat\Behat\Context\ClosuredContextInterface,
    Behat\Behat\Context\TranslatedContextInterface,
    Behat\Behat\Context\BehatContext,
    Behat\Behat\Exception\PendingException;
use Behat\Gherkin\Node\PyStringNode,
    Behat\Gherkin\Node\TableNode;
require_once 'bootstrap.php';
use Behat\Mink\Behat\Context\MinkContext;
use Behat\Mink\Session;
use Behat\Mink\Driver\DriverInterface;

/**
 * Features context.
 */
class FeatureContext extends MinkContext
{ /**
     * @Given /^I am on Category page$/
     */
    public function iAmOnCategoryPage()
    {
      $this->visit('/shop/women/aw11/skirts.html');
    }

    /**
     * @When /^I choose product$/
     */
    public function iChooseProduct()
    {
      $this->getMink()->getSession()->getDriver()->click("//a/div/img");
      $this->getMink()->getSession()->wait("10000");
    }

    /**
     * @Given /^I configure it$/
     */
    public function iConfigureIt()
    {
      $this->getMink()->getSession()->getDriver()->click("//*[@id='bodycontainer']/article/div/section[1]/div/form/div[1]/div/a");
      $this->getMink()->getSession()->getDriver()->click("//*[@id='bodycontainer']/article/div/section[1]/div/form/div[1]/ul/li[2]/a");
      $this->getMink()->getSession()->getDriver()->click("//*[@id='bodycontainer']/article/div/section[1]/div/form/div[2]/div/a");
      $this->getMink()->getSession()->getDriver()->click("//*[@id='bodycontainer']/article/div/section[1]/div/form/div[2]/ul/li[2]/a");
      $this->getMink()->getSession()->getDriver()->click("//*[@id='bodycontainer']/article/div/section[1]/div/form/button");
      $this->getMink()->getSession()->wait("10000");
    }

    /**
     * @Then /^I should see product added to shopping cart$/
     */
    public function iShouldSeeProductAddedToShoppingCart()
    {
     $this->assertElementOnPage('#cartHeader');
     $this->getMink()->getSession()->getDriver()->click(".//*[@id='cartHeader']");
     $this->assertElementOnPage('#topCartContent');    
    }
```
After implementation, we have to run “behat” command again. You will see this:

We managed to get all our scenario/steps “passed”. It’s time to login into your Sauce Labs account to see this scenario running on Sauce Labs.

Now we have to use “sauce.yml” config file. We will run below mentioned command from terminal and we can see output on Sauce Labs as shown below:
```
 behat -c sauce.yml --name addToCart
```
Screenshots for feature running on Sauce Labs:

feature started running on sauce labs

feature completed on Sauce Labs

You can see detail steps, screenshots and video of this job on Sauce Labs here.
See terminal output as shown below:

Scenario Passed at terminal

Building Features with Jenkins

Continuous Integration (CI) is one of the best practice in agile projects to detect bugs early. Each integration is verified by an automated build to detect integration errors as quickly as possible. Now we will see how we can integrate Jenkins to our behat project.

Don’t forget to start your engine before building project in Jenkins. You have start Sahi/Selenium/WebDriver before starting Jenkins.

There is “/report” directory to save JUnit reports in xml format generated by Behat. You can generate reports using this command:
```
behat -f junit --out ~/pathto/report
```
To continue, you need Jenkins installed on your machine. Now start Jenkins by executing *.war file from terminal on port 8080. You also need to visit “http://localhost:8080″ to see Jenkins GUI.
```
$  java -jar jenkins.war
```
- Visit http://localhost:8080/ and you should see Jenkins
Jenkins
- Create “New Job” for behat project.
- Specify your SCM to create Jenkin’s workspace on your system.
- You can specify build file as per your project structure. In here I have configured in “Acne” project by specifying build file path and test reports like this.
Jenkins Config
- Save your configuration. Go back to project and click”Build Now”, you will see:
build started

Console output
- You can see features running in browser.
- Sit back and enjoy your features running on Jenkins till it finishes job.
- Few minutes later, you will see your build is “Green”:
Build Success
- Watch your test results:
Jenkin Test Result

Behat, Sauce Labs and Jenkins

You can run all your features on Sauce Labs just by updating ANT file like:
```
<project name="behat" default= "behat" basedir="">
   <delete dir="${basedir}/report" />
   <mkdir dir="${basedir}/report"/>
   <target name="behat">
    <exec dir="${basedir}" executable="behat" failonerror="true">
     <arg line="-c sauce.yml -f junit --out ${basedir}/report"/>
    </exec>
   </target>
   <target name="create-test-report"  
          description="Generate reports for executed JUnit tests.">  
       <junitreport todir="./report">  
          <fileset dir="${basedir}/report">  
             <include name="TEST-*.xml"/>  
          </fileset>  
          <report format="frames" todir="./report/html"/>  
       </junitreport>  
   </target>  
</project>
```
Alternatively, you can configure Sauce On demand tunnel in Jenkins.

You can watch demo of this blog on vimeo.

Conclusion:

We can write web acceptance tests with Behat and Mink combination. We can plug them into Sauce Labs with config file (“sauce.yml”) and run them on a CI server. Now you can implement BDD practices for PHP applications by taking benefit from Bahat and Sauce Labs.
[Devis Lucato]Select: Inversion of Control December 13, 2011

Inversion of control is the answer to more maintainable, testable, modular code, a common pattern in OOP adopted by frameworks and enterprise projects. The main idea is to separate configuration (class names and initialisation parameters) from implementation (class instantiations and static calls), avoiding hard coded class names and parameters, so that they can be replaced by third parties and during tests.

There are at least three common ways of injecting dependencies (see here for a detailed description).

Constructor Injection: injection through constructor parameters

For every object, all the dependencies are passed as constructor arguments. Constructor injection is fairly straitforward and works quite well on small projects, but as a project and the number of dependecies grow, so do constructors' signatures length. Complex constructors are eventually refactored to receive an array of dependecies or moving them up in the hierarchy, in some base class, which evolves into a registry of dependencies (among the other responsibilities).

Setter Injection: public setters for every dependency

Every dependency is set using a public method inside a class. For instance three classes depending on a Mailer would have each a `setMailer($object)` method. Fairly simple to implement but leads to duplication and maintainability issues, every time an object is create all the setters must be called.

Service locator: holder/registry of components/services

All the dependencies are provided by a builder, which serves as a registry of dependencies and/or service definitions. The service locator knows how to instantiate each dependency. Such service exposes methods like `getMailer()`, `getLogger()` etc. A service locator centralises the configuration detailing classes and parameters involved on objects instantiations.

Select is a static Service Locator implementation with PHP method overloading. It allows to replace classes and can be used to hold components/services, identified by unique names and automatically exposed with getter methods.

Select is designed to be subclassed with a custom class name, as opposed to the common injection through constructors. To replace Select you subclass the main class. For instance: during tests you can either use a different set of definitions (suggested) or use a mocked Service Locator class implementing the same interface iSelect.

read more
[Mark van der Velden]A world without cookies June 29, 2011
Imagine a world wide web without cookies. And this might not be the strangest thing, since the use of cookies is severely limited since Wednesday the 22nd of June 2011 in the Netherlands. And more countries will follow: http://www.bbc.co.uk/news/technology-12668552.

The Law

The (Dutch) law, that requires a user to agree before storing data, doesn't only apply on HTTP cookies. But in fact any kind of data that is stored on the users computer. Such as; HTML5 storage, flash cookies. But also desktop applications, etc.. The law also states that cookies "required" for certain functionality, are allowed without confirmation. Personally I don't see how anything will change, with this exception in place. And I wonder how many experts were involved into making this law. But that is a subject for another article perhaps...

What are cookies

Cookies are little packages of information stored in the browser of a website visitor, they can contain "small" amounts of data such as an identifiable token or a user preference.

What purpose do cookies serve

Cookies are very generic and can be used for many things, good and bad. The most popular probably being tracking your activity and advertisement. But they are also used to keep a state between requests and to store a preference. Such as "remember me" at a login form, or perhaps "no I do not want to participate in your survey".

Cookie problems

If When the law becomes international, you are potentially violating laws by simply calling something like: setcookie() without the users consent. There are hacks around the typical HTTP cookies, by using flash cookies for example (Oh and btw, here a guide on how to clear those: http://www.macromedia.com/...ngs_manager07.html). And possibly HTML5 storage could play a role here (or any of it's derivatives).

Another thing that has been happening, is visitor awareness and thus browser features. More and more people block cookies to stop advertisement tracking, but unfortunately this also prevents a user to use the features he or she wants to use (such as login sessions, etc.). There is an answer for this and quite a few browser vendor's plan on implementing the "Do Not Track" (http://donottrack.us/) feature, or have already done so. But I'm not too happy with it. The downside of "Do Not Track" is that it's voluntary for website owners and advertisement companies to respect this feature. Other tools include projects such as "Ad Blocker", that only block cookies (and more) for advertisement purposes. It works pretty good, but that is hardly user-friendly.

But, back to "no more cookies"... How do you solve the problem of keeping a state between requests over a stateless protocol?

Some ideas
Well in short, I have some ideas but definitely no real answers. I don't think there is a real answer just yet.

Let's take the example of a login session. Where you want to offer a secure section to your visitors, where they can (e.g.) read their e-mail, privately. A few things come to mind:
1. Identifying yourself in every relevant request, using headers for example or an argument &userId=42 in the request. This however has some big problems on its own.
2. Digest authentication on every relevant request.
3. Take a look at other technologies, such as IPv6 and move authentication to a lower level.
4. Perhaps use a commit and rollback system. Where you can do a variety of actions and only your password will be asked in the end, either applying or discarding your actions. Requiring you only to enter your login once. Not the most ideal solution for e-mail though, I have to admit. With this system you would re-post your data to next screens in a similar fashion as e.g. a wizard.
5. Or perhaps we need to move away from complete server-side applications and move to the client, Javascript applications and use the HTTP spec a little better as it was intended and only have web-services.
Many, if not all, of the things I mentioned above would require secure connections (SSL/TLS) to avoid other security problems. Which might not be a bad move anyway.

Personally I think that there is a future, in an improved implementation of digest authentication over SSL. One that uses HMAC and stronger algorithms, SSL would then supply the missing server validation feature. It should also be more strict and not fall back to insecure legacy features.

Conclusion

All in all I firmly believe that the browser should play a big role in this new cookie recipe and should (partially) solve these problems. Also there should be a more clear separation between "generic storage" and authentication versus a simulated persistency. In more perfect world I would vote for a solution that works on other (underlying) layers and make it application agnostic.

I suppose the point I'm trying to make with this article is the following: Take away a feature the entire world uses (since 1996), and wait for the brilliant and creative minds, perhaps such as yourself, to come up with a more innovative feature. It's time for something better!
Another interesting read:

http://www.vsecurity.com/download/papers/WeaningTheWebOffOfSessionCookies.pdf
I made some updates to this article, based on some comments.

[Devis Lucato]PHP BDD with Cucumber/Cuke4Php April 6, 2011

This is a quick reference on how to start using Cucumber with PHP, thanks to Cuke4Php.

Cucumber is a Behavioural Driven Development framework written in Ruby. Using a wire it allows to write BDD steps in PHP to test PHP applications. Cuke4php is in fact a wire protocol implementation of Cucumber written in PHP. If you are interested on BDD you might want to have a look at Behat too, which is written entirely in PHP.

As an exercise this guide should guide through the installation of Ruby, Cucumber, Cuke4php, and PHPUnit to implement the classic Calculator's Addition feature example. Please consider this just a demo. The system used is a Debian 6 Squeeze. Zend Framework is used only to simulate a real world project scenario.

Here we go! [N.B. working as root]

Install Ruby

# apt-get install ruby ruby1.8-dev rdoc1.8 irb libxml2-dev libxslt1-dev libc6-dev-i386 libopenssl-ruby
...
Note, selecting 'ruby1.8' instead of 'rdoc1.8'
Note, selecting 'ruby' instead of 'irb'
Note, selecting 'libruby' instead of 'libopenssl-ruby'
...

Manual installation of Ruby gems

# cd /tmp
# wget http://rubyforge.org/frs/download.php/60718/rubygems-1.3.5.tgz
# tar xvf rubygems-1.3.5.tgz
# cd rubygems-1.3.5
# ruby setup.rb
RubyGems 1.3.5 installed

Add gems

# gem sources -a http&#58//gems.github.com
http://gems.github.com added to sources

# gem install cucumber mechanize rspec webrat
Building native extensions. This could take a while...
ERROR: Error installing cucumber: cucumber requires gherkin (>= 2.3.5, runtime)
....

# gem install gherkin cucumber
Building native extensions. This could take a while...
.......
Successfully installed gherkin-2.3.5
Successfully installed cucumber-0.10.2

Install a custom copy of Zend Framework

# wget http://framework.zend.com/releases/ZendFramework-1.11.4/ZendFramework-1.11.4.tar.gz
# tar xzf ZendFramework-1.11.4.tar.gz
# mkdir ~/bddtest
# mv ZendFramework-1.11.4 ~/bddtest/ZF
# PATH=$PATH:~/bddtest/ZF/bin/
# chmod u+x ~/bddtest/ZF/bin/zf.sh
# alias zf='~/bddtest/ZF/bin/zf.sh'

Create a new ZF project

# cd ~/bddtest
# zf create project zfapp
Creating project at ~/bddtest/zfapp
.....

Create Features tree:

# cd ~/bddtest/zfapp
# mkdir -p features/support
# mkdir -p features/step_definitions

Ruby configuration

# cd ~/bddtest
# nano features/support/env.rb

File content:

# RSpec
require 'rspec/expectations' # Webrat
require 'webrat' require 'test/unit/assertions'
World(Test::Unit::Assertions) Webrat.configure do |config| config.mode = :mechanize
end World do session = Webrat::Session.new session.extend(Webrat::Methods) session.extend(Webrat::Matchers) session
end

Create the first requirement, the Calculator Addition feature

nano features/addition.feature

File content, the Calculator Addition feature to implement:

Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two number Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen

And test the feature with cucumber:

# cucumber features/addition.feature

The second command verifies the feature which is not implemented yet and returns some errors:

Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two number Scenario: Add two numbers # features/addition.feature:6 Given I have entered 50 into the calculator # features/addition.feature:7 And I have entered 70 into the calculator # features/addition.feature:8 When I press add # features/addition.feature:9 Then the result should be 120 on the screen # features/addition.feature:10 1 scenario (1 undefined)
4 steps (4 undefined)
0m0.003s

Install Cuke4php

# gem install cuke4php
******************************************************************************** Please install PHPUnit >= 3.0 if you've not already done it! Add PEAR channels: pear channel-discover pear.phpunit.de pear channel-discover components.ez.no pear channel-discover pear.symfony-project.com Install PHPUnit: pear install phpunit/PHPUnit ********************************************************************************
Successfully installed cuke4php-0.9.5
1 gem installed

Cuke4php requires PHPUnit, let's install it, in this case the installation failed because PEAR was not up to date:

# pear channel-discover pear.phpunit.de
Adding Channel "pear.phpunit.de" succeeded
Discovery of channel "pear.phpunit.de" succeeded # pear channel-discover components.ez.no
Adding Channel "components.ez.no" succeeded
Discovery of channel "components.ez.no" succeeded # pear channel-discover pear.symfony-project.com
Adding Channel "pear.symfony-project.com" succeeded
Discovery of channel "pear.symfony-project.com" succeeded # pear install phpunit/PHPUnit
...
phpunit/PHPUnit requires PEAR Installer (version >= 1.9.2), installed version is 1.9.1
...

Upgrade PEAR and install PHPUnit

# pear upgrade
# pear install phpunit/PHPUnit

Create the wire protocol to allow Cucumber to test the application PHP code

# nano features/Cuke4Php.wire

File content:

host: localhost
port: <%= ENV['CUKE4PHP_PORT'] %>

Enable PHPUnit autoloader, to avoid issues later in the tests

# nano features/support/Env.php

File content:

<?php
require "PHPUnit/Autoload.php";

Let's test the features with Cuek4php to see if Cuke4php is working fine.

# cuke4php features/addition.feature Cuke4Php listening on port 16816
Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two number Scenario: Add two numbers # features/addition.feature:6 Given I have entered 50 into the calculator # features/addition.feature:7 And I have entered 70 into the calculator # features/addition.feature:8 When I press add # features/addition.feature:9 Then the result should be 120 on the screen # features/addition.feature:10 1 scenario (1 undefined)
4 steps (4 undefined)
0m0.086s You can implement step definitions for undefined steps with these snippets: Given /^I have entered (\d+) into the calculator$/ do |arg1| pending # express the regexp above with the code you wish you had
end /**
* Given /^I have entered 50 into the calculator$/
**/
public function stepIHaveEntered50IntoTheCalculator() { self::markPending();
} Given /^I have entered (\d+) into the calculator$/ do |arg1| pending # express the regexp above with the code you wish you had
end /**
* Given /^I have entered 70 into the calculator$/
**/
public function stepIHaveEntered70IntoTheCalculator() { self::markPending();
} When /^I press add$/ do pending # express the regexp above with the code you wish you had
end /**
* When /^I press add$/
**/
public function stepIPressAdd() { self::markPending();
} Then /^the result should be (\d+) on the screen$/ do |arg1| pending # express the regexp above with the code you wish you had
end /**
* Then /^the result should be 120 on the screen$/
**/
public function stepTheResultShouldBe120OnTheScreen() { self::markPending();
}

The only missing bits now are the steps required to implement the feature. They are defined in a php file:

# nano features/step_definitions/calculator_steps.php

Temporary file content:

<?php class CalculatorSteps extends CucumberSteps
{ /** * Given /^I have entered 50 into the calculator$/ **/ public function stepIHaveEntered50IntoTheCalculator() { self::markPending(); } /** * Given /^I have entered 70 into the calculator$/ **/ public function stepIHaveEntered70IntoTheCalculator() { self::markPending(); } /** * When /^I press add$/ **/ public function stepIPressAdd() { self::markPending(); } /** * Then /^the result should be 120 on the screen$/ **/ public function stepTheResultShouldBe120OnTheScreen() { self::markPending(); }
}

Testing cuke4php now

# cuke4php features/addition.feature Cuke4Php listening on port 16816
Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two number Scenario: Add two numbers # features/addition.feature:6 Given I have entered 50 into the calculator # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:8 Not Implemented (Cucumber::Pending) features/addition.feature:7:in `Given I have entered 50 into the calculator' And I have entered 70 into the calculator # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:15 When I press add # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:22 Then the result should be 120 on the screen # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:29 1 scenario (1 pending)
4 steps (3 skipped, 1 pending)
0m0.129s

Let's define the real steps which describe the feature and to verify that the feature is implemented:

# nano features/step_definitions/calculator_steps.php

Here's the final file content. Note how PHPDoc is used to map a method to a step description and to extract the required parameters:

<?php class CalculatorSteps extends CucumberSteps
{ public function beforeAll() { require 'application/models/Calculator.php'; $this->aGlobals['calculator'] = new Application_Model_Calculator(); } /** * Given /^I have entered 50 into the calculator$/ **/ public function stepIHaveEntered50IntoTheCalculator() { $this->aGlobals['calculator']->enter(50); } /** * Given /^I have entered 70 into the calculator$/ **/ public function stepIHaveEntered70IntoTheCalculator() { $this->aGlobals['calculator']->enter(70); } /** * When /^I press add$/ **/ public function stepIPressAdd() { $this->aGlobals['calculator']->doSum(); } /** * Then /^the result should be 120 on the screen$/ **/ public function stepTheResultShouldBe120OnTheScreen() { $this->assertEquals(120, $this->aGlobals['calculator']->getResult()); }
}

And create the model which encapsulate the Calculator logic:

# zf create model Calculator
Creating a model at ~/bddtest/zfapp/application/models/Calculator.php
Updating project profile '~/bddtest/zfapp/.zfproject.xml' # nano application/models/Calculator.php

File content:

<?php class Application_Model_Calculator
{ protected $result; protected $values; public function __construct() { $this->result = 0; $this->values = array(); } public function enter($value) { $this->values[] = $value; } public function doSum() { for ($this->result = reset($this->values); $value = next($this->values);) $this->result += $value; } public function getResult() { return $this->result; }
}

...and testing the feature now...

# cuke4php features/addition.feature Cuke4Php listening on port 16816
Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two number Scenario: Add two numbers # features/addition.feature:6 Given I have entered 50 into the calculator # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:13 And I have entered 70 into the calculator # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:20 When I press add # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:27 Then the result should be 120 on the screen # /root/bddtest/zfapp/features/step_definitions/calculator_steps.php:34 1 scenario (1 passed)
4 steps (4 passed)
0m0.173s

Done! Mission accomplished! Now in order to implement another feature, ie Multiply, it's just a matter of creating a feature-file, the steps required to verify the feature, and implement the business logic in some model.

[Felix de Vliegher]Goodbye Ibuildings, hello Egeniq April 5, 2011

A couple of years ago, when I joined Ibuildings, it was because I had a passion for PHP and the PHP community, and Ibuildings was THE company embodying this spirit. I’ve got the opportunity to work at some of the most amazing projects, and met some of the smartest people in PHP-land. However since about a year, I’ve noticed my interests shifting a lot to mobile and mobile development. Of course I’m still as passionate about PHP as I was a few years ago, but I feel that mobile development is quite close to web development in a couple of ways, especially nowadays when people often expect to have internet available at all times. After focussing more and more on mobile development and seeing a huge future in it, I decided it was time for me to move on and leave Ibuildings.

To better match my combined interest of PHP and mobile development, I decided to join Egeniq. For those who don’t know Egeniq, they’re an awesome mobile technology company, founded 7 months ago by my former colleagues Peter, Bas and Ivo. Other than your average mobile app shop, they focus on proper software engineering, architecture, performance and scalability. They build on the years of experience they have in the PHP world and bring that expertise to the mobile business. They’re also really into geek stuff, and mobile devices fit nicely into that category, too :-)

Even though deciding on leaving Ibuildings was probably the hardest career decision I ever had to make, I very much look forward playing around with all things mobile and working closely with my former colleagues again. Exciting! :-) I’d like to thank Ibuildings for the opportunities they gave me and all the wonderful people over there, from whom I’ve learned so much. I will be at Ibuildings until the end of April, starting at Egeniq at the start of May.

This is quite a big step, and I thought about taking this step for a long time. But you know you should pursue something, if you have a very strong gut feeling that you just need to do this. The feeling that, if you don’t do this, you will regret it. The feeling that you get when considering all the pros and cons, but deep down inside yourself you know you already made up your mind. It goes without saying that I’m very much looking forward to the new things to come at Egeniq!
[Devis Lucato]Drizzle: Big-Data-happy MySQL fork debuts March 17, 2011

Drizzle, a lightweight fork of Oracle's MySQL database for cloud computing, has been released by open sourcers. Version 2011.03.13 has been released as general availability version. It comes nearly three years after the project was announced by Brian Aker, one of MySQL's key architects.

Drizzle aims to be different from MySQL, stripping out "unnecessary" features loved by enterprise and OEMs in the name of greater speed and simplicity and for reduced management overhead.

Drizzle has no stored procedures, triggers, or views and, in a blow to a large chunk of the computing and IT establishment, it doesn't run on Microsoft's Windows. Also, there's no embedded server.

Drizzle has been optimized for "massively concurrent" environments and is designed for "modern" POSIX systems, and there aren't any installation scripts.

The GA includes log-based replication, the HailDB relational database engine instead of the Oracle-owned InnoDB, and "easy migration" from MySQL using the Drizzledump.

read more
[Devis Lucato]Constrained Application Protocol (CoAP) March 15, 2011

Constrained Application Protocol (CoAP), is a specialized RESTful (Representational State Transfer) protocol for use with constrained networks and nodes for machine-to-machine applications such as smart energy and building automation.

These constrained nodes often have 8-bit microcontrollers with small amounts of ROM and RAM, while networks such as 6LoWPAN often have high packet error rates and a typical throughput of 10s of kbit/s.

CoAP provides the REST Method/Response interaction model between application end-points, supports built-in resource discovery, and includes key web concepts such as URIs and content-types.

CoAP easily translates to HTTP for integration with the web while meeting specialized requirements such as multicast support, very low overhead and simplicity for constrained environments.

The goal of CoAP is not to blindly compress HTTP, but rather to realize a subset of REST common with HTTP but optimized for M2M applications. Although CoRE could be used for compressing simple HTTP interfaces, it more importantly also offers features for M2M such as built-in discovery, multicast support and asynchronous transactions.

Unlike HTTP, CoAP deals with these REST interchanges asynchronously over a UDP transport with support for both unicast and multicast interactions. This is achieved using transaction messages (CON=Confirmable, NON=Non-Confirmable, ACK=Acknowledgment, RST=Reset) supporting optional reliability (with exponential back-off) and transaction IDs between end-points to carry REST requests and responses. These transactions are transparent to the REST interchanges. The only difference being that responses may arrive asynchronously.

The goal of binding CoAP to UDP is to provide the bare minimum features for the protocol to operate over UDP, without trying to re-create the full feature set of a transport like TCP.

See http://tools.ietf.org/html/draft-ietf-core-coap-01 for the current Internet-Draft and a complete description.

read more
[Harrie Verveer]DB migrations: rename instead of drop March 2, 2011
Reverting a dropped column

In my talks and article I always mention that when it comes to database migrations it is generally a bad idea to rely on undo-patches for rollbacks. As an example I always use the same story: “Imagine you write a database patch that removes a column from a table, and then you write an undo-patch that adds the column back again. Sure, your database schema is now the same as it was before – but the content is gone!” And then I would move on by saying that you probably shouldn’t rely on undo-patches anyway for rollbacks, because it is better to thoroughly test everything before updating so you are absolutely certain that you won’t have to rollback. Also, I recommended to make backups first, and use that to revert database migrations when needed. Easier said than done!

The fact that I am in the luxurious position where this approach usually is an option, doesn’t mean that this is a solution for everyone. For other projects where, for example, larger databases are involved, rolling back by restoring a backup would be considered highly inefficient (and if the database is in use it would actually result in data loss). Being able to rollback patches can be convenient, but reverting dropped columns can’t be done off course.

When I did my talk at DPC10 in Amsterdam however, someone in the audience (sorry, I don’t remember who it was) suggested a very simple but very effective solution. In fact, it was such a simple solution that I couldn’t stand I hadn’t thought of it before: If you rename your column or table first instead of dropping it you can still just use an undo-patch to revert the change. In a later stage, when you are absolutely certain you don’t need to do a rollback anymore, you can simply drop the renamed column or table by writing another patch. For example in a next release.

Reverting a renamed column

As you can see, sometimes the simplest solution is the best. Beware though, as in some situations this approach might still cause problems: Records added to this table between the moment the patch was applied and the moment the undo file was applied won’t have a value in the age column, to name just one. As always when we are talking database version control, it all depends on the specific situation you are in.

I myself still refuse to write undo-patches as most patches create new tables or columns anyway, instead of removing them – and if patches actually do drop tables or columns they probably haven’t been very significant for quite some time anyway. Maybe the main reason is that I have simply never found myself in the situation where an undo-patch would have helped me. This might be different for you though, and if you are looking for a way to rollback dropped columns or tables, renaming them first might be a pretty good idea.
[Ben Longden]PHP on Azure February 13, 2011

When I first started out playing with PHP a number of years ago, I did it on my home PC running windows. I remember manually setting up apache (I never tried with IIS), configuring php to work with it and getting a mysql database up and running. The whole process was time consuming and frustrating – without package management it’s not a particularly pleasant experience.

With Microsofts Azure cloud platform available on a trial [link], and with PHP Benelux running the PHP on Azure contest [link] I decided it would be a good opportunity to have another go at using the windows environment as a hosting platform – though this time using the Web Platform Installer (WebPI) with IIS and SQL Server instead of my traditional LAMP stack.

WebPI promised that this would be a simple process. It certainly looks the part, and has a good range of PHP apps that can be setup out the box. Unfortunately as I was getting all the pre-requisites installed in my Windows 7 VM, the PHP command line tools didn’t appear to have the Windows Azure SDK available as a dependency. Nor was it available when I searched for it within WebPI.

SQL Server did install ok though, so I followed the manual instructions [link] to get IIS set up with PHP, and get the SDK installed.

That was fine – up until the point that I was building my first test deployment to my local development ‘cloud’. It transpires that the package.php script (used to build an azure package out of your php project ready for deployment) assumes that your username does not contain a space. Mine was ‘Ben Longden’.

So I created a new user account and tried again. None of the Azure SDK tools worked under my new user and just claimed that ‘A fatal error occured’, but reinstalling them helped. The next issue was that my new user doesn’t have access to connect to the SQL server installation (this is something that I have not yet resolved!).

So not plain sailing so far! I found it quite frustrating in all (compared to ‘sudo tasksel install lamp-server’), however my initial deployment to the Azure Cloud went fine in the end, once the environment was set up.

Initial concerns are with how the development process will work. Up next, will be setting up IIS as a development environment that I can work with, and automating the deployment to my local cloud.

[Harrie Verveer]Benchmarking Xdebug February 6, 2011

A very popular and widely used PHP extension for debugging and profiling PHP applications is Xdebug. Normally this tool would be used on your development environment, or in rare occasions on some remote environment that is similar to the production environment. However, a little while ago one of my co-workers found a production environment running Xdebug, and we wondered how severe this was influencing performance. Since you are adding extra overhead it is very likely things will run slower. But how significant is this difference. Would it be something worth considering?

Disclaimer
Now I know that debugging a production environment is generally considered unnecessary and a really bad idea. I myself can’t really think of a situation where I would actually want to have Xdebug on my production machine – but that’s not the point. I just had a new question that I wanted to answer: “How much does installing Xdebug affect the performance of PHP?”

The hardware
To be able to test this I first needed to find a proper testing environment. Unfortunately I don’t have any webservers laying around, and testing directly on my macbook would probably be a bad idea – since there are a million processes running that all influence the performance of the machine, and therefore would influence the results. After some pondering I realised that I had an old (well… more like ancient) laptop laying around that would actually suit this purpose quite well. When it comes to speed it doesn’t come anywhere close to a modern production environment – but since we will be looking at relative changes rather than absolute measurements I figured it would be just fine.

The software
I downloaded and installed a fresh new copy of Debian. Once it was installed I decided to take the easy way and install PHP simply by using apt-get install php5. Finally, I installed xdebug using the PECL installer. With just this installed and nothing else, I was convinced my benchmark results would be quite reliable.

Software installed:

Debian 5.0.8
PHP 5.2.6 with Suhosin-patch
Xdebug 2.1.0

The benchmark
Why write a benchmark when they already exist? For testing purposes I decided to use the bench.php script that comes with the PHP source. This script does not run out-of-the-box when Xdebug is installed, because one test (Ackermann function) exceeds the default Xdebug limit of 100 nested function calls. Now I could of course increase this number, but I wanted to test how Xdebug performs without any modifications to the configuration – so I decided to simply skip this one test and just run the remaining ones.

Finally, after a lot of preparation, I was ready to run the script. \o/

The results
I decided to run the script three times with the xdebug extension enabled, and then another three times with the xdebug extension disabled. The results were pretty clear:

	No Xdebug (time in seconds)			Xdebug (time in seconds)
	Run 1	Run 2	Run 3	Run 1	Run 2	Run 3
simple	0.687	0.648	0.681	1.327	1.361	1.367
simplecall	0.813	0.744	0.786	3.581	3.770	3.875
simpleucall	1.138	1.225	1.169	4.439	4.510	4.877
simpleudcall	1.289	1.469	1.336	4.621	4.609	4.747
mandel	2.016	2.259	2.161	3.646	4.028	4.036
mandel2	2.774	2.753	2.766	3.495	4.008	4.215
ary(50000)	0.171	0.175	0.171	0.210	0.224	0.224
ary2(50000)	0.136	0.136	0.136	0.172	0.180	0.184
ary3(2000)	1.363	1.365	1.416	2.133	2.463	2.450
fibo(30)	3.694	3.517	3.538	10.418	12.364	12.943
hash1(50000)	0.212	0.216	0.214	0.515	0.618	0.638
hash2(500)	0.264	0.266	0.266	0.283	0.357	0.342
heapsort(20000)	0.676	0.667	0.671	1.026	1.198	1.196
matrix(20)	0.556	0.552	0.555	0.810	0.960	0.936
nestedloop(12)	1.221	1.208	1.311	2.197	2.570	2.573
sieve(30)	0.786	0.786	0.789	1.105	1.231	1.226
strcat(200000)	0.090	0.093	0.096	0.148	0.173	0.169
Total:	17.886	18.079	18.062	40.126	44.624	45.998

Average:	18.009 seconds			43.583 seconds

So there we go. Without Xdebug it takes about 18 seconds to run the script, and with Xdebug enabled is takes about 44 seconds: 241% of the original time (or: it took 2.41 times as long to run the script).

So what does this mean?
In this situation adding Xdebug slowed down the execution of the benchmark script quite significantly.

So what doesn’t this mean?
This doesn’t mean that your application will slow down with the same relative percentages, nor does it mean that you can compensate for this by buying 2.41x more servers. When we look at the breakdown per test it’s clear to see that the differences per test are quite significant as well. For example: the time needed to run the ‘ary’-test increased with only 27%, while the time needed for the ‘simplecall’ test increased with a whopping 379%.

Apparently the overhead caused by running Xdebug varies and is related to what kind of operations you are running. Calling functions seems to be affected a lot by Xdebug, while the effect of manipulating strings or arrays is less severe.

Test	No xdebug	Xdebug	Percentage
simplecall	0.78	3.74	479%
simpleucall	1.18	4.61	391%
simpleudcall	1.36	4.66	341%
fibo(30)	3.58	11.91	332%
hash1(50000)	0.21	0.59	276%
simple	0.67	1.35	201%
nestedloop(12)	1.25	2.45	196%
mandel	2.15	3.9	182%
strcat(200000)	0.09	0.16	176%
ary3(2000)	1.38	2.35	170%
heapsort(20000)	0.67	1.14	170%
matrix(20)	0.55	0.9	163%
sieve(30)	0.79	1.19	151%
mandel2	2.76	3.91	141%
ary2(50000)	0.14	0.18	131%
ary(50000)	0.17	0.22	127%
hash2(500)	0.27	0.33	123%

A more realistic setup – Testing WordPress with ApacheBench
The numbers from the previous test are quite clear – but they hardly represent a real world example of an application. To get a bit more insight in how enabling or disabling Xdebug would affect an actual real life application I decided to add Apache, MySQL and WordPress into the mix, and benchmark the default homepage of a standard WordPress installation using the benchmark tool ApacheBench (ab), which comes with Apache. Adding these applications means there will be a lot more factors to influence the results – but I was hoping the results would somehow give an indication on how much an average application would be affected by installing Xdebug.

I ran ApacheBench a number of times. Every time it fired a total of 100 requests and every time I changed the number of maximum concurrent requests. I assumed that when Xdebug was enabled the requests would take longer – but because of that it’s also more likely the maximum number of concurrent requests that’s still acceptable is reached sooner. After a couple of test runs I had the following figures:

Max. concurrent requests	No Xdebug (req./sec.)	Xdebug (req./sec.)	Difference (req./sec.)	Difference %
1	2,79	2,22	0,57	20,43%
5	2,75	2,16	0,59	21,45%
10	2,73	2,13	0,6	21,98%
15	2,69	2,11	0,58	21,56%
20	2,64	2,1	0,54	20,45%
25	2,61	1,34	1,27	48,66%
30	0,95	- timeout -	0,95	100,00%

or in a pretty graph:

So not only does it take about 20% more time to respond to requests in normal situations, the maximum number of requests your server can handle is also influenced significantly.

Conclusion
So as a conclusion we can say what we actually already suspected, Xdebug influences performance quite a lot. How much exactly is hard to tell and depends on what your application does. Xdebug is a great tool, and pretty much irreplaceable when you are looking for a proper tool to debug or profile an application on your development- or testing environment. Running it on your production machines however is something that would only make sense in very specific situations, as it is typically unnecessary and it slows down your application a lot. Also, make sure you turn off Xdebug on environments that you want to use for benchmarking your application – as it will influence your results.

Edit: Directly after publishing this article Xdebug mentioned on twitter that the current version in SVN is supposed to be a lot faster. I’ll benchmark this new version soon – and let you know if there is any significant different.

[Devis Lucato]Debian 6.0 "Squeeze" with FreeBSD ports February 6, 2011

After 24 months of constant development, the Debian Project is proud to present its new stable version 6.0 (code name "Squeeze"). Debian 6.0 is a free operating system, coming for the first time in two flavours. Alongside Debian GNU/Linux, Debian GNU/kFreeBSD is introduced with this version as a "technology preview".

Debian 6.0 includes the KDE Plasma Desktop and Applications, the GNOME, Xfce, and LXDE desktop environments as well as all kinds of server applications. It also features compatibility with the FHS v2.3 and software developed for version 3.2 of the LSB.

Debian 6.0 "Squeeze" introduces technical previews of two new ports to the kernel of the FreeBSD project using the known Debian/GNU userland: Debian GNU/kFreeBSD for the 32-bit PC (kfreebsd-i386) and the 64-bit PC (kfreebsd-amd64). These ports are the first ones ever to be included in a Debian release which are not based on the Linux kernel. The support of common server software is strong and combines the existing features of Linux-based Debian versions with the unique features known from the BSD world.

Another first is the completely free Linux kernel, which no longer contains problematic firmware files. These were split out into separate packages and moved out of the Debian main archive into the non-free area of our archive, which is not enabled by default. In this way Debian users have the possibility of running a completely free operating system, but may still choose to use non-free firmware files if necessary.

Furthermore, Debian 6.0 introduces a dependency based boot system, making system start-up faster and more robust due to parallel execution of boot scripts and correct dependency tracking between them. Various other changes make Debian more suitable for small form factor notebooks, like the introduction of the KDE Plasma Netbook shell.

Debian 6.0 includes over 10,000 new packages like the browser Chromium, the monitoring solution Icinga, the package management frontend Software Center, the network manager wicd, the Linux container tools lxc and the cluster framework Corosync.

The installation process for Debian GNU/Linux 6.0 has been improved in various ways, including easier selection of language and keyboard settings, and partitioning of logical volumes, RAID and encrypted systems. Support has also been added for the ext4 and Btrfs filesystems and - on the kFreeBSD architecture - the Zettabyte filesystem (ZFS).

As always, Debian GNU/Linux systems may be upgraded painlessly, in place, without any forced downtime, but it is strongly recommended to read the release notes as well as the installation guide for possible issues, and for detailed instructions on installing and upgrading.

read more
[Devis Lucato]Changes in the TIOBE index January 31, 2011
The TIOBE Programming Community index is an indicator of the popularity of programming languages. The index is updated every month and is based on the number of skilled engineers world-wide, courses and vendors. Data is extracted from the popular search engines. The index is not about the best programming language or a measure of LOC.

While PHP (-2.24%) has lost one position to C++ (-0.93%), moving from third to fourth position, Java (+0.29%) and C (-0.39%) are pretty stable at the top.

However some relevant changes happened in 2010. Python (+1.81%) has been declared the programming language of 2010, moving from 7th to 5th position, and it is now the "de facto" standard in system scripting. Python was declared language of the year also in 2007.

Some big step forward also for Objective-C (+1.63%), while it's surprising (to me at least) that Javascript (-1.12%) is losing momentum moving from 9th to 11th position.

Looking at the last 5 years trend:
- Java, VB and Perl are slowly losing ground
- C, C++, PHP and Ruby are stable
- Python, C# and Object-C are speeding up
read more
[Devis Lucato]Silverstripe's modules moved to Github January 20, 2011

The announce from SilverStripe Core Development that all modules marked as Silverstripe supported, have been moved from svn to GitHub.

Modules were previously hosted on svn.silverstripe.org and are now hosted at https://github.com/silverstripe . This doesn't include the Silverstripe "core" though (phpinstaller, sapphire, cms) which will be migrated over the next days/weeks.

Other modules available at svn.silverstripe.org will progressively be migrated to their owners' github account when possible, or other places such as Google code.

For people using svn:externals to the trunk, the core team suggest to use Piston, which copies a snapshot of the code into your own repository and knows how to update this snapshot. For people using a specific tag instead, the SVN repository is still available (read only).

Downloads at http://silverstripe.org/modules will continue to be available.

Previous SVN repositories are now read only and contributions will be managed via pull requests (see http://help.github.com/pull-requests for a good tutorial).

read more
[Harrie Verveer]TechPortal article on database version control January 20, 2011
Last year I spoke at different conferences throughout Europe about database version control. However, a while ago I decided that I did the talk often enough and that it’s time to move on. Therefor I wrote a big wrap-up article that summarizes everything I told (and learned) during these events. I’m proud to announce that this article was published on ibuildings’ techPortal site today!

You can find the article here:
http://techportal.ibuildings.com/2011/01/11/database-version-control/
[Harrie Verveer]Video: me doing my database version control talk at PHPNW10 December 9, 2010
As mentioned before I’ve done my talk “database version control without pain” a couple of times last year. When I did my talk at PHPNW10 in Manchester the organisation made videos of the different speakers, and last week they published the video of me doing my talk on blip.tv.

I found it quite shocking to see myself doing my talk. I’m sure we’ve all been there: you record your own voice (for example on you answering machine) and when you hear it back it sounds really awkward. It’s that feeling, but then with video… in a foreign language… 1 hour long. Anyway, I learned a lot from seeing this back, and I can definitely use the video to improve my future talks. For example: I now see I really need to be more aware of where I keep my hands when I’m speaking, and that my English still has this annoying dutch accent going on

Anyway, if you missed the talk or if you want to see it again, or if you simply want to see me speak in front of an audience (hi mom!) this is your chance!

View the video on blip.tv

The slides hard to read on the video because of the low picture quality. Therefor I’d recommend keeping the associated slides next to it and click along with me.

Many thanks to PHPNW, Magma Digital and everybody else involved in organizing this great conference and making and publishing the videos.