Mindquarry recently entered a phase of intensive testing. This shall ensure that the first release of Mindquarry, which is targeted to the beginning of the next year, is working well and that the high quality pretensions of Mindquarry are fulfilled.

For this purpose we started looking around for open source tools that can be used for automated testing of web applications. After reading a bit about commercial solutions and trying HtmlUnit, I got a tip from Jeremy Quinn about Selenium.

Selenium is an open source framework for recording and executing unit tests for web applications in different browsers. It uses a proxy approach, which means the requests are send to a browser instance first. Afterwards the browser forwards the request to your web system. This allows reusing your unit tests for different browser without changing your test code (only a system property in our environment).

First of all, it took us some time to get Selenium running, but after that I was really impressed of it. Selenium provides a Firefox plugin which can be used for recording (and executing) test cases. Additionally you can export your tests to Java, C# or Ruby in order to run them within your test suite, e.g. as JUnit test.

Using this recording feature we were able to build a complete test suite for our application very fast. After having our test suite, we could run it against different browsers to check if the application is working with it. This reduced the testing effort enormously.

Another cool thing is, that the proxy browser approach supports testing all our AJAX and JavaScript stuff, which is nearly impossible with tools like HtmlUnit.

Finally one could say Selenium is a good candidate for web based open source projects, if they want to build a test suite. But one problem remains: the layout. Every web developer knows the problem of testing web layouts and styling in different browsers. Doing this by hand requires huge efforts.

At least is should be possible to reduce this efforts. We had the idea to automatically generate browser screenshots while executing our test suite and compare the results of different browsers afterwards.

Additionally our chief architect had the idea to compare the screenshots using fuzzy logic algorithms. But that's just an idea and we don't know if this is really possible, sounds like a research issue.

Next time I will write more about the results of our investigations.

Posted at 06:08PM Dec 27, 2006 (Permalink) by Alexander Saar with tags ajax development selenium softwarequality tests  |  Comments[1] Share This Link