Testing Methods – a web perspective

Here is another code-club presentation focusing on testing methods including JSUnit, Selenium and dependency testing.

Types of testing

  • Unit testing – tests the minimal software component, module or function. Each unit (basic component) of the software is tested to verify that the detailed design for the unit has been correctly implemented. WEB: Javascript code & server-side library functions
  • Integration/dependency testing verifies that a system is integrated to any external or third party systems defined in the system requirements.
  • System/functional testing tests the complete system to verify that it meets its requirements.
  • Acceptance testing can be conducted by the end-user, customer, or client to validate whether or not to accept the product. Acceptance testing may be performed after the testing and before the implementation phase.

Unit Testing

Philosophy

  • Testing the smallest functional component
  • Never delete tests – always build them up
  • Group tests together and run as part of larger ‘test suite’
  • Some people even advocate writing tests before writing the functions.
  • I’ve found writing tests after finishing an isolated function works best (for me)
  • One of the central underpinning requirements for agile development methodologies

Writing tests

  • Create a function which will test a specific object (module/function/component)
  • Set up the environment inside a setUp() method
  • Write the test function to execute the object under test and use a set of assertions to check the results.
    • e.g.assertTrue()
    • assertEquals()
    • assertNull()
  • You can test the direct output of a function (for more procedural functions), or it’s affect on other components (better for declarative or event-driven functions).
  • After running all the tests, a final method tearDown() gets executed allowing you to destroy the environment and revert to the original state of the system prior to the test
  • Most test frameworks can be integrated into a build process and executed automatically (e.g. before check-in to repository or final deployment)

Advantages of Unit Testing

  • Find all sorts of bugs (quite possible to find ones you never expected)
  • Promotes good programming practice – it really makes you think about the code your writing.
    • What are the inputs – what type, how many etc?
    • How should I handle errors? (esp and issue with JS)
    • Forces you to consider border scenarios (nulls, missing parameters, ‘assumed’ dependencies etc)
    • Makes you consider the design of your code (function grouping & organisation)
  • Allows you to rapidly check that new functionality has not affected existing code.
    • This lends itself well to team projects where you might not be able to accurately predict the impact your changes have on existing code. You can then run all existing unit tests before check-in to a repository.
    • Also useful as an aid for future programmers
    • Can support (or even form?) documentation
  • Enhances existing practice – everyone already tests their code. I guesstimate ~80% of the unit tests are ones you would have written/tested for (or maybe should have tested for!) anyway.

Negatives of unit testing

  • Writing unit tests can become tedious
  • You can become too anal about your code
  • Can only test for foreseeable events
  • Can become too reliant on the tests!
  • Difficult to test heavily dynamic or event driven functionality (e.g. Ajax)
    • To do such requires development of Mock Objects – this takes a lot more effort & time to set up and can be another cause for mistakes

Question

Should tests know what is happening inside the function or does it treat each as a blackbox?

I.e. do I/should I care how startsWith() works inside the function

Blackbox
Quicker
Just need requirement docs
Good enough to confirm code works

White box
Fully explore all possible code paths
More technical knowledge needed
Language specific

JSUnit – a UnitTesting framework for JavaScript

Why use unit tests with javascript?

  • Good practice (see above) – helps to organise your code into manageable libraries.
  • In JS it’s too easy to write a piece of code that can break other code (especially look out for modifications to the global namespace or changes to object prototypes)
  • Helps you check for browser compatibilities
  • JS unit’s command line interface allows you to run your tests in multiple browsers (see example below) and on different machines
  • JSUnit can help sanity check code if reducing size using a cruncher/compressor

Functional testing with Selenium

Selenium Core is a functional testing tool to check web applications.

Selenium IDE is a Firefox extension that acts as a selenium script macro-like recorder (you start it up, interact with the browser directly and it records your actions. These can then be played back).

  • Built on top of JSUnit
  • Runs directly in any javascript-enabled browser
  • You test the application from the end user’s perspective
  • Easy to simulate user login, page access, website searches etc
  • Verify page content (page titles, form elements, dom elements etc
  • Makes it easy to test AJAX events
  • Selenium scripts can be written in a variety of languages (HTML/Text, C#, Java, Perl, PHP, Ruby, and Python).
  • The core library provides a variety of functions and assertions:
    • open ( url )
    • assertTitle ( pattern )
    • assertElementPresent ( locator )
    • assertText ( locator, pattern )
    • fireEvent( locator,eventName )
    • click( locator )
    • etc.
  • Can extend library with home-grown javascript functions.
  • Element locators can be referenced using a variety of mechanisms:
    • Element ID/Name
    • Javascript DOM querie
    • XPATH
    • CSS style

Issues with Selenium

  • I had quite a bit of difficulty transferring scripts created in the IDE into IE (specifically in the xpath references the IDE used)
  • Any form of system output testing is going to be dependent on the entire chain of processing. All too easy to implement a new feature in code that fundamentally changes the format of the output and breaks the test. As selenium IDE scripts are mostly interpreted text strings, this makes it hard to re-jig a script once broken.
    • This is in part dependent on the method of locator used (ID, xpath etc).
    • E.g. Automatic element id naming is prone to new element insertions, xpath references are dependent on dom positions etc. Therefore think carefully about which referencing mechanism you want to use.
  • Due to scripting limitations you need to host the core library on the same application server as the one under test
  • Doesn’t handle dialog/alert boxes very well

Dependency Testing

  • These are popular in mainstream applications (see MediaWiki installation)
  • For this I created a script/webpage that tests each required component of the system e.g. email, code libraries, logging, database access etc.
  • Very handy to keep all these tests in a single place – run once then forget.
  • Can form part of documentation – allows developers to see all app dependencies
  • Issues:
    • Some failure conditions are hard to recover from and display nice message (e.g. xforms testing)
    • Difficult to test if dependent system is working correctly (usually just testing access)
    • Still require human interaction as can’t test all dependent features (check email was sent, file uploading facilities, SSO login)
  • Must remember to delete dependency script after use
Dependency Testing (test selection) Dependency Testing (test results)

References/Further reading

http://en.wikipedia.org/wiki/Unit_testing

JSUnit

http://www.jsunit.net/

YUI Test framework

http://developer.yahoo.com/yui/yuitest/

Selenium

http://www-128.ibm.com/developerworks/library/wa-selenium-ajax/

Original Kent Beck testing framework for Smalltalk

http://www.xprogramming.com/testfram.htm