Mark Gilbert's Blog

Science and technology, served light and fluffy.

Web Service Testing – Part 4 of 4

In this final post, I’ll discuss an inherent problem with the test suite I constructed for a web service I wrote.  The first three posts covered the basic structure of the test suite, a performance issue that I worked around, and the value that the suite brought to the project.

Each test would hit the web service twice, once to perform a search of “include by X” and the other “exclude by X” where “X” was some search criteria.  I would count the number of records returned by each and add them together.  I expected to get the total number of records.  The inherent problem with this is that all three numbers were obtained via the web service.  I am effectively testing the web service using the web service.  The tests aren’t providing an independent critique of the functionality under test.

As I thought about the problem, I came to the conclusion that the ideal way to test the service would be to insert a few test records with known values, use the web service to check that I can retrieve those records by searching on those values, and then delete the test records when the test was complete.  Assuming that my test-record-insert and test-record-delete methods were working properly, this would be an independent test of the service.  Unfortunately, I don’t have direct access to the pre-production and production databases, so my “ideal” test suite wouldn’t work in all environments.  I wanted to be able to run the majority of my suite against all environments, so that I could verify the functionality in development, and verify that the service was being deployed correctly to the other environments.

What we ended up doing to provide this independent test was to take one of our web sites that currently accesses the database and rewire it to pull data instead from the production web service.  Then, we checked the rewired site against the current production site, and compare the data being brought back.  This became our independent check on the service (which, by the way performed beautifully; no new issues were discovered in this final test).


August 5, 2008 - Posted by | Agile, Visual Studio/.NET

Sorry, the comment form is closed at this time.

%d bloggers like this: