Testing in the Trenches (TinT) is an occasional series recounting some of the situations I encountered and advice I gave real people and real teams on real projects to help them buy into and improve at applying the concepts of Unit Testing. Any identifying data has been changed to protect them.
Not so very long ago, I worked with a software team whose application had some core functionality that the clients would purchase and install on their own machines (we did not host the system). This core functionality could then be expanded through about a dozen separate "Licenses" that the clients could purchase in whatever combination suited their needs. If the client wanted a web interface to the core desktop system, they could purchase one or more of the web-functionality licenses. Or if they needed more powerful tools for the financial processing side of the core application, they could buy the license that would allow access to those expanded features.
The state of the Licenses for any given installation was accessible to the code through public static methods. So for the imaginary MoneyPlus license, the code could check if the client held that license or not through a call such as:
if ( App.isMoneyPlusLicensed() )
There were no setters. At system launch, a process would do its secret work, using database settings, registry key lookups, calls to our own server or other secret acts to determine exactly what licenses the client had purchased, and would set the private license state flags accordingly.
License-setting was deeply buried in the core system and by design almost inaccessible, partly out of concerns for security and sales, partly out of concern (paranoia?) that clients might find a way to hack around and fake a license without paying.
This presented some challenges when we wanted to unit-test a piece of code, and verify that it behaved one way with the MoneyPlus license, and another way without it. There was no easy way for our tests to control the state of the licenses.
One idea we had was to separate the tests into different suites, and do the setup work for a given license state before running the suite. But we ruled out that idea; since there were more than a dozen different licenses, clients could pick and choose among them. That meant a very large number of possible license combinations. There had to be an easier way.
Another idea was to use the PowerMock tool to mock and thereby control the return value of a given license-check method. This worked, but tests that used PowerMock were inevitably slower and more complex. Assuming that simpler technique would be more likely to be adopted by the team, could we find such a simpler way?
Another idea was to create a test utility that would allow the licenses to be set from within an individual test. Since we knew that the license state was held as private static boolean fields in one of the core classes, we could write a utility method that would use Java's Reflection to set the value.
We ran with this idea, and created a utility method. It would live in our Test infrastructure, which would not be shipped with the production code and risk opening a back-door to the licenses, yet would allow our tests to set the licenses as needed:
public static void setLicense(String licenseFieldName, boolean isLicensed)
It took a String representation of the name of the private field, and a boolean value indicating whether the license was on or off. And through Java Reflection the value of the private static field was changed.
Since then, this method has been used it many times in tests. But there is at least one design flaw: To set the license, one parameter needs to be a String holding the name of the private field related to the given license.
This means that, to use such a test, I need to know and remember that secret name, or else take the time to look it up.
And, as significantly, by using it, the tests become more fragile: if ever that private field is renamed, for example, many tests would break, and create a tedious effort to fix them all.
Going back to the drawing board, we added a license enumeration to the test harness. This enumeration made it easier to set the licenses in our tests. No need to remember field names, and the tests become less fragile: if ever the field name changes, the enumeration is the only place that will need to be changed in our test framework.
The enumeration is UnitTest.AppLicense and it has an item for each of the licenses handled by the application:
UnitTestUtilities.setLicense(AppLicense license, boolean isLicensed);
Not so very long ago, I worked with a software team whose application had some core functionality that the clients would purchase and install on their own machines (we did not host the system). This core functionality could then be expanded through about a dozen separate "Licenses" that the clients could purchase in whatever combination suited their needs. If the client wanted a web interface to the core desktop system, they could purchase one or more of the web-functionality licenses. Or if they needed more powerful tools for the financial processing side of the core application, they could buy the license that would allow access to those expanded features.
The state of the Licenses for any given installation was accessible to the code through public static methods. So for the imaginary MoneyPlus license, the code could check if the client held that license or not through a call such as:
if ( App.isMoneyPlusLicensed() )
There were no setters. At system launch, a process would do its secret work, using database settings, registry key lookups, calls to our own server or other secret acts to determine exactly what licenses the client had purchased, and would set the private license state flags accordingly.
License-setting was deeply buried in the core system and by design almost inaccessible, partly out of concerns for security and sales, partly out of concern (paranoia?) that clients might find a way to hack around and fake a license without paying.
This presented some challenges when we wanted to unit-test a piece of code, and verify that it behaved one way with the MoneyPlus license, and another way without it. There was no easy way for our tests to control the state of the licenses.
One idea we had was to separate the tests into different suites, and do the setup work for a given license state before running the suite. But we ruled out that idea; since there were more than a dozen different licenses, clients could pick and choose among them. That meant a very large number of possible license combinations. There had to be an easier way.
Another idea was to use the PowerMock tool to mock and thereby control the return value of a given license-check method. This worked, but tests that used PowerMock were inevitably slower and more complex. Assuming that simpler technique would be more likely to be adopted by the team, could we find such a simpler way?
Another idea was to create a test utility that would allow the licenses to be set from within an individual test. Since we knew that the license state was held as private static boolean fields in one of the core classes, we could write a utility method that would use Java's Reflection to set the value.
We ran with this idea, and created a utility method. It would live in our Test infrastructure, which would not be shipped with the production code and risk opening a back-door to the licenses, yet would allow our tests to set the licenses as needed:
public static void setLicense(String licenseFieldName, boolean isLicensed)
Since then, this method has been used it many times in tests. But there is at least one design flaw: To set the license, one parameter needs to be a String holding the name of the private field related to the given license.
This means that, to use such a test, I need to know and remember that secret name, or else take the time to look it up.
And, as significantly, by using it, the tests become more fragile: if ever that private field is renamed, for example, many tests would break, and create a tedious effort to fix them all.
Going back to the drawing board, we added a license enumeration to the test harness. This enumeration made it easier to set the licenses in our tests. No need to remember field names, and the tests become less fragile: if ever the field name changes, the enumeration is the only place that will need to be changed in our test framework.
The enumeration is UnitTest.AppLicense and it has an item for each of the licenses handled by the application:
UnitTestUtilities.setLicense(AppLicense license, boolean isLicensed);