Private beta of ObMimic for out-of-container servlet testing

30 05 2008

Updated Feb 2018: OpenBrace Limited has closed down, and its ObMimic product is no longer supported.

The ObMimic library for out-of-container servlet testing is now being made available to a small number of users as a private “beta” release, in advance of a more public beta.

We’re ready for a few more people to start trying it out, so if you’re interested just let me know – either via this blog’s “contact me” page or via my company e-mail address of mike-at-openbrace-dot-com.

In outline, ObMimic provides a comprehensive set of fully-configurable test doubles for the Servlet API, so that you can use normal “plain java” tools and techniques to test servlets, filters, listeners and any other code that depends on the Servlet API. We call these test doubles “mimics”, because they “mimic” the behaviour of the real object.

We see this as the ultimate set of “test doubles” for this specific API: a set of plain Java objects that completely and accurately mimic the behaviour of the “real” Servlet API objects, whilst being fully configurable and inspectable and with additional instrumentation to support both “state-based” and “interaction-based” testing.

If you find servlet code harder to test than plain Java, ObMimic might be just what you’re looking for.

With ObMimic, you can create instances of any Servlet API interface or abstract class using plain no-argument constructors; configure and inspect all relevant details of their internal state as necessary; and pass them into your code wherever Servlet API objects are needed. This makes it easy to do detailed testing of servlets, filters, listeners and other code that depends on the Servlet API, without needing a servlet container and without any of the complexities and overheads of packaging, deployment, restarts/reloads, networking etc.

ObMimic includes facilities for:

  • Setting values that are “read-only” in the Servlet API (including full programmatic control over “deployment descriptor” values and other values that are normally fixed during packaging/deployment, or that have fixed values in each servlet container).
  • Examining values that are normally “write-only” in the Servlet API (such as a response’s body content).
  • Optionally recording and retrieving details of the Servlet API calls made to each object (with ability to turn this on and off on individual objects).
  • Controlling which version of the Servlet API is simulated, with versions 2.3, 2.4 and 2.5 currently supported (for example, you can programmatically repeat a test using different Servlet API versions).
  • Detecting and reporting any calls to Servlet API methods whose handling isn’t strictly defined by the API (e.g. passing null arguments to Servlet API methods whose Javadoc doesn’t specify whether nulls are permitted or how they are handled).
  • Controlling the simulation of container-specific behaviour (i.e. where the Servlet API allows variations or leaves this open).
  • Explicitly forcing Servlet API methods to throw a checked exception (e.g. so that you can test any code that handles such exceptions).
  • Handling JNDI look-ups using a built-in, in-memory JNDI simulation.

There are no dependencies on any particular testing framework or third-party libraries (other than Java SE 5 or higher and the Servlet API itself), so you can freely use ObMimic with JUnit, TestNG or any other testing framework or tool.

In contrast to traditional “mock” or “stub” objects, ObMimic provides complete, ready-made implementations of the Servlet API interfaces and abstract classes as defined by their Javadoc. As a result, your tests don’t have to depend on your own assumptions about the Servlet API’s behaviour, and both state-based and interaction-based tests can be supported. ObMimic can even handle complex sequences of Servlet API calls, such as for session-handling, request dispatching, incorporation of “POST” body content into request parameters, notification to listeners, and other such complex interactions between Servlet API objects. It can thus be used not only for testing individual components in isolation, but also for testing more complete paths through your code and third-party libraries.

With the appropriate configuration, it’s even possible to test code that uses other frameworks on top of the Servlet API. For example, we’ve been able to use ObMimic to test “Struts 1” code, and to run ZeroTurnaround’s JspWeaver on top of ObMimic to provide out-of-container testing of JSPs (as documented previously).

As a somewhat arbitrary example, the following code illustrates a very simple use of ObMimic to test a servlet (just to show the basics of how Servlet API objects can be created, configured and used):

import com.openbrace.obmimic.mimic.servlet.http.HttpServletRequestMimic;
import com.openbrace.obmimic.mimic.servlet.http.HttpServletResponseMimic;
import com.openbrace.obmimic.mimic.servlet.ServletConfigMimic;
import javax.servlet.Servlet;
import javax.servlet.ServletException;


/* Create a request and configure it as needed by the test. */    
HttpServletRequestMimic request = new HttpServletRequestMimic();
request.getMimicState().getRequestParameters().set("name", "foo");
request.getMimicState().getAttributes().set("bar", 123);
... further request set-up as desired ...

/* Create a response. */
HttpServletResponseMimic response = new HttpServletResponseMimic();

 * Create and initialize the servlet to be tested (assumed to be a
 * class called "MyHttpServlet"), using a dummy/minimal 
 * ServletConfig.
Servlet myServlet = new MyHttpServlet();
try {
    myServlet.init(new ServletConfigMimic());
} catch (ServletException e) {
    ... report that test failed with unexpected ServletException ...

/* Invoke the servlet to process the request and response. */
try {
    myServlet.service(request, response);
} catch (ServletException e) {
    ... report that test failed with unexpected ServletException ...
} catch (IOException e) {
    ... report that test failed with unexpected IOException ...

 * Retrieve the response's resulting status code and body content,
 * as examples of how the resulting state of the relevant mimic 
 * instances can be examined.
int statusCode 
    = response.getMimicState().getEffectiveHttpStatusCode();
String bodyContent
    = response.getMimicState().getBodyContentAsString();
... then check them as appropriate for the test ...

For further examples and details, refer to the previous posts “First experiments with out-of-container testing of Servlet code using ObMimic” part 1 and part 2, “Out-of-container JSP testing with ObMimic and JspWeaver”, and the related post “Mocking an API should be somebody else’s problem”.

There are also more extensive examples in ObMimic’s documentation.

ObMimic isn’t open-source, but it will have a zero-cost version (full API coverage but a few overall features disabled, such as the ability to configure the Servlet API version, control over how incorrect/ambiguous API calls are handled, and recording of API calls). There will also be a low-cost per-user “Professional” version with full functionality, and an “Enterprise” version that includes all of ObMimic’s source-code and internal tests (with an Ant build script) as well as a licence for up to 200 users.

At the moment there’s no web-site, discussion forums or bug-reporting mechanisms (all still being prepared), but ObMimic already comes with full documentation including both short and detailed “getting started” guides, “how to”s with example code, and extensive Javadoc – and for this private beta I’m providing direct support by e-mail.

Anyway, if you’d like to try out ObMimic, or have any questions or comments, or would like to be informed when there’s a more public release, just let me know via the “contact me” page or by e-mail.

Out-of-container JSP testing with ObMimic and JspWeaver

19 02 2008

Updated Feb 2018: OpenBrace Limited has closed down, and its ObMimic product is no longer supported.

Updated May 2013: JspWeaver seems to no longer exist.


I’ve been experimenting with the use of ZeroTurnaround’s JspWeaver tool on top of my own ObMimic library so as to provide out-of-container testing of JSPs. It’s been relatively straightforward so far, though I’ve only tried fairly simple JSPs.

JspWeaver from ZeroTurnaround aims to speed up the development of JSP code by using on-the-fly interpretation of JSP code instead of the usual translate-and-compile each time it is changed. As I understand it, the argument is that when you’re repeatedly editing and trying out a JSP, the translate-and-compile delay for each run interrupts your flow and they all mount up over time. So even where the delays are acceptable, it still helps and saves time overall if it can be made faster. I imagine some people will find this a boon whilst others won’t see much point, depending on what delays they are seeing from JSP translation and compilation. It’s a commercial product but only $49 per seat (and with volume discounts), so the price shouldn’t be an issue if it’s helpful to you.

Anyway, what interested me about this was the possibility of combining it with my own “ObMimic” library. This is a library of test-doubles for the Servlet API so as to support out-of-container testing of servlets and other code that depends on Servlet API objects. It’s not yet released, but approaching beta (documentation still being worked on). ObMimic’s test-doubles, which I call “mimics”, provide complete and accurate POJO simulations of all of the Servlet API’s interfaces and abstract classes. This includes full support for the various complex interactions between Servlet API classes (e.g. “forwards” and “includes”, session handling etc), with all their little quirks and oddities. The mimics are also fully configurable, so for example you can set up a ServletContext instance to use specified servlet mappings, static resources etc.

This allows you to use normal JUnit/TestNG tests and plain Java code to do detailed testing of servlets, filters, and any other code that depends on the Servlet API, without having to deploy or run in a servlet container, and with full ability to programmatically configure and inspect the Servlet API objects.

However, whilst ObMimic can be used for out-of-container testing of servlet code, it doesn’t currently have any explicit support for JSPs. I intend to add that in future, but for the time being you can’t use ObMimic to test JSP code. In contrast, JspWeaver can process JSP files but needs to be run in a servlet container as it depends on the Servlet API.

When I heard about JspWeaver, it seemed natural to try running it on top of ObMimic’s simulation of the Servlet API. With ObMimic for out-of-container simulation of the Servlet API, and JspWeaver for JSP processing on top of the Servlet API, the two together ought to allow out-of-container testing of JSP code.

So far I’ve only tried this on some simple JSPs that use a variety of JSP features, but it has all been fairly straightforward and seems to work quite well.

Basic Approach and Example Code

The basic approach to running JspWeaver on top of ObMimic is to configure an ObMimic “ServletContextMimic” in much the same way as one would configure a real web-application to use JspWeaver. This involves defining the JspWeaver servlet and mapping all JSP paths to it, and making the relevant JSP files and other resources available at the appropriate context-relative paths within the servlet context.

In addition, JspWeaver also requires its licence file to be available at a context-relative path of /WEB-INF/lib, and also issues a warning if it can’t find a web.xml file at /WEB-INF/web.xml (though even a dummy web.xml file with minimal content proved sufficient for these tests).

Here’s some example code to illustrate one way to configure such a ServletContextMimic to use JspWeaver:

import com.openbrace.obmimic.mimic.servlet.ServletContextMimic;
import com.openbrace.obmimic.state.servlet.ServletContextState;
import com.openbrace.obmimic.substate.servlet.WebAppConfig;
import com.openbrace.obmimic.substate.servlet.WebAppResources;
import com.openbrace.obmimic.substate.servlet.InitParameters;
import com.openbrace.obmimic.substate.servlet.ServletDefinition;
import com.openbrace.obmimic.substate.servlet.ServletMapping;
import com.openbrace.obmimic.lifecycle.servlet.ServletContextMimicManager;
import com.zeroturnaround.jspweaver.JspInterpretingServlet;


// Create ServletContextMimic and retrieve its "mimicState",
// (which represents its internal state) and relevant
// subcomponents of its mimicState.
ServletContextMimic context = new ServletContextMimic();
ServletContextState contextState 
    = context.getMimicState();
WebAppConfig webAppConfig 
    = contextState.getWebAppConfig();
WebAppResources webAppResources 
    = contextState.getWebAppResources();

// Add a servlet definition for the JspWeaver servlet.
String servletName = "jspWeaverServlet";
String servletClass = JspInterpretingServlet.class.getName();
InitParameters initParams = null;
int loadOnStartup = 1;
    new ServletDefinition(servletName, servletClass,
        initParams, loadOnStartup));

// Add a servlet mapping for ".jsp" paths.
    new ServletMapping(servletName, "*.jsp"));

// Use the contents of a specified directory as the 
// servlet context's "resources".
String webAppRoot = ...path to root of web-application files...

// Explicitly "initialize" the servlet context so as to force
// creation and initialization of its servlets, filters etc
// (otherwise ObMimic does this automatically when the first 
// Servlet API call occurs, but for this example it's shown as
// being done explicitly at this point).
new ServletContextMimicManager(context).initializeContext();


Note that:

  • This example code configures the servlet context to obtain its context-relative resources from a real directory structure that contains the web-application’s files (i.e. corresponding to an expanded “war” archive, or at least the subset of its files that are actually needed for the particular tests being carried out). This needs to include the JspWeaver licence file in its /WEB-INF/lib, and to avoid JspWeaver warnings it needs to include at least a “dummy” web.xml in its /WEB-INF directory.
  • Although this example shows the use of a whole directory structure to provide the web-application’s resources, ObMimic also lets you configure individual context-relative paths to use a specific resource. The resource itself can be provided by a file, or a classpath resource, or an in-memory byte array, or the content obtained from a URL. So you could, for example, provide the JspWeaver licence file and web.xml as classpath resources held alongside the test case’s class files. Similarly you could use the “real” web-application directory structure but then set particular context-relative paths to have different content for testing purposes.
  • A dummy web.xml proved sufficient to prevent “web.xml not found” warnings from JspWeaver, but more complex situations might require appropriate content within the web.xml – basically this would be necessary for any values that JspWeaver actually reads directly from the web.xml as opposed to accessing via the Servlet API (e.g. explicit TLD declarations). Of course, if you’re using a “real” web-application directory structure, it probably already has a suitable web.xml file.

The above approach is very general, in the sense that it configures the servlet context to use JspWeaver for any JSP files. As an alternative, ObMimic could also be used to explicitly construct and initialize an instance of the JspWeaver servlet which could then be used directly. That might be simpler when testing a single self-contained JSP, but it wouldn’t handle any request dispatching to other JSP files, or testing of servlets that “forward” to JSPs, or when also including some other framework that applies filters or servlets before dispatching to the JSP page (e.g. JSF).

With a suitably-configured ServletContextMimic, JSPs can then be tested by using normal ObMimic facilities to create and configure a suitable HttpServletRequest and HttpServletResponse and using ObMimic’s request-dispatching facilities to process them. This can include testing of JSPs in combination with servlets, filters etc (for example, testing a Servlet that “forwards” to a JSP).

For example, if you simply want to use the JspWeaver servlet to process a specific JSP this can be done by retrieving the JspWeaver servlet from the ServletContextMimic and directly invoking its “service” method with a request that has the appropriate URL. Alternatively (or if a more complex combination of servlets, filters, JSPs etc is to be tested) you can use the normal Servlet API facilities to obtain a RequestDispatcher from the servlet context for the relevant path and then “forward” to it. More generally, you can also use various ObMimic facilities to construct and invoke the appropriate FilterChain for a given context-relative path.

To illustrate this, here’s some example code that shows one way to configure a request with a URL for a particular context-relative path, and then directly invokes the JspWeaver servlet to process it (assuming that “context” is a ServletContextMimic configured as shown above):

import com.openbrace.obmimic.mimic.servlet.http.HttpServletRequestMimic;
import com.openbrace.obmimic.mimic.servlet.http.HttpServletResponseMimic;
import javax.servlet.Servlet;
import javax.servlet.ServletException;


// Create a request and response, with the 
// ServletContextMimic as their servlet context.
HttpServletRequestMimic request 
    = new HttpServletRequestMimic();
HttpServletResponseMimic response 
    = new HttpServletResponseMimic();

// Configure the request, including all of its URL-related
// properties (request URL, context path, servlet path, path 
// info etc).
String contextPath = context.getMimicState().getContextPath();
String contextRelativePath = "/pages/jstl.jsp";
String serverRelativePath = contextPath + contextRelativePath;

... further configuration of the request as desired ...

// Retrieve the JspWeaver servlet from the servlet context
// and invoke it.
Servlet target = context.getMimicState().getServlets().get(
try {
    target.service(request, response);
} catch (ServletException e) {
    ... failed with a ServletException ...
} catch (IOException e) {
    ... failed with an IOException ...

... examine the response, context, HTTP session etc
    as desired ...


On completion, the normal ObMimic facilities can be used to examine all relevant details of the response, servlet context, HTTP session etc. For example, you can use “response.getMimicState().getBodyContentAsString()” to retrieve the response’s body content.

General Findings

I’ve successfully used the combination of JspWeaver and ObMimic to test simple examples of:

  • A very simple JSP with plain-text content.
  • A JSP with embedded JSP declarations, scriptlets and expressions.
  • A JSP with static and dynamic includes.
  • A JSP that uses a custom tag file.
  • A JSP that makes use of JSTL tags and EL expressions.
  • A Servlet that forwards to a JSP.

In theory this should be able to cope with any JSP code that JspWeaver is able to successfully interpret.

Performance is potentially a concern. JspWeaver’s interpretation of JSP is faster than translate+compile when repeatedly editing and manually viewing a page, and in general its use for “out-of-container” testing should be faster and more convenient than HTTP-based “in-container” tests. But it’s bound to be slower than using a pre-compiled page when repeatedly running many tests against an unchanging page. First impressions are that the performance is good enough for reasonable use, but I wouldn’t want to have huge numbers of tests for each page as part of my frequent/detailed test runs. Overall it looks like a reasonable approach until a suitable translate+compile out-of-container solution is available – at which point one might still want a choice between the two approaches (e.g. use “translate+compile” when running suites of tests in build scripts, but “interpret” when manually running individual tests within an IDE).

Detailed Findings

Whilst this has all basically worked without undue difficulty, there have been a few minor issues and other findings:

  • The “bsh” parser used by JspWeaver to parse JSP pages doesn’t appear to cope with generics yet. Any use of generics within JSP scripting seems to result in parsing exceptions, presumably because it is thrown by the presence of “<” and “>” characters (to the extent that they result in rather misleading or unhelpful error messages). Maybe this just isn’t allowed in JSP scripting, or just depends on whether the JSP translator/interpreter copes with generics. But off-hand I’m not aware of any general JSP restriction on this.
  • More generally, syntax errors in the JSP can be somewhat hard to find from the error messages produced by JspWeaver when trying to interpret the files (especially when using static/dynamic includes, tag files etc).
  • There might be issues over how best to integrate this into one’s overall build process. For example, if the JSP code needs tag libraries that are built by the same project, these would need to be already built and jar’d before testing the JSPs, even though normally you’d probably run all such tests before “packaging” the libraries. This shouldn’t be a show-stopper, but might need some adjustments and re-thinking of the order in which components are built and tested, and which tests are run at which point.
  • When locating tag files, JspWeaver uses the ServletContext.getResourcePaths method but passes it subdirectory paths without a trailing “/” (for example, “/WEB-INF/tags” rather than “/WEB-INF/tags/”). Although this will generally work, the Javadoc for this Servlet API method isn’t entirely explicit about the form of such paths, and arguably implies that they should end with a trailing “/”, as do all of its examples. By default ObMimic therefore rejects calls to this method for paths that don’t have a trailing “/”, to highlight the possibly suspect use of the Servlet API (i.e. behaviour might vary depending on the servlet container implementation). ObMimic therefore has to be configuring to ignore this and permit such calls (for which its behaviour is then to treat the given path as if it did have the trailing “/”).
  • It can be tricky getting the classpath correct for all the libraries needed for JSP processing. The libraries needed for JSTL and EL support have changed as JSTL and EL have evolved, EL has been moved into the JSP API, and there are incompatible versions of libraries knocking around. It’s just the usual “jar hell”. In particular, I’ve not yet found a combination of Glassfish jars that work with its “javaee.jar” (I always seem to get a java.lang.AbstractMethodError for javax.servlet.jsp.PageContext.getELContext(ELContext). The only working solution that I’ve found so far is to use the Tomcat 6 jstl.jar and standard.jar. This is presumably solvable with a bit more research and tinkering, unless it’s something in JspWeaver that specifically depends on “old” versions of these libraries, but I haven’t followed this up yet.
  • More generally, it can be hard to work out which jars are needed when any are missing, as the error messages and the nature of the failures aren’t always as useful as they could be (e.g. console shows an exception/stack trace from some point in the internal JspWeaver processing of the page, but the JSP completes successfully with truncated output but no exception).

Next Steps

I haven’t yet tried any particularly complex JSPs or “complete” real-life applications, but would like to get around to this if I ever find time. I’ve also got an existing Struts 1 example where the servlet processing is tested using ObMimic but with its JSPs “dummied out”, and at some point I’d like to go back to this and see if its tests can now properly handle the JSP invocations.

I’d also like to see if JSF pages can be tested this way. I’ve had a brief dabble with this, using Sun’s JSF 1.2 implementation as provided by Glassfish. I’ve got as far as seeing JSF “initialize” itself OK (once I’d realized that it needs an implementation-specific context listener, which for the Sun implementation is provided by class com.sun.faces.ConfigureListener). But JspWeaver doesn’t seem to be finding the TLDs for the JSF tags, even if the tag libraries are present in both the classpath and within the web-application directory structure. Providing the TLDs as separate files and explicitly specifying these in the web.xml does result in them being found, but then complains that the actual tag classes don’t have the relevant “setter” methods for the tag’s attributes. I’d guess this is a classpath problem (maybe related to which JSTL/EL libraries are used), or otherwise depends on exactly how JspWeaver searches for TLDs and tag libraries (or maybe what classloader it uses). But I haven’t yet got round to digging any deeper. Also, whilst I’d like to see tests of JSF pages working, I’m not even sure what testing JSF pages like this would mean, given their rather opaque URLs and “behind-the-scenes” processing (e.g. what request details are needed to test a particular feature of the page, and what would you examine to see if it worked correctly?).

Another thought is that ObMimic could be made to automatically detect the presence of JspWeaver and configure itself to automatically use it for all JSP files if not otherwise configured. Maybe this could be a configurable option for which servlet to use for processing JSPs, with the default being a servlet that checks for the presence of JspWeaver and delegates to it if present. That would let you automatically use JspWeaver if you have it, whilst also allowing for other JSP processors to be introduced.

%d bloggers like this: