Mocking an API should be somebody else’s problem

11 03 2008

In an interview about his book with Cédric Beust, Next Generation Java Testing: TestNT and Advanced Concepts, Hani Suleiman says:

“I’m fairly strongly against the use of mocks for Java EE constructs. These APIs are often complicated and come with whole swathes of tests to verify compliance. End users never see what a pain in the ass it actually is to certify a product as compatible to a given EE API. Mock implementations on the other hand are most certainly not certified. The danger of using them is that over time, people start implementing more and more of the API, its more code that can go wrong, and more code that’s written just for the sake of making your tests look good. Increasingly, it becomes more and more divorced from the real implementation.”

You might think that I would disagree with that, in view of my current work on an ObMimic library of test-doubles for the Servlet API (in the broad xunitpatterns.com meaning of “test-double”).

But actually I strongly agree with it, and it’s one of the motivations behind ObMimic.

Yes, if you write your own stubs or mocks or use a general-purpose “mocking” tool, it can be extremely difficult to accurately simulate or predict the real behaviour of an API, and over time you’re likely to encounter more and more of the API. You can also find yourself needing more and more scaffolding and instrumentation to serve the needs of your tests. So it can be problematic and uneconomical to do this as part of an individual application’s testing. Even when it remains simple and doesn’t grow over time, it’s still additional thought and effort for each individual application, and very easy to get wrong. Whilst it’s all theoretically possible, and can seem very simple at the start, the long-term economics of it don’t look good.

But it looks rather different if the necessary facilities are all provided for you by a specialist library developed by someone else. Then it’s up to them to figure out all the quirks of the API and provide complete and accurate coverage, and all you have to do in individual applications is to use it.

This is much like the need for a “container” or other such API implementations in the first place. For example, given the Servlet API, it’s neither feasible nor economic for each application to implement its own servlet container, but it’s perfectly reasonable to have separate, dedicated projects that produce servlet containers that everybody else just uses.

My own opinion is that the same goes for test-doubles for APIs such as the Servlet API: it’s not worth everyone doing this half-heartedly themselves, but it is worth somebody doing it well and producing a comprehensive, high-quality library that everyone else can just use.

Of course, this only works if the resulting library is complete enough and of good enough quality for you to be able to rely on it. This points to the kind of criteria on which to judge such suites of test-doubles:

  • How complete is the API coverage?
  • How accurate is the API simulation?
  • How configurable are the test-doubles?
  • How examinable are the test-doubles?
  • How well documented is it?
  • How easy is it to use?
  • What extra features to support testing does it provide? (e.g. strict or configurable validation of API calls, tracking the API calls made, configurable for different versions of the API etc).
  • What dependencies does it have? (e.g. is it limited to only being used with specific tools or frameworks or only in certain scenarios, or is it more generally applicable).

Unfortunately, we don’t seem to have many API-specific libraries of test-doubles at the moment, and in my own limited experience those that we do have aren’t generally good enough.

That’s understandable, as it’s a huge amount of work to do this well for any substantial API that wasn’t written with testing in mind. Especially for APIs as complex, imperfect and subject to change as some of the older Java EE APIs.

Apart from my own ObMimic library for the Servlet API, I’m aware of some other attempts at doing this for the Servlet API, such as HttpUnit’s ServletUnit and the Spring framework’s org.springframework.mock.web package. However, in general these tend to be somewhat incomplete, inadequately documented, and lacking in configurability and test-instrumentation. Some are also outdated or defunct, limited to a particular web-app framework, or are a very minor and secondary component within a broader product that has a rather different purpose and priorities (and is thus unlikely to get much attention or maintenance).

In terms of other APIs, I’m aware of MockEJB for EJB, Mock Javamail for JavaMail, and a few such libraries for JNDI. There’s also a discussion of this issue in James Strachan’s blog article “Mocking out protocols and services is damn useful” (though some of the solutions mentioned are lightweight “real” implementations rather than test-doubles as such). But that seems to be about it.

Does anyone know any more? Or have any general views on the quality and suitability of any of these libraries?

As an ideal, I’d like to see the provision of a suitable library of any necessary test-doubles as a mandatory part of any Java API, in the same way that the JCP demands not just a specification but also a reference implementation and a compatibility kit.

That sounds like a big extra burden on API development. However, most new APIs ought to be designed so that test-doubles aren’t generally necessary in the first place, or can be relatively simple. For example, EJB 3.0 has far less need for this sort of thing than EJB 2.0, due to being more “POJO”-based. As another example, I believe the new JSR 310 Date and Time API is being designed so that the API itself will allow you to “stop” or control the time for testing purposes (for example, see slides 89-91 of Stephen Colebourne’s Javopolis 2007 presentation on JSR 310).

More generally, if this was always tackled as an intrinsic part of each API’s design then it ought to result in APIs that are more amenable to testing, and developing any test-doubles that are still necessary for such an API ought to be far simpler than trying to provide this retrospectively for an API that has ignored this issue. Having any necessary test-doubles should also be helpful in the development and testing of real implementations. In any case, this ought to be a more efficient division of labour than leaving everybody to hack their own way around the absence of such facilities.

As an absolute ideal, I’d want the resulting libraries of API-specific test-doubles to all take a similar form, with common features and facilities, terminology, naming conventions, usage patterns etc. But that’s probably getting into the realm of fantasy.

Advertisements




Out-of-container JSP testing with ObMimic and JspWeaver

19 02 2008

Updated Feb 2018: OpenBrace Limited has closed down, and its ObMimic product is no longer supported.

Updated May 2013: JspWeaver seems to no longer exist.

Background

I’ve been experimenting with the use of ZeroTurnaround’s JspWeaver tool on top of my own ObMimic library so as to provide out-of-container testing of JSPs. It’s been relatively straightforward so far, though I’ve only tried fairly simple JSPs.

JspWeaver from ZeroTurnaround aims to speed up the development of JSP code by using on-the-fly interpretation of JSP code instead of the usual translate-and-compile each time it is changed. As I understand it, the argument is that when you’re repeatedly editing and trying out a JSP, the translate-and-compile delay for each run interrupts your flow and they all mount up over time. So even where the delays are acceptable, it still helps and saves time overall if it can be made faster. I imagine some people will find this a boon whilst others won’t see much point, depending on what delays they are seeing from JSP translation and compilation. It’s a commercial product but only $49 per seat (and with volume discounts), so the price shouldn’t be an issue if it’s helpful to you.

Anyway, what interested me about this was the possibility of combining it with my own “ObMimic” library. This is a library of test-doubles for the Servlet API so as to support out-of-container testing of servlets and other code that depends on Servlet API objects. It’s not yet released, but approaching beta (documentation still being worked on). ObMimic’s test-doubles, which I call “mimics”, provide complete and accurate POJO simulations of all of the Servlet API’s interfaces and abstract classes. This includes full support for the various complex interactions between Servlet API classes (e.g. “forwards” and “includes”, session handling etc), with all their little quirks and oddities. The mimics are also fully configurable, so for example you can set up a ServletContext instance to use specified servlet mappings, static resources etc.

This allows you to use normal JUnit/TestNG tests and plain Java code to do detailed testing of servlets, filters, and any other code that depends on the Servlet API, without having to deploy or run in a servlet container, and with full ability to programmatically configure and inspect the Servlet API objects.

However, whilst ObMimic can be used for out-of-container testing of servlet code, it doesn’t currently have any explicit support for JSPs. I intend to add that in future, but for the time being you can’t use ObMimic to test JSP code. In contrast, JspWeaver can process JSP files but needs to be run in a servlet container as it depends on the Servlet API.

When I heard about JspWeaver, it seemed natural to try running it on top of ObMimic’s simulation of the Servlet API. With ObMimic for out-of-container simulation of the Servlet API, and JspWeaver for JSP processing on top of the Servlet API, the two together ought to allow out-of-container testing of JSP code.

So far I’ve only tried this on some simple JSPs that use a variety of JSP features, but it has all been fairly straightforward and seems to work quite well.

Basic Approach and Example Code

The basic approach to running JspWeaver on top of ObMimic is to configure an ObMimic “ServletContextMimic” in much the same way as one would configure a real web-application to use JspWeaver. This involves defining the JspWeaver servlet and mapping all JSP paths to it, and making the relevant JSP files and other resources available at the appropriate context-relative paths within the servlet context.

In addition, JspWeaver also requires its licence file to be available at a context-relative path of /WEB-INF/lib, and also issues a warning if it can’t find a web.xml file at /WEB-INF/web.xml (though even a dummy web.xml file with minimal content proved sufficient for these tests).

Here’s some example code to illustrate one way to configure such a ServletContextMimic to use JspWeaver:

import com.openbrace.obmimic.mimic.servlet.ServletContextMimic;
import com.openbrace.obmimic.state.servlet.ServletContextState;
import com.openbrace.obmimic.substate.servlet.WebAppConfig;
import com.openbrace.obmimic.substate.servlet.WebAppResources;
import com.openbrace.obmimic.substate.servlet.InitParameters;
import com.openbrace.obmimic.substate.servlet.ServletDefinition;
import com.openbrace.obmimic.substate.servlet.ServletMapping;
import com.openbrace.obmimic.lifecycle.servlet.ServletContextMimicManager;
import com.zeroturnaround.jspweaver.JspInterpretingServlet;

...

// Create ServletContextMimic and retrieve its "mimicState",
// (which represents its internal state) and relevant
// subcomponents of its mimicState.
ServletContextMimic context = new ServletContextMimic();
ServletContextState contextState 
    = context.getMimicState();
WebAppConfig webAppConfig 
    = contextState.getWebAppConfig();
WebAppResources webAppResources 
    = contextState.getWebAppResources();

// Add a servlet definition for the JspWeaver servlet.
String servletName = "jspWeaverServlet";
String servletClass = JspInterpretingServlet.class.getName();
InitParameters initParams = null;
int loadOnStartup = 1;
webAppConfig.getServletDefinitions().add(
    new ServletDefinition(servletName, servletClass,
        initParams, loadOnStartup));

// Add a servlet mapping for ".jsp" paths.
webAppConfig.getServletMappings().add(
    new ServletMapping(servletName, "*.jsp"));

// Use the contents of a specified directory as the 
// servlet context's "resources".
String webAppRoot = ...path to root of web-application files...
webAppResources.loadResourcesFromDirectory(webAppRoot);

// Explicitly "initialize" the servlet context so as to force
// creation and initialization of its servlets, filters etc
// (otherwise ObMimic does this automatically when the first 
// Servlet API call occurs, but for this example it's shown as
// being done explicitly at this point).
new ServletContextMimicManager(context).initializeContext();

...

Note that:

  • This example code configures the servlet context to obtain its context-relative resources from a real directory structure that contains the web-application’s files (i.e. corresponding to an expanded “war” archive, or at least the subset of its files that are actually needed for the particular tests being carried out). This needs to include the JspWeaver licence file in its /WEB-INF/lib, and to avoid JspWeaver warnings it needs to include at least a “dummy” web.xml in its /WEB-INF directory.
  • Although this example shows the use of a whole directory structure to provide the web-application’s resources, ObMimic also lets you configure individual context-relative paths to use a specific resource. The resource itself can be provided by a file, or a classpath resource, or an in-memory byte array, or the content obtained from a URL. So you could, for example, provide the JspWeaver licence file and web.xml as classpath resources held alongside the test case’s class files. Similarly you could use the “real” web-application directory structure but then set particular context-relative paths to have different content for testing purposes.
  • A dummy web.xml proved sufficient to prevent “web.xml not found” warnings from JspWeaver, but more complex situations might require appropriate content within the web.xml – basically this would be necessary for any values that JspWeaver actually reads directly from the web.xml as opposed to accessing via the Servlet API (e.g. explicit TLD declarations). Of course, if you’re using a “real” web-application directory structure, it probably already has a suitable web.xml file.

The above approach is very general, in the sense that it configures the servlet context to use JspWeaver for any JSP files. As an alternative, ObMimic could also be used to explicitly construct and initialize an instance of the JspWeaver servlet which could then be used directly. That might be simpler when testing a single self-contained JSP, but it wouldn’t handle any request dispatching to other JSP files, or testing of servlets that “forward” to JSPs, or when also including some other framework that applies filters or servlets before dispatching to the JSP page (e.g. JSF).

With a suitably-configured ServletContextMimic, JSPs can then be tested by using normal ObMimic facilities to create and configure a suitable HttpServletRequest and HttpServletResponse and using ObMimic’s request-dispatching facilities to process them. This can include testing of JSPs in combination with servlets, filters etc (for example, testing a Servlet that “forwards” to a JSP).

For example, if you simply want to use the JspWeaver servlet to process a specific JSP this can be done by retrieving the JspWeaver servlet from the ServletContextMimic and directly invoking its “service” method with a request that has the appropriate URL. Alternatively (or if a more complex combination of servlets, filters, JSPs etc is to be tested) you can use the normal Servlet API facilities to obtain a RequestDispatcher from the servlet context for the relevant path and then “forward” to it. More generally, you can also use various ObMimic facilities to construct and invoke the appropriate FilterChain for a given context-relative path.

To illustrate this, here’s some example code that shows one way to configure a request with a URL for a particular context-relative path, and then directly invokes the JspWeaver servlet to process it (assuming that “context” is a ServletContextMimic configured as shown above):

import com.openbrace.obmimic.mimic.servlet.http.HttpServletRequestMimic;
import com.openbrace.obmimic.mimic.servlet.http.HttpServletResponseMimic;
import javax.servlet.Servlet;
import javax.servlet.ServletException;
import java.io.IOException;

...

// Create a request and response, with the 
// ServletContextMimic as their servlet context.
HttpServletRequestMimic request 
    = new HttpServletRequestMimic();
HttpServletResponseMimic response 
    = new HttpServletResponseMimic();
request.getMimicState().setServletContext(context);
response.getMimicState().setServletContext(context);

// Configure the request, including all of its URL-related
// properties (request URL, context path, servlet path, path 
// info etc).
String contextPath = context.getMimicState().getContextPath();
String contextRelativePath = "/pages/jstl.jsp";
String serverRelativePath = contextPath + contextRelativePath;
request.getMimicState().populateRelativeURIFromUnencodedURI(
    serverRelativePath);

... further configuration of the request as desired ...

// Retrieve the JspWeaver servlet from the servlet context
// and invoke it.
Servlet target = context.getMimicState().getServlets().get(
    "jspWeaverServlet");
try {
    target.service(request, response);
} catch (ServletException e) {
    ... failed with a ServletException ...
} catch (IOException e) {
    ... failed with an IOException ...
}

... examine the response, context, HTTP session etc
    as desired ...

...

On completion, the normal ObMimic facilities can be used to examine all relevant details of the response, servlet context, HTTP session etc. For example, you can use “response.getMimicState().getBodyContentAsString()” to retrieve the response’s body content.

General Findings

I’ve successfully used the combination of JspWeaver and ObMimic to test simple examples of:

  • A very simple JSP with plain-text content.
  • A JSP with embedded JSP declarations, scriptlets and expressions.
  • A JSP with static and dynamic includes.
  • A JSP that uses a custom tag file.
  • A JSP that makes use of JSTL tags and EL expressions.
  • A Servlet that forwards to a JSP.

In theory this should be able to cope with any JSP code that JspWeaver is able to successfully interpret.

Performance is potentially a concern. JspWeaver’s interpretation of JSP is faster than translate+compile when repeatedly editing and manually viewing a page, and in general its use for “out-of-container” testing should be faster and more convenient than HTTP-based “in-container” tests. But it’s bound to be slower than using a pre-compiled page when repeatedly running many tests against an unchanging page. First impressions are that the performance is good enough for reasonable use, but I wouldn’t want to have huge numbers of tests for each page as part of my frequent/detailed test runs. Overall it looks like a reasonable approach until a suitable translate+compile out-of-container solution is available – at which point one might still want a choice between the two approaches (e.g. use “translate+compile” when running suites of tests in build scripts, but “interpret” when manually running individual tests within an IDE).

Detailed Findings

Whilst this has all basically worked without undue difficulty, there have been a few minor issues and other findings:

  • The “bsh” parser used by JspWeaver to parse JSP pages doesn’t appear to cope with generics yet. Any use of generics within JSP scripting seems to result in parsing exceptions, presumably because it is thrown by the presence of “<” and “>” characters (to the extent that they result in rather misleading or unhelpful error messages). Maybe this just isn’t allowed in JSP scripting, or just depends on whether the JSP translator/interpreter copes with generics. But off-hand I’m not aware of any general JSP restriction on this.
  • More generally, syntax errors in the JSP can be somewhat hard to find from the error messages produced by JspWeaver when trying to interpret the files (especially when using static/dynamic includes, tag files etc).
  • There might be issues over how best to integrate this into one’s overall build process. For example, if the JSP code needs tag libraries that are built by the same project, these would need to be already built and jar’d before testing the JSPs, even though normally you’d probably run all such tests before “packaging” the libraries. This shouldn’t be a show-stopper, but might need some adjustments and re-thinking of the order in which components are built and tested, and which tests are run at which point.
  • When locating tag files, JspWeaver uses the ServletContext.getResourcePaths method but passes it subdirectory paths without a trailing “/” (for example, “/WEB-INF/tags” rather than “/WEB-INF/tags/”). Although this will generally work, the Javadoc for this Servlet API method isn’t entirely explicit about the form of such paths, and arguably implies that they should end with a trailing “/”, as do all of its examples. By default ObMimic therefore rejects calls to this method for paths that don’t have a trailing “/”, to highlight the possibly suspect use of the Servlet API (i.e. behaviour might vary depending on the servlet container implementation). ObMimic therefore has to be configuring to ignore this and permit such calls (for which its behaviour is then to treat the given path as if it did have the trailing “/”).
  • It can be tricky getting the classpath correct for all the libraries needed for JSP processing. The libraries needed for JSTL and EL support have changed as JSTL and EL have evolved, EL has been moved into the JSP API, and there are incompatible versions of libraries knocking around. It’s just the usual “jar hell”. In particular, I’ve not yet found a combination of Glassfish jars that work with its “javaee.jar” (I always seem to get a java.lang.AbstractMethodError for javax.servlet.jsp.PageContext.getELContext(ELContext). The only working solution that I’ve found so far is to use the Tomcat 6 jstl.jar and standard.jar. This is presumably solvable with a bit more research and tinkering, unless it’s something in JspWeaver that specifically depends on “old” versions of these libraries, but I haven’t followed this up yet.
  • More generally, it can be hard to work out which jars are needed when any are missing, as the error messages and the nature of the failures aren’t always as useful as they could be (e.g. console shows an exception/stack trace from some point in the internal JspWeaver processing of the page, but the JSP completes successfully with truncated output but no exception).

Next Steps

I haven’t yet tried any particularly complex JSPs or “complete” real-life applications, but would like to get around to this if I ever find time. I’ve also got an existing Struts 1 example where the servlet processing is tested using ObMimic but with its JSPs “dummied out”, and at some point I’d like to go back to this and see if its tests can now properly handle the JSP invocations.

I’d also like to see if JSF pages can be tested this way. I’ve had a brief dabble with this, using Sun’s JSF 1.2 implementation as provided by Glassfish. I’ve got as far as seeing JSF “initialize” itself OK (once I’d realized that it needs an implementation-specific context listener, which for the Sun implementation is provided by class com.sun.faces.ConfigureListener). But JspWeaver doesn’t seem to be finding the TLDs for the JSF tags, even if the tag libraries are present in both the classpath and within the web-application directory structure. Providing the TLDs as separate files and explicitly specifying these in the web.xml does result in them being found, but then complains that the actual tag classes don’t have the relevant “setter” methods for the tag’s attributes. I’d guess this is a classpath problem (maybe related to which JSTL/EL libraries are used), or otherwise depends on exactly how JspWeaver searches for TLDs and tag libraries (or maybe what classloader it uses). But I haven’t yet got round to digging any deeper. Also, whilst I’d like to see tests of JSF pages working, I’m not even sure what testing JSF pages like this would mean, given their rather opaque URLs and “behind-the-scenes” processing (e.g. what request details are needed to test a particular feature of the page, and what would you examine to see if it worked correctly?).

Another thought is that ObMimic could be made to automatically detect the presence of JspWeaver and configure itself to automatically use it for all JSP files if not otherwise configured. Maybe this could be a configurable option for which servlet to use for processing JSPs, with the default being a servlet that checks for the presence of JspWeaver and delegates to it if present. That would let you automatically use JspWeaver if you have it, whilst also allowing for other JSP processors to be introduced.





ServletException rootCause vs cause

27 11 2007

A recent failed test-case when testing some code against different Servlet API libraries has made me look more closely at how ServletException deals with exception chaining.

I’d pretty much taken this for granted before, but it turns out to be not entirely trivial. So it seems worth spelling out exactly how it works, and what steps you need to take to use it safely.

The underlying problem is that the ServletException class pre-dates the introduction of standardised support for a “cause” within Throwable. As with many such classes from before the introduction of “cause”, ServletException provides its own mechanism for exception chaining – in its own case, a “rootCause” property.

Since the introduction of standard “cause” support in Throwable, ServletException thus has both its own “rootCause” property and the inherited “cause” property. However, the relationship between these two mechanisms is left entirely unspecified, with the Javadoc for ServletException’s constructor and “getRootCause” methods making no mention of whether or how they interact with the inherited “cause” mechanism.

The result is that the interaction between “rootCause” and “cause” may differ between Servlet API implementations, and care may be necessary when populating these properties.

Specifically:

  • ServletException provides constructors that take a “rootCause”, together with a “getRootCause” method that returns it, as well as other constructors that do not take a “rootCause” and for which “getRootCause” returns null.
  • ServletException’s underlying Throwable base class provides constructors that take a “cause” and other constructors that do not, plus an “initCause” method that can be used to populate the cause after construction if and only if it is not already set, together with a “getCause” method that returns the “cause” (if any).
  • ServletException does not specify whether supplying a “rootCause” also populates its “cause” or leaves it unpopulated. Different Servlet API implementations are potentially free to handle this differently.
  • ServletException does not specify whether setting a “cause” via its “initCause” method also sets its “rootCause” (and if so, whether this is only permitted if no “rootCause” was supplied on construction, or whether it can be called even if a “rootCause” is already present). Different Servlet API implementations are potentially free to handle this differently.

So when you’re creating a ServletException with an underlying cause, this leaves you not knowing:

  • Whether to just supply the cause to the ServletException’s constructor as its “rootCause” (on the basis that this should “obviously” take the given value as being both the “rootCause” and the “cause”).
  • Or whether to also call the resulting instance’s “initCause” method so as to ensure that the “cause” is populated properly (on the basis that ServletException doesn’t explicitly say that it populates the “cause”, so there is no reason to think that supplying a “rootCause” has any effect on its “cause”).
  • Or whether to ignore the ServletException “rootCause” constructor and instead just call the inherited “initCause” method (on the basis that in the absence of an existing “rootCause” this can safely be called to set the “cause” and should “obviously” also set the “rootCause”).

If you get this wrong, you can potentially end up with a ServletException that has a “rootCause” but no “cause” (and thus, for example, doesn’t show the underlying cause in its stack trace); or with a call to “initCause” that fails with an IllegalStateException due to the “cause” having already being populated during construction; or with a ServletException that has a “cause” but no “rootCause”.

This isn’t entirely academic, because Servlet API implementations do actually differ in how they handle this. For example:

  • The Servlet API 2.5 reference implementation in Glassfish does automatically populate the “cause” when you supply a “rootCause” (by passing the “rootCause” to the appropriate superclass constructor during the ServletException’s construction).
  • The reference implementations provided by Apache Tomcat for earlier Servlet API versions do not popluate the “cause” when you supply a “rootCause”, and the “cause” remains empty unless and until you populate it yourself. (Theoretically, nothing prevents the use of two entirely different exceptions as the “rootCause” and “cause”, which could get rather confusing, especially if done accidentally).
  • Other implementations and subsequent releases remain free to take either approach.
  • Although I’ve not seen, and can’t really imagine, a ServletException implementation whose “initCause” takes the “rootCause” into account (e.g fails if it is already set but otherwise populates it in addition to the “cause”), this is potentially possible and stranger things have happened.

So as far as I can see, the safest way to ensure that both the “rootCause” and “cause” are populated (without being dependent on the particular implementation of a particular Servlet API library) is to supply the underlying exception to the ServletException constructor but then also call the “initCause” method to populate the “cause” if and only if it has not already been populated by the constructor.

That is, you need code along the lines of:

ServletException e = new ServletException(message, cause);
if (e.getCause() == null) {
    e.initCause(cause);
};

Obviously, if you need to do this in multiple places it makes sense to encapsulate it in a method or to provide your own subclass of ServletException that does this in its constructor.





First use of ObMimic for out-of-container testing of Servlets and Struts (Part 2)

27 06 2007

Updated Feb 2018: OpenBrace Limited has closed down, and its ObMimic product is no longer supported.

As explained in part 1 of this posting, I’ve recently started trying out my newly-completed ObMimic library for out-of-container POJO-like testing of servlet code.

So, as promised, here are some of my early experiences from starting to use ObMimic for testing of some simple existing filters, listeners, an old “Struts 1” application (including out-of-container running of the Struts 1 controller servlet and configuration file), and tuckey.org’s “UrlRewriteFilter”.

These are just some initial “smoke test” experiments to check how ObMimic copes with simple situations, and to evaluate its usability. The ObMimic code itself has been tested in detail during its development, and its use for more complex and useful scenarios will be examined later.

Also note that this is primarily intended for my own historical records – actual ObMimic documentation, tutorials, Javadoc etc will be published when it’s ready for public release.

Experiment 1: Some simple listeners, filters and other basic Servlet API code

As a gentle first step, I revisited some listeners and filters in various old projects, and some other utility methods that take instances of Servlet API interfaces as arguments. Some of these had existing tests using mock objects, others didn’t. None of them do anything particularly complicated.

Writing out-of-container tests for them using ObMimic was straightforward and all worked as intended. As you’d expect, it basically just involves:

  • Creating and configuring the various objects needed as arguments. For example, using “new ServletContextMimic()” to create a ServletContext for use in a ServletContextEvent, and configuring it via the ServletContextState object returned by its getMimicState() method. Or creating and configuring an appropriate FilterChainMimic for passing to a filter.
  • Making the call to an instance of the class being tested.
  • Carrying out whatever checks are necessary to see if the code being tested has worked correctly.

The details, of course, depend on exactly what the code being tested is supposed to be doing.

For simple listeners, filters and other such code this is pretty straightforward. For simple unit-tests of such code, one could do much the same thing using mock objects, though from my (admittedly biased) point of view I think that the ObMimic code is slightly simpler and more direct than the same tests using mock objects, even for these simple cases. At any rate, it better fits my general preference for “state-based” rather than “interaction-based” testing, and as we’ll see later it can also handle far more complex situations.

For testing more complex code, there’d typically be more set-up involved to get the context, request, response etc configured as required (for example, so as to appropriately handle any “forwarding” done by the code being tested). Similarly, checking of results can become arbitrarily complicated depending on what the code being tested actually does. But that’s just the usual joy of testing something. We’ll see a more complex example later on.

Experiment 2: Struts 1 “Actions”

The next set of components examined were a handful of “Actions” in an old Struts 1 application. Actually, this was a bit of an anti-climax. The Struts “ActionForm” classes are “POJO”s anyway and already had test cases, and the “execute” method of each Struts “Action” class just needs:

  • A suitable HttpServletRequestMimic and HttpServletResponseMimic for use as the request and response (with appropriately-configured ServletContextMimic, HttpSessionMimic etc as necessary for each test).
  • An instance of the relevant ActionForm subclass, configured with the required property values.
  • A Struts ActionMapping instance, configured to map the relevant result strings to a suitable “ActionForward” instance.

The Struts ActionMapping and ActionForward classes are both suitably POJO-like, so are easily configured. There isn’t even any need to configure mapping of the ActionForward’s path to an actual target resource, as the “execute” method just returns the relevant ActionForward rather than actually carrying out the forwarding.

A few of the Action classes did need a fair bit of configuration of the HttpServletRequestMimic, its ServletContextMimic and the relevant HttpSessionMimic for some of the individual tests, but this was all relatively straightforward.

Although such tests check the Action’s “execute” method in isolation, it would also seem useful (and, for purposes of these experiments, more challenging) to be able to test the broader overall handling of a request. That is, including the mapping of a request to the correct ActionForm and Action and their combined operation. So the next experiment was to try and execute the Struts “controller” servlet, so as to be able to do “out-of-container” testing of the Struts configuration file, ActionForm and Action all together.

Experiment 3: Struts 1 controller servlet

The aim for this experiment was to try to get the Struts 1 controller servlet running “out-of-container” using ObMimic. This is partly motivated by wanting to be able to “integration test” the combination of a Struts configuration file, ActionForm and Action. But more importantly this seemed like a useful more general and more challenging test of what ObMimic can cope with, and an indication of how easy or hard it might be to get ObMimic working for other web frameworks.

The first step is to configure a ServletContext to be able to run Struts, in much the same way as one would configure a real web-application for Struts. Whilst there are several ways to do this, and the following example includes some things that aren’t stricly necessary, for purposes of this experiment I chose to do this as closely as possible to how it would be done in a web.xml. This resulted in code of the following form (adjusted a bit to help illustrate it):

// Create the ServletContextMimic and (for convenience) retrieve
// its MimicState and relevant objects within its MimicState.

ServletContextMimic servletContext = new ServletContextMimic();
ServletContextState contextState = servletContext.getMimicState();
WebAppConfig webAppConfig = contextState.getWebAppConfig();
WebAppResources webAppResources = contextState.getWebAppResources();

// Give the web-app a context path.

contextState.setContextPath("/examples");

// Add the struts-config file (provided as a system resource file
// in this class's package) as a static resource at the 
// appropriate location.

String strutsConfigResourceName
    = getClass().getPackage().getName().replace('.', '/') 
        + "/ExampleStrutsConfig.xml";
webAppResources.setResource("/WEB-INF/struts-config.xml",
    new SystemReadableResource(strutsConfigResourceName));

// Add a servlet definition for the struts controller, including 
// an init-parameter giving the location of the struts-config file.

InitParameters strutsControllerParameters = new InitParameters();
strutsControllerParameters.set("config", 
    "/WEB-INF/struts-config.xml");
int loadOnStartupOrder = 10;
ServletDefinition strutsController = new ServletDefinition(
    "strutsController", 
    ActionServlet.class.getCanonicalName(),
    strutsControllerParameters, 
    loadOnStartupOrder);
webAppConfig.getServletDefinitions().add(strutsController);

// Add a servlet mapping for the struts controller.

ServletMapping strutsControllerMapping 
    = new ServletMapping("strutsController", "*.do");
webAppConfig.getServletMappings().add(strutsControllerMapping);

// Initialize the context ("load-on-startup" servlets are 
// created and initialized etc).

ServletContextMimicManager contextManager 
    = new ServletContextMimicManager(servletContext);
contextManager.initializeContext();

Here, the SystemReadableResource class used to access the Struts config file is a ReadableResource as described in a previous article. It reads the content of the Struts config file to be used in the test, with this being supplied as a file in the same package as the above code. (Alternatively, the application’s existing Struts config file could accessed using a “FileReadableResource”, but the details would depend on the project’s directory structures, whereas the approach shown here also allows individual tests to use their own specific Struts configuration and keep it with the test-case code).

The rest of the classes involved are ObMimic classes. Hopefully the gist of this is fairly clear even without their full details.

One slight concession is that ObMimic doesn’t yet support JSP, so where the struts-config file specifies a path to a JSP file, the test needs to map such paths to a Servlet instead. This involves defining a suitable servlet (e.g. an HttpServlet subclass with a “doPost” method that sets the response’s status code to OK and writes some identifying text into the response’s body content, so that the test can check that the right servlet was reached). The corresponding servlet definition and servlet mapping can then be added to the above configuration of the ServletContextMimic (similar to those for the Struts controller servlet).

Then we just need a suitable request to process. Again, there are various ways to do this, and the particular details depend on the needs of the individual test. In outline, the code used for this experiment was along these lines (demonstrating a POST with body-content request parameters):

HttpServletRequestMimic request = new HttpServletRequestMimic();
request.getMimicState().setServletContext(servletContext);
request.getMimicState().setHttpMethodName("POST");
request.getMimicState().populateRelativeURIFromUnencodedURI(
    "/examples/exampleStrutsFormSubmit.do");
request.getMimicState().setContentTypeMimeType(
    "application/x-www-form-urlencoded");
try {
    request.getMimicState().setBodyContent("a=1&b=2", 
        "ISO-8859-1");
} catch (UnsupportedEncodingException e) {
    fail("Attempt to configure request body content "
        + "for a POST failed due to unexpected " + e);
}

Here, the “populateRelativeURIFromUnencodedURI” method is one of various such short-cuts provided by ObMimic for setting request URI/URL details from various types of overall URL strings. This one takes a non-URL-encoded container-relative path, interprets it based on the ServletContext’s mappings etc, and populates the request’s context-path, servlet-path and path-info accordingly.

The response can start out as just a plain HttpServletResponseMimic with the correct ServletContext:

HttpServletResponseMimic response = new HttpServletResponseMimic();
response.getMimicState().setServletContext(servletContext);

So then we can invoke the Struts controller servlet, and it should all work just as it would within a servlet container, based on the supplied struts-config file and the ServletContextMimic’s configuration.

We could get hold of the Struts controller servlet from the ServletContextMimic by name, or maybe even just use a new instance of it. However, as we’ve gone to the effort of configuring a mapping for it, we might as well start with the request’s URI and do the actual look-up. For this I use a convenience method on ObMimic’s ServletContextMimicManager class that returns the target resource for a given context-relative path (again, there are various ways to do this, with or without any necessary filter chain etc, but this will do for these purposes):

ServletContextMimicManager contextManager 
    = new ServletContextMimicManager(servletContext());
Servlet actionServlet 
    = contextManager.getServletForRelativePath(
        "/exampleStrutsFormSubmit.do");
try {
    actionServlet.service(request, response);
} catch (IOException e) {
    fail(...suitable message...);
} catch (ServletException e) {
    fail(...suitable message...);
}

Then it’s just a matter of checking the response’s content (using, for example, calls such as “response.getMimicState().getBodyContentAsString()”), and anything else necessary to check that the request has been processed correctly.

Well, that’s the theory. So what happened in practice? A couple of minor problems were encountered, but easily overcome:

  • The version of Struts used appears to issue “removeAttribute” calls even where the attribute is not present. Although the Servlet API Javadoc for HttpSession specifies that its removeAttribute method does nothing if the attribute is not present, the Javadoc for ServletContext and ServletRequest don’t explicitly specify whether this is permitted or how it should be handled. ObMimic therefore treats such calls to ServletContext.removeAttribute and ServletRequest.removeAttribute as “API ambiguities”. Its default behaviour for these is to throw an exception to indicate a questionable call. But ObMimic’s handling of such ambiguous API calls is configurable, so the immediate work-around was just to have the test-case programmatically configure ObMimic to ignore this particular ambiguity for these particular methods, such that the removeAttribute calls do nothing if the attribute doesn’t exist. In retrospect it’s probably way too strict to treat this as an ambiguity – it’s a reasonable assumption that removeAttribute should succeed but do nothing if the attribute doesn’t exist, and there is probably lots of code that does this. So I’ve relented on this, and gone back and changed ObMimic so that this isn’t treated as an ambiguity anymore.
  • It turns out that the version of Struts used actually reads the contents of the /WEB-INF/web.xml file. This took a bit of hunting down, as the resulting exception wasn’t particularly explicit, but because the run is all “out-of-container” it was easy to step through test and into the Struts code in a debugger and find where it failed. The solution is to add a suitable web.xml file to the test class’s package and make this available to the ServletContext as a static resource at /WEB-INF/web.xml (in the same way as the struts-config.xml file). Actually, at least for this particular test, the precise content of the web.xml doesn’t seem to matter – Struts seems perfectly happy with a dummy web.xml file with a valid top-level <web-app> element but no content within it.

And that’s it. Having added a suitable /WEB-INF/web.xml static resource into the ServletContextMimic, Struts happily processes the request, pushes it through the right ActionForm and Action, and forwards it to the servlet that’s standing in for the target JSP. All within a “plain” JUnit test, with no servlet container involved (and easily repeatable with different struts-config.xml files, different context init-parameters, or with ObMimic simulating different Servlet API versions etc etc).

Experiment 4: URL Rewrite Filter

I’ve a few example/demo applications where I’ve played around with the UrlRewriteFilter library from tuckey.org to present “clean” URLs and hide technology-specific extensions such as “.jsp”. So I thought I’d try out-of-container testing of this as well.

The rules files that control the URL rewriting are fairly straightforward, but once you have multiple rules with wildcards etc it can become a bit fiddly to get exactly what you want. Tracking down anything that isn’t as intended can be a bit clumsy when it’s running in a servlet container, just from the nature of being in a deployed and running application. So I like the idea of being able to write and debug normal out-of-container test-cases for the config file, and using dummy or diagnostic servlets instead of the “normal” application resources.

This was pretty quick and straightforward after tackling the Struts controller servlet.

Although the details were very different, it again involves configuring a ServletContextMimic with the definitions, mappings and static resources for the UrlRewriteFilter and its configuration file. Much of the code was just copied and edited from the Struts experiment. Again, it proved useful to write a little servlet to which the “rewritten” URLs can be directed, with this having a “doGet” method that writes a message into the response’s body content, so as to indicate that it was invoked and what request URL it saw.

Then each actual test consists of using the relevant ObMimic facilities to obtain and invoke the filter chain for an example URL, with the filter chain’s ultimate target being a static resource whose content just shows if it was reached. After invoking the filter chain, the response’s body content can be examined to check which servlet processed it and what URL the servlet saw.

This wasn’t a very extensive test, as I just wanted to quickly see if it was basically possible, but it all worked without a hitch.

As with the preceding Struts experiment, the key issues are finding your way around the ObMimic classes in order to get the configuration you need, and figuring out what servlets and stuff you need in order to check the results.

Conclusions

So far, I’m happy with ObMimic technically. It’s particularly encouraging to have got both the Struts 1 controller servlet and the URL-rewrite filter running “out-of-container” so easily, as this suggests that it should be feasible to do the same for a variety of web frameworks and tools (especially once JSP support is implemented, which will be a priority for future versions of ObMimic).

On the other hand, I think the ObMimic Javadoc and other documentation needs more work. In practice, the key to using ObMimic is find your way around the MimicState classes that encapsulate each Mimic’s internal state. IDE code-completion is hugely useful for all this, as you can hunt around within each MimicState to look for the relevant properties and methods. However, it helps to have a rough idea of the general scheme of things – what’s available, what you’re looking for, and where things are most likely to be found. To a lesser extent it’s also helpful to know your way around the various supporting classes and methods that provide shortcuts for some of the more complex tasks. The documentation needs to provide some high-level help with all this.

Then there’s the Javadoc. This provides a comprehensive and detailed reference, but unfortunately it’s just too big and detailed. As it stands I think it would be too daunting for new users, or for casual use of the Javadoc. The first problem is that the standard Javadoc main index gives a full list of packages in alphabetical order. I’m hoping to deliver ObMimic as a single self-contained library, so there are a lot of packages, and the most useful routes into the Javadoc end up being scatered around the middle of a long list.

More generally, there are lots of specific tasks which are straightforward once you know how to do them, but hard to figure out from scratch. Things like how to make a static resource available within a ServletContext, or set up “POST” requests, or support JNDI look-ups, or maintain sessions across requests, or the easiest way to populate an HttpServletRequestMimic given the text of an HTTP request…

So my initial lessons from these experiments are:

  • ObMimic’s Javadoc needs to be made more approachable. One idea might be to supplement the standard Javadoc index page with a hand-written index page that groups the packages into meaningful categories and shows everything in a more sensible order.
  • It’d be useful to provide some kind of outline “map” of the MimicState classes, summarizing the properties and key methods of each class.
  • The ObMimic Javadoc needs to be supplemented by a set of task-oriented “how-to” guides.




Experiments with out-of-container testing of Servlet code using ObMimic (Part 1)

4 06 2007

Updated Feb 2018: OpenBrace Limited has closed down, and its ObMimic product is no longer supported.

At long last I’ve reached the stage where I can “eat my own dogfood” and use my ObMimic library for out-of-container testing of servlet code, so I’ve been trying it out on some existing applications.

But to start with, I guess I’d better explain just what “ObMimic” is, as I’ve not said anything much about it publicly yet.

So this article introduces ObMimic, and will be followed shortly by another one explaining some findings from my own initial use of it.

ObMimic is an as-yet-unreleased library of classes that supports “out of container” testing of code that depends on the Servlet API, by providing a complete set of fully-configurable POJO implementations of all of the Servlet API’s interfaces and abstract classes.

For every interface and abstract class of the Servlet API, ObMimic provides a concrete class (with a simple no-argument constructor) that is a complete and accurate simulation of the relevant Servlet API interface or class, based on an “internal state” object through which you can configure and query all of the relevant details.

This lets you test servlets, filters, listeners, and any other code that depends on the Servlet API (including, at least to some extent, higher-level frameworks that run on top of it), in the same way as you would for “plain” Java code. Typically, you construct and configure the Servlet API objects you need, pass them to the code being tested, and examine the results – with no need for any servlet containers, deployment, networking overheads, or complex “in-container” test frameworks.

Compared to HTTP-based “end-to-end” tests, this supports finer-grained and faster tests, and makes it far easier to test the effect of different deployment-descriptor values such as “init parameters” (because the relevant details can be changed at any time via normal “setter” methods, instead of requiring changes to the web.xml file and redeployment). You can also readily use ObMimic with JUnit, TestNG or any other test framework, as it doesn’t depend on any special base-class for tests and is entirely orthogonal to any test-framework facilities.

This approach is similar to using mocks or stubs for the Servlet API classes, but unlike mocks or stubs, ObMimic provides ready-made, complete and accurate implementations of the Servlet API functionality as defined by the Servlet API’s Javadoc. This includes proper handling of features such as request-dispatching, session-handling, listener notifications, automatic “commit” of responses when their specified content-length is reached, servlet and filter mapping, access to static resources at context-relative paths, merging of HTTP “POST” body-content and query-string request parameters, the effects of different sequences of Servlet API method calls, and all the other myriad and complex interactions between different Servlet API methods.

I call these implementation classes “mimics” in order to distinguish them from “mocks” and “stubs” and on the basis that they “mimic” the behaviour of real Servlet API implementations. Technically, they are “fake” objects as described by xUnit Patterns and Martin Fowler, with the addition of some stub/mock-like facilities. But the term “fake” doesn’t quite feel right, doesn’t seem to be very widely used, and some people use it as a synonym for “stub” (for example, Wikipedia as at the time of writing). Inventing yet another term isn’t ideal either, but at least it shouldn’t lead to any pre-conceptions or confusion with anything else.

Whilst mocks and stubs are fine for “interaction-based” testing or for arbitrary interfaces for which you don’t have “real” implementations, using “mimic” implementations seems a simpler, more natural and more useful approach for “state-based” testing or when you have access to appropriate “mimic” classes. At least, that’s my personal take on it. For a broader discussion of some of the relevant issues, see Martin Fowler’s article Mocks Aren’t Stubs.

By way of an example, at its simplest ObMimic lets you write test-case code like the following (where everything uses a “default” ServletContextMimic as the relevant ServletContext, and all details not explicitly configured start out with reasonable default values, such as the request being a “GET”):

HttpServletRequestMimic request = new HttpServletRequestMimic();
HttpServletResponseMimic response = new HttpServletResponseMimic();
request.getMimicState().getRequestParameters().set("a", "1"); // just for example
Servlet myServlet = new SomeExampleServletClass();
myServlet.init(new ServletConfigMimic());
myServlet.service(request, response);
// ... check contents of request, response etc...

Actually, the very simplest example is that if you just need, say, a ServletContext to pass as an argument to some method but its content doesn’t matter, you can just do “new ServletContextMimic()” – which must be about as simple as this could ever be.

ObMimic is also potentially usable for higher-level frameworks that run on top of the Servlet API, such as Struts. Such frameworks generally just need a suitably configured ServletContext and the relevant servlet/filter/listener definitions and mappings, plus various configuration files as static resources within the context – all of which are supported by ObMimic’s ServletContextMimic. And in many cases you can test components without needing the whole framework anyway – just requests and responses together with framework components that are themselves POJOs or otherwise suitably configurable. That’s the theory anyway. In practice this will depend on the details of the particular framework and the nature of its own classes and other API dependencies. But more of that in the next article…

Other current features of ObMimic include:

  • Configurability to simulate different Servlet API versions (2.3, 2.4 or 2.5).
  • A “mimic history” feature for recording and inspecting the Servlet API calls made to individual mimics.
  • Explicit checking and control over the many ambiguities in the Servlet API. That is, where the Servlet API Javadoc is ambiguous about how a particular argument value or sequence of calls should be treated, the ObMimic Javadoc documents the ambiguity and by default ObMimic throws an exception if the code being tested issues such a call, but can also be configured to ignore the call, throw a specified exception, or ignore the ambiguity and process the call in some “reasonable” manner.
  • A basic “in memory” JNDI simulation to support JNDI look-ups by the code being tested.
  • Easy to add to projects, as it consists of a single jar archive with no dependences other than Java 5 or higher and the Servlet API itself (which the code being tested will already need anyway).

Features not yet present but intended for future versions include:

  • Mimics for the JSP API, to support “out-of-container” testing of JSP pages, tag handlers etc.
  • Population of ServletContextMimics from web.xml deployment descriptors.
  • Population of HttpServletRequestMimics from the text of HTTP requests.
  • Production of HTTP response texts from HttpServletResponseMimics.
  • Specific support for particular web-frameworks (depending on demand and any particular issues encountered).

I guess I’ll be writing a lot more about these and other features over the next few months.

Anyway, the ObMimic code has been fully tested during its development, and has certainly been useful during its own testing. However, it’s in the nature of the Servlet API that any non-trivial code tends to depend on a large subset of the API and the interactions between its classes. So it hasn’t seemed particularly worth trying out ObMimic on any “real” projects whilst it was incomplete.

Now that ObMimic has reached the stage where it covers the entire Servlet API, I’ve finally been able to take it for a spin and try it out on some previously-written code. In particular, I’m keen to see how it copes with framework such as Struts, as this is likely to be a good way to shake out any problems.

So the next article will look at my initial experiences with using ObMimic to test some existing filters, listeners, Struts components, and overall Struts 1 operation (including out-of-container execution of the Struts 1 “ActionServlet” controller). I hope to follow this with further articles as I try it out for other web-frameworks, and progress towards a beta-test and public release.

By the way, in case you were wondering, the “Ob” in “ObMimic” is based on our not-yet-officially-announced company name.





Some possible defects in HttpServletResponse Javadoc?

22 04 2007

I’ve noticed what I think are some mistakes, inconsistencies and omissions in the Servlet API Javadoc for the javax.servlet.http.HttpServletResponse class’s sendError, sendRedirect and setStatus methods. Of course, this might just be me, but I think it’s worth raising.

So I’ve just raised the following issues as a bug report on the Glassfish issue tracker (issue number 2885), and thought I’d also document these issues here. Primarily just as a handy place to keep track of my own current assumptions and ideas about them, but also just in case this helps anyone else when trying to work out what these methods are supposed to do – with the caveat that everything here is just my own opinion at this point in time, whilst awaiting a response to the Glassfish bug report. If I do find I’ve got anything badly wrong, I’ll come back and add a comment afterwards.

So here are the problems, and what I personally think the javadoc is intended to mean (or ought to be saying):

1 setStatus(int) sets the location header.

The setStatus(int) javadoc includes the line “The container clears the buffer and sets the Location header, preserving cookies and other headers.”. This looks completely out of place – I can’t see why setting a non-error status code should need to clear the buffer, or what the Location header might have to do with this, or what the Location header is supposed to be set to.

My guess is that this is just in the wrong place, and is supposed to be part of the sendRedirect javadoc (which doesn’t currently say anything about clearing the buffer or setting the Location header).

2 sendError(int, String) and sendError(int) differences.

sendError(int, String) describes the nature of the default error response, the use of error-page declarations etc, but doesn’t say anything about clearing any existing buffered content. In contrast, sendError(int) says nothing about the nature of the error response or the use of error-page declarations, but does say “and clearing the buffer” (though it’s unclear whether this means before writing the error response, or with an empty response as the result).

My assumption is that despite the differences in their javadoc, these two sendError methods should be doing the same thing, just with or without an explicitly-given error message string. That is, the javadoc for both of these methods should actually say that they clear the buffer and then write an error page taking into account error-page declarations etc, with the default text/html page if there is no relevant error-page mapping.

3 sendRedirect doesn’t specify response content.

sendRedirect doesn’t say anything about what body content is sent. That is, whether existing buffer content is cleared or remains in the response, and whether or not the redirect writes any particular body content (e.g. a text/html page explaining the redirect and giving the link).

I tend to think that sendRedirect should be sending a page explaining the redirect and giving the link, as is the convention for HTTP redirects. But then it is a bit unclear what format should be used for such a page. Maybe it should be a text/html page, as is the case for the default error page. But then what if the client isn’t using HTML, and maybe isn’t accepting HTML? This isn’t just an “it’s all gone horribly wrong” error page, and might be part of the application’s normal flow, and any page sent is supposed to be a human-readable explanation of the redirect in case it isn’t followed automatically. So sending a specific type of page regardless of the client might be inappropriate. Also, maybe the application wants to produce its own customized redirect pages – in which case leaving any existing content in place would allow this, except that it’s risky because any caller that isn’t expecting this and doesn’t clear the buffer itself could end up sending a partially-built normal page by mistake.

So in the absence of anything in the javadoc saying what should be sent, for the time being I tend to think that clearing the buffer and sending an empty body content is the safest bet, but that the actual behaviour is implementation-dependent.

4 sendError implies Content-Length unchanged.

sendError(int, String) says “leaving cookies and other headers unmodified”, but presumably any Content-Length header should be cleared, as any existing value would presumably be wrong for the resulting error page, and could potentially cause it to be “auto committed” before being fully written. Or maybe the Content-Length header ought to get set to the actual length of the resulting error page? This also applies to sendRedirect (assuming it doesn’t just leave the existing buffered content unchanged), and anything else that clears existing content from the buffer.

My assumption is that if an error page is written or the buffer is otherwise cleared, the Content-Length header should then reflect the resulting page’s actual length regardless of any explicit content length that was set prior to the error, even if the javadoc says that headers should be unchanged.

5 setStatus(int) for error codes.

setStatus(int) says it is used when no error, but “if there is an error AND the caller wishes to invoke an error-page defined in the web application” then sendError “should” be used instead. This leaves it unclear whether the method can or cannot be used for error codes if the caller does NOT wish to invoke the error-page mechanism (i.e. the caller is setting the status code on its own, prior to explicitly writing the error page content for itself). So if called with an error code, what does this method actually do? Throw an exception? Ignore it? Just set the status code to it? Treat it as per sendError? Treat it as per sendError but with no default error page if there’s no error-page mapping? Implementation dependent?

My guess would be that this method is intended to accept error codes as well as non-error codes, and should just set the status code to whatever value is given.

But the javadoc doesn’t guarantee this, and it’s possible that what it’s trying to say is that error codes should/must only be set via sendError, and that setStatus should not be used for error codes and has undefined behaviour for them.

6 Valid status codes not specified.

More generally, none of the methods that take a status code specify what values are valid, or how any out-of-range values are treated (for example, whether integers outside the HTTP 1xx – 5xx range are allowed, what happens if sendError is called with non-error status codes, or what happens if a negative value is given).

In the absence of anything in the javadoc, I’d assume that it’s only safe to send 4xx-5xx values to sendError, 1xx-3xx and possibly also 4xx-5xx values to setStatus(int), and any 1xx-5xx value to the deprecated setStatus(int, message), with the handling of anything else being entirely implementation-dependent.

7 sendError(int, String) clarification.

As a minor clarification, where sendError(int, String) says that if an error-page is configured “it will be served back in preference to the suggested msg parameter”, maybe this should be “..in preference to the default error page” (especially as the resulting error page might actually make use of the given message).

8 sendRedirect status code?

It might also be useful for sendRedirect to specify the actual status code used (e.g. 302 as given by SC_MOVED_TEMPORARILY or SC_FOUND, or 307 as given by SC_TEMPORARY_REDIRECT, or whether this depends on circumstances).

For the time being I’m assuming that sendRedirect always sends a 302 as given by SC_FOUND.

9 setStatus(int, String) inadequately specified.

Whilst setStatus(int, String) is deprecated due to the ambiguous meaning of the “message” argument, it doesn’t adequately define what the method actually does (e.g. what does setting the “message” for this response actually mean? Is behaviour as per setStatus(int) if given a non-error code and as per sendError(int, String) if given an error code? Or something else?).

For the time being I’m assuming that this has been deliberately left as entirely unspecified and implementation dependent, and the behaviour of any calls to this deprecated method can’t be relied on (though a reasonable implementation might be to treat non-error codes as per setStatus(int) and error codes as per sendError(int, String) or sendError(int) depending on whether the message argument is present or null).

Conclusion

As with all such javadoc, there are probably lots of other things too, but the ones listed above are the ones that leave me most puzzled over what their javadoc really means and what behaviour can or can’t be relied on from these methods.

We’ll see what response this gets on the Glassfish bug list (which presumably needs to vet these issues and pass any suggested changes on to the Servlet API expert group, if I’ve understood how javax.servlet code is maintained at the moment – though I’m not really very sure about that at all).

In the meantime I’d be interested if anyone else has encountered any actual problems with the definition of these methods or their implementation by various servlet containers, or can set me straight or provide other interpretations of the Javadoc.





Is it time for a next-generation Servlet API?

20 03 2007

The Servlet API has clearly been hugely successful and influential. It has been a key part of Java’s success on the server, and is the underlying base on which a broad range of higher-level web-frameworks have been built. It has also stood the test of time, surviving largely unchanged from the early days of programmatically printing lines of HTML through to the world of tag libraries, portlets, JSF, AJAX, mash-ups, web-services and the like. That’s something that can’t be said about all APIs from that far back.

But I’m starting to wonder if the time has come to take a fresh look at this API and modernize it – or even replace it with something radically different.

So what’s wrong?

(Some of this bit’ is rather dry… feel free to skip to the wild-and-wacky stuff after it if that’s more your thing)

As with any API, over time there has been a gradual accretion of features that weren’t originally catered for, or weren’t adequately understood, and there has been the usual general wear-and-tear on the API due to deprecations, adjustments, clarifications, assorted tweaks etc. But in addition, the underlying Java platform and its libraries have improved significantly; the nature of the API’s clients and related technologies have changed quite dramatically (as has the API’s general place in the overall scheme of things); and ideas on API design have progressed.

So here are some particular examples of things that I think look outdated or otherwise in need of improvements or redesign (in somewhat random order, and by no means intended to be comprehensive):

  • The current API was designed before Collections and generics. There’s at least one use of Map, but in general the support for attributes, request parameters, and init parameters and the like all use repetitive patterns of get/set/getName/getValues methods, and they return arrays and untyped Enumerations. This could surely be simplified and improved by taking full advantage of Collections and generics. There may also be other places where generics could tighten things up (e.g. to enforce matching request/response subclasses).
  • The current API design also precedes JDK 1.4’s introduction of the java.net.URI class. It uses strings to represent URLs, URIs and their component parts, with lots of inconsistent terminology and incomplete rules and explanations (for example, where path parameters and/or fragments are permitted and how they are handled, which strings are and aren’t URL-encoded, how absolute/context-relative/request-relative paths are identified and “resolved” etc). A lot of this could probably be made more general, more explicit, and probably rather simpler by using java.net.URI wherever appropriate.
  • Java SE 6 also introduces a java.net.Cookie class and supporting classes for managing sets of cookies, which look rather more capable than javax.servlet.Cookie and presumably render it redundant. There are probably lots of other Java SE classes that weren’t available when the Servlet API was designed but that it could now take advantage of (subject to which Java SE versions have to be supported).
  • The use of inheritance to separate HTTP-specific features from more general protocol-independent base classes looked like a good idea originally but the more I see of it the worse it seems. In practice, everthing seems to be designed around HTTP anyway, and the notional ability to support other protocols is rather vague and under-developed whilst adding considerable complexity. Despite the logical division, the protocol-independent base classes often specify particular behaviour that is required when the protocol is HTTP, or worse just silently assume HTTP and refer to HTTP-only features (for example, request-dispatching deals with requests of any kind, but specifies the “adjustment” of features that are only present in HTTP requests). Conversely, other features are restricted to the HTTP classes even when they might apply more broadly (such as the request having a URL). Similarly, many deployment descriptor settings are only relevant for HTTP and have uncertain meaning for non-HTTP requests (what do URL-patterns mean for non-HTTP requests if such requests aren’t considered to have a URL? how do you “map” such requests?). More generally, there doesn’t seem to be any clear picture of just how broad a range of protocols the API is and isn’t intended to accommodate. As a result, the choice of what features are common to all protocols seems somewhat arbitrary, whilst still leaving the suitability of the base classes for any actual protocols other than HTTP rather uncertain. There are lots of other problems with this inheritance-based approach (for example, the use of “wrapper” classes also becomes rather awkward once you need separate wrapper classes for HTTP and non-HTTP classes, and this can end up leading to some horrible class hierarchies). In practice, the API seems to be almost synonymous with HTTP processing. Maybe it would be simpler overall if it really were just HTTP-specific, with other protocols being considered as outside its scope. Or maybe the protocol-specific support should be provided by composition rather than inheritance (e.g. using a set of protocol-specific “extensions” as pluggable sub-components to provide the relevant combination of additional features).
  • The general quality of the Javadoc seems surprisingly poor. Maybe it’s just me, but there seems to be a lot of it that looks ok at first glance but falls apart under detailed examination. Lots of minor mistakes, typos and inconsistencies, of course (including methods that appear to mistakenly have some or all of another method’s Javadoc). But also too many methods that don’t specify what range of argument values are allowed (for example, whether null or empty-string arguments are allowed, and if so how such values are treated). Too many under-specified interactions between different parts of the API (like whether an HTTP response’s “getCharacterEncoding” method is or isn’t affected by explicit use of the “setHeader” method to set its content-type header). Too many vague terms and references to things that aren’t accessible through the API (like references to “the current request URI” in classes that don’t otherwise have any explicit relationship to a request and don’t explain what “current” actually means in the face of request dispatching and wrappers).
  • Unit-testing of code that uses the Servlet API is far too difficult. The application code depends on objects that are “magically” constructed by the container, typically with lots of read-only features and complex internal behaviour and interactions, and potentially dependent on container configuration. You can test an application’s overall behaviour by deploying it and throwing HTTP requests at it, or you can use a tool such as Cactus to run more detailed tests inside the container, or you can use mocks/stubs of some kind to simulate what you think the objects will be, how they will behave, and what the container will do. But all of these approaches have limitations and shortcomings, and make testing Servlet code far harder than normal testing of POJOs. I don’t know the entire answer to this, but there must be way of making the API more “POJO-friendly” and more amenable to normal testing tools and techniques.
  • There are a few areas that seem to have turned out to be rather more complex than originally thought, and which seem to have continual adjustments and clarifications. Some of these have become a bit more complex than is desirable, whilst other still have rough edges and uncertainties. Particular things that come to mind are the handling of content-type and character-encoding (interaction of setters/getters, HTTP headers, request/response state, explict resetting of request, implicit resetting of request due to forwarding etc); the interaction between cross-context request dispatching and session handling (and its implications for session-tracking mechanisms); and how to properly “wrap” a response bearing in mind all the complex interactions between its methods and the possibility of other wrappers around it (even the standard HttpServlet implementation gets this wrong). I’d imagine that having been through all this once, a redesign of the API could come up with a simpler and more complete solution for many of these issues.
  • The request and response are represented by separate objects but are always processed together. Normally the same type-checks, casts, wrapping etc must be applied to both. They also each have methods that depend on particular aspects of the other’s state, despite neither having any explicit access to each other (for example, there are situations where the response’s sendRedirect method depends on the request’s URL, and there are methods on the request that depend on whether the response has been committed or not). It might be simpler to combine them into a single object, or at least have them as sub-components of a single object (as per the “Exchange” object in Sun’s com.sun.net.httpserver package).
  • Although much of the API implementation must be left to the container, there are quite a few API methods that are related or can be based on each other. Often these are minor variations on each other or have slightly different arguments but provide the same functionality, or are roughly equivalent methods provided in different places. There are also probably parts of the API that could reasonably be provided independently of the container implementation (such as “attribute” handling). So maybe there’s an opportunity to separate out a “Service Provider Interface” that defines a minimal set of facilities that the container must provide, and the Servlet API itself can provide a standardized implementation of how the full API is built up from these. This might also make it easier to provide a mock/stub “container” for use during testing.
  • There are also lots of new and long-outstanding requirements that might heavily influence the API if it were being designed from scratch. For example, maybe more direct and explicit support for web-services and SOAP, AJAX and XMLHttpRequest would make sense, or better support for component-based frameworks such as JSF (I’m not immediately aware of any particular areas where the current API design is a problem for this, or where a different API design could help, but I suspect there might be some). More generally, we’re probably due a change of emphasis from sending complete HTML pages to the client towards providing services and data for other applications and “rich” clients.

You may well disagree with some of the above, but you can probably also think of many more.

I think it’s quite telling that the com.sun.net.httpserver package introduced in Sun’s JDK 6 is very different in design from the Servlet API. OK, it’s intended for rather different purposes, and it is effectively the “container” itself rather than the API for talking to the “container” (and being a com.sun package, presumably isn’t actually an official part of Java SE). But its features do substantially overlap with the Servlet API, and its design does seem to recognise and address some of the issues listed above. For example, it uses an HttpExchange class to represent a request and response together; it uses a distinct Map-based “Header” class to represent a set of HTTP headers; and it makes no pretense of being for anything other than HTTP. If all was well with the Servlet API, maybe this wouldn’t be needed as an entirely separate and unrelated package, or maybe it wouldn’t use such a completely different design. Maybe it would just be a lightweight servlet-container and implement the Servlet API, or at least the relevant subset of it.

On a slight side-track, isn’t it a bit worrying that we now have the Servlet API classes in Java EE, overlapping HTTP-related classes in Java SE that the Servlet API doesn’t actually use, and a separate com.sun HttpServer in the JDK that takes a quite different and unrelated approach?

The bigger picture, and speculations.

In more general terms, I’m no longer sure exactly what type of “service” the Servlet API is intended to address, what particular protocols (if any) it should be tied to, or even whether the Java SE vs. EE split really makes sense anymore.

Back in the late 90’s it clearly made sense to provide an API through which code could run inside a web-server and respond to HTTP requests by writing out lines of HTML. Server-side web programming was largely CGI-based, beyond which Microsoft had ISAPI and Netscape had NSAPI. Java needed a standard API for doing something along the same lines. The clients were all “dumb” browsers that needed to be spoon-fed the actual text to be displayed, with no real “presentation layer” processing of their own.

But I don’t think much of that holds any more, or at least not to the point where we need a major API focused purely on that particular model. What we now have are increasingly smart clients, even where they are web browsers – and increasingly the client is just as likely to be another server application, or a “thick” client on a desktop, phone, or some other device.

So I suspect Google’s GWT, the various “rich client” approaches (including, still, Java applets), and other such browser-based presentation layers are the way forward. That is, provide a presentation layer that the client can run itself, and the back-end services are just that: back-end services. The communication might still be over HTTP for convenience, but the meaningful bit is its content – be it web-service XML stuff, XMLHttpRequest, JSON, or whatever.

From that point of view, the nature and purpose of a “Servlet API” starts to look rather different, and it’s easy to see how the current approach is ignoring where the real action is and leaving it to a confusing mess of non-standardized, “higher-level” frameworks. It’s also hard to see any fundamental differentiation between the Servlet API and EJB Session Beans, or why such services should be considered “web tier”.

To mix things up even further, we then also have an HTTP server appearing in the JDK, and desktop applications that might increasingly need simple ways to utilize multi-core CPUs without getting bogged down in concurrency issues or worrying about synchronization (sound familiar?).

So I’m wondering if what we really need is a more general, protocol-independent concept of a “service” that supports multiple concurrent requests. Typically but not necessarily over a network, possibly but not necessarily with support for sessions/conversations, and that can be called either synchronously or asynchronously. Maybe suitable for internal services within desktop apps just as much as for server-based access over the internet. Ideally all POJO-based. It’s almost a language-level thing. Then have protocol-specific adaptors for talking to/from such services. OK, I’m dreaming and completely out of my depth here, but it feels like it might unify a lot of things if the differences in their characteristics doesn’t render it unworkable.

So where do we go from here?

Back to reality: I’m not expecting anyone to jump up and down and immediately start replacing the Servlet API with a whole new API. Let alone replace the Servlet API, Session Beans, Message Beans, the com.sun.net.httpserver package, and all sorts of other stuff with some kind of wizzy new native language support for POJO-based “services”… maybe aim for Java 9 or 10 instead :-).

But I do think that even if we stick with a recognizable Servlet API, there’s a lot that could be done to substantially modernize it and provide a better design from which it can move forward. We’re surely long overdue the kind of revamp and simplification that we’ve seen in EJB. I’m convinced there’s a really small and tidy API buried in there somewhere!

Alternatively, maybe it would be more practical to implement a redesigned API as a layer on top of the existing API, whilst remaining highly interoperable with it (any takers?).

Or maybe it’s all beside the point… maybe the Servlet API now just sits there gathering dust as the long-ignored but somewhat cranky foundation on which everything else is built, until eventually something else grows to the point where the Servlet API can ride off into the sunset.








%d bloggers like this: