Introduction

Ever since annotations officially made it in the Java language, the debate has been raging about where configuration should go: in annotations or XML files.

The pros and cons of each approach are easy to summarize: annotations allow you to put your configuration system close to the Java source it applies to, but changing it requires Java knowledge and a recompilation. On the other hand, XML files are usually easy to modify and they can be reread at runtime (sometimes even without relaunching your application), but they are very verbose and the edition can be error prone.

As usual in such situations, the correct course of action is to use a mix of these approaches. For example, a rule of thumb that I have been following is that "any configuration parameter that refers to a Java element (package, class, method) should be specified in an annotation" (e.g. @Test or @Transaction) and "any configuration parameter that applies to your application as a whole probably belongs to an XML file" (e.g. the name of an output log file or a target Web server).

Another approach that blends these two techniques has also been under scrutiny recently: being able to override Java annotations with XML files. I haven't come across an implementation of this idea that can claim it's the defacto standard, and at this point, I suspect that this approach is unlikely to take off because it suffers from the "worst of both worlds" syndrome.

Here is why.

Code

Consider the following simple code:

public class Mytest {
    @Test(invocationCount = 10)
        public void verify() {
            // ...
    }
}

This instructs TestNG to invoke the verify() test method ten times.

If I want to override this value at runtime, I find myself having to write a fairly complex piece of XML code in order to pinpoint the exact annotation I'm trying to override. It would probably look like this:

<override-annotation>
  <package name="org.foo.tests">
    <class name="MyClass">
      <method name="verify">
        <annotation name="org.testng.Test">
          <attribute name="invocationCount" value="15" />
        </annotation>
      </method>
    </class>
  </package>
</override-annotation>

Of course, you might want to come up with a way to capture the override (the annotation/attribute part) so you can reuse it somewhere else in your XML file, in case you want this override to apply to more than just one method. This would probably be achieved with the definition of an "override-ref" that you would define at the top of your XML file and that you would repeatedly use throughout your XML file (exactly like ant's classpath-ref, for example).

That's already a lot of work (and it's quite hard to read), but it's also extremely fragile since it will most likely break if you decide to rename your class or your method name (IDE's are beginning to extend refactorings to non-Java files, but we're not quite there yet).

Because of these shortcomings, I have been working on a slightly different approach with TestNG that lets you specify this runtime override in Java. Again, it's not a silver bullet and suffers from some of the compromises expressed above, but if you can live with the idea that you have to recompile your override if you modify it (but not the annotated code you are trying to override, so it's already a progress), it's actually fairly simple to achieve.

TestNG introduces the following interface:

public interface IAnnotationTransformer {
    public void transform(ITest annotation, Class testClass,
                          Constructor testConstructor, Method testMethod);
}

You use it as follows:

public class MyTransformer implements IAnnotationTransformer
{
    public void transform(ITest annotation, Class testClass,
                          Constructor testConstructor, Method testMethod)  {
        if ("verify".equals(testMethod.getName())) {
            annotation.setInvocationCount(15);
        }
    }
}

Observations

Here are a few observations about this technique:

The idea is that whenever TestNG parses a @Test annotation, it will run it through the annotation transformer before using it.
The three extra parameters of the method let you know where this annotation was found (on a class, a constructor or a method). Only one of these three parameters will be non-null.
Notice that the parameter to the transform() method is an ITest. This is because TestNG supports both standard and JavaDoc annotations, so ITest is a simple façade object that represents the @Test (JDK) or @testng.test (JavaDoc) annotation. In a more general framework, the parameter would probably directly be the type of the annotation, although there are still a few problems to solve with this (see the next point).
This approach requires write access to the annotation, which is not possible with the current JDK implementation. You can work around this limitation by using a façade, which could possibly be generated as a dynamic proxy.

The advantage of annotation transformers is that they can be very powerful: it's very easy to apply them to several methods, several classes, all methods of a class or of a package, etc.

Annotation Transformers are already implemented in the TestNG Subversion depot, and they will be available publicly when TestNG 5.3 comes out (very soon).

Annotation Transformers in Java