Extension with @TestTemplate

[svenkost]

I'm creating an extension for running tests to different environments.

The test class has an annotation that has all information to discover which environments should be tested. The annotation is a consul service name. I discover all the environments with the BeforeAllCallback and save them for later in the store that is in the ExtensionContext. (step 1)

The test class has a @TestTemplate test. With the TestTemplateInvocationContextProvider I generate a stream of TestTemplateInvocationContextObjects. In the object I have overridden the getDisplayName(final int invocationIndex). The stream is based on the list of environments I have created in step 1. (Let's call this step 2).

Then what happens is that the class will be instantiated with the TestInstanceFactory interface. The method createTestInstance(TestInstanceFactoryContext factoryContext, ExtensionContext extensionContext) is called; but here starts the problem: I have no indication for which test-template-instance the class is instantiated, so I have no reference for which environment this object is instantiated. (Step 3). I need to know why the object is to be instantiated, so I can call the constructor with the consul-information from step 1.

I made a 'dirty hack' to just have an internal counter to give every construction another environment; which works okay for 1 @TestTemplate, but if you have more than one and do it multithreaded, the environments are divided totally wrong: Test1-Env1 gets Env1, Test2-Env1 get Env2, Test2-Env2 gets Env3, Test1-Env2 gets Env4.

To run the test I use interceptTestTemplateMethod(Invocation<Void> invocation, ReflectiveInvocationContext<Method> invocationContext, ExtensionContext extensionContext) because I need to have initialization around the actual run of the test method.

What I need and can't find is that I need some sort of connection between the stream of tests that are created in step 2 and the construction of the test object for that specific test in step 3.

@RunAllInstances
@ServiceName("myservice")
@ExtendWith(ConsulExtension.class)
@Execution(ExecutionMode.CONCURRENT)
class ExtensionTest  {
  private static final Logger logger = LoggerFactory.getLogger(ExtensionTest);

  private final ConsulTestContext consulTestContext;

  public ExtensionTest(final ConsulTestContext ctx) {
    logger.info("Constructor with ConsulTestContext: " + ctx);
    consulTestContext = ctx;
  }

  @TestTemplate
  public void myTestTemplate() {
    final long wait = Math.round(Math.random() * 3_000d) + 1_000L;
    logger.info("TestTemplate test " + consulTestContext.getMachine() + " class = " + this + ": waiting " + wait + " ms");
    try {
      Thread.sleep(wait);
    } catch (final InterruptedException ignored) {
    }
  }
}

[portyu9]

Hi @svenkost

There is now a direct solution for this. The missing connection in the 2022 API has since been added to JUnit Jupiter.

The two relevant pieces are:

• TestTemplateInvocationContext.prepareInvocation(...), which receives the context for that specific template invocation
• getTestInstantiationExtensionContextScope(...), which lets your TestInstanceFactory receive the test-method/invocation-scoped context

So you no longer need a counter.

Store the environment when preparing the invocation:

private static final ExtensionContext.Namespace NAMESPACE =
        ExtensionContext.Namespace.create(ConsulExtension.class);

private static final String ENVIRONMENT = "environment";

TestTemplateInvocationContext invocationFor(ConsulTestContext env) {
    return new TestTemplateInvocationContext() {

        @Override
        public String getDisplayName(int invocationIndex) {
            return env.getMachine();
        }

        @Override
        public void prepareInvocation(ExtensionContext context) {
            context.getStore(NAMESPACE).put(ENVIRONMENT, env);
        }
    };
}

Then make the TestInstanceFactory request test-method scope:

@Override
public ExtensionContextScope getTestInstantiationExtensionContextScope(
        ExtensionContext rootContext) {
    return ExtensionContextScope.TEST_METHOD;
}

and retrieve the value when the instance is created:

@Override
public Object createTestInstance(
        TestInstanceFactoryContext factoryContext,
        ExtensionContext context) {

    ConsulTestContext env = context.getStore(NAMESPACE)
            .get(ENVIRONMENT, ConsulTestContext.class);

    // Use `env` in your existing test-instance construction logic.
    ...
}

ExtensionContextScope is:

org.junit.jupiter.api.extension.TestInstantiationAwareExtension.ExtensionContextScope

Because the state belongs to the individual invocation's ExtensionContext, this also avoids the race/order problem caused by a shared counter during concurrent execution.

This requires a recent JUnit version; these APIs did not exist in this form when the question was originally posted.

Also keep the normal PER_METHOD test-instance lifecycle for this design. Test-method-scoped instantiation context is not available in the same way for a PER_CLASS test instance that must be created before individual template invocations exist.

Please mark this answer as accepted if it helped, thank you.