Broadleaf Microservices
  • v1.0.0-latest-prod

Java Backend Testing

Key Considerations

Generally the following is true:

  • Fast tests are better than slow tests

  • Tests that test a small number of things are better than tests that bake in a large number of things

  • Unit tests (non-Spring) are better than integration testing (Spring)

  • If something is hard to test, it is likely that your component under test should be redesigned

Testing Technologies

  • JUnit (prefer JUnit 5/JUnit Jupiter, but there are exceptions for compatibility where JUnit 4 is required)

  • Mockito

Using JUnit 5

Usually you will add spring-boot-starter-test as a <scope>test</scope> dependency to your pom. However, Spring’s default is JUnit 4. For Junit 5, you need this:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
    <exclusions>
        <!-- Excluding for conflict with JUnit Jupiter which is
        under a different groupId/artifactId -->
        <exclusion>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
        </exclusion>
    </exclusions>
</dependency>

JUnit 5 still has the potential to be [the default in Spring Boot 2.1](https://github.com/spring-projects/spring-boot/issues/6402#issuecomment-398164055) with the resolution of the referenced Surefire issue

Package Structure for Tests

Driving factors:

  • All tests (unit and integration) should reside in the service’s src/test/java directory, not under src/main/java.

  • Unit tests should end in *Test, integration tests in *IT

  • Unit tests and integration tests can reside in any package, but it is preferred that a test class resides in the same package as the class it is testing.

Example, your src/main/java and src/test/java should be structured like this:

|-- src
  |-- main
      |-- java
          |-- com
              |-- broadleafcommerce
                  |-- catalog
                      |-- service
                          |-- ProductService.java // Service you are testing
                      |-- controller
                          |-- ProductEndpoint.java // @Controller you are trying to test


  |-- test
      |-- java
          |-- com
              |-- broadleafcommerce
                  |-- catalog
                     |-- service
                         |-- ProductServiceTest.java // Service unit tests
                     |-- controller
                         |-- ProductEndpointIT.java // integration tests for the ProductEndpoint

Maven Configuration

Unit Testing

Unit tests are executed by the [Surefire plugin](https://maven.apache.org/surefire/maven-surefire-plugin/) and in the [test phase](https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#A_Build_Lifecycle_is_Made_Up_of_Phases). This means that whenever you package or install then these tests are executed.

Integration Testing

Integration tests are executed by the [Failsafe Plugin](https://maven.apache.org/surefire/maven-failsafe-plugin/) and runs under the [verify phase](https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#A_Build_Lifecycle_is_Made_Up_of_Phases). This does not execute when you package or install, you have to mvn verify to execute them in that phase.

Also, the failsafe plugin is not hooked up by default. To do so, add this in your pom.xml:

<build>
    <plugins>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
        </plugin>
    </plugins>
</build>

Unit Testing

Because integration tests can take a very long time to run, unit tests should generally be your default choice when writing tests. Use your best judgment when determining if a test really benefits from an integration test.

Unit testing runs under the surefire plugin which is activated automatically for all classes [ending in Test](https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#includes). To run these tests, simply use mvn test or run them individually from your IDE.

Unit tests do not use a live Spring Application Context. This means that if your class file ends in Test, you should not have an @ExtendWith(SpringExtension.class) (JUnit 5) or @Runner(SpringRunner.class) (Junit 4), @SpringBootTest, @DataJpaTest, or any similar annotation that starts Spring or interacts with a database.

Mocking dependencies using Mockito

This still leaves a requirement for managing dependencies of a service, and in this situation, we prefer using [Mockito](https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/Mockito.html). We will only cover some basic concepts here, as there are tons of third party resources on how to use Mockito, and covering all that is beyond the scope of this article.

Mockito allows us to mock out a service’s dependencies, and there are multiple ways of instantiating mocks. First, we can use @ExtendWith:

@ExtendWith(org.mockito.junit.jupiter.MockitoExtension.class)
public class ProductServiceTest {

  private ProductService classUnderTest;

  @org.mockito.Mock
  private ProductRepository repository;

  @BeforeEach
  void setup() {
    classUnderTest = new ProductService(repository);
  }
}

Using @ExtendWith(MockitoExtension.class) in the code above instructs Mockito to initialize any variable annotated with @Mock as a mock instance. We can then inject this mock into our class under test.

This scenario is fine for the majority of cases. Sometimes, @ExtendWith may not be available if we need to extend our test class with some other extension. There are two potential solutions here. We can manually instantiate our mocks:

public class ProductServiceTest {

  private ProductService classUnderTest;

  private ProductRepository repository;

  @BeforeEach
  void setup() {
    repository = Mockito.mock(ProductRepository.class);
    classUnderTest = new ProductService(repository);
  }
}

And we can also continue using the @Mock annotation, and manually initialize our annotated mocks:

public class ProductServiceTest {

  private ProductService classUnderTest;

  @org.mockito.Mock
  private ProductRepository repository;

  @BeforeEach
  void setup() {
    MockitoAnnotations.initMocks(this); // This is functionally the same as @ExtendWith(MockitoExtension.class)
    classUnderTest = new ProductService(repository);
  }
}

Finally, there are a few key methods we’ll use when mocking out method calls. These methods will cover the majority of our use cases when mocking. We’ll cover them briefly.

If we have the following methods in the ProductRepository class: * Product readProduct(String id) * void deleteProduct(String id)

  1. Mockito.when(repository).readProduct(any(String.class)).thenReturn(new Product()) This allows us to mock a simple method and its return value

  2. Mockito.when(repository).readProduct(any(String.class)).thenAnswer(invocation → // some complex code) This allows more advanced logic when mocking out methods that return values.

  3. Mockito.doAnswer(invocation → // some kind of complex code).when(repository).deleteProduct(any(String.class)) This allows us to perform some sort of logic when a void method is called. Alternatively, there is doNothing and doThrow.

  4. Mockito.verify(repository, times(1)).deleteProduct(any(String.class)) Verify that a method is called 1 time. Note: Using verify should be avoided when at all possible, as this directly ties our test to the logic of the class under test. However, sometimes it is unavoidable when dealing with void methods.

The above methods should cover the overwhelming majority of our use cases when writing unit tests. In the case that we need more complex tests, Mockito can likely handle it. Refer to the Mockito documentation for more details.

Fake repositories

The final topic we’ll discuss is repositories. In many cases, the classes we’re testing have some sort of repository instance that interacts with a database in the real world, and we generally don’t want to use an in memory database in our test cases for performance reasons.

One option we have is to mock out the repository, which is generally fine for very basic use cases, but that quickly breaks down with more complex services. What often ends up happening is we’ll end up with a test class that is very hard to maintain with large amounts of mocking code.

In cases like this, it’s often better (from a readability and maintainability perspective) to just create a 'fake' version of the repository.

Consider this repository:

@Repository
public interface ProductRepository {

  Product create(Product product);

  List<Product> readAll();

  Optional<Product> readById(String id);

  void delete(String id);
}

Looking at this, it should be easy to see how this can quickly get out of hand if we’re going to mock each one of these methods for each of our test cases, and this is a very basic example. Real repositories have even more methods that we’d have to mock.

Instead, what we can do is just create our own concrete implementation of the repository.

public class FakeProductRepository implements ProductRepository {

  private Map<String, Product> productMap = new HashMap<>();

  public Product create(Product product){
    productMap.put(product.getId(), product);
  }

  public List<Product> readAll() {
    return new ArrayList<>(productMap.values());
  }

  public Optional<Product> readById(String id) {
    return Optional.ofNullable(productMap.get(id));
  }

  public void delete(String id) {
    productMap.remove(id);
  }
}

Now, instead of mocking out ProductRepository, we just instantiate an instance FakeProductRepository and inject it into our class under test. It functions very similarly to an actual, real-world, repository. This does take a bit longer to set up, but this makes our tests far more maintainable and easier to read.

Integration (Spring Application Context and beyond) testing

Integration tests assume that you have a Spring Application context, or even a live web application with endpoints you can hit. Oftentimes we will have components that only make sense to test with a live Spring ApplicationContext.

The easiest test you can write is something like this:

@SpringBootTest
public class MyIT {

    @Test
    public void gotAnAppctx(ApplicationContext ctx) {
        assertNotNull(ctx);
    }
}

This has some key advantages:

  • Easy to write your first few tests before you iron out what you’re trying to accomplish and why

  • Likely Just Works™

  • Usually the exact same environment at runtime and test time, easier to configure

  • Super easy to test a full flow from end to end

  • Good starting point allowing you to iterate quickly

There are also some disadvantages:

  • Slow as it can start up the entirety of Spring Boot’s AutoConfiguration ecosystem. This test only ensures that the ApplicationContext is injected; this can be done faster (as written, this takes 10s to run in a non-trivial application like the Auth Services)

  • Easier for your test to fail in an esoteric way that doesn’t really inform the maintainer why it’s failing (although targeted configuration can have this effect too)

  • Initializes a lot of extra things that you are not trying to test

A faster test would only use @ContextConfiguration to do the same thing:

@ContextConfiguration
@ExtendWith(SpringExtension.class)
public class MyIT {

    @Test
    public void gotAnAppctx(ApplicationContext ctx) {
        assertNotNull(ctx);
    }
}

This test runs in 1s and still provides the exact same verification. Faster is better. But, there are plenty of legitimate cases to use @SpringBootTest, so don’t feel like you have to completely avoid it.

Configuration Slicing

Spring Boot ships a fair amount of annotations designed to only start up exactly what you need. Generally, integration testing is usually silo’d to something like "the controller layer" or "the data layer". Here’s an example:

@DataJpaTest
public class DataIT {

    @Test
    public void queryMethodWorks(SomeRepository repo) {
        repo.findByNameIgnoreCase("A Name");
    }
}

Note the lack of @SpringBootTest on this test. This test only boots up exactly what you would need within the Spring ApplicationContext to perform a @DataJpaTest, which excludes anything in the controller layer, like Spring MVC.

If you only want to boot up the controller layer to do some testing with MockMVC:

@WebMvcTest
public class NavigationEndpointTest {

    @Autowired
    MockMvc mockMvc;

    @Test
    public void requiresAuthentication() throws Exception {
        mockMvc.perform(get("/navigation")).andDo(print())
            .andExpect(status().isUnauthorized());
    }
}

However, this test could be better by further restricting the controller to test in @WebMvcTest(NavigationEndpoint.class) since it will run faster (assuming there are other controllers in your project).

Spring has a number of other of these types of 'Configuration Slicing' annotations in the spring-boot-test-autoconfigure, which is automatically brought in via spring-boot-starter-test.

Configuration Slicing vs @SpringBootTest

The next obvious question is "Okay, well which annotation should I be using for my test?" Generally, software development is done in 3 phases:

  1. Make it correct and easy to read your code; naive solutions,

  2. Optimize for performance

  3. GOTO 1

Practically, this can be applied to tests as well. Likely an @SpringBootTest is the best starting point, and then tests can be optimized from there. However, if you know all you are going to test is an @Controller, consider starting with @WebMvcTest and expanding as you need it.

Configuration Slicing with @SpringBootTest

There are some slice annotations that use a different bootstrapper than the SpringBootTestContextBootstrapper.class. If this happens, the normal Spring Boot Autoconfiguration may or may not be loaded. An example is the @WebMvcTest. This uses a WebContextBootstrapper which is similar but not the exact same. Therefore, it might make more sense to use this:

@SpringBootTest
@AutoConfigureMockMvc
public class MyIT { }

Read the Javadocs for the annotations you are using as they usually have descriptions and where and how to use them.

What if I don’t have an @SpringBootApplication in src/main/java?

In general, adding an @SpringBootApplication as either a static inner class of your test or something that can be referred to from multiple tests solely in src/test/java is fine. However, a well-optimized test might not have to include any @SpringBootApplication to run at all.

Customizing Configuration Slicing

When you start to use configuration slicing, you might notice that your configuration classes that hook up other functionality are not included, causing missing beans, etc. The issue is that Spring does not yet know about your classes

If we look at the @WebMvcTest annotation, you will see the following Spring annotations:

@BootstrapWith(WebMvcTestContextBootstrapper.class)
@OverrideAutoConfiguration(enabled = false)
@TypeExcludeFilters(WebMvcTypeExcludeFilter.class)
@AutoConfigureCache
@AutoConfigureWebMvc
@AutoConfigureMockMvc
@ImportAutoConfiguration
public @interface WebMvcTest {

Going further, @AutoConfigureWebMvc is annotated like this:

@ImportAutoConfiguration
public @interface AutoConfigureWebMvc {

@ImportAutoConfiguration is the magic that reads out of META-INF/spring.factories and allows additional classes to participate in Spring Boot’s auto configuration. In the naked version of the annotation, this will look for keys in META-INF/spring.factories that correspond to the fully-qualified class name of the class it resides on, so org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureWebMvc:

org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureWebMvc=\
    will.be.an.AutoConfigurationClass

Practical Example

Let’s say you have this example:

// JpaConfig.java
package com.broadleafcommerce.catalog.provider.jpa;

@Configuration
@EnableJpaRepositories(repositoryFactoryBeanClass = JpaTrackableRepositoryFactoryBean.class)
public class JpaConfig { }

==================

// JpaProductRepository.java
package com.broadleafcommerce.catalog.provider.jpa.repository;

@Repository
@Narrow(JpaNarrowExecutor.class)
public interface JpaProductRepository extends ProductRepository<JpaProduct> { }

You were a good citizen and made sure this was in a normal src/main/resources/META-INF/spring.factories to participate in normal AutoConfiguration:

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
        com.broadleafcommerce.catalog.provider.jpa.JpaConfig

Because of that, we can write a test that does this:

@SpringBootTest
public class ProductRepoIT {

    @Test
    public void gotARepo(JpaProductRepository repo) {
        assertNotNull(repo);
    }
}

But this test fails:

@DataJpaTest
public class ProductRepoIT {

    @Test
    public void gotARepo(JpaProductRepository repo) {
        // failed ApplicationContext startup
        assertNotNull(repo);
    }
}

@DataJpaTest is annotated like this:

@BootstrapWith(DataJpaTestContextBootstrapper.class)
@ExtendWith({SpringExtension.class})
@OverrideAutoConfiguration(
    enabled = false
)
@TypeExcludeFilters({DataJpaTypeExcludeFilter.class})
@Transactional
@AutoConfigureCache
@AutoConfigureDataJpa
@AutoConfigureTestDatabase
@AutoConfigureTestEntityManager
@ImportAutoConfiguration
public @interface DataJpaTest {

Which means we can add an entry in src/test/resources/META-INF/spring.factories under the AutoConfigureDataJpa fully qualified class name key to participate in this "slice":

org.springframework.boot.test.autoconfigure.data.jpa.AutoConfigureDataJpa=\
    com.broadleafcommerce.catalog.provider.jpa.JpaConfig

Now our test passes!

Avoiding AutoConfiguration

There is an argument for ignoring the complexity in spring.factories of the autoconfiguration altogether. The previous example can manually @Import the @Configuration class to make it arguably simpler to grok:

@DataJpaTest
@Import(JpaConfig.class)
public class ProductRepoIT {

    @Test
    public void gotARepo(JpaProductRepository repo) {
        // success!
        assertNotNull(repo);
    }
}

Accessing Broadleaf Test Sources

From Broadleaf 1.7.2 onward, test sources will be published & available for download. Using AuthServices as an example, the release now includes the following artifacts:

  • broadleaf-authentication-services-1.7.2-GA-test-sources.jar

  • broadleaf-authentication-services-1.7.2-GA-tests.jar

To make use of these jars, add the following dependency to your pom:

<dependency>
    <groupId>com.broadleafcommerce.microservices</groupId>
    <artifactId>broadleaf-authentication-services</artifactId>
    <version>1.7.2-GA</version>
    <classifier>tests</classifier>
    <type>test-jar</type>
    <scope>test</scope>
</dependency>

If you then prompt your IDE to download sources for a test class, it will automatically resolve the test sources jar. This is similar behavior to what you see with regular classes.