Dynamic and Parameterised Tests
Estimated time to read: 13 minutes
Dynamic Tests¶
@Test
¶
With JUnit, test methods are specified using the @Test
annotation, these can not be changed at runtime.
For example, if the below method needed to be run on multiple sets of data...
... one way would be to create a list and iterate it to act and assert on different data each time.
@Test
void testRewardProgram() {
/// ...
List<TestData> list = createTestData();
for(TestDate data : list) {
// ...
assertEquals(type, reward.getType());
}
}
@RepeatedTest
¶
The @RepeatedTest
annotation could also be used along with the RepetitionInfo
interface and removal of the for
loop. However, this still acts and asserts on different data each time.
@RepeatedTest(10)
void testRewardProgram(RepetitionInfo repetitionInfo) {
// ...
TestData data = list.get(repetitionInfo.getCurrentRepetition() -1));
// ...
assertEquals(type, reward.getType());
}
@TestFactory
(Dynamic Tests)¶
JUnit 5 introduced dynamic tests, which are generated by a method annotated with @TestFactory
. @TestFactory
is not a test method by itself, but rather a factory of tests.
Note
The @TestFactory
method must not be private or static. Otherwise, it will not be executed by JUnit.
Data Sources¶
The @TestFactory
can use the following as data sources:
- Collections
- Iterable Interfaces
- Iterator Interfaces
- Streams
- Arrays
DynamicNode¶
Data sources must contain elements of type DynamicNode
. DynamicNode
is an abstract class that is the parent of DynamicContainer
and DynamicTest
.
DynamicContainer¶
DynamicContainer
is a container of DynamicTest
that has its own display name and contains either an iterable or a stream of DynamicNodes. It can contain other DynamicContainers
or DynamicTest
s.
DynamicTest¶
DynamicTest
represents the tests generated at runtime. It is composed of a display name and an Executable
. The Executable
is a functional interface that wraps the code of the test so it can be provided as a lambda expression or method reference.
Note
@BeforeEach
and @AfterEach
are not executed for each dynamic test!
Note
Dynamic tests have no notion of the lifecycle.
Example¶
public class RewardByGiftServiceDynamicTest {
@BeforeEach
void setUp() { // The setUp method will only be executed once for all tests within the @TestFactory, not for each dynamic test.
System.out.prinln("BeforeEach");
}
@TestFactory // Add the `@TestFactory` annotation.
Collection<DynamicTest> dynamicTestsFromCollection() { // Create a collection of DynamicTest objects
return Arrays.asList(
dynamicTest( // Static dynamic test method
"1st dynamic test" // Display name
() -> assertEquals(1,1)), // Lambda with implements the `Executable` interfaces.
dynamicTest(
"2nd dynamic test"
() -> assertEquals(1,1))
);
}
private LongStream getStreamOfRandomNumbers() {
Random r = new Random();
return r.longs(1000, 2000)
}
private List<Product> getSampleOrder() {
Product smallDecaf = new Product(1, "Small Decaff", 1.99);
Product bigDecaf = new Product(2, "Big Decaff", 2.49);
Product bigLatte = new Product(3, "Big Latte", 2.99);
Product bigTea = new Product(4, "Big Tea", 2.99);
Product espresso = new Product(5, "Espresso", 2.99);
return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
}
}
public class RewardByGiftServiceDynamicTest {
@BeforeEach
void setUp() { // The setUp method will only be executed once for all tests within the @TestFactory, not for each dynamic test.
System.out.prinln("BeforeEach");
}
@TestFactory
Iterator<DynamicTest> dynamicTestsFromCollection() {
return Arrays.asList(
dynamicTest( // Static dynamic test method
"1st dynamic test" // Display name
() -> assertEquals(1,1)), // Lambda with implements the `Executable` interfaces.
dynamicTest(
"2nd dynamic test"
() -> assertEquals(1,1))
).iterator();
}
private LongStream getStreamOfRandomNumbers() {
Random r = new Random();
return r.longs(1000, 2000)
}
private List<Product> getSampleOrder() {
Product smallDecaf = new Product(1, "Small Decaff", 1.99);
Product bigDecaf = new Product(2, "Big Decaff", 2.49);
Product bigLatte = new Product(3, "Big Latte", 2.99);
Product bigTea = new Product(4, "Big Tea", 2.99);
Product espresso = new Product(5, "Espresso", 2.99);
return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
}
}
public class RewardByGiftServiceDynamicTest {
private RewardByGiftService reward;
@BeforeEach
void setUp() {
reward = new RewardByGiftService();
reward.setNeededPoints(100);
System.out.prinln("BeforeEach");
}
@TestFactory
Stream<DynamicTest> giftProductNotInOrderRewardNotApplied() {
return getSTreamOfRandomNumbers() // Get a stream of random numbers
.limit(5) // Limit the stream to five results
.mapToObj(random Id -> // Use a map method to take the randomId and call the `DynamicTest` static method
dyanmicTest(
"Testing Product ID" + randomId, // Set the display name
() -> { // Lambda expression
reward.setGiftProductId(randomId); // Set the random number as the giftProductId
RewardInformation info = reward.applyReward(getSampleOrder(), 200); // Apply the reward
assertEquals(0, info.getDiscount()); // Assert that the discount is 0
assertEquals(0. info.getPointsRedeemed()); // Assert that the pointsRedeemed is 0
}
)
}
private LongStream getStreamOfRandomNumbers() {
Random r = new Random();
return r.longs(1000, 2000)
}
private List<Product> getSampleOrder() {
Product smallDecaf = new Product(1, "Small Decaff", 1.99);
Product bigDecaf = new Product(2, "Big Decaff", 2.49);
Product bigLatte = new Product(3, "Big Latte", 2.99);
Product bigTea = new Product(4, "Big Tea", 2.99);
Product espresso = new Product(5, "Espresso", 2.99);
return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
}
}
public class RewardByGiftServiceDynamicTest {
private RewardByGiftService reward;
@BeforeEach
void setUp() {
reward = new RewardByGiftService();
reward.setNeededPoints(100);
System.out.prinln("BeforeEach");
}
// @TestFactory
// Stream<DynamicTest> giftProductNotInOrderRewardNotApplied() {
// return DynamicTest.stream( // DynamicTest.stream takes three parameters. The latter two must take the same type as the initial generator. See the example.
// inputGeneratorIterator, // An iterator the serves as a dynamic input generator.
// displayNameGeneratorFuction, //The interface function generates a displayName based on the input value.
// testExecutorThrowingConsumer // A consumer the executes a test based on the input value.
// )
// }
@TestFactory
Stream<DynamicTest> giftProductNotInOrderRewardNotApplied() {
Iterator<Long> inputGeneratorIterator = getStreamOfRandomNumbers().limit(5).iterator(); // Streams have a method to convert a stream to an iterator.
Function<Long, String> displayNameGeneratorFunction = randomId - > "Testing Product ID " + randomId;
ThrowingConsumer<Long> testExecutorThrowingConsumer = randomId -> {
reward.setGiftProductId(randomId);
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertEquals(0, info.getDiscount());
assertEquals(0, info.getPointsRedeemed());
};
return DynamicTest.stream(
inputGeneratorIterator,
displayNameGeneratorFunction,
testExecutorThrowingConsumer
);
}
private LongStream getStreamOfRandomNumbers() {
Random r = new Random();
return r.longs(1000, 2000)
}
private List<Product> getSampleOrder() {
Product smallDecaf = new Product(1, "Small Decaff", 1.99);
Product bigDecaf = new Product(2, "Big Decaff", 2.49);
Product bigLatte = new Product(3, "Big Latte", 2.99);
Product bigTea = new Product(4, "Big Tea", 2.99);
Product espresso = new Product(5, "Espresso", 2.99);
return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
}
}
public class RewardByGiftServiceDynamicTest {
private RewardByGiftService reward;
@BeforeEach
void setUp() {
reward = new RewardByGiftService();
reward.setNeededPoints(100);
System.out.println("BeforeEach");
}
@TestFactory
Stream<DynamicContainer> dynamicTestsWithContainers() {
return longStream.range(1,6) // Create a stream of numbers from 1 to 6
.mapToObj(productId -> dynamicContainer( // Map / convert the objects to DynamicContainer objects.
"Container for ID" + productId, // Display name
Stream.of( // DynamicContainer can take a stream of DynamicNodes
dynamicTest("Valid Id", () -> assertTrue(productId > 0)) // DynamicTest to assert that productId is greater than 0
dynamicContainer("Test", Stream.of( // dynamicContainer which contains a dynamicTest, to test that a discount has been applied as the numbers generated are the ones that happen to be in the sample order
dynamicTest("Discount applied", () -> {
reward.setGiftProductId(productId);
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() > 0);
})
)
)
));
}
private LongStream getStreamOfRandomNumbers() {
Random r = new Random();
return r.longs(1000, 2000)
}
private List<Product> getSampleOrder() {
Product smallDecaf = new Product(1, "Small Decaf", 1.99);
Product bigDecaf = new Product(2, "Big Decaf", 2.49);
Product bigLatte = new Product(3, "Big Latte", 2.99);
Product bigTea = new Product(4, "Big Tea", 2.99);
Product espresso = new Product(5, "Espresso", 2.99);
return Arrays.asList(smallDecaf, bigDecaf, bigLatte, bigTea, espresso);
}
}
Parameterised Tests¶
Tests can be run multiple times with different arguments by using parameterised tests.
This can be done through the @ParameterizedTest
annotation.
Note
@ParameterizedTest
replaces the @Test
annotation.
Note
@ParamterizedTests
have the same life cycle as regular test methods.
Consuming Parameters¶
A source must be provided to @ParamterizedTest
which will provide the arguments for each test invocation.
Parameterised tests consume arguments form a source whilst adhering to the following rules:
- Zero or more indexed arguments must be declared. There is typically a 1-1 relationship between the argument source index and the method parameter index.
- Zero or more aggregators must be declared next. Multiple parameters can be aggregated into one.
- Zero or more arguments supplied by a ParameterResolved must be declared last.
Indexed Arguments¶
Indexed arguments are provided by an implementation of the ArgumentsProvider
interface. ArgumentsProvider
provides indexed arguments to a @ParameterizedTest
method.
Each argument provided corresponds to a single method parameter; arguments are passed at the same index in the method's parameter list.
An ArgumentsProvider
can be registered with the @ArgumentsSource
annotation.
Argument Aggregators¶
ArgumentsAccessor
can be used to aggregate multiple arguments into one. An instance of the object is automatically injected into any parameter of the same type. ArgumentsAccessor
defined an API for accessing multiple arguments through a single one passed to the test method.
Custom aggregators can also be created by implementing the ArgumentsAggregator
interface. It can then be registered with the @AggregateWith
annotation.
Inject Paramaters¶
The ParameterResolver
interface can be used to inject other types of parameters into a test. The ParameterResolver
interface defines an API to dynamically resolve parameters at runtime.
Three built in resolvers are registered automatically, these are:
TestInfoParameterResolver
- To inject parameters of typeTestInfo
in all types of test and lifecycle methods;@Test
,@RepeatedTest
,@ParameterizedTest
,@TestFactory
.RepetitionInfoParameterResolver
- To inject parameters of type RepetitionInfo for methods annotated with@RepeatedTest
,@BeforeEach
and@AfterEach
TestReporterParameterResolver
- To inject parameters of typeTestReporter
in all types of test methods;@Test
,@RepeatedTest
,@ParameterizedTest
,@TestFactory
,@BeforeEach
,@AfterEach
JUnit Dependency¶
To use parameterised tests, the following dependency is required.
Group ID: org.junit.jupiter Artifact ID: junit-jupiter-params Version: 5.x.x
Custom Display Names¶
@ParameterizedTest
's can have custom display names, like other types of test. There is a default format, however, this can be overridden.
- {displayName} - The displayName of the method
- {index} - The current invocation index of the parameter source, starting from one
- {arguments} - The complete comma separated arguments list
- {argumentsWithNames} - The complete comma separated argument list with names included
- {0}, {1}, ... - Each argument can be called by using its identifier. Note that these start at 0
Example¶
junit-jupiter
includes junit-jupiter-params
as a transitive dependency. Ensure that JUnit is included in the pom.xml for the project.
// ...
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
// ...
// ...
public class RewardByGiftServiceParameterizedTest {
private RewardByGiftService = null;
@BeforeEach
void setUp() {
reward = new RewardByGiftService();
reward.setNeededPoints(100);
System.out.println("BeforeEach");
}
// @ParameterizedTest // Annotate the test with @ParamterizedTest
@ParameterizedTest(name = "Test #{index}: productId={0}") // Passing the name argument to the annotation, along with a value, will name each test individually using the variables for ParameterizedTest.
@ValueSource(longs = {1, 2, 3, 4}) // ValueSource allows specifying an array of strings or primitive types. It provides a single parameter to the method. This just happens to be the single argument expected in the code below.
// Note that the ValueSource has been created with `longs`, this means the argument value must also be long.
void discountShouldBeApplied(long productId) {
reward.setGiftProductId(productId);
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() > 0);
}
@ParameterizedTest(name = "Test #{index}: productId={0}")
@ValueSource(longs = {5, 6, 7, 8})
@DisplayName("Display Name")
void discountShouldBeApplied(long productId, TestInfo testInfo, TestReporter testReporter) {
System.out.println("Display name: " + testInfo.getDisplayName());
testReporter.publishEntry("ProductID", String.valueOf(productId));
reward.setGiftProductId(productId);
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() > 0);
}
// ...
}
Note
The @BeforeEach
method will still run before each parameterized test as it follows the lifecycle of regular test methods. This is in contrast to @TestFactory
!
Argument Sources¶
In parameterized tests, arguments can be provided by sources via the use of annotations.
There are three rules to consider when using argument sources:
- For every test method, there should be at least one source, else the test will not be executed
- Each source must provide arguments for all expected method parameters
- The test will be executed once for each group of arguments
@ValueSource¶
@ValueSource
allows defining arrays of type:
- java.lang.String
- java.lang.Class
- Primitives (int, boolean, et al)
Can only be used on test methods that accept a single parameter.
Example¶
@ParameterizedTest(name = "Test #{index}: productId={0}")
@ValueSource(longs = {5, 6, 7, 8})
@DisplayName("Display Name")
void discountShouldBeApplied(long productId, TestInfo testInfo, TestReporter testReporter) {
System.out.println("Display name: " + testInfo.getDisplayName());
testReporter.publishEntry("ProductID", String.valueOf(productId));
reward.setGiftProductId(productId);
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() > 0);
}
@EnumSource¶
@EnumSource
can be used to run a test with the values of a provided enum.
It takes, as optional parameters:
- Names of the enum values to be included / excluded
- Mode of the parameter, depending on the value
Can only be used on test methods that accept a single parameter.
Example¶
@ParameterizedTest
@EnumSource(SpecialProductsEnum.class)
void discountShouldBeAppliedEnumSource(SpecialProductsEnum product) {
reward.setGiftProductId(product.getProductId());
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() >0);
}
@ParameterizedTest
@EnumSource(SpecialProductsEnum.class, names= = {"BIG_LATTE", "BIG_TEA"}) // Specifying names allows only certain values to be tested from the Enum.
void discountShouldBeAppliedEnumSourceLimited(SpecialProductsEnum product) {
reward.setGiftProductId(product.getProductId());
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() >0);
}
public enum SpecialProductsEnum {
SMALL_DECAF(1),
BIG_DECAF(2),
BIG_LATTE(3),
BIG_TEA(4),
ESPRESSO(5);
private final int productId;
private SpecialProductsEnum(int productId) {
this.productId = productId;
}
public int getProductId(){
return productId;
}
}
@MethodSource¶
@MethodSource
allows the specification of one or more methods that will provide the arguments for the test.
For single parameter tests, the parsed methods can:
- Return a Stream of the parameter type
- Return a Stream of primitive types
For multiple parameter tests, the parsed methods can:
- Return a Stream, Iterable, Iterator, or array of elements with type
Arguments
.Arguments
is an interface for wrapping an array of objects.
The methods parsed by @MethodSource
must be static, unless a @TestInstance(Lifecycle.PER_CLASS)
lifecycle is in use; in this case, methods can be defined in an external class as long as a fully qualified method name is given.
Example¶
@ParameterizedTest
@MethodSource("productIds")
void discountShouldBeAppliedSource(long productId) {
reward.setGiftProductId(product.getProductId());
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() >0);
}
private static LongStream productIds() { // This method can be private, however, it must be static!
return LongStream.range(1,6);
}
@ParameterizedTest
@MethodSource("productIdsCustomerPoints")
void discountShouldBeAppliedSourceMultiParam(long productId, long customerPoints) {
reward.setGiftProductId(product.getProductId());
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() >0);
}
static Stream<Arguments> productIdsCustomerPoints() { // Returns a string of `Arguments` object.
return productIds().mapToObj(productId -> Arguments.of(productId, 100 * productId));
// Outputs 2, 200
}
@CsvSource¶
@CsvSource
allows the declaration of arguments as a comma-separated list of strings.
It takes:
- delimiter - To specify the delimiting character. This is
,
by default. - delimiterString
Single quotes '
are used as quote characters.
@CsvFileSource¶
@CsvFileSource
takes one or more CSV files from the CLASSPATH or the local file system.
It takes:
- files - To specify files from the file system
- resources - To specify files from the CLASSPATH
- attributes - To specify the files' encoding
- lineSeparator
- delimiter - To specify the delimiting character. This is
,
by default. - delimiterString
Each line of the CSV files results in a test invocation.
Lines beginning as a #
will interpreted as a comment within the file.
Double quotes "
are used as quote characters.
Example¶
@ParameterizedTest
@CsvFileSource(resources ="/product-point-data.csv")
void discountShouldBeAppliedCsvFileSource(long productId, long customerPoints) {
reward.setGiftProductId(product.getProductId());
RewardInformation info = reward.applyReward(getSampleOrder(), 200);
assertTrue(info.getDiscount() >0);
}
Null and Empty sources¶
@NullSource
provides a null argument. It can not be used for primitive type arguments.@EmptySource
provides an empty value for parameters of type String, List, Set, Map or arrays.@NullAndEmptySource
combines the functionality of@NullSource
and@EmptySource
.
@ArgumentsSource¶
@ArgumentsSrouce
allows defining custom sources through an ArgumentsProvider
interface implementation. This interface returns a string of elements of type Arguments
and takes an object of type ExtensionContext
.
interface ArgumentsProvider {
Stream<? extends Arguments>
provideArguments(ExtensionContext context)
throws Exception;
}
Example¶
public class ProductIdArgumentsProvider implements ArgumentsProvider {
@Override
public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
return LongStream.range(1, 6)
.mapToObj(
productId ->
Arguments.of(productId, 200 * productId)
);
}
}
@ParameterizedTest
@ArgumentsSource(ProductIdArgumentsProvider.class)
void discountShouldBeAppliedArgumentsSource(long productId, long customerPoints) {
reward.setGiftProductId(productId);
RewardInformation info = reward.applyReward(
getSampleOrder(), customerPoints);
assertTrue(info.getDiscount() > 0);
}
Argument Conversion¶
When working with parameterized tests, if the argument and the parameter types do not match, the argument type can be converted implicitly.
String to Object Conversion¶
Implicit Conversion¶
In particular, for the annotations @ValueSource, @CsvSource and @CsvFileSource, the implicit conversion is performed when the argument is provided as a string.
A String type can be converted implicitly to:
- Primitive types and their wrapper classes
- Enums
- java.time classes
- localDate
- period
- et al
- Path
- Currency
- Locale
- And more...
Factory Conversion¶
Strings can also be converted to a particular type if the type declares exactly one non-private static factory method that accepts a single string argument and returns an instance of the type.
The name of the method is irrelevant. However, if there are multiple factory methods with these characteristics, no conversion will happen.
public class MyTest {
@ParameterizedTest
@ValueSource(strings = "Latte")
void testProduct(Product p) {
// ...
}
}
public class Product {
private long id;
private String name;
private double price;
// ...
public static Product factoryMethod(String name) {
return new Product(-1, name, 0.0);
}
}
Constructor Conversion¶
JUnit can also use a non-private constructor of the type that accepts a single string argument, but the type must be declared either as a top-level class or as a static nested class.
If a factory method and a constructor are discovered, then the factory method will be used.
public class MyTest {
@ParameterizedTest
@ValueSource(strings = "Latte")
void testProduct(Product p) {
// ...
}
}
public class Product {
private long id;
private String name;
private double price;
// ...
public Product(String name) {
this.id = -1;
this.name = name;
this.price = 0.0;
}
}
Custom Converters¶
A custom converter can be created using the ArgumentConverter
interface and then put into use by using the @ConvertWith
annotation.
interface ArgumentConverter {
Object convert(Object source, ParameterContext context)
throws ArgumentConversionException;
}
The abstract class SimpleArgumentConverter
extends ArgumentConverter
. Its convert method receives the target type that the source object should be converted in to.
abstract class SimpleArgumentConverter implements ArgumentConverter {
protected abstract Object convert(Object source, Class<?> targetType)
throws ArgumentConversionException;
}
The abstract class TypedArgumentConverter
extends ArgumentConverter
and avoid type checks
abstract class TypedArgumentConverter<S, T> implements ArgumentConverter {
protected abstract T convert(S source)
throws ArgumentConversionException;
}
Conversion Example¶
@ParameterizedTest
@ValueSource(string = { "1; Small Decaf; 1..99", "2; Big Decaf; 2.49"})
void discountShouldBeAppliedCustomConverter(@ConvertWith(ProductArgumentConverter.class)Product product) {
System.out.println("Testing product " + product.getName());
reward.setGiftProductId(product.getId());
RewardInformation info - reward.applyReward(getSampleOrder(), 200)
assertTrue(info.getDiscount() > 0);
}
ProductArgumentConverter
extends from TypedArgumentConveter
. It takes a String
as the input type and Product
as the output type.
// ...
public class ProductArgumentConverter extends TypedArgumentConverter<String, Product> {
protected ProductArgumentConverter() { // Implement the constructor and include the input/output types
super(String.class, Product.class);
}
@Override
protected Product convert(String source) {
String[] productString = source.split(";");
Product product = new Product(
Long.parseLong(productString[0]),
productString[1].trim(),
Double.parseDouble(productString[2])
);
return product;
}