PIT is available from maven central since version 0.20.
Add the plugin to build/plugins in your pom.xml
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>LATEST</version>
</plugin>
That’s it, you’re up and running.
By default pitest will mutate all code in your project. You can limit which code is mutated and which tests are run using targetClasses and targetTests. Be sure to read the globs section if you want to use exact class names.
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>LATEST</version>
<configuration>
<!-- no longer required -->
<targetClasses>
<param>com.your.package.root.want.to.mutate*</param>
</targetClasses>
<!-- no longer required -->
<targetTests>
<param>com.your.package.root*</param>
</targetTests>
</configuration>
</plugin>
If no targetClasses are provided in versions before 1.2.0, pitest assumes that your classes live in a package matching your projects group id. In 1.2.0 and later versions pitest will scan your project to determine which classes are present.
PIT provides two goals
The mutation coverage goal analyses all classes in the codebase that match the target tests and target classes filters.
It can be run directly from the command-line
mvn test-compile org.pitest:pitest-maven:mutationCoverage
This will output an html report to target/pit-reports/YYYYMMDDHHMI.
To speed-up repeated analysis of the same codebase set the withHistory parameter to true.
mvn -DwithHistory test-compile org.pitest:pitest-maven:mutationCoverage
The scm mutation coverage goal analyses only classes that match the filters and the source file has a given status within the project source control system (by default ADDED or MODIFIED). This provides a quick way to check the coverage of changes prior to checking code in / pushing code to a repository.
mvn org.pitest:pitest-maven:scmMutationCoverage -Dinclude=ADDED,UNKNOWN -DmutationThreshold=85
To use this goal the maven scm plugin must be correctly configured for the project
This goal does not currently guarantee to analyse changes made to non public classes that are not inner classes.
Globs are pretty simple and will work as expected as long as you match packages (like com.your.package.root.want.to.mutate*). But if you match exact class names, inner classes won’t be included. If you need them you’ll have to either add a ‘*’ at the end of the glob to also match them (com.package.Class* instead of com.package.Class) or to add another rule for it (com.package.Class.* in addition to com.package.Class).
Introduced in 1.17.3, dry run mode configures pitest to gather coverage for all tests, and generate all mutants, without running any tests against the mutants.
This mode can be useful when first setting up pitest to iron out any problems pitest has with the test suite.
To set dry run mode from within the pom add.
<dryRun>true</dryRun>
Or from the command line set pit.dryRun
mvn -Ppitest -Dpit.dryRun=true test
By default, pitest assumes that the unit tests for your code live in the same maven module as the code.
Since 1.17.1 limited support is available for projects where unit tests are within different modules, but this must be explicitly configured.
If the property crossModule is set for a module, then pitest will mutate both code within that module and the code of any modules from the same project that the module depends on.
It is important to understand that this could result in multiple results for each class. If your project has three modules
common
a
b
Where a and b both depend on common, then if common contained a single class named Foo, there would be three results for Foo. One in the report for common, and one each in the reports for the a and b modules (assuming crossModule had been set for true in both cases).
If report aggregation is used without ensuring that each class is reported only once, the results are undefined.
To ensure each class is reported only once, the user must configure the project to match their test strategy.
If all the tests for common are contained in a, and b simply uses the code in common, then crossModule should be set for a only. If the relationship is more complicated, the targetClasses and targetClasses parameters must be set for each module so that only the intended code is mutated.
PIT tries to work sensibly out of the box, but also provides many configuration options.
The number of threads and list of mutation operators are both worth having a play with.
List of pitest features to enable or disable. Available options are shown in the console output when verbose logging is enabled. Additional features may be added by pitest plugins.
Some features are enabled by default and must be disabled with -featureName, others must be enabled explicitly with +featureName.
For example
<features>
<feature>-frecord</feature>
<feature>+auto_threads</feature>
</features>
Will disable the filtering of junk mutations in code the compiler generates to support Java records, and enable the automatic setting of the number of threads based on the number of cores reported by the current machine.
List of additional features to enable or disable. This parameter allows additional features to be activated from the command-line or build scripts without overwriting existing configuration.
e.g.
mvn -Ppitest -DextraFeatures="+GIT(from[HEAD~1]), +arcmutate_history(run_tests[false])" test-compile
Here the parameter is used to activate arcmutate’s git integration and history implementations without unintentionally deactivating features already enabled within the pom.xml.
Output directory for the reports
The classes to be mutated. This is expressed as a list of globs.
For example
<targetClasses>
<param>com.mycompany.*</param>
</targetClasses>
or
<targetClasses>
<param>com.mycompany.package.*</param>
<param>com.mycompany.packageB.Foo*</param>
<param>com.partner.*</param>
</targetClasses>
If no targetClasses are supplied pitest will automatically determine what to mutate.
Before 1.2.0 pitest assumed that all code lives in a package matching the maven group id. In 1.2.0 and later versions, the classes to mutate are determined by scanning the maven output directory.
A list of globs can be supplied to this parameter to limit the tests available to be run.
This parameter can be used to point PIT to a top level suite or suites. Custom suites such as ClassPathSuite are supported. Tests found via these suites can also be limited by the distance filter (see below).
Removed in 1.9.0
PIT can optionally apply an additional filter to the supplied tests, such that only tests a certain distance from a mutated class will be considered for running. e.g A test that directly calls a method on a mutated class has a distance of 1, a test that calls a method on a class that uses the mutee as an implementation detail has a distance of 2 etc.
This filter will not work for tests that utilise classes via interfaces, reflection or other methods where the dependencies between classes cannot be determined from the byte code.
The distance filter is particularly useful when performing a targeted mutation test of a subset of classes within a large project as it avoids the overheads of calculating the times and coverage of tests that cannot exercise the mutees.
The number of threads to use when mutation testing. By default a single thread will be used.
Support for mutating static initializers was removed in pitest 1.3.0
Whether or not to create mutations in static initializers. Defaults to false.
List of mutation operators to apply.
for example
<configuration>
<mutators>
<mutator>CONSTRUCTOR_CALLS</mutator>
<mutator>NON_VOID_METHOD_CALLS</mutator>
</mutators>
</configuration>
For details of the available mutators and the default set applied see mutation operators.
List of globs to match against method names. Methods matching the globs will be excluded from mutation.
List of globs to match against class names. Matching classes will be excluded from mutation.
Prior to 1.3.0 matching test classes were also not run. From 1.3.0 onwards tests are excluded with the excludedTests parameter
List of globs to match against test class names. Matching tests will not be run (note if a suite includes an excluded class, then it will “leak” back in).
List of packages and classes which are to be considered outside the scope of mutation. Any lines of code containing calls to these classes will not be mutated.
If a list is not explicitly supplied then PIT will default to a list of common logging packages as follows
So, the configuration section must look like:
<avoidCallsTo>
<avoidCallsTo>java.util.logging</avoidCallsTo>
<avoidCallsTo>org.apache.log4j</avoidCallsTo>
<avoidCallsTo>org.slf4j</avoidCallsTo>
<avoidCallsTo>org.apache.commons.logging</avoidCallsTo>
</avoidCallsTo>
If the feature FLOGCALL is disabled, this parameter is ignored and logging calls are also mutated.
Output verbose logging. Defaults to off/false.
A factor to apply to the normal runtime of a test when considering if it is stuck in an infinite loop.
Defaults to 1.25
Constant amount of additional time, in milliseconds, to allow a test to run for (after the application of the timeoutFactor) before considering it to be stuck in an infinite loop.
Defaults to 4000
This parameter was removed in release 1.2.3 and has been replaced by the CLASSLIMIT feature.
This can be configured as follows
features="+CLASSLIMIT(limit[42])"
The maximum number of mutations to create per class. Use 0 or -ve number to set no limit.
List of arguments to use when PIT launches child processes. This is most commonly used to increase the amount of memory available to the process, but may be used to pass any valid JVM argument.
For example when running on OpenJDK 7 the it is sometimes necessary to disable the split verifier.
<jvmArgs>
<jvmArg>-XX:-UseSplitVerifier</jvmArg>
</jvmArgs>
The path to the java executable to be used to launch test with. If none is supplied defaults to the one pointed to by JAVA_HOME.
List of formats in which to write mutation results as the mutations are analysed. Supported formats are HTML, XML, CSV.
Defaults to HTML.
Whether to throw error when no mutations found.
Defaults to true.
List of TestNG groups or JUnit Categories to exclude from mutation analysis.
List of TestNG groups or JUnit Categories to include in mutation analysis.
Maximum number of mutations to include in a single analysis unit.
Defaults to 0 (unlimited)
Export line coverage data.
Defaults to false
Engine to use when generating mutations. Additional engines may be added via plugins.
Defaults to gregor
No longer required. Test plugins are now selected automatically.
List of additional classpath entries to use when looking for tests and mutable code. These will be used in addition to the classpath with which PIT is launched.
Enabled by default since 0.29.
Indicates if PIT should attempt to detect the inlined code generated by the java compiler in order to implement finally blocks. Each copy of the inlined code would normally be mutated separately, resulting in multiple identical looking mutations. When inlined code detection is enabled PIT will attempt to spot inlined code and create only a single mutation that mutates all affected instructions simultaneously.
The algorithm cannot easily distinguish between inlined copies of code, and genuine duplicate instructions on the same line within a finally block.
In the case of any doubt PIT will act cautiously and assume that the code is not inlined.
This will be detected as two separate inlined instructions
finally {
int++;
int++;
}
But this will look confusing so PIT will assume no in-lining is taking place.
finally {
int++; int++;
}
This sort of pattern might not be common with integer addition, but things like string concatenation are likely to produce multiple similar instructions on the same line.
Defaults to false.
When set to true, PIT will create a date and time stamped folder for each output each time it runs.
Defaults to false.
Mutation score threshold at which to fail the build.
Please bear in mind that your build may contain equivalent mutations. Careful thought must therefore be given when selecting a threshold.
Line coverage threshold at which to fail the build.
Path to a file containing history information for incremental analysis.
Path to write history information for incremental analysis. May be the same as historyInputFile.
Sets the history input and output files to point a project specific file within the temp directory.
This is a convenient way of using history files to speed up local analysis.
You can skip the launch by adding the parameter skip on configuration section:
<configuration>
<skip>true</skip>
</configuration>
It’s very useful on maven module: when you need to skip an entire module, you can add this setting on the declaration of the plugin to ignore it.
When set to true causes pitest to continue processing after a test fails and record addition failing tests when XML output is enabled. This is a partially supported feature added due to demand from the research community. Other pitest features are not guaranteed to work correctly when it is enabled.
Defaults to false.
Starting with version 1.1.6, the pit maven plugin has a maven report goal. This goal should only be invoked as part of the maven site lifecycle. To execute this goal, the mutationCoverage goal must have already been executed to produce an HTML report (i.e. the outputFormat parameter must have HTML in it if the parameter is specified. The report goal then copies the latest HTML report to the site directory. If multiple reports exist (as in the case where timestampedReports is set to true), then only the report with the latest create time is used.
To generate the pit site report, set up the pitest-maven plugin in the project’s pom as explained in the Getting Started section above and the <reporting> section as explained below. Then, execute both the mutationCoverage goal and the site lifecycle. For example:
mvn clean org.pitest:pitest-maven:mutationCoverage site
The following configuration is the minimum required to generate the pit site report:
<reporting>
<plugins>
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>LATEST</version>
<reportSets>
<reportSet>
<reports>
<report>report</report>
</reports>
</reportSet>
</reportSets>
</plugin>
</plugins>
</reporting>
Additional parameters exist to customize the generation of the report. They are:
false${pit.report.skip}mutationCoverage goal wrote the pit HTML reports. This parameter does not need to be set unless the reportsDirectory parameter was set during the execution of the mutationCoverage goal. The value in this parameter must be an absolute path to the directory where the pit HTML report is located.${project.build.directory}/pit-reports${reportsDirectory}${pit.report.sourceDataFormats}${pit.report.name}${pit.report.description}target/site/pit-reports${pit.report.outputdir}<reporting>
<plugins>
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>LATEST</version>
<configuration>
<skip>false</skip>
<reportsDirectory>${project.build.directory}/pit-custom-output-dir</reportsDirectory>
<sourceDataFormats>
<sourceDataFormat>HTML</sourceDataFormat>
</sourceDataFormats>
<siteReportName>my-pit-report-name</siteReportName>
<siteReportDescription>my pit report custom description</siteReportDescription>
<siteReportDirectory>pit-custom-site-directory</siteReportDirectory>
</configuration>
<reportSets>
<reportSet>
<reports>
<report>report</report>
</reports>
</reportSet>
</reportSets>
</plugin>
</plugins>
</reporting>
PitMP (PIT for Multi-module Project) is a Maven plugin to run PIT on multi-module projects.
By default, PIT mutates only the classes defined in the same module as the test suite.
Meanwhile, PitMP runs PIT on a complete project: the test suites are executed against the mutants generated in all classes of the project. In return, it produces a global mutation score for the project. The key rationale for the plugin is that some projects include test cases that are meant to assess the correctness of code regions that are in other modules.
PitMP extends PIT, it doesn’t rewrite PIT features. So, all PIT properties can be used. PitMP runs test suite as PIT does, just extending the list of classes to be mutated to the whole project tree, instead of mutating only the classes of the test suite module.
PitMP is available in Maven Central, and source and documentation are available in PitMP github.
Arcmutate provides pull request integration and other advanced features.