赞
踩
遇到surefire问题,最好是去官方文档看资料,看说明,因为网上关于surefire的使用经验几乎没有的。
官方说明地址:http://maven.apache.org/plugins/maven-surefire-plugin/examples/system-properties.html
在这篇文章的最末尾,可以看到<argLine>….</argLine>的配置,于是在单元测试的工程中加了插件的配置,如下所示:
这里的配置还有一个provider配置的问题,默认情况下surefire会根据工程中junit的版本来选择provider等,具体的解释可以看:http://maven.apache.org/plugins/maven-surefire-plugin/examples/junit.html
这个网页内容如下:
To get started with JUnit, you need to add the required version of JUnit to your project:
<dependencies> [...] <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> <scope>test</scope> </dependency> [...] </dependencies>
This is the only step that is required to get started - you can now create tests in your test source directory (e.g., src/test/java).
Surefire supports three different generations of JUnit: JUnit 3.8.x, JUnit 4.x (serial provider) and JUnit 4.7 (junit-core provider with parallel support). The provider is selected based on the JUnit version in your project and the configuration parameters (for parallel).
As of Surefire version 2.7, the algorithm for choosing which tests to run has changed. From 2.7 and on, only valid JUnit tests are run for all versions of JUnit, where older versions of the plugin would also run invalid tests that satisfied the naming convention.
When upgrading from a Surefire version prior to 2.7, the build can be run with the flag -Dsurefire.junit4.upgradecheck. This will perform a check and notify you of any invalid tests that will not be run with this version of Surefire (and the build fails). This is only meant to be used as a tool when upgrading to check that all expected tests will be run. It is a transitional feature that will be removed in a future version of surefire.
If nothing is configured, Surefire detects which JUnit version to use by the following algorithm:
if the JUnit version in the project >= 4.7 and the parallel attribute has ANY value use junit47 provider if JUnit >= 4.0 is present use junit4 provider else use junit3.8.1
Please note that the "else" part of this algorithm is also a FAQ response:
You depend on the appropriate version of JUnit being present in the project dependencies, or Surefire may choose the wrong provider. If, for instance, one of your dependencies pulls in JUnit 3.8.1 you risk that surefire chooses the 3.8.1 provider, which will not support annotations or any of the 4.x features.
Use mvn dependency:tree, POM dependency ordering and/or exclusion of transitive dependencies to fix this problem.
You can also manually force a specific provider by adding it as a dependency to Surefire itself:
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <dependencies> <dependency> <groupId>org.apache.maven.surefire</groupId> <artifactId>surefire-junit47</artifactId> <version>2.20</version> </dependency> </dependencies> </plugin> [...] </plugins>
When using this technique there is no check that the proper test-frameworks are present on your project's classpath. Failing to add the proper test-frameworks will result in a build failure.
From JUnit 4.7 onwards you can run your tests in parallel. To do this, you must set the parallel parameter, and may change the threadCount or useUnlimitedThreads attribute. For example:
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <parallel>methods</parallel> <threadCount>10</threadCount> </configuration> </plugin> [...] </plugins>
If your tests specify any value for the parallel attribute and your project uses JUnit 4.7+, your request will be routed to the concurrent JUnit provider, which uses the JUnit JUnitCore test runner.
This is particularly useful for slow tests that can have high concurrency.
As of Surefire 2.7, no additional dependencies are needed to use the full set of options with parallel. As of Surefire 2.16, new thread-count attributes are introduced, namely threadCountSuites, threadCountClasses and threadCountMethods. Additionally, the new attributes parallelTestsTimeoutInSeconds and parallelTestsTimeoutForcedInSeconds are used to shut down the parallel execution after an elapsed timeout, and the attribute parallel specifies new values.
See also Fork Options and Parallel Test Execution.
The junit4 and junit47 providers provide support for attaching custom RunListeners to your tests.
You can configure multiple custom listeners like this:
<dependencies> [...] <dependency> <groupId>your-junit-listener-artifact-groupid</groupId> <artifactId>your-junit-listener-artifact-artifactid</artifactId> <version>your-junit-listener-artifact-version</version> <scope>test</scope> </dependency> [...] </dependencies> [...] <plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <properties> <property> <name>listener</name> <value>com.mycompany.MyResultListener,com.mycompany.MyResultListener2</value> </property> </properties> </configuration> </plugin> [...] </plugins>
For more information on JUnit, see the JUnit web site.
You can implement JUnit listener interface org.junit.runner.notification.RunListener in a separate test artifact your-junit-listener-artifact with scope=test, or in project test source code src/test/java. You can filter test artifacts by the parameter dependenciesToScan to load its classes in current ClassLoader of surefire-junit* providers.
Since JUnit 4.12 thread safe listener class should be annotated by org.junit.runner.notification.RunListener.ThreadSafe which avoids unnecessary locks in JUnit.
As long as forkCount is not 0 and you use JUnit3, you can run your tests with a Java security manager enabled. The class name of the security manager must be sent as a system property variable to the JUnit3 provider.
JUnit4 uses mechanisms internally that are not compatible with the tested security managers and thus this means of configuring a security manager with JUnit4 is not supported by Surefire.
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <systemPropertyVariables> <surefire.security.manager>java.lang.SecurityManager</surefire.security.manager> </systemPropertyVariables> </configuration> </plugin> [...] </plugins>
Alternatively you can define a policy file that allows all providers to run with Surefire and configure it using the argLine parameter and two system properties:
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <argLine>-Djava.security.manager -Djava.security.policy=${basedir}/src/test/resources/java.policy</argLine> </configuration> </plugin> [...] </plugins>
The disadvantage of this solution is that the policy changes will affect the tests too, which make the security environment less realistic.
JUnit 4.8 introduced the notion of Categories. You can use JUnit categories by using the groups parameter. As long as the JUnit version in the project is 4.8 or higher, the presence of the "groups" parameter will automatically make Surefire select the junit47 provider, which supports groups.
<plugins> [...] <plugin> <artifactId>maven-surefire-plugin</artifactId> <version>2.11</version> <configuration> <groups>com.mycompany.SlowTests</groups> </configuration> </plugin> [...] </plugins>
This will execute only those tests annotated with the @Category(com.mycompany.SlowTests.class) annotation and those tests annotated with @Category(com.mycompany.SlowerTests.class) if class/interface SlowerTests is subclass of SlowTests:
public interface SlowTests{} public interface SlowerTests extends SlowTests{}
public class AppTest { @Test @Category(com.mycompany.SlowTests.class) public void testSlow() { System.out.println("slow"); } @Test @Category(com.mycompany.SlowerTests.class) public void testSlower() { System.out.println("slower"); } @Test @Category(com.cmycompany.FastTests.class) public void testSlow() { System.out.println("fast"); } }
The @Category annotation can also be applied at class-level.
For more information on JUnit, see the JUnit web site.
Since version 2.18.1 and JUnit 4.12, the @Category annotation type is automatically inherited from superclasses, see @java.lang.annotation.Inherited. Make sure that test class inheritance still makes sense together with @Category annotation of the JUnit 4.12 or higher appeared in superclas
关于surefire集成TestNG的官方说明:http://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html 内容如下:
- TestNG 5.14.3: Bad formatted pom.xml. - TestNG 5.14.4 and 5.14.5: TestNG is using a missing dependency (org.testng:guice:2.0). Excluding it, may break some features.
To get started with TestNG, include the following dependency in your project (replacing the version with the one you wish to use):
<dependencies> [...] <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.9.8</version> <scope>test</scope> </dependency> [...] </dependencies>
If you are using an older version of TestNG (<= 5.11), the dependency would instead look like this:
<dependencies> [...] <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>5.11</version> <scope>test</scope> <classifier>jdk15</classifier> </dependency> [...] </dependencies>
Note: if you are using JDK 1.4 Javadoc annotations for your TestNG tests, replace the classifier jdk15 with jdk14 above.
This is the only step that is required to get started - you can now create tests in your test source directory (e.g., src/test/java). As long as they are named in accordance with the defaults such as *Test.java they will be run by Surefire as TestNG tests.
If you'd like to use a different naming scheme, you can change the includes parameter, as discussed in the Inclusions and Exclusions of Tests example.
Another alternative is to use TestNG suite XML files. This allows flexible configuration of the tests to be run. These files are created in the normal way, and then added to the Surefire Plugin configuration:
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <suiteXmlFiles> <suiteXmlFile>testng.xml</suiteXmlFile> </suiteXmlFiles> </configuration> </plugin> [...] </plugins>
This configuration will override the includes and excludes patterns and run all tests in the suite files.
Your TestNG test can accept parameters with the @Parameters annotation. You can also pass parameters from Maven into your TestNG test, by specifying them as system properties, like this:
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <systemPropertyVariables> <propertyName>firefox</propertyName> </systemPropertyVariables> </configuration> </plugin> [...] </plugins>
For more information about setting system properties in Surefire tests, see System Properties.
TestNG allows you to group your tests. You can then execute one or more specific groups. To do this with Surefire, use the groups parameter, for example:
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <groups>functest,perftest</groups> </configuration> </plugin> [...] </plugins>
Likewise, the excludedGroups parameter can be used to run all but a certain set of groups.
TestNG allows you to run your tests in parallel, including JUnit tests. To do this, you must set the parallel parameter, and may change the threadCount parameter if the default of 5 is not sufficient. For example:
</plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <parallel>methods</parallel> <threadCount>10</threadCount> </configuration> </plugin> [...] </plugins>
This is particularly useful for slow tests that can have high concurrency, or to quickly and roughly assess the independence and thread safety of your tests and code.
TestNG 5.10 and plugin version 2.19 or higher allows you to run methods in parallel test using data provider.
</plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <properties> <property> <name>parallel</name> <value>methods</value> </property> <property> <name>dataproviderthreadcount</name> <value>30</value> </property> </properties> </configuration> </plugin> [...] </plugins>
TestNG 6.9.8 (JRE 1.7) and plugin version 2.19 or higher allows you to run suites in parallel.
</plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <suiteXmlFiles> <file>src/test/resources/testng1.xml</file> <file>src/test/resources/testng2.xml</file> </suiteXmlFiles> <properties> <property> <name>suitethreadpoolsize</name> <value>2</value> </property> </properties> </configuration> </plugin> [...] </plugins>
See also Fork Options and Parallel Test Execution.
TestNG provides support for attaching custom listeners, reporters, annotation transformers and method interceptors to your tests. By default, TestNG attaches a few basic listeners to generate HTML and XML reports.
Unsupported versions: - TestNG 5.14.1 and 5.14.2: Due to an internal TestNG issue, listeners and reporters are not working with TestNG. Please upgrade TestNG to version 5.14.9 or higher. Note: It may be fixed in a future surefire version.
You can configure multiple custom listeners like this:
<dependencies> [...] <dependency> <groupId>your-testng-listener-artifact-groupid</groupId> <artifactId>your-testng-listener-artifact-artifactid</artifactId> <version>your-testng-listener-artifact-version</version> <scope>test</scope> </dependency> [...] </dependencies> [...] </plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> <properties> <property> <name>usedefaultlisteners</name> <value>false</value> <!-- disabling default listeners is optional --> </property> <property> <name>listener</name> <value>com.mycompany.MyResultListener,com.mycompany.MyAnnotationTransformer,com.mycompany.MyMethodInterceptor</value> </property> <property> <name>reporter</name> <value>listenReport.Reporter</value> </property> </properties> </configuration> </plugin> [...] </plugins>
For more information on TestNG, see the TestNG web site.
You can implement TestNG listener interface org.testng.ITestListener in a separate test artifact your-testng-listener-artifact with scope=test, or in project test source code src/test/java. You can filter test artifacts by the parameter dependenciesToScan to load its classes in current ClassLoader of surefire-testng provider. The TestNG reporter class should implement org.testng.IReporter.
Since the plugin version 2.19 or higher the verbosity level can be configured in provider property surefire.testng.verbose. The verbosity level is between 0 and 10 where 10 is the most detailed. You can specify -1 and this will put TestNG in debug mode (no longer slicing off stack traces and all). The default level is 0.
</plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> [...] <properties> <property> <name>surefire.testng.verbose</name> <value>10</value> </property> </properties> [...] </configuration> </plugin> [...] </plugins>
Since the plugin version 2.19 and TestNG 5.7 or higher you can customize TestNG object factory by implementing org.testng.IObjectFactory and binding the class name to the key objectfactory in provider properties:
</plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> [...] <properties> <property> <name>objectfactory</name> <value>testng.objectfactory.TestNGCustomObjectFactory</value> </property> </properties> [...] </configuration> </plugin> [...] </plugins>
Since the plugin version 2.19 and TestNG 5.9 or higher you can customize TestNG runner factory by implementing org.testng.ITestRunnerFactory and binding the class name to the key testrunfactory in provider properties:
</plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> [...] <properties> <property> <name>testrunfactory</name> <value>testng.testrunnerfactory.TestNGCustomTestRunnerFactory</value> </property> </properties> [...] </configuration> </plugin> [...] </plugins>
Only tests defined in a test tag matching one of these names will be run. In this example two tests run out of 7; namely InstallTest and ATest. The test a-t3 does not match any test in suite.xml.
</plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> [...] <suiteXmlFiles> <file>src/test/resources/suite.xml</file> </suiteXmlFiles> <properties> <property> <name>testnames</name> <value>a-t1,a-t3</value> </property> </properties> [...] </configuration> </plugin> [...] </plugins>
The test suite 'suite.xml' :
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="Component Tests" verbose="2" annotations="JDK"> <test name="a-t1" preserve-order="true" > <classes> <class name="server.InstallTest" /> <class name="server.ATest" /> </classes> </test> <test name="a-t2" preserve-order="true" > <classes> <class name="server.SCHTest" /> <class name="server.PRGTest" /> <class name="server.SIBBTest" /> <class name="server.EDNTest" /> <class name="server.PPVTest" /> </classes> </test> </suite>
TestNG 6.5.1 and higher provides support to run TestNG and JUnit 4.x in current Maven project. All you need is to introduce TestNG and JUnit dependency in POM.
<dependencies> [...] <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.9.4</version> <scope>test</scope> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> <scope>test</scope> </dependency> [...] </dependencies>
You may want to run two providers, e.g. surefire-junit47 and surefire-testng, and avoid running JUnit tests within surefire-testng provider by setting property junit=false.
<plugins> [...] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.20</version> <configuration> [...] <properties> <property> <name>junit</name> <value>false</value> </property> </properties> <threadCount>1</threadCount> [...] </configuration> <dependencies> <dependency> <groupId>org.apache.maven.surefire</groupId> <artifactId>surefire-junit47</artifactId> <version>2.20</version> </dependency> <dependency> <groupId>org.apache.maven.surefire</groupId> <artifactId>surefire-testng</artifactId> <version>2.20</version> </dependency> </dependencies> </plugin> [...] </plugins>
关于FAQ介绍 http://maven.apache.org/surefire/maven-surefire-plugin/faq.html#dumpfiles
maven-surefire-plugin is designed for running unit tests and if any of the tests fail then it will fail the build immediately.
maven-failsafe-plugin is designed for running integration tests, and decouples failing the build if there are test failures from actually running the tests.
Visit this link for your reference, | Attaching tests
Surefire does not support tests or any referenced libraries calling System.exit() at any time. If they do so, they are incompatible with Surefire and you should probably file an issue with the library/vendor. Alternatively the forked VM could also have crashed for a number of reasons. Look for the classical "hs_err*" files indicating VM crashes or examine the Maven log output when the tests execute. Some "extraordinary" output from crashing processes may be dumped to the console/log. If this happens on a CI environment and only after it runs for some time, there is a fair chance your test suite is leaking some kind of OS-level resource that makes things worse at every run. Regular OS-level monitoring tools may give you some indication.
After a forked JVM has crashed the console of forked JVM prints Crashed tests: and lists the last test which has crashed. In the console log you can find the message The forked VM terminated without properly saying goodbye.
Use the following configuration:
<useSystemClassLoader>true</useSystemClassLoader>
<useManifestOnlyJar>false</useManifestOnlyJar>
<forkCount>1</forkCount>
Try reuseForks=true and if it doesn't work, fall back to reuseForks=false
Maven does property replacement for
${...}
Since the Version 2.17 using an alternate syntax for these properties,
@{...}
By default maven-failsafe-plugin uses project artifact file in test classpath if packaging is set to "jar" in pom.xml. This can be modified and for instance set to main project classes if you use configuration parameter "classesDirectory". This would mean that you set value "${project.build.outputDirectory}" for the parameter "classesDirectory" in the configuration of plugin.
By default maven-failsafe-plugin and maven-surefire-plugin dumps fatal errors in dump files and these are located in target/failsafe-reports and target/surefire-reports. Names of dump files are formatted as follows:
[date]-jvmRun[N].dump
[date]-jvmRun[N].dumpstream
[date].dumpstream
Forked JVM process and plugin process communicate via std/out. If this channel is corrupted, for a whatever reason, the dump of the corrupted stream appears in *.dumpstream.
在这里甚至发现了一个forkMode的参数,原来运行mvn test也不是一定要启动另外一个fork进程的,通过forkMode可以修改这个配置,forkMode默认值是once,这个配置会启动一个新的jvm fork进程用来跑单元测试,把forkMode参数设置成never,单元测试跑的时候就会由maven的主进程来跑了,也就是说这时候再通过MAVEN_OPTS来进行设置jvm参数也是可以行得通的了。
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。