Ideally we would like to get perf reports out of every build, which means we would like to do reporting as part of our Maven build, with results available as easily readable charts on our build server. The top hit you're likely to get from searching for "maven jmeter" is the awful JMeterMavenPlugin. I say awful because it wasn't easy to integrate, and if you look at the source code it's obvious that the project was done in spare time. There are a number of comments in the source like "this mess is necessary because..." which makes me think the whole thing is poorly designed, and if you search around you will indeed find that there are a number of problems people have encountered trying to use it. Finally, the output from the plugin is just the simple JMeter log, and not the reports I'd like.
All the way down in the middle of the second page of the Google results I found this gem: chronos-maven-plugin. Not only does this look like a well-designed and well-executed project, it produces wonderful HTML reports, perfect for plugging into our build server! This is a snippet of what the Chronos output looks like:
The only downside is that the Chronos plugin requires an external install of JMeter, which kind of defeats the whole purpose of Maven. Fortunately, inspired by an Atlassian post, I worked out a way to use the Chronos plugin without making JMeter a manual install by using the maven-dependency-plugin. First I deployed the JMeter ZIP file as an artifact on our Artifcatory repository:
<?xml version="1.0" encoding="UTF-8"?> <project> <modelVersion>4.0.0</modelVersion> <groupId>org.apache.jmeter</groupId> <artifactId>jmeter</artifactId> <version>2.3.4</version> <packaging>zip</packaging> <description>Artifactory auto generated POM</description> </project>
In my POM, I set jmeter.home to the location that we'll be unpacking JMeter into:
<properties> <jmeter-version>2.3.4</jmeter-version> <jmeter.home>${project.build.directory}/jakarta-jmeter-${jmeter-version}</jmeter.home> </properties>
Next I use the dependency plugin in the pre-integration-test step to unpack JMeter into the target folder:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <version>2.1</version> <executions> <execution> <id>unpack-jmeter</id> <phase>pre-integration-test</phase> <goals> <goal>unpack</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>org.apache.jmeter</groupId> <artifactId>jmeter</artifactId> <version>${jmeter-version}</version> <type>zip</type> </artifactItem> </artifactItems> <outputDirectory>${project.build.directory}</outputDirectory> </configuration> </execution> </executions> </plugin>
Finally I configure Chronos to run:
<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>chronos-maven-plugin</artifactId> <version>1.0-SNAPSHOT</version> <configuration> <input>${basedir}/src/test/jmeter/UserSession.jmx</input> </configuration> <executions> <execution> <goals> <goal>jmeter</goal> <goal>savehistory</goal> </goals> </execution> </executions> </plugin>
Bingo. Now anyone running our build can get the JMeter performance reports with nothing more complex than running "mvn verify chronos:report".
Hi
ReplyDeleteCould you please help me? I have ready jmeter report: aggregate_jmeter.jtl and I want to build graphs with maven-chronos-plugin. What I need to do?
ps: I have pure knowledge of java
It would be great if you-ll email me: koldaevav@gmail.com
Maven runs Chronos, and Chronos runs Jmeter. It's meant to take your Jmeter test plan (your .jmx file), and run it and generate the graphs. It doesn't take a Jmeter .jtl file as input. If you're not already using Maven then Chronos won't be much help to you.
ReplyDeleteHello,
ReplyDeleteCould you please share complete examples of needed Maven's xmls.
I'm very nub in Maven, but I need continuous testing with jMeter, even without builds of the application
Thanks!
hi Gabe, did you uploaded the complete jmeter.zip File to your maven repository(~18MB)? The zip contains a directories/files not needed by JMeter e.g. "docs". I'm not sure which files have to remain in the zip. On http://wiki.apache.org/jakarta-jmeter/JMeterMavenPlugin I found a JMeter.jar and a pom but this is for JMeter 2.2
ReplyDeleteMartin -- more than just JMeter.jar is required because there are a number of dependencies inside the ZIP file. I like just putting the ZIP in the repository because then it makes the process of upgrading to a new version of JMeter very simple -- just upload the new ZIP and update the version in the pom.xml
ReplyDeleteHi Gabe
ReplyDeleteHow customizable have you found Chronos, if at all? If you want to generate custom reports, with more than the bubbled up pieces of information (such as per sample or group of sample records etc), is Chronos a good tool? I did not see any such features listed in its doc
Thanks
The output report is pretty much fixed in code. You can tweak some very minor elements of the report, such as whether to include garbage collection logs.
ReplyDeleteChronos is great for getting measurements and a simple, readable report out of the build without too much work. If you need more, you need to download the source and modify it to suit your needs. I made some minor modifications in this way. But my company is probably going to move to an in-house reporting system that isn't tied to JMeter as a harness and lets us generate the graphs exactly how we want 'em.
Thanks Gabe. That is probably the direction we are headed in too. We will probably use JMeter for logging only, and use other tools or in-house tools to generate reports
ReplyDeleteSince we may open-source our reporting system, I'm curious to see what scenarios other people would be interested in. What sort of customization are you looking to do? What information do you want to extract beyond the simple graphs/data chronos provides?
ReplyDeleteThanks!
Sorry for not posting back on your last question. I have not yet figured out the reporting needs exactly.
ReplyDeleteHowever, I tried the method you mentioned, and it takes loooooooooooooooooong time (of the order of 4-5 minutes) just to unpack the zip into my test directory. Did you see this issue too?
If this is the case, I may have to resort to having local jmeter installs, and provide the location using the jmeter.home property as a command line (system property) parameter.
Thanks
No, it's quite fast for me. Are you sure it's the unpack step that's slow? Maven isn't doing other things like downloading dependencies?
ReplyDeleteIn my logs the time between the "Unpacking" step and the next one is 4 seconds.
You might try running mvn with --debug to figure out why it's so slow.
Out of interest have you had a look at the Jmeter Maven Plugin recently? Quite a bit of development effort has gone into it since you last looked at it and there is also a JMeter Analysis Maven Plugin available now as well.
ReplyDeleteI'd be interested to know if you opinion has changed:
https://github.com/Ronnie76er/jmeter-maven-plugin
I just had to say that this saved me hours of work. Thanks Gabe!
ReplyDelete