New features in JMeter 2.13?

JMeter 2.13 is now here for sometime now. There are many new features in JMeter 2.13 . Some of the important and major enhancements are listed below.

Following are the Major enhancements in JMeter 2.13 version.

New Element – New Async BackendListener with Graphite implementation

A new Async BackendListener has been added to allow sending result data to a backend listener. JMeter ships with a GraphiteBackendListenerClient that allows sending results to a Graphite server using Pickle of Plaintext protocols.

New JMeter 2.13

New connect time metric

Starting with this version a new metric called connectTime has been added. It represents the time to establish connection. By default it is not saved to CSV or XML, to have it saved add to user.properties:
jmeter.save.saveservice.connect_time=true

ConnectedTIme

95 and 99 percentile addition on Aggregate Graph and Report

The listeners Aggregate Graph and Aggregate Report previously showed only the 90 percentile (historical behavior), the 95 percentile and the 99 percentile have been added and are customizable. To setup the percentiles value you want, add to user.properties:
aggregate_rpt_pct1=90
aggregate_rpt_pct2=95
aggregate_rpt_pct3=99

New JMeter 2.13

Https test script modification

Now component is able to detect authentication schemes and automatically adds a pre-configured HTTP Authorization Manager with the correct Mechanism

Difference between __CSVRead and __Stringfromfile function

__CSVRead() function

The CSVRead function returns a string from a CSV file.
When a filename is first encountered, the file is opened and read into an internal array. If a blank line is detected, this is treated as end of file – this allows trailing comments to be used.
All subsequent references to the same file name use the same internal array.

Each thread has its own internal pointer to its current row in the file array. When a thread first refers to the file it will be allocated the next free row in the array, so each thread will access a different row from all other threads.

eg: You need to test login to an app with different user login credentials for different threads.

__Stringfromfile()

The StringFromFile function can be used to read strings from a text file. Each time it is called it reads the next line from the file. All threads share the same instance, so different threads will get data from different lines. When the end of the file is reached, it will start reading again from the beginning, unless the maximum loop count has been reached.

Note: If there are multiple references to the function in a test script, each will open the file independently, even if the file names are the same.  So it is advisable to use different file name across same scripts otherwise the output will be unpredictable.
Eg: You need to add different products to a shopping cart for the same user. In this case you will be using CSVRead to read user credentials and Stringfromfile to read product info that need to be added to each user.
So in a nutshell, Difference between __CSVRead and __Stringfromfile function is that Stringfromfile should be used when different data are required in a loop for same thread and CSVRead should be used when different data are required for different threads(users).

What’s new in JMeter 2.10

jmeter-210_httpsrecorder

JMeter 2.10 has been released. There are few notable changes/improvements in the new JMeter 2.10. Some of the changes are listed below:

 

New CSS/JQuery Tester in View Tree Results

A new CSS/JQuery tester has been added in view tree result. This will help test expressions very easily.

jmeter 2.10 css jquery tester

Improvement in HTTP(S) recorder

“HTTP proxy server” element and has been renamed to “HTTP(S) test script recorder”.

HTTP(S) recording have been improved with many bug fixes around this feature.

jmeter 2.10 httpsrecorder

MongoDB support

Now you can load test MongoDB  using new MongoDB Source Config.

jmeter 2.10 mongodb

Kerberos Authentication support

Authorization Manager now support Kerberos authentication along with Basic_DIGEST

jmeter 2.10 kerberos

New Functions

New functions (__urlencode and __urldecode) are now available to encode/decode URL encoded chars.

jmeter 2.10 newfunctions

Improvement in Distributed testing

  • Number of threads on each node is now reported to controller.
  • Performance improvement on BatchSampleSender
  • Addition of 2 SampleSender modes (StrippedAsynch and StrippedDiskStore)

jmeter 2.10 distributedtesting_threadsumarizer.

Miscellaneous

  • Webservice (SOAP) Request has been removed by default from GUI as Element is deprecated. (Use HTTP Request with Body Data , see also the Template Building a SOAP Webservice Test Plan ), if you need to show it, see property not_in_menu in jmeter.properties
  • HTTP(S) Test Script Recorder if Grouping is set to Put each group in a new Transaction Controller , the Recorder will create Transaction Controller instances with Include duration of timer and pre-post processors in generated sample set to false. This default value reflect more accurately response time.
  • Transaction Controller now sets Response Code of Generated Parent Sampler (if Generated Parent Sampler is checked) to response code of first failing child in case of failure of one of the children, in previous versions Response Code was empty.

For more detail feature click here.

Using Selenium with JMeter

To use Selenium Webdriver with JMeter, install “Webdriver” plugins. The WebDriver sampler is useful if you want to test for performance AJAX or GWT based web applications.

Download WebDriver plugins from http://jmeter-plugins.org/downloads/all/

Unzip the files and copy in the lib folder under JMeter home directory.

To test if WebDriver plugins installed open Jmeter and test if there is jp@gc – Firefox Driver Config.

*If not, check your JMeter’s lib folder.
 Selenium Webdriver SamplerAdd the Webdriver sampler and Firefox or chrome driver config element. Insert the following code to launch Bing search and search for a text. Also add tree listener to view the result.
var pkg = JavaImporter(org.openqa.selenium)
var support_ui = JavaImporter(org.openqa.selenium.support.ui.WebDriverWait)
var wait = new support_ui.WebDriverWait(WDS.browser, 20000)
WDS.sampleResult.sampleStart()
WDS.browser.get('http://www.bing.com/')
var searchField = WDS.browser.findElement(pkg.By.id('sb_form_q'))
searchField.click()
searchField.sendKeys(['testing'])
var button = WDS.browser.findElement(pkg.By.id('sb_form_go'))
button.click()
WDS.sampleResult.sampleEnd()
selenium_jmeter_2
Explanation of the code
  • The code starts with the import Java packages “org.openqa.selenium” and “org.openqa.selenium.support.ui.WebDriverWait” which will allow you to use the WebDriver classes.
  • WDS.sampleResult.sampleStart() and WDS.sampleResult.sampleEnd() captures sampler’s time and will track it. You can remove them, the script will still work but you will not get load time
  • WDS.browser.get(‘http://www.bing.com/’) – opens the browser with Bing search website
  • var searchField = WDS.browser.findElement(pkg.By.id(‘sb_form_q’)) – saves the search text box in searchField varialble
  • searchField.click() – Clicks the search field to make it active
  • searchField.sendKeys([‘testing’]) – Enters text to search
  • var button = WDS.browser.findElement(pkg.By.id(‘sb_form_go’)) – Saves search button in a variable
  • button.click() – clicks on the search button.

You are now ready to go. Do let us know your experience in using Selenium with JMeter

JMeter Tuning Tips

Context

You require to do Load test that requires thousands of thread. How will you configure JMeter so that JMeter performs well without any issues? Issues that you normally encounter with JMeter when using large number of threads are

  • Out of Memory error
  • High CPU usage
  • High Memory usage
  • JMeter scripts not responding or not ending

If you have faced above problems or require to perform load test with higher number of threads( 100 or more) you are at the right place.  This article will help you get most out of JMeter.

 

JMeter tuning tips

Listeners consume more resources.

Listeners receive Sample Results and do some processing with it, this takes resources (memory, CPU).  REMOVE ALL Listeners from your script once you are ready to load test. Listeners should be used only while creating and debugging your scripts.  You can view your results after your  execution is complete. The result file is saved using  -l option in non GUI mode. The next recommendation is in using Non-GUI mode.

Use Non-GUI mode

GUI mode is for creating the test plan, checking it, debugging it. Once you are ready with the script, remove all listeners as mentioned above and run in Non-GUI mode . The command for the same is as follows:

jmeter –t  <path of .jmx file> -n -l  <Path for jtl file name name>

Configure JMeter java options

Default heap size is 512 Mb. Increase it to 1024 or more.  On how to configure Heap Size refer the following Post

http://digitalab.org/2013/03/jmeter-out-of-memory-error/

Use CSV output rather than xml

CSV is less CPU and memory intensive. To use CSV output update user.properties, as shown below

 jmeter.save.saveservice.output_format=csv

Do not use functional mode for load testing

Save data that you need.

Use few assertions and Post-Processors

Always use latest version of JMeter

Use distributed mode for very high load.

Once you reach the limits of one machine, you can switch to distributed or remote testing. When using distributed load testing make sure to update mode properties correctly. In user.properties file update

mode=strippedbatch

To know more on distributed testing click here.

If the above points are not enough and you still face issue contact me details.

Results Visualization using xsl stylesheet in JMeter

Using xsl stylesheet in JMeter with .jtl files

Jmeter provide several .xsl files to visualize the result in human readable format outside JMeter tool.
These are located in %APACHE_JMETER_HOME/extras folder

  • jmeter-results-detail-report.xsl
  • jmeter-results-detail-report_21.xsl
  • jmeter-results-report.xsl
  • jmeter-results-report_21.xsl

1. Open the Jtl file in wordpad or any other text editor and inser the following line:
<?xml-stylesheet type=”text/xsl” href=”<Path of Jmeter home>\extras\jmeter-results-report_21.xsl”?>
eg: <?xml-stylesheet type=”text/xsl” href=”D:\jakarta-jmeter-2.9\extras\jmeter-results-report_21.xsl”?>

This line should be inserted between the line <?xml version=”1.0″ encoding=”UTF-8″?> and <testResults version=”1.2″> as  shown below:

JTL file

2. Save the jtl file and open an excel worksheet.Drag the Jtl file into it. You are there.

 

XSL example

You see that *.jtl file is parsed and converted to Excel worksheet.

But wait a minute. You shoud save your result in xml format and not csv. To do so either change the default settings in JMeter properties files or in the configure option in Listeners as shown below:

 

Result Configure

 

Concurrent pool size and browser emulation in JMeter

Since JMeter 2.5 there is a new option called “Concurrent pool size”.

Whats so important about this feature?

To answer this lets first understand how browser works and what an end user response time means.

Once the browser fetches a page, it subsequently fetches all the page’s resources like images, css, javascript etc. During the fetch time, the browser can open series of parallel connections to the remote server(s) hosting the embeded resources. Number of parallel connections vary from browser to browser. Chrome opens 6 parallel connections where as IE10 opens 8 connections.

For more information on parallel http connections in a browser go to following link.

http://stackoverflow.com/questions/985431/max-parallel-http-connections-in-a-browser

Fetching the content of a page using parallel connections dramatically reduces the render time.

Concurrent pool size

 

Now, in JMeter,

Concurrent pool size is the number of parallel connection that JMeter will open while fetching the resources. if we set this as 4 connections, then it means 4 connections per threads (users). So if the load generator can support 200 user threads, using connection pool option you should reduce the user threads per machine accordingly.

 

 

JMeter – Transaction Controller

Transaction Controller

The Transaction Controller generates an additional sample which measures the overall time taken to perform the nested test elements.

Note that this time by default includes all processing within the controller scope, not just the samples; this can be changed by unchecking “Include duration of timer and pre-post processors in generated sample“.

Transaction Controller

 

The generated sample time includes all the times for the nested samplers, and any timers etc. Depending on the clock resolution, it may be slightly longer than the sum of the individual samplers plus timers. The clock might tick after the controller recorded the start time but before the first sample starts. Similarly at the end.

It is only regarded successful if all the sub samples are successful.

Transaction Controller Results

If “Generate Parent sample” is selected individual samples no longer appear as separate entities in listeners except tree listener. Also, the sub-samples do not appear in CSV log files, but they can be saved to XML files.

Benefit of having Transaction controller

  • It will help you to get response time of the page, provided you have grouped all the elements of the page correctly.
  • Do not consider this as browser page load time. JMeter does not emulate browser behavior.

There is an option in JMeter 2.5+ to get close to browser rendering time by using thread pool option.

In JMeter 2.5+ there is an option to emulate browser behavior of parallel connections. As in browser all resources / page elements are not loaded sequentially.

For more detail on this feature, click here to view my post on concurrent pool size and browser emulation.

JMeter – Out of memory error

Are you getting “out of memory” error in JMeter while executing your tests?

This is most likely because of heap size. Your heap memory allocated to JMeter may be too less. By default this set to 512 MB. This setting is available in jmeter.bat file as shown below.

  • set HEAP=-Xms512m -Xmx512m
If increasing the heap size does not solve the problem, you may need to look into your scripts and remove unnecessary listeners. Some listeners are memory intensive like “View results in Tree”. As a best practice you should remove this once your script is ready for execution and only used while debugging the script.
Also, try to minimize the number of request in one Test Plan. Having everything in one Test Plan is not recommended unless the scenario requires you to do. In that case have very high end machine with good RAM.

Whats new in JMeter 2.9

Its been quite some time since JMeter 2.9 is available. There have been few notable changes in the new version of JMeter.

JMeter can now handle different types of documents (PDF, MsOffice files, Apache OpenOffice’s files…) within different elements. 

What does this mean?

  • Regular expression extractor can extract text from such documents
  • Assertion response can check text in such documents
  • Vew text of such documents in Result Tree.

pdfdocument-resulttreee-JMeter 2.9

 

JMS Publisher can now send Bytes Messages

JMSPublisher-JMeter 2.9

 

XPath Extractor now enables using a JMeter variable as input

Xpathextractor-JMeter 2.9

 

Copy/paste is now possible between 2 JMeter instances

HTTP Proxy Server now automatically uses HTTP Request with Raw Post Body mode for samples that only have one unnamed argument (JSON, XML, GWT…)

HTTP Proxy Server does not force user to select the type of Sampler in HTTP Sampler Settings.

This allows easier switch between implementations as Sampler do not have this information set anymore

JMeter now required Java 6 runtime or higher

 

For details list of changes and bug fixes in JMeter 2.9 vist http://jmeter.apache.org/changes.html