Automate UI Testing using Instruments
An instrument is a powerful tool that can be used to automate UI testing. Instruments can also be used to collect data about the performance and behavior of one or more processes on the system and track that data over time. Although most instruments are geared toward gathering trace data, the User Interface instrument helps automate data collection. With it you can record user events while you gather your trace data. You can use this recording to reliably reproduce the same sequence of events over and over again.
Instruments is contained within the Xcode 4 toolset. Download Xcode from the App Store and install it onto your computer. After you have installed Xcode, you are ready to run Instruments. Instruments can be launched in one of three ways.
To run Instruments from Xcode
- Open Xcode.
- Choose Xcode > Open Developer Tool > Instruments.
Even though each instrument is different, there is one general workflow when collecting information from your app. This workflow is a four-step process.
- Choose a trace template
- Direct Instruments to your app
- Collect information from about your app
- Examine the collected information
For UI automation we will be using Automation template in Instruments to execute scripts. An important benefit of the Automation instrument is that you can use it with other instruments to perform sophisticated tests such as tracking down memory leaks and isolating causes of performance problems.
Note: The Automation instrument only works with apps that have been code signed with a development provisioning profile. Apps signed with a distribution provisioning profile cannot be automated with the UI Automation programming interface.
Writing an automation test script
- Select the Automation trace template.
- Click Add > Create.
- Double-click New Script to change the name of the script.
- In the Detail pane, select Console to enter the code for your script.
- Choose a target for your script.
- Click the Play button at the bottom of the Console.
Selecting trace template
After you create the script, it can be used throughout the development of your app. You can do this by importing your saved script and running it with the Automation instrument.
To import a previously saved script
- Select the Automation trace template.
- Click Add > Import.
- Navigate to your saved script file and click Open
Accessing and Manipulating UI Elements
To perform an action on an element in your app, you explicitly identify that element in terms of the app’s element hierarchy. Each accessible element is inherited from the base element, UIAElement. Every element can contain zero or more other elements. Script can access individual elements by their position within the element hierarchy. However, you can assign a unique name to each element by setting the label attribute and making sure Accessibility is selected in Interface Builder for the control represented by that element.
The four properties used in the scripts to access elements are
- name. Derived from the accessibility label
- value. The current value of the control, for example, the text in a text field
- elements. Any child elements contained within the current element, for example, the cells in a table view
- parent. The element that contains the current element
You want to install Cucumber on your mac machine. Or if you are having trouble installing Cucumber and waitr or selenium-webdriver then you are at the right place.
Following are the steps you need to follow in order to install cucumber with waitr/selenium-webdriver.
Step 1 – Install RVM and Ruby
curl -L get.rvm.io | bash -s stable
$ rvm list known
$rvm install 1.9.3
# This installs Ruby 1.9.3
Step 2 – Install the required gems
gem update --system
gem install rspec --no-ri --no-rdoc
gem install watir-webdriver --no-ri --no-rdoc
gem install cucumber --no-ri --no-rdoc
watir-webdriver requires selenium webdriver, so that one will be installed always. If you only want selenium-webdriver, just do gem install selenium-webdriver instead.
When creating an install_gems.sh file inlcuding these commands, e.g: gem update –system
Note: You need to have Xcode and Git already installed.
For details on using Cucumber refer the following url: https://github.com/cucumber/cucumber/wiki
JMeter 2.10 has been released. There are few notable changes/improvements in the new JMeter 2.10. Some of the changes are listed below:
New CSS/JQuery Tester in View Tree Results
A new CSS/JQuery tester has been added in view tree result. This will help test expressions very easily.
Improvement in HTTP(S) recorder
“HTTP proxy server” element and has been renamed to “HTTP(S) test script recorder”.
HTTP(S) recording have been improved with many bug fixes around this feature.
Now you can load test MongoDB using new MongoDB Source Config.
Kerberos Authentication support
Authorization Manager now support Kerberos authentication along with Basic_DIGEST
New functions (__urlencode and __urldecode) are now available to encode/decode URL encoded chars.
Improvement in Distributed testing
- Number of threads on each node is now reported to controller.
- Performance improvement on BatchSampleSender
- Addition of 2 SampleSender modes (StrippedAsynch and StrippedDiskStore)
- Webservice (SOAP) Request has been removed by default from GUI as Element is deprecated. (Use HTTP Request with Body Data , see also the Template Building a SOAP Webservice Test Plan ), if you need to show it, see property not_in_menu in jmeter.properties
- HTTP(S) Test Script Recorder if Grouping is set to Put each group in a new Transaction Controller , the Recorder will create Transaction Controller instances with Include duration of timer and pre-post processors in generated sample set to false. This default value reflect more accurately response time.
- Transaction Controller now sets Response Code of Generated Parent Sampler (if Generated Parent Sampler is checked) to response code of first failing child in case of failure of one of the children, in previous versions Response Code was empty.
For more detail feature click here.
MaxQ web functional testing tool is an open source free tool based on Python.
How it Works?
Advantage of using MaxQ
- It is a free and open source too.
- MaxQ scripts can be integrated and reused with CLIF load testing framework.
- Supports commandline and hence can be executed unattended.
- Scripts can run as JUnit tests as it is in jython an implementation of Python.
Download the binaries from here (http://maxq.tigris.org/servlets/ProjectDocumentList?folderID=695) and extract the zip file.
Make sure you have java 1.4 or later in your system and ensure that JAVA HOME and PATH variables are properly set.
Recording your first Script
Before you start recording, change the HTTP Proxy setting in your favourite browser. How to do this will change depending on the browser. On Internet explorer follow the following step.s
- Select Tools->Internet Options
- Click on connections tab
- Click on LAN Settings button
- Select the Proxy Server checkbox and enter the Address and port as localhost and 8090
- Click OK to save it.
Start MaxQ by double clicking maxq.bat file under bin folder
Select File->New->Standard Script. This will create a template for the script.
Select Test->Start Recording to start your actual recording process. Any action performed on the browser after this will be recorded in the script.
Once you are done recording you can stop recording by selecting Test -> Stop Recording.
Enter the filename for the script when prompted and use extension as *.py ( phython files)
Replaying the Script
To replay the script, Start Maxq as above and open the script file. From the menu select Test -> Run.
You can also run the script from command line using the following command.
maxq –r sampletest.py
The next part of the blog will have more insight into the script and how to edit the script and integration with CLIF.
SpecFlow Installation and Setup
What is SpecFlow?
SpecFlow is an open-source .NET tool for automated testing that lets you write specifications using 100%-Cucumber-compatible Gherkin syntax, and has a number of advantages over Cucumber itself.
- It integrates with Visual Studio.
- It gives complete VS debugger support, so you can set breakpoints on Given/When/Then lines in your .feature files and step through their execution.
- You can implement your step definitions in any .NET language.
- When you compile a project containing SpecFlow feature files, the output is an NUnit test assembly, so you can use your favourite NUnit-compatible test runner or existing CI infrastructure to run the specifications with no additional configuration.
- Installation involves installing IDE Integration and Setting up project to work
- For VS 2012 and VS 2010, use extension manager to search “SpecFlow”
- Refer the screenshots below.
Setup SpecFlow Project using NUGET
The SpecFlow tests are usually placed into one or more separate project in the solution: The easiest and most convenient way to setup these projects is to use NuGet package: SpecFlow or one of the specific helper packages, like SpecFlow.NUnit or SpecRun.SpecFlow.
Since SpecFlow can use different unit testing frameworks to execute the SpecFlow tests, the SpecFlow projects have to be either Test Projects (in case of MsTest) or a simple Class Libraries.
- Add reference for the SpecFlow runtime
SpecFlow projects need the TechTalk.SpecFlow.dll in order to compile. You can setup this by installing NuGet package: SpecFlow or one of the specific helper packages, like SpecFlow.NUnit or SpecRun.SpecFlow. The helper packages group all necessary dependencies and apply the necessary configuration.
- Configure SpecFlow project
If you have used one of the helper NuGet packages you don’t need to configure anything. If you have used the SpecFlow package or want to fine tune SpecFlow, you need to change the settings in the App.config file.
- Add your first feature file
Your project is ready to use SpecFlow. You can add the first feature files (with Add / New Item in Visual Studio for example) and implement your applications in Specification by Example (Behavior Driven Development (BDD),Acceptance Test Driven Development (ATDD)) style.
Difference between “AndWait” and the “waitFor” Commands
The difference between a command and its AndWait alternative is that the regular command (e.g. click) will do the action and continue with the following command as fast as it can, while the AndWait alternative (e.g. clickAndWait) tells Selenium to wait for the page to load after the action has been done.
The AndWait alternative is always used when the action causes the browser to navigate to another page or reload the current page.
Note that, if you use an AndWait command for an action that does not trigger a navigation/refresh, your test will fail. This happens because Selenium will reach the AndWait‘s timeout without seeing any navigation or refresh being made, causing Selenium to raise a timeout exception.
In AJAX driven web applications, data is retrieved from server without refreshing the page. Using andWait commands will not work as the page is not actually refreshed. Pausing the test execution for a certain period of time is also not a good approach as web element might appear later or earlier than the stipulated period depending on the system’s responsiveness/Performance etc, leading to test failures. The best approach would be to wait for the needed element in a dynamic period and then continue the execution as soon as the element is found.
This is done using waitFor commands, as waitForElementPresent or waitForVisible, which wait dynamically, checking for the desired condition every second and continuing to the next command in the script as soon as the condition is met.