What do you mean by Economies of Software testing?
Why software defects are costly to find and fix?
How early defect detection can save much cost in fixing defects?
There is a definite economic impact of software testing. One economic impact is from the cost of defects and the another is the way we perform testing. Cost of defect is a very real and very tangible cost. The second point the way we perform testing will not consider it now.
Where defects originate?
One of the most commonly understood facts about defects is that most defects originate in the requirements definition phase of a project. The next runner-up is the design phase.
Some problems in getting accurate, clear, and testable requirements are:
Many people do not have a solid requirements gathering process
Few people have been trained in or understand the dynamics of requirements
Projects, people, and the world around us change very quickly
The English language is ambiguous and even what we consider clear language can be interpreted differently by different people.
Where testing resources are used?
In the previous section we saw that most defects originate in requirements and design, but most of the testing effort occurs in a traditional “testing” phase toward the end of the project. The problem with the big bang approach to testing is that defects are not found until toward the end of the project. This is the most costly and risky time to fix defects. Some complex defects may even be impossible to fix.
Relative cost of fixing defects
One of the known facts about software defects is that the longer they go undetected, the more expensive they are to fix. Although research differs on the exact ratios, the general rule is 1:10:100. That is, if a defect costs one unit (hour, dollar, etc.) to fix in requirements and design, it costs 10 units to fix in testing (system/acceptance) and over 100 times to fix in production. The cosThis cost of defects doesn’t even take into account the impact cost of defects. These t of fixing in production may be even higher than 100 times. Costs could be attributed to lost revenue, reimbursements, fraud, lost customers, bad public relations, and litigation.
Most defects are created in the early stages of a project
Most defects are found in the later stages of a project
It costs 10 to 100 times as much to fix a defect in the later phases of a project.
So, what does all of this mean? The main conclusion is that most people perform testing too late in the process. These people wonder why testing is so expensive and why their projects are often over budget.
If you really want to make your testing more efficient and reduce the overall cost of testing and defects, test early in the project and continue testing throughout the project.
SQL injection is a technique often used to attack data driven applications. This is done by including portions of SQL statements in an entry field in an attempt to get the website to pass a newly formed SQL command to the database. ….
What is Security testing? Security testing is a process to determine that an information system protects data and maintains functionality as intended. It is the process that determines that confidential data stays confidential and users can perform only those tasks that they are authorized to perform….
Good part of selenium is that the selenium tests can be written in multiple programming languages like c#, Java, Perl, PHP, Ruby etc. I normally hear saying that “This Java project, so we can write tests in Java as well”. The only advantage I see with this is that you can get help from development team incase you are stuck somewhere. But if you are good at c# or any other language it should not be an hindrance. Test framework will be different that coding framework unless you want to integrate it.
I covered Java and C# in most of my previous posts. Here I will be using Ruby to write the Selenium tests.
require “selenium-webdriver” – This is similar to import in Java and using in c#
The above example is without using any Test framework. In Ruby you can use Test Framework as you do with C# ( NUNIT test framework) and Java( JUNIT or TestNG framework).
In Ruby you can use minitest or BDD framework RSpec or Cucumber
Structure of RSpec test
load File.dirname(__FILE__) + '/test_helper.rb'
#'describe' marks the a test group
describe "Selenium Ruby Tests" do
# 'it' marks the start of a test case, ends with the matching 'end'
it "Start Chrome" do
browser = Selenium::WebDriver.for(:chrome) browser.navigate.to(site_url)
it "Start FireFox" do
browser = Selenium::WebDriver.for(:firefox) browser.navigate.to(site_url)
it "Start IE" do
browser = Selenium::WebDriver.for(:ie) browser.navigate.to(site_url)
Pair testing is a way of approaching a test design process by having two people test the same time and place continuously.
The dynamics of pairing enables the generation of more and different ideas than either tester is likely to produce on his own.
It is an effective complement to individual testing.
Composer/ lyricist pair
Pilot and Co-pilot in a aeroplane
Two tester working together produce tests, over a period of time continuously exchanging ideas. Assuming that the conditions exists that enables test design, successful pair testing requires three specific conditions
At least one tester is available who can be trusted to test without supervision
Another tester is available who can join the first tester for a session of test design.
The two testers are otherwise capable of working together
Idea Exchange – The process of explaining and questioning helps pollinate new ideas. This is true even when one of the testers is much less knowledgeable than the other one.
Attention flow – The core dynamic of pair testing is the flow of attention. Pair testing requires that the testers synchronize their pace of work. They continuously share ideas and direct themselves to various problems.
Test strategy – If each tester specializes in a different sub-systems, then as a pair they may be especially effective at system testing that examines the interaction among those sub-systems.
Scenario testing is one of the example of Test Patterns
The objective of scenario testing is to prove that the program will fail when asked to do real work ( significant tasks) by an experienced user. A failure at this level is a validation failure ( a failure to meet the stated or implicit program requirements.)
“All” of the features have been tested in isolation. More precisely, all of the features that will be called within this scenario have been tested on their own and as fas we can tell, none of them has an error that will block this scenario test.)
The tester must have sufficient knowledge of the domain( eg: accounting, if this is an accounting application) and many of the ways in which skilled users will use the program.
Time trade off
There is often much time pressure to develop “realistic” tests quickly.
Because complex tests are expensive. You can not develop many of them. What level of coverage will you get from these tets? What level should you expect.
Ideally scenario test has four attributes
The test is realistic
It is complex
It is easy to tell whether the program passed of failed the test
Serious failure if the program will not pass a given scenario
The key message of this pattern is that you should keep this four attributes in mind when you design a scenario test and try hard to achieve them.
Next we will cover another example of test pattern.
The CSVRead function returns a string from a CSV file.
When a filename is first encountered, the file is opened and read into an internal array. If a blank line is detected, this is treated as end of file – this allows trailing comments to be used.
All subsequent references to the same file name use the same internal array.
Each thread has its own internal pointer to its current row in the file array. When a thread first refers to the file it will be allocated the next free row in the array, so each thread will access a different row from all other threads.
eg: You need to test login to an app with different user login credentials for different threads.
The StringFromFile function can be used to read strings from a text file. Each time it is called it reads the next line from the file. All threads share the same instance, so different threads will get data from different lines. When the end of the file is reached, it will start reading again from the beginning, unless the maximum loop count has been reached.
Note: If there are multiple references to the function in a test script, each will open the file independently, even if the file names are the same. So it is advisable to use different file name across same scripts otherwise the output will be unpredictable.
Eg: You need to add different products to a shopping cart for the same user. In this case you will be using CSVRead to read user credentials and Stringfromfile to read product info that need to be added to each user.
So in a nutshell, Difference between __CSVRead and __Stringfromfile function is that Stringfromfile should be used when different data are required in a loop for same thread and CSVRead should be used when different data are required for different threads(users).
The following example uses VBScript to add counters whose values are retrieved from the local computer, modifies some of the SYSMON properties that control how the monitor is displayed, and processes the OnCounterAdd event. The example uses the wildcard character (*) to add all instances of the process counter.
Save the bleow code in a HTML page and open in browser.
<HTML> <BODY BGCOLOR=#C0C0C0> <SCRIPT LANGUAGE="VBScript">
Sub Monitor_OnCounterAdded(index) Monitor.Counters.Item(1).Width = 8 End Sub </Script>
<OBJECT CLASSID="clsid:C4D2D8E0-D1DD-11CE-940F-008029004347" ID="Monitor" HEIGHT=80% WIDTH=100%>
Sub Window_OnLoad On Error Resume Next Monitor.ShowValueBar = False Monitor.ShowHorizontalGrid = True Monitor.Counters.Add("\Process(*)\% Processor Time") Monitor.DisplayType=sysmonLineGraph Monitor.GraphTitle="System Performance Overview" End Sub
Note: You need to enable ActiveX in your browser, to run the html file.
An instrument is a powerful tool that can be used to automate UI testing. Instruments can also be used to collect data about the performance and behavior of one or more processes on the system and track that data over time. Although most instruments are geared toward gathering trace data, the User Interface instrument helps automate data collection. With it you can record user events while you gather your trace data. You can use this recording to reliably reproduce the same sequence of events over and over again.
Instruments is contained within the Xcode 4 toolset. Download Xcode from the App Store and install it onto your computer. After you have installed Xcode, you are ready to run Instruments. Instruments can be launched in one of three ways.
To run Instruments from Xcode
Choose Xcode > Open Developer Tool > Instruments.
Even though each instrument is different, there is one general workflow when collecting information from your app. This workflow is a four-step process.
Choose a trace template
Direct Instruments to your app
Collect information from about your app
Examine the collected information
For UI automation we will be using Automation template in Instruments to execute scripts. An important benefit of the Automation instrument is that you can use it with other instruments to perform sophisticated tests such as tracking down memory leaks and isolating causes of performance problems.
Note: The Automation instrument only works with apps that have been code signed with a development provisioning profile. Apps signed with a distribution provisioning profile cannot be automated with the UI Automation programming interface.
Writing an automation test script
Select the Automation trace template.
Click Add > Create.
Double-click New Script to change the name of the script.
In the Detail pane, select Console to enter the code for your script.
Choose a target for your script.
Click the Play button at the bottom of the Console.
Selecting trace template
After you create the script, it can be used throughout the development of your app. You can do this by importing your saved script and running it with the Automation instrument.
To import a previously saved script
Select the Automation trace template.
Click Add > Import.
Navigate to your saved script file and click Open
Accessing and Manipulating UI Elements
To perform an action on an element in your app, you explicitly identify that element in terms of the app’s element hierarchy. Each accessible element is inherited from the base element, UIAElement. Every element can contain zero or more other elements. Script can access individual elements by their position within the element hierarchy. However, you can assign a unique name to each element by setting the label attribute and making sure Accessibility is selected in Interface Builder for the control represented by that element.
The four properties used in the scripts to access elements are
name. Derived from the accessibility label
value. The current value of the control, for example, the text in a text field
elements. Any child elements contained within the current element, for example, the cells in a table view
parent. The element that contains the current element