Load testing: Difference between revisions

Content deleted Content added

Inline

Revision as of 03:18, 8 February 2018

Load testing is the process of putting demand on a software system or computing device and measuring its response. Load testing is performed to determine a system's behavior under both normal and anticipated peak load conditions. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation. When the load placed on the system is raised beyond normal usage patterns to test the system's response at unusually high or peak loads, it is known as stress testing. The load is usually so great that error conditions are the expected result, but there is no clear boundary when an activity ceases to be a load test and becomes a stress test.

The term "load testing" is often used synonymously with concurrency testing, software performance testing, reliability testing, and volume testing for specific scenarios. All of these are types of non-functional testing that are not part of functionality testing used to validate suitability for use of any given software.

Software load testing

The term load testing is used in different ways in the professional software testing community. Load testing generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the program concurrently.^[1] As such, this testing is most relevant for multi-user systems; often one built using a client/server model, such as web servers. However, other types of software systems can also be load tested. For example, a word processor or graphics editor can be forced to read an extremely large document; or a financial package can be forced to generate a report based on several years' worth of data. The most accurate load testing simulates actual use, as opposed to testing using theoretical or analytical modeling.

Load testing lets you measure your website's quality of service (QOS) performance based on actual customer behavior. Nearly all the load testing tools and frame-works follow the classical load testing paradigm: when customers visit your web site, a script recorder records the communication and then creates related interaction scripts. A load generator tries to replay the recorded scripts, which could possibly be modified with different test parameters before replay. In the replay procedure, both the hardware and software statistics will be monitored and collected by the conductor, these statistics include the CPU, memory, disk IO of the physical servers and the response time, throughput of the system under test (SUT), etc. And at last, all these statistics will be analyzed and a load testing report will be generated.

Load and performance testing analyzes software intended for a multi-user audience by subjecting the software to different numbers of virtual and live users while monitoring performance measurements under these different loads. Load and performance testing is usually conducted in a test environment identical to the production environment before the software system is permitted to go live.

As an example, a web site with shopping cart capability is required to support 100 concurrent users broken out into following activities:

25 virtual users (VUsers) log in, browse through items and then log off
25 VUsers log in, add items to their shopping cart, check out and then log off
25 VUsers log in, return items previously purchased and then log off
25 VUsers just log in without any subsequent activity

A test analyst can use various load testing tools to create these VUsers and their activities. Once the test has started and reached a steady state, the application is being tested at the 100 VUser load as described above. The application’s performance can then be monitored and captured.

The specifics of a load test plan or script will generally vary across organizations. For example, in the bulleted list above, the first item could represent 25 VUsers browsing unique items, random items, or a selected set of items depending upon the test plan or script developed. However, all load test plans attempt to simulate system performance across a range of anticipated peak workflows and volumes. The criteria for passing or failing a load test (pass/fail criteria) are generally different across organizations as well. There are no standards specifying acceptable load testing performance metrics.

A common misconception is that load testing software provides record and playback capabilities like regression testing tools. Load testing tools analyze the entire OSI protocol stack whereas most regression testing tools focus on GUI performance. For example, a regression testing tool will record and playback a mouse click on a button on a web browser, but a load testing tool will send out hypertext the web browser sends after the user clicks the button. In a multiple-user environment, load testing tools can send out hypertext for multiple users with each user having a unique login ID, password, etc.

The popular load testing tools available also provide insight into the causes for slow performance. There are numerous possible causes for slow system performance, including, but not limited to, the following:

Application server(s) or software
Database server(s)
Network – latency, congestion, etc.
Client-side processing
Load balancing between multiple servers

Load testing is especially important if the application, system or service will be subject to a service level agreement or SLA.

User Experience Under Load test

In the example above, while the device under test (DUT) is under production load - 100 VUsers, run the target application. The performance of the target application here would be the User Experience Under Load. It describes how fast or slow the DUT responds, and how satisfied or how the user actually perceives performance.

Load testing tools

Tool name	Company name	License	Type	Pricing	Focus	Notes
Apache JMeter	An Apache Jakarta open source project	Apache License 2.0	Local	Free		Java desktop application for load testing and performance measurement.
BlazeMeter	BlazeMeter Ltd.	Proprietary	Cloud based	Free plan available	Website, web app, mobile, database and other scenarios	Scalable load up to 200,000 concurrent simulated browser users from across eight geographical locations. Can also be used for integration and functional testing.
Blitz	Spirent Communications	Proprietary	Cloud based	Trial available	Websites, mobile and REST APIs	It allows to simulate up to 50,000 simultaneous virtual users from different worldwide locations.
CloudTest	SOASTA	Proprietary	Cloud-based	Free, trial and licensed versions	Mobile and web applications
Flood.io	Flood IO	Proprietary	Cloud based	Trial available	Website, web app, mobile, database and other scenarios	Scalable load up to millions of concurrent users from across 14 geographical locations. Pricing is based on infrastructure used, not tests or virtual users. ^[2] ^[3]
Gatling	Open Source	Apache License 2.0	Local	Free	Java web applications	Portable HTML reports.
Loader.io	SendGrid Labs	Proprietary	Cloud based	Free plan available	Web applications and API
LoadRunner	HP	Proprietary	Local	Trial available		Primarily used for executing large numbers of tests (or a large number of virtual users) concurrently. Can be used for unit and integration testing as well.
Load Test (included with SOAtest)	Parasoft	Proprietary	Local	Demo available		Performance testing tool that verifies functionality and performance under load. Supports SOAtest tests, JUnits, lightweight socket-based components. Detects concurrency issues.
loadUI	SmartBear Software	EUPL	Local	Free		Cross-platform load testing tool, targeted mainly at web services. Integrates with soapUI.
Login VSI	Login VSI, Inc.	Proprietary	Local	Trial available	Virtual windows desktops workloads
NeoLoad	Neotys	Proprietary	Local	Free plan available	Web and mobile	Load can be generated from local agents or from the cloud.
OpenSTA	Open System Testing Architecture	GNU General Public License 2.0	Local	Free	Web server	Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.
Rational Performance Tester	IBM	Proprietary	Local	Trial available		Eclipse based large scale performance testing tool primarily used for executing large volume performance tests to measure system response time for server based applications.
Siege	Open source	Open source GPLv3 or later	Local	Free	Web servers	Supports basic authentication, cookies, HTTP, HTTPS and FTP protocols.
Silk Performer	Borland	Proprietary	Local	Trial available		Application performance tool with cloud and local virtual agents. Supports most protocols and applications. Licensed.
Test Studio	Telerik	Proprietary	Local	Trial available		Assess how website would perform if visited by a large number of users at the same time.
Visual Studio Ultimate edition	Microsoft	Proprietary	Local	Trial available		Visual Studio Ultimate edition includes a load test tool which enables a developer to execute a variety of tests (web, unit etc...) with a combination of configurations to simulate real user load.^[4]
WebLOAD	RadView	Proprietary	Local	Free plan available	Web and Mobile applications and API	Load testing tool for web and mobile applications including Web Dashboard for performance test analysis. Used for large-scale loads which can be generated also from the cloud. Licensed.

Physical load testing

Many types of machinery, engines,^[5] structures,^[6] and motors^[7] are load tested. The load may be at a designated safe working load (SWL), full load, or at an aggravated level of load. The governing contract, technical specification or test method contains the details of conducting the test. The purpose of a mechanical load test is to verify that all the component parts of a structure including materials, base-fixings are fit for task and loading it is designed for.

Several types of load testing are employed

Static testing is when a designated constant load is applied for a specified time.
Dynamic testing is when a variable or moving load is applied.
Cyclical testing consists of repeated loading and unloading for specified cycles, durations and conditions.

The Supply of Machinery (Safety) Regulation 1992 UK state that load testing is undertaken before the equipment is put into service for the first time. Performance testing applies a safe working load (SWL), or other specified load, for a designated time in a governing test method, specification, or contract. Under the Lifting Operations and Lifting Equipment Regulations 1998 UK load testing after the initial test is required if a major component is replaced, if the item is moved from one location to another or as dictated by the competent person.

Car charging system

A load test can be used to evaluate the health of a car's battery. The tester consists of a large resistor that has a resistance similar to a car's starter motor and a meter to read the battery's output voltage both in the unloaded and loaded state. When the tester is used, the battery's open circuit voltage is checked first. If the open circuit voltage is below spec (12.6 volts for a fully charged battery), the battery is charged first. After reading the battery's open circuit voltage, the load is applied. When applied, it draws approximately the same current the car's starter motor would draw during cranking. Based on the specified cold cranking amperes of the battery, if the voltage under load falls below a certain point, the battery is bad. Load tests are also used on running cars to check the output of the car's alternator.

References

^ Wescott, Bob (2013). The Every Computer Performance Book, Chapter 6: Load Testing. CreateSpace. ISBN 1482657759.
^ Erinle, Bayo (2014). JMeter Cookbook. Packt Publishing. ISBN 978-1783988280.
^ Erinle, Bayo (2015). Performance Testing with JMeter. Packt Publishing. ISBN 978-1784394813.
^ "Load Testing ASP.NET Applications with Visual Studio 2010". Eggheadcafe.com. Retrieved 2013-01-13.
^ Harper, David; Devin Martin, Harold Miller, Robert Grimley and Frédéric Greiner (2003), Design of the 6C Heavy-Duty Gas Turbine, ASME Turbo Expo 2003, collocated with the 2003 International Joint Power Generation Conference, vol. Volume 2: Turbo Expo 2003, Atlanta GA: ASME 1., pp. 833–841, ISBN 0-7918-3685-1, retrieved 2013-07-14 {{citation}}: |volume= has extra text (help)CS1 maint: multiple names: authors list (link)
^ Raines, Richard; Garnier, Jacques (2004), Physical Modeling of Suction Piles in Clay, 23rd International Conference on Offshore Mechanics and Arctic Engineering, vol. 1, Vancouver, BC: ASME, pp. 621–631, doi:10.1115/OMAE2004-51343, retrieved 2013-07-14
^ DETERMINING ELECTRIC MOTOR LOAD AND EFFICIENCY (PDF), vol. DOE/GO-10097-517, US Department of Energy, 2010, ISBN 978-0-9709500-6-2, retrieved 2013-07-14

External links

Modeling the Real World for Load Testing Web Sites by Steven Splaine
What is Load Testing? by Tom Huston
4 Types of Load Testing and when each should be used by David Buch
Performance, Load, Stress or Endurance Test? Which do you want? by Chris Jones

[1] Wescott, Bob (2013). The Every Computer Performance Book, Chapter 6: Load Testing. CreateSpace. ISBN 1482657759.

[Cook-2] Erinle, Bayo (2014). JMeter Cookbook. Packt Publishing. ISBN 978-1783988280.

[Test-3] Erinle, Bayo (2015). Performance Testing with JMeter. Packt Publishing. ISBN 978-1784394813.

[4] "Load Testing ASP.NET Applications with Visual Studio 2010". Eggheadcafe.com. Retrieved 2013-01-13.

[5] Harper, David; Devin Martin, Harold Miller, Robert Grimley and Frédéric Greiner (2003), Design of the 6C Heavy-Duty Gas Turbine, ASME Turbo Expo 2003, collocated with the 2003 International Joint Power Generation Conference, vol. Volume 2: Turbo Expo 2003, Atlanta GA: ASME 1., pp. 833–841, ISBN 0-7918-3685-1, retrieved 2013-07-14 {{citation}}: |volume= has extra text (help)CS1 maint: multiple names: authors list (link)

[6] Raines, Richard; Garnier, Jacques (2004), Physical Modeling of Suction Piles in Clay, 23rd International Conference on Offshore Mechanics and Arctic Engineering, vol. 1, Vancouver, BC: ASME, pp. 621–631, doi:10.1115/OMAE2004-51343, retrieved 2013-07-14

[7] DETERMINING ELECTRIC MOTOR LOAD AND EFFICIENCY (PDF), vol. DOE/GO-10097-517, US Department of Energy, 2010, ISBN 978-0-9709500-6-2, retrieved 2013-07-14

[1]

[2]

[3]

[4]

[5]

[6]

[7]

@@ Line 2: / Line 2: @@
 '''Load testing''' is the process of putting demand on a [[software system]] or [[computer|computing device]] and measuring its response. Load testing is performed to determine a system's behavior under both normal and anticipated peak load conditions. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation. When the load placed on the system is raised beyond normal usage patterns to test the system's response at unusually high or peak loads, it is known as [[stress testing]]. The load is usually so great that error conditions are the expected result, but there is no clear boundary when an activity ceases to be a load test and becomes a stress test.
-The term "load testing" is often used synonymously with concurrency testing, [[software performance testing]], [[reliability testing]], and [[volume testing]] for specific scenarios. All of these are types of [[non-functional testing]] that are not part of functionality testing used to validate suitability for use of any given software.
+The term "load testing" is often used synonymously with [[Concurrent testing|concurrency testing]], [[software performance testing]], [[reliability testing]], and [[volume testing]] for specific scenarios. All of these are types of [[non-functional testing]] that are not part of functionality testing used to validate suitability for use of any given software.
 == Software load testing ==