Non-Functional testing refers to any testing that does not cover the testing of functionality, such as Performance, Stress, Volume, Soak, Security and Intrusion. Generally, performance type testing is run using a number of concurrent users.
These types of testing are usually very technical and not readily conducted by resource without the relevant skills.
Conducting these types of testing requires a different mindset and skillset to that required of Functional Testing, and is really done following the successful completion of Functional Testing. For Performance type testing we are interested in monitoring things like CPU, Memory, Network, Load Balancing, Multi-User experience, Application Servers, Web Servers, Databases and assisting in tuning the system.
It should be noted that testing things as just one user rarely shows up the defects that are produced by running the test as multiple concurrent users. This exposes such as issues as the ability to log on multiple users, individual memory issues, database issues, sizing issues, speed issues etc. In fact, it is during performance type testing that we find the majority of defects.
Micro Focus Loadrunner
The practicalities of conducting Performance testing and producing a Load on a system require the use of an automated tool and Loadrunner is probably the best one on the market. Loadrunner can emulate hundreds or thousands of users and also has the most protocols. The scripts can be bundled into scenarios for stress, volume, soak, etc. testing
Scripts are generally recorded against a GUI, enhanced using ANSI C and subsequently run against an API without a GUI, allowing multiple vusers to be generated by one server.
Scripts are enhanced to add parameterization, text checking and manipulation, timing measurements, etc.
Scripts are grouped together into Scenarios, which emulate realistic user behavior harmonics.
Agentless monitors can be arranged on the system under test to record such things as CPU usage, Memory usage, Network delay, etc.
Scenarios are run for differing lengths of time and load, depending upon the test type being run and can include Load, Stress or Soak tests.
The results are carefully analyzed and interpreted so that any bottlenecks can be found and rectified.
Clear reporting is produced.