This is the second of two articles discussing the topic of Test Environments and Testing Practices. The first one, "Baselining--Stress Testing--Performance Testing--Oh My--Part One--Environments," focused on proper testing environment design. This article is focused on what you do with them once they have been created--Network and Application Testing.



When one is performance testing, what is REALLY being tested? Is it the Application, the Network, the Desktop, the WAN? Having done a performance test of one, do you then know something about the others? Not really.

What is the difference between Stress Testing, Load Testing and Baselining? Are they just different ways to say the same thing? No.

What, if any, is the consequence of using these terms interchangeably? After all, we all know what we mean--don't we? It is common for various teams that are really trying to cooperate with each other to founder. Sometimes it is because they thought that they had all agreed with each other--but discovered that each of them had a very different understanding of what is was to which they had all agreed. Semantics matter. This is not limited to IT by any means, yet, this article is focused on IT--one very specific part of IT, namely the processes used by IT to Test Performance.

TESTING DEFINITIONS:

Baseline Testing: Focus is on how long a single transaction, or set of transactions, take for a normal user during normal conditions.

Only a single user is tested. Best if can be done SAFELY during production hours. After all, that is when users will experience any issues. A baseline that tells how the transaction functions under ideal conditions will only make any production environment seem too slow.

- Test from each satellite location (if applicable).

- Use a Packet- Sniffer running locally to the test user. Something like Wireshark running on the

host PC--if using live test users.

- A packet-sniffer like WireShark listening to a port mirror in a switch in front of the device running the script--if using a form of virtual user.

- Capture the transaction while testing.

- Save capture, test again.

BESTline Testing: (Our Term), describes a test that is performed to determine the best possible time an application can be reasonably expected to provide.

- Attempts to eliminate all non-application issues such as the Network and WAN.

- Use workstation most local to first tier servers

- If possible, relocate switch port for workstation onto the same ASIC.

- If possible, recreate the second and other tiers, as locally to the first tier as possible.

Application Profiling: Provides a packet level and protocol level description of what is taking place with an application--across a network wire.

- How many TCP connections are used?

- What is the nature of the communication at the packet level?

- Many small connections or a few large connections?

- What form of SMB is used?

- What is the size of the files it “must” transport such as DLLs, EXEs, Java Applets, etc.

- What are the size of required Cookies and Tokens?

- HTTP usage For example, POST or GET? etc.

- May also result in an Interpath Application Flow Diagram.

Application Profiling will be the topic of a dedicated article/podcast soon.

Performance Testing: Focus is on DEV, QA, or UAT.

Stress Testing: Attempts to see how much a system can handle before the system degrades below acceptable parameters or fails. Utilizes simulated users generated by a tool such as LoadRunner or some other tool that generates virtual users.

Load Testing: Different than Stress Testing in that it attempts to induce Minimal Load. Steered towards a specific target based on a Capacity Planning goal.

Single Transactional Testing: Attempts to see exactly what is happening in a transaction from the packet level.

- May involve many monitoring locations along the application's pathway.

- Requires in-depth packet analysis, but provides the best possible vision of how the application is performing the transaction(s) in question.

If one team is planning a Stress Test, but calling it a Baseline Test-- would it matter? Well, do they intend to try it in PROD? Ouch! You probably wouldn't want any kind of load in PROD. What if they really meant Baseline, but said Stress Test? While their actions would be safe, someone would jump up on a chair and say, "Stop!" That could damage team rapport and cause long and unnecessary delays.

Creating valid testing environments and then designing appropriate test plans are critical to creating stable architecture and applications. Many IT organizations of all sizes do not have what they need in this regard.

What about a real lab? Do you do any form of Device Certification on new load balancers, or WAN Optimizers, or Switches, or Routers, or Firewalls, or NIDS, or other Testing Tools themselves? Accurate testing, (accurate being the key word here) is far less common than it should be.

Of course, now you have another question. What about Capacity Planning?