TOOLS CREATE NEEDS



There is an old vaudeville routine about a man who finds another man, a bit inebriated, crawling around on the cement under a street light looking for something. He asks him, "What are you looking for?" The man on the cement says, "My keys!" The kind passerby asks where the man thinks he dropped them, at which point the man on the cement points to another location around 30 feet away. The man standing says, "I don't understand. Why are you looking for them here under this street light? The man on the cement says, "What? Are you nuts? It's dark over there! I wouldn't stand a chance!"

Application or Network Performance Analysis is, in many ways, still a young industry. Its terminology and best practices are still growing, as are even its most basic concepts. This is in part due to a learning curve that is usually led by vendors seeking a niche market. Claims about the usefulness of new products are made and IT Departments purchase them. However, often these products aren't really what the buyer expected. This provides opportunity for a new product, and so on. This is not a bad thing; it is market-side evolution. It is how an industry grows.

Vendors really do want to provide what is needed, and be the first one to do so. But they also need to be able to sell it. In a marketplace where IT departments often want the tool to be fully automated itself, buyers often lead vendors into quick and easy solutions. Those solutions sell and IT workers have to make due.

One side effect of this process is that Best Practices often follow the tools limitations/abilities. If we need to measure RED and the tool can only measure BLUE--we end up with a process that knows about BLUE. So what happens? We try to describe RED issues in BLUE terms, despite the fact that RED issues were never actually monitored. We feel that we have to do it that way. After all, "It's dark over there on RED and we wouldn't stand a chance!"

IMPACT ON INDUSTRY

We tend to be a bit confused or disagree on some key concepts; especially those that are constantly redefined by vendors with a special take on the topic. Why is it that ideas like Baselining are well known--yet seldom practiced? For example, how does one go about doing a proper baseline? Which tool do you use for which job? What is most important?

-- Do you want to see how the various parts of the transaction execute? Maybe your tools are best at providing this type of information so you are focused on this viewpoint. But that is Application Profiling, not Baselining

-- Do you want to see how far you can push the application or network before they cry? Maybe you have tools that induce load from scripts, so you usually do things that way. That is Stress Testing, also not Baselining.

-- Do you want to see the best performance you can get? That is BESTlining (my term), also not baselining.

Sounds pretty subjective huh? Well, it is. That is the situation we are in. We perform the tests that our tool can perform, not necessarily the tests that we need to perform. Then we try to apply the results to our problem. Unfortunately, this often doesn't work very well.

SOLUTION?

There is no real solution but there can be more successful practices.

-- Think about what you really want to see, not what the tool you normally use can show you.

-- Think about where whatever it is that you want to see would be visible.

-- Select the tool that can be deployed in the proper location and capture the right information.

-- Buy or borrow it, along with the people that know how to use it.

-- Think in terms of skills and teams, not tools--people--not applications. Ultimately, your success is dependant on people, not tools.