Thursday, 21 October 2010

What about testing infrastructure?

I've been in some infrastructural projects the last few years and still wondering how these projects survive and produce products that are of 'good enough' quality. With testing infrastructure I mean: Upgrade of the Operating System for the complete company, new Multifunctional printers, Citrix environments, corporate virus tools and so on.

Testing is not a part of developing infrastructure
Wish I could say testing is really a part of this infrastructure world. But it isn't. There are not many testers around specialized in this area. I did lead a team of 10 testers at a company, those people were all testing infrastructure. The complete team of testers in the company had 35 (or so) infrastructural testers. Well... How many of them really studied the craft of testing, how many knew at least one methodology or testing technique? Maybe five or six? And what are they doing a few years later? Testing software again at another company or getting another job, system engineer, designer, bus driver or something. De motivated by the lack of interest in the infrastructure world for testing.

Methodology for testing infrastructure
One company tried to work out a methodology for testing infrastructure. They came up with their 750 page book about structured testing and made a light version of it. So the structure is there, what about testing? Well, let the technicians test within the boundaries of this methodology. What if they don't want to be in this structural hazard of this methodology? What if they really do not want to test? Because they are technicians, they want to create a system that makes it possible to connect several thousands of computers together, or build a software distribution system that enables them to distribute applications with one press on the button. "Please don't tell us to test!"

Infrastructure is becoming software
But more and more the infrastructure world is realizing costs of solving problems in production go up. Also, infrastructure is becoming more and more complex. Your Windows computer is not on your desk but on a server some miles away. The little box on your desk has also an operating system, but that's only there to pass keyboard strokes and mouse gliding to that server a few miles away. Virtualization of infrastructure (servers, switches, desktops) makes infrastructure look more and more like software products. The combination of hardware and software does not matter anymore.

Realizing the costs of problems in production
While the software world already knows (at least more companies understand) that going to find out there are problems with your system when it is already in production isn't going to work for the long term, in the infrastructure world it is still a hidden cost.

Let the technicians test
I tried starting to explain exploratory testing to the technicians that would be testing the release, because that was part of my strategy, some scripted testing, for the basic checks and then we would test the highest product risks with exploratory testing. And I would explain how to do that...

And then realized something I already knew, but forgot for a while, that good exploratory testing can only be done by testers with some experience in testing.

"Testing? Bug tracking tools?"
And also realize that IT companies that sell you an infrastructure solution even don't know yet what a bug database is.

If the level of testing in the infrastructure world is still on "not even knowing what testing is and not ever heard of bug tracking tools", it will be a hard day’s work to keep up with technology that works 'good enough'.

I need you!
As I see it, we will need a lot of testers in the infrastructure world who are capable of exploring all the attributes, searching for the flaws in infrastructure systems, just as we do with software.

So please technical people that want to learn about testing: don't leave me here alone and join me in the testworld, I don't know if I can hold out much longer here!

No comments:

Post a Comment