Overcoming Irrationality

The past few months have been rewarding. On every aspects.

From a change in work, to work place to be able to spend time ‘thinking’ about testing. It is always nice to have time on hand, peers to bounce ideas off, advise from experts, knowledge from intellectuals to be able to better yourself; be it as a tester or as a human.

I remember, it was just over a year at the BCS SIGIST conference that I came across James Lyndsay’s paper “Irrational Tester“. It immediately strikes a chord. It immediately lights the bulb – I could almost feel it over my head at the time! Not only was it a wake up call, it also acts, since that day, as a check list for me.

I’ve been trying to experiment with my own version of exploratory testing at the workplace – old and new – and have been having varying degree of success. Its what suits best that counts, to be honest. Testing Data Warehouses has tremendous scope for exploratory testing (something for later day) as well as automation. I’d say a lot of exploratory testing *before* automation checks!

It is during one of these automation checks I have been undertaking recently that reminded me of the irrational tester. We have a pool of resources (I wont categorise them as ‘developers’ or ‘testers’ yet) who have been testing a part of data warehouse. Nothing wrong there; Even the Analysts have been having a poke around, which is really good to see. Minutest of details are being questioned, logged etc.

The situation now is that we have many many tests (read SQLs) that poke around at various places and do their own bit of testing.

Let me rephrase that. There are many ‘test’ SQLs that do their own bit of ‘checking’.

And because everyone has been doing their ownbit of testing, there’s no one who has bothered to assemble/write a regression pack.

Enter yours truly. After deciding on an approach to be taken, I set myself down saying – how hard can it be – looking at already written code, merging them after tidying/optimising them into a pack.

No sooner had I started merging multiple SQLs, I could see myself succumbing to almost every facet of the Confirmation Bias James mentions in his paper. Its not just limited to finding defects, but is also with other things ‘test’ related – design, approach and even execution.

I fell at the first hurdle. I assumed I knew the exact SQL script (of the hundreds) required, had a good understanding of what I had ‘expected’ it to do just by looking at a few lines of code and was overconfident because I had to get the script done in a day!

James describes this as Illusion of Control.

Even when I did get the right scripts and merged them to do a multitude of things – I couldnt get the script to return the same results as the individual ones did, I had fallen prey to Congruence bias. This was because of multiple joins to different tables with unions produced a cartesian product which wasn’t desired – which was also meant Inattentional bias!

It was not long before I realised I had to stop. Take a step (or two) back. The first thing I did was delete the script I had created (read stiched together).

Doing a retrospection, I could feel a light bulb over my head. It wasn’t exactly rocket science to realise everyone tests differently and so everyone will have their own version of a script / SQL, doing things in their own way. Some developers will write efficient SQL, analysts won’t. If I have a task to do my own thing, I should do it my own way.

Another ‘ping’ and another light bulb lit up. No better opportunity for me to use exploratory testing. Spoke to the business analyst and developer over lunch (and later at the coffee machine) to get an understanding of what was required, what tables, columns were required, what business rules were to be followed and I had my mind map (in my mind) of what I needed to do.

It is essential to point out that the strategies James mentioned were critically important in helping me de-biase. Those who might have jumped to the document might notice some of them including seeking independent explanations, keeping mind engaged by avoiding tiredness (in this case focus away from the clutter of SQLs), seeking feedback and even recognising a failing strategy and cutting one’s losses!

So, overall I’d say I ‘invested’ first half of that day learning.

A classic example of “At first, we cognize; then we recognize”.

Advertisements