The thrill of testing

I gave myself 60 minutes to write a blog about my day and the excitement that testing brings me. For better or worse, here it is.

I’ve written in previous posts and articles that I find testing enormously satisfying and challenging. Every challenge is unique in some ways but often similar in other ways. Testing is one big jigsaw puzzle, a sudoku, a cryptic crossword puzzle, or a “whatever floats your boat” as a puzzle. You never quite know what skills you will need to pull from your toolkit, what roadblocks you might hit, and, unlike that jigsaw puzzle, there isn’t always a picture of what the completed product (or feature) looks like (and there isn’t always a single “right” answer).

Today I had one of those days that reminds about how exciting and engaging testing really is. One of those days where I got to hypothesise, explore, test, take onboard feedback from my actions, adjust, try again. Today I was a detective. I was given a crime scene but there was no weapon, no clues as to how the crime was committed, just the aftermath.

Today I was given a SQL query and asked to execute it. The SQL query revealed a number of database records that had a value in a table column that should not have been there. The number of records impacted by the “crime” was small compared to the total number of database records writing to the table. This in itself makes the problem more intriguing – why so few? My mission – determine the cause of the problem so it can be replicated and then resolved.

My first reaction was basically “wow, where do I go with this?”. Time to assess. What do I know at the moment. I know there are a bunch of incorrect records. What don’t I know? I was unsure of the relationship of one of the tables columns to our problem. OK, let’s interview the reporting witness and query that relationship. With that done, the problem was actually quite clear. That helps but it’s not completing the mission. Now the fun really starts.

As I now know the type of transaction that creates the data on this table, I created a few of those transactions and completed things I imagined the customer would or might do. Hopes were high but my hypotheses didn’t hold. That’s bad, right? Actually, it’s not. I observed some relationships between the data and the database table that I didn’t previously have knowledge of – that’s useful. I also eliminated some potential suspects.

What to do next? We keep a bunch of full copies of production data (fully masked). What might these tell me. I start querying a database that is around 2 months old. It has less instances of this problem but there’s still a number of them to consider. I was really hoping to have a smaller field if possible. Although the data volume did allow me to observe that some potential key values differed considerably. No clear discernible pattern. Dismissed totally, no – noted for later if required – but put to one side as it felt more like a distraction, a red herring. The search through the data, via SQL queries continued until I queried a copy of production data that is 4 days older than our current production copy. Then – you beauty!!! – I had only 1 less problem record in the 4 day old database than in our latest production copy. I had stumbled across an occurrence of the problem and within a 4 day window. Not only that I could see the problem when I compared the 2 records within each databases GUI.

Soooo, we have logs, let’s use them. I eagerly plug in a reference Id to help me find the suspect transactions. Drats, there is zero there that is even vaguely suspicious and nothing that has any clear alignment with the problem data state. Thinking, mulling, reflecting for a moment, boom. There are effectively 2 parties to this transaction. Doh!! I run the same log search this time on the other reference and…….boom, I can see the type of transaction that I was initially suspecting. I can see that it happened within the right timeframe and I can see that the transaction could not have been updated in the way I had hypothesised.

I’d love to close this story out with a classic “then I solved it”, but alas, that part of the story awaits for me to arrive at work tomorrow morning. So, yeah, the mission is not yet complete but I’m confident it will be tomorrow based on what I learnt today. As important as it is to find the cause and then a solution (a proper conclusion) I wanted to write this for no other reason than to celebrate how exciting testing is, and remind those who might need a reminder. There is a “rush” in going from “what the hell” to “OK, this is what’s happening” to helping derive a solution that improves your product. There were no test cases, there was minimal background, there was just evidence of “something” being wrong. Constructing a picture of that “something” is amazing fun and satisfying beyond belief. This is one reason why, after almost 20 years of testing, I keep coming back for more and celebrating the opportunities.