Changing context for generating better testing idea’s

As a tester one part of your job is to come up with testing idea’s.  A way that is very often used is take the template used in a previous testcase. The test cases can have the format like excel or something like As a user I want so that. It is very comfortable to do, it gives you guidance. The result is often good.

I tend to use a different approach. The tool I use to execute the test, this can be manual or automated, I will not use for brainstorming / test case generation. In a project where I use cucumber-js for checking acceptance criteria. I use a mind map for generating the testing idea’s. The tool that I use is Xmind.
I tend to start with a blank document, this is done so that I am not biased by the previous example. After this test example generation, I have a nice set of tests. Then I open a previous test to see if I have something forgotten or I need to add test from the previous test. This follows the same process (look, see, image, show) as described in MinmapTestingIdea'son the The back of the napkin from Dan Roam.
Other great sources of tests idea’s are the domain testing workbook from Cem Kamer, testing tours, the heuristic test strategy model, You Are Not Done Yet list.

I also use the mind map in Xmind to mark al test that (in this case) are automated, see example image.
When these idea’s do not suites you an other option is to use a little technique to play with words. Let me give an example: As an site admin I want to see if my site is operational so that …

As an site admin I want to see if my site is operational.
As an site owner I want to see if the site is operational.
As an site user I want to see if the site is operational.
As an site developer I want to see if the site is operational.
As an site … I want to see if the site is operational.

As an site admin I want to get a mail if the site is operational.
As an site admin I want to get a mail if the site is not operational.
As an site admin I want to get text message if the site becomes not operational.
As an site admin I want to get text message if the site becomes operational again.
As an site admin I want to get mail if the site is degraded.
As an site admin I want to get mail if the site is slow.

You got the idea (I think), so you can create many more testing idea’s.

Tip: Try each word to changed with an inverse one, an alternative word like in this example change see into / call / text message /mail /… .

Chancing the format / the view helps to get new idea’s. Try to print and use a pen and paper. Try to go where your product is used or go outside in a park, at least go to a different room.

In my case grabbing a guitar will also help. Be surprised about what happens.

Posted in Agile, practical things, Specification, testing | 1 Comment

Shorten your Feedback cycle while testing.

During the last months I confronted my selve with the fact that I have material to share but where? Was it in a presentation a document or a piece of code? somewhere I have it?

I will start a serie with small post on practical things / techniques that help teams to continue. This is probably just as helpful for you then it is for me.

Shorten your feedback cycle while testing.

If your testing and you observe the status of the GUI but do you also observe the status of the backend or hardware? An example:

You have an import function. It will probably have successful and unsuccessful messages after importing. In your backend there are probably also messages of succes or not. I recommend to view live (like tail message.log) on the server if it really works. If an import seems to work on UI level it will not say that is dit not give acceptions in the backend. If you want to enhance it, you change the color of the terminal or logging window or add a sound. This will shorten the feedback cycle. The problem is that after 10 different import it is hard to sync which error occurred on which step. If an error occurs you direct see the color change. The investigation can start.

You make it visible

Below you see an example which uses this code (tailored for a mac). In the upper terminal I edit a file (message.log). I save the file and add more text this time with the word error. The lower terminal changes the color when the text error is present in the message.log file.

tail -F message.log | grep -m 1 "error";tput setab 13; clear; tail message.log

Posted in practical things, testing | Leave a comment

My approach on getting grip on exploratory testing

How to do a decent job on Exploratory testing. That is the question I gave my self at the beginning of this year.

What did I do.

My colleague Huib Schoot explained me his view on exploratory testing, as a context driving tester, he pointed out that there are more views on exploratory testing. Like the overview on exploratory testing by  Gitte Ottosen  See slideshare

Huib also pointed me to the course Getting a grip on exploratory testing by James Lyndsay.

It is a training that triggered me in an excellent way. I was already excited about exploratory testing but it gave me a boost. The relaxed way of teaching by James and the extreme amount of experience made it great to follow. It was a training with experienced students like me therefore the coffee breaks were also worthwhile.
James creates testmachine for understanding exploratory testing. They are great tool to let you experience that testing something this simplistic like a one button application is not always easy to test. I found out that it can be challenging to test something that simple. All the participants explained their test idea’s and together with James ideas we got the idea we have a good start.  James explained  the framework In / Out which uses inputs and outputs to discover the behavior of the program.

If you want to know more about exploratory testing it is a great course. They only downside I noticed that the available time for the course could be more. Two days for such a great training is short.

After the course it was time to practice at my daily job. After this, it was time to give training together with Huib Schoots and later give another training and a couple workshops. Giving training is a great way of learning.  At TestNet Huib and I gave a workshop and we gave a tutorial at Agile Testing Days where exploratory testing was a part of the tutorial.

The great things about giving training it lets you practice and tests your knowledge. Especially when having a colleague like Huib Schoots, he is a walking wikipedia on Exploratory testing. It is great to explore exploratory testing with a person like him.

All this gave me an experience boost on exploratory testing this year.  

Posted in coaching, testing | Tagged | 2 Comments

User Story Risk Quadrants

On my way back from Agile Testing Days 2013 I had an conversation with Gitte Ottosen of course it was about testing. We talked a bout risk and how to address that in agile. Therefore I have a very simple but effect method to get the team more involved.

I call it User Story Risk Quadrants it is designed to let the team discus how much risk is involved with the user story.
In the past I gave presentation on risk in agile so I search for an old presentation and found out that I did not have anything online to point out to. I send her an email with an example. Her response to my email  was “I just wanted to let you know that people are really loving the risk quadrant you showed me.
This is why I wrote this blog.

PRA User Story Risk Quadrants

What is it?
It is a detailed version of a standard Product Risk Analysis or a graph to show the combination of likelihood and impact.
This can be used in a project if they want to use a risk based approach. The idea is focus on the more riskier part of the software. You can find these graphs for example in a master testplan. As in Agile there is no focus on the master testplan so I moved the addressing risk to a lower level, the user stories part. 
Why should I use it? 

I see two problems and if you like to solve them

  1. The high and low risk areas are tested the same way.
  2. There is no conversation about risk in the sprint
1. Isn’t it strange that risky areas are tested the same way?
There is a golden rule I use. Diversify in technique and in data. The problem sometime is that you as an tester should at least know 15 techniques to diversify. If you know one you need to learn more first.
The use of User Story Risk Quadrants is a good way to show the rest of the team that this user story will be tested more in dept and the other only by a simple error guessing. It is hard for some team member to diverse the test. Therefor this helps them to remind to diversify. Example PPT  Example key

2. As an tester you are the test mind in the team. When the risk is high the level of testing is chaining accordingly and you can have a discussion why this user story is needs more testing.
How to use it?
User story risk quadrants beside a scrumboard
I start by asking the Product owner and the development team about the impact and likelihood of that user story. Add the result to the user story.  As a big fan of visualization I like put it on the wall
When to use it?
Most likely I use it during the preparation of the user story with the Product Owner. User stories are not done when they do not have a risk attribute.
And during the planning I tell the team that this user story is a quadrant 4 user story and that means these kind of tests.
Posted in Agile, conference, Specification, testing | Leave a comment

Getting a plan and start building

Third iteration Getting a plan.

We known that we are allowed to build and we have an assumption that we can afford it.

The next questions that should be solved are: What is the time frame? And when should we start building.
The plan was to do the most by our selves. The plan is simple

  • Remove the old shed 2 weeks.
  • Brick layering 1 month
  • Carpenters work  1 month
  • Garden reconstruction 2 weeks.

This means a total of 3 month work.  The quality is fixed for us and we also wanted to fixed the time. The time is more or less fixed because at an certain moment you want to finnish. I hate projects of a year or longer. So with these assumption we found out we want needed help if we want to meet the deadline. We need specialist to help us. We asked a carpenter and a bricklayer to help us. Both we asked what do you think of our assumptions and are you able to help us.  The bricklayer has 40 years of experience and said it could be done in less then 2 weeks. The carpenter with 25 years of experience said between 6 and 8 weeks. Both said that we had to help them to keep the amount of hours low. (Both get paid by the hour).  Because I know both persons for a long time we made a deal with a handshake. TRUST (value people)  We had a shared understanding of what my vision is (the prototype and the drawing) With an handshake and an estimation in weeks I went home with a smile. With my neighbor we made the agreement that he should do all the non technical stuff, like carrying all the stuff to the garden  and I will do all the technical stuff. Again with a handshake.

Iteration successful with an agreement (my plan).

Iteration number four: starting to build

Everything was perfect. We had an agreement, we were  allowed, we had the money. Nothing can go wrong. But it can go wrong. We planned it to start early in 2010. During Christmas (2009 ) my neighbor fell while playing soccer with his grand child. He broke his hip. So If everything was postponed 3 months  The facts is that  surgeons can makes mistakes so he was operated 3 times and it took until the end of summer before he was recovered. After this negative delay we had positive delay. I was becoming father for the second time. That means from experience, It impacts the available time for a job like this. So it got postponed again.

Iteration four failed.

Iteration number five : Starting to build, again. 

In the summer 2012 we planned to start again. My daughter is then almost a year  and she is doing great.  My neighbor is healthy so we started.IMG_2900

In the first two weeks we removed the shed from me and my neighbor. We put the new concrete floor on top of the old concrete floor to make it stronger to hold the mansion. The bricklayer came and build all the walls in two weeks. My neighbor and I could help a lot and everything went fine and within the estimation of hours. We requested to build the brick wall instead of the wooden wall so we changed the “plan” a little back to the original brick wall idea. It took 2 additional days but that was not a problem. All the bricklaying was done in 2 and half week. Perfect.

everything still ok

It was time to get started with all the wood. Unlucky I had injured my shoulder badly. It slowed me down. The carpenter worked hard and there was a very interesting thing that was going on. He used the prototype and the drawing just for imagination.  He was very experienced and told me drawings are nice but this is the real world and that world changes.

It became a way of living: wakeup, go to work, come back home and go to work in the garden until late in the evening and go to sleep. Next day same patern. The idea was to finish in 8 weeks. At the end it was 8 weeks but with an huge amount of hours for my but it was done. I was committed to get it done.

Iteration 5 : done

Iteration number six : finishing . 

The mansion is finished now with even electricity and water.  I call it done but the garden is not that pretty anymore. In the next month we will see how nature survived it.


When I look back there are so many similarities with building the mansion with building software.

  1. You need good people.
  2. You need a clear vision of the future
  3. fun
  4. commitment.
  5. Things will change

I was writing this while sitting on the porch.  For me is physical work great to get my brain relaxed.

Posted in mansion | Leave a comment

Speaking at STAREAST 2013

The first time speaking on STAREAST. My talk was about thinking visual ad visualisation tools for testers. I’ve arrived there well prepared, was it good enough? I’ve ask James Bach and Michael Bolton to “test” my talk. Not only the content but also the way how I present. James was not able to make it due to an interview but he did send his brother John.  Which I thought was very nice.speakerStarEast2013

I talked to Derk jan de Grood and Ruud Teunissen in the morning and both said we will go through our talk. Then I talk to James Lindsay and he said prepare your talk.  After 3 experienced speakers I’ve got the message, ill go through it one more time. I did discoverd that when I use my phone as a timer you need to set the auto lock to off.
Being well prepared I checked the room an hour before to check if I need additional extension material or connectors / converters. I also went to the presentation of Paul Holland prior to me in the room that I speak in. Paul Holland did the session about PCO, he had some audio problems. Therefore I took extra time to check that. Everything was setup the way I wanted it to be. The room was full of people the presentation could start.
It was an honor to talk to a fully pack room. The presentation went very well, the feedback should tell if it is true.  There were quite some people telling me they liked it and I’m pleased with that. Like here on twitter!/hattsoff_inc/status/329772370531336193. Got feedback from Jon Bach and from Michael Bolton on my talk and they where positive about my presentation. A few improvements that is alway good, I am not perfect. Like forgetting to take a picture of the audience to show you.

What did STAREAST bring me?

It confirmed that it is good to rehearsal and take the time to get your presentation setup.
Presentation was my main point to go to stareast, there was more that made it a great conference. I was very pleased with the invitation from James Lindsay for the test lab. Later I’ve got invited for an interview as well for the virtual conference. And I’ll already signed up for meet the speakers.  There were also a lot of very interesting people over there.  And off course there some great presentations. (In the picture  you see Derkjan Jan de Grood speaking, same room ).

James Lindsay ask me to do a session on the testlab. It was great to do a handson session. The software under test was the aloha editor. The approach I took how visualization can help you to, to understand the system.
The first step was draw a tiny PCO. This to make clear what we test. In an normal situation I would take the next steps.
  1. Collect as  much information as I can
  2. Try to understand relation between the gatherd information
  3. Try to visualize what I want to test
  4. Create a test report / create dashboard

In the testlab I did focus on one part of my approach.

After the tiny PCO the next step was pick one function of the PCO, in this case it was enabling and disabling bold text. Then we zoomed in on the feature how it works. We used a flip over to draw an overview.  We explored the design / architecture of the application by sketching together. This sketching together is an nessecary step to understanding the situation. You as a tester needs to know where it can go wrong therefor the design / architecture is important .  We must understand how it works see slide visual test strategy in my presentation. The last part of my approach is get all information I need for the test report. It is always good to know what your story is that you like to tell. It is also a check if I am missing information during testing. For example what are the average response time during testing. If you do not measure it during testing you can not put it in your report.
The testlab was for me a lot of fun.
Next time I lke to do it again!!
The interview
It was my first interview, hope I did we’ll . When I know the link I will publish it.


Meet the speaker at the table.  This time not with visualization as title but with my other favorite title developers are my agile friends.
This was for me a nice experience there were nice colleague testers that could not make it to the  presentation but had questions. We had some nice discussions on visualization. There were also two testers by acident were sitting at the table. With them I got great discussion on documenting the test. How they do it. How I do it.
It where two great days at the conference.  Next year ill like to go again.
Posted in conference, testing | Leave a comment

Magic Estimation can work!

Today a team asked me on how to do a good grooming session. They were arguing on details and process.

After the team had spend an hour or two on estimation I asked them to join me in the afternoon to do a different kind of estimation session. What I did was suggesting magic estimation I did it some years before but it seemed right for this context , were the product owner wants an first estimation on his product backlog, which was filled with new epics. A couple epic with details some epics pretty vague.


I started with a little game (this to fresh up the brain). And explained how magic estimation works.

All the poker cards are on the table from 1 to 100. I gave everybody a set of stories / epics. In pure silence they lay down the stories on the table with corresponds to the correct number. After this all story have an initial estimation. Everybody had a good look at each estimation and we discussed a couple strange ones. Within 45 minutes we had estimated 30 stories / epics.

The product owner wrote down all the points and prioritized the list. Of course it is not a perfect and detailed estimation but that is not what the scrum team wanted.

DONE and a happy team.

Posted in Agile, coaching | 3 Comments