3 of 3: Eurostar Testconference 2012 – Lessons learned and to learn...

The testlab and the community Hub

I didn't write down all the presentations I have seen or all discussions I've had during the conference. Nor all ideas that popped up.

But I would like to mention the Test lab, which was positioned in the expo hall. This was great, until now the test lab was hidden away somewhere in a conference room, but now it was  open and visible to see. They had computers to test on and real robots you could test. If you found a bug, you got a nice t-shirt and I wear it with proud. When you get to a conference where the Test Lab of Bart Knaack or James Lyndsay is, reserve a few hours to spend there. You could be testing software with some of the well known testers in the world and learn something new.

Testing the Robots in the testlab

Community Hub
Something new at the Eurostar conference, a room where five minutes talks were executed, discussion and to relax. Also the Eurostar minmap of learning was made in this room. This is something like a big beautiful mindmap where the visitors of Eurostar could put subjects (by yellow papers) on the map, that are drawn visually by Nathalie Roosenboom.

I visited the community hub a few times, even got a free book there: "Who wants this book?": Experiences of sofware test automation The authors (Dorothy Graham and Mark Fewster) were there, so they signed my copy. This was one of the new books I wanted to read, and didn't buy yet.

Detail of the mindmap 'what did I learn at Eurostar'

The workshop of Marcus Gärtner
I like subjects that are not really about testing but help you in your daily work. I also like to be challenged in thinking and this workshop had both aspects. Marcus Gärtner named his workshop "Beyond testing". It was really way beyond testing and we all had difficulty keeping up the pace.

We started of with some questions about testing, but very quickly we had to form groups and start thinking and discussing the question: What is System thinking? We created a mind map and got even to complexity theory and other subjects.

Later on I talked about systems thinking and was advised by to read Quality software systems Volume 1-2 by Jerry Weinberg. I knew the books, but now I finally ordered them.

The discussion in our group in mind map format.

The workshop continued about communication and aspects of communication, in combination with playing role games in this workshop. Also in blog 1 of 3 of this series I mentioned the cause effect diagram. This was discussed and tried out by ourselves. After that Marcus talked about change in organizations and a way to support that process via a certain method. And even the break in between was a communication exercise (talk to a member of another group how that group had interpreted systems thinking).

So this was a fully filled 2,5 hour, very interactive, much to learn and think about session. OK, I was lost there a few times, but the subjects are in my notes and will certainly guide me for more learning.

The end
A conference ends and then there is time to think and reflect. I guess the conference was better then I expected beforehand. So maybe next year again to Eurostar... Well, as I really like the 'big cheese'  who runs the show next time, there is no hesitating in that and I expect even a better conference next time.

To summarize the lessons learned and to learn
So to summarize this, also to check on later in the next year, these are things, because of this conference I will actively use. If other ideas pop up, that's great, but for now:

Some books were mentioned that I will have to read now

• Little black book on test design (R. Edgren) Well I read this already, but will have to revisit it, being some time ago.
• Quality software systems Volume 1-2 (Jerry Weinberg) In fact I ordered alsy volume 3
• The lean startup (Eric Ries)

Solving problems (is innovating)

• Is it really a problem, maybe there is a problem behind the problem, what is the real cause of the problem?
• Always check if the problem still exists.
• Defocus on what is perceived and try to see the bigger issue behind it.
• Start using cause-effect diagrams

Test reporting, explaining testing

• Study reporting by for example visualizing testing and testing results (Research blogs that write about this subject)
• Practice status reporting of a project: Stop where you are while working in a project and explain the status of the project in 30 seconds.
• Stop using as much as possible the ISTQB words and use the business language for this.
• Try to explain testing without using the word “test” in it. This is also something I heard from other presentations and on the Agile Testing Days. (Did the challenge this week, creating a plan to improve and structure testing, without using the word testing. That was a challenge, still have to refine this :-))

Create improving Testing within organizations checklist

• testing dojos, focus on learning and training for testers
• let people speak on conferences, introducing test work groups
• less documentation by using mind mapping techniques and low tech dashboards
• test improvement by session (session based test improvement?)

Cognitive biases

• create checklist for overcoming biases and he mentioned
• always be cautious when making decisions
• Reference class forecasting (check with other industries and compare).
• Double check the Michael Bolton presentation about confirmation bias

Skills I will work on the next months

• Influencing other people
• systems thinking

Lots of more ideas were going around, but I have to channelize this in this way. I hope it helps you too in some odd way that I wrote this down. See you!

And now on to my notes for the Agile Testing Days of last week... Pfew.

Pictures from Eurostar 2012

OK, I took some pictures during the conference with my phone. Mainly I use them for taking notes and use them to reflect later. But they also give a feeling of the overall conference I guess. So I share some pictures here that were good enough to publish on the blog. (if you want the original picture, don't hesitate to mail me)

Ideas is about solving the right problem, by Alan Page

Catch the whole potato, by Rikard Edgren

The passion of the tester. 'If you don't like testing, quit your job', by Huib Schoots

The testlab

Squeeze the chicken in the test lab when you  found a bug.

 

The community hub, small room. People sharing ideas.

The mind map of 'What did you learn at Eurostar', by Nathalie Rooseboom de Vries- van Delft

Community hub again, a five minute talk

Thinking and discussion at the workshop "Beyond testing" by Marcus Gärtner

Workshop "Beyond testing"

2 of 3: Eurostar Testconference 2012 – Lessons learned and to learn...

Part 1 of this posting is here

Huib Schoots and changing the context at the Rabobank
The world of testing is changing: As we have been using methods like TMap and ISTQB in the past, which could fit perfectly in your project or organization, the big differences in organizations and projects lead us to defer from these methods or even throw them away completely. This to rethink testing and get a better fit with testing in certain ways. The Rabobank is doing so and is a good example of introducing these (new) ideas in bigger organizations, where you would expect it not to be possible or at least too difficult to get this accomplished.

This because organizations such as banks have big IT machines that exist for a long time and have certain ways of working and people that do not want to change after working for 40 years in the same job.

Not only testing, but the world of IT is changing and the way we think about software development. The agile revolution is one of these changes. And this was also introduced in the bank and was as a hook to improve testing too.

Low tech dashboards as a way to replace comprehensive reporting

Some actions were taken, as doing testing dojos, focus on learning and training for testers, let people speak on conferences, experiment with Exploratory testing, less documentation by using mind mapping techniques and low tech dashboards, introducing test workgroups to figure things out and get the testability on a higher focus within the organisation.

These are all things I can put on my own test improvement checklist. I will do that and search out some more about those subjects and will think about how to introduce those within organizations when I'm on a project of such kind. 

Graham Freeburn and mapping your way to better testing
And the change is also happening in the way we look at test process improvement. Graham Freeburn had used Test Proces Improvement (TPI) in the past. But he learned that the TPI method is too much focused on processes and has less focus on actual skills.

So he created another way to improve testing processes. Now he uses sessions such as kick offs, presentations and creates an action plan from these sessions. Because of this there is constant communication between him and the people and between the teams in  organisations. He uses the Goals problem tree and checks constantly with the business goals. For working out and collaborating he mainly uses mind maps.

• Best quote: If you deliver value every time, the customer will worry less about costs
• Important skill he mentioned: Influencing other people.

The sessions he mentioned that he uses are good to put on my test improvement list. So instead of asking questions and do interviews, communicate, think, explore within these sessions to get the quality of testing higher. The outcome would fit an organization better than the standard formal process of TPI.

Cognitive biases
Martin Jansson (from the test lab) already mentioned this to me while talking in the test lab: “Knowing the requirements could cause you to not see certain aspects of a product and could eventually lead to missing some bugs.”

I checked ‘cognitive biases’ on Wikipedia: “A cognitive bias is a pattern of deviation in judgment that occurs in particular situations, which may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or what is broadly called irrationality.”

The inaccurate judgment and perceptual distortion would be issues we as testers are dealing with every day.

"It is the human tendency to make systematic errors in certain
circumstances based on cognitive factors rather than evidence."

 
A human is easily influenced. The speaker gave an example of a scientific research project. People were asked to write down (or think) about the last two digits of their social security number. After they were asked about this, they participated in an auction. The result was that people with higher numbers in their head because of a higher number at the end of the social security number ended up bidding more than the others.

Some ways to overcome biases and he mentioned are the use of checklists, always be cautious when making decisions (black hat thinking), reference class forecasting (check with other industries and compare).

Later that day I spoke with Srikanth and about the difficulty I have with biases, namely not recognizing them, he said, that it is good to be aware at least these exist. (so a bit of curing from biases here in his talk)

I will have to update my checklists I already have and will use them also constantly. Until now my own checklists are all over the place and not structured to my needs. Rethink, use new ideas and structure this for future use.

Good to check: Michael Bolton has a presentation about the confirmation bias here http://www.developsense.com/presentations/2010-04-ConfirmationBias.pdf

To be continued in part 3

1 of 3: Eurostar Testconference 2012 – Lessons learned and to learn...

I’ve been to the Eurostar 2012 test conference in Amsterdam (November 2012). There were so many presentations about lots of subjects. If I would try to learn from all those wonderful ideas I would certainly get lost in this overload of information. But as Alan Page said in his keynote, ideas come from other ideas, so I will let that be a guidance.

So in the end these blog posts leads to:
-    some ideas to put on my self-study list
-    a list of books to read (or to put on my everlasting book pile).
-    some ideas I can use immediately in my daily practice as a tester.

So what did I learn?

The keynote of Alan Page was about ideas and innovation. I combined some learning from Alan Page and the workshop I followed on the third day: Beyond Testing, by Marcus Gärtner.

What is innovation? To solve the ‘right’ problem. But one should know what the problem is and then you should follow it up by questioning the problem: Is it really a problem, maybe there is a problem behind the problem, what is the real cause of the problem?

Furthermore: Always check if the problem still exists. Is there a bigger problem here than what we acknowledge until now. Defocus on what is perceived and try to see the bigger issue behind it.

One way to start solving issues and create some ideas is drawing up a a cause-effect diagram: What are the different aspects that influence each other, are these positive or negative influences. Do they increase the problem or are there any positive side effects we can think of? How can we change the negative relations between activities to get out of this downward spiral.

Cause-effect diagram, drawn by M. Gärtner in workshop

The binary disease

Am I on my way to complete health from the binary disease? Richard Edgren did a talk about some diseases we have as testers. The binary disease is one of them: Reporting test results in the form of pass/fail lists. Because real life isn’t that black and white, it is actually very strange to report test results in this binary way of “passed” and “failed” test cases.

In my current job I had discussions with the project managers about test reports I created. Instead of a big list of OK and Not OK they only wanted the most important defects, and no red blinking Not OK signs.

But only reporting on blocking issues wouldn’t give the overall picture, I still wanted more in the reports, but was not sure what to do about it in this context. Eventually it was a defect report combined with mentioning possible risks for going into production. Now I should study reporting more by checking those various blogs that are written about this. There are some blogs going on about visualizing testing and testing results, so that would be something to check out. Richard also mentioned a great exercise that I’m going to use: Stop where you are while working in a project and explain the status of the project in 30 seconds.

I connected the Richard talk with the Alan Richardson (aka Evil Tester) talk later on with the subject of using other words to describe testing, not using the standard words we all learn during ISTQB or TMap courses and trainings. Use words that fit the customer in a better way.

For example: system testing is an official standard, but in practice could mean something completely different from organization to organization. So instead of trying to use certain words from testing standards and methods , use the words that best describe the situation for a certain organization, project, software product. On what system testing really means for example, simply communicate and discuss with the people that are dealing with that kind of testing.

End of part 1 of 3

30% korting op Exploratory workshop van James Lyndsay!!

Op de Eurostar test conference 2012 dit jaar kwam ik James Lyndsay tegen...

Ik ben naar de Eurostar Conferences 2012 geweest in november en ik heb daar aardig wat geleerd. Daarover volgt nog een blog.

Een van de organisatoren is James Lyndsay. Ik ken hem via Twitter en zijn blog en volg hem al een tijdje. Ook heb ik hem ontmoet op de Agile testing Days van vorig jaar. Hij geeft, waar de vraag is, Exploratory Test workshops en ik wilde eigenlijk dit jaar al naar zijn workshop, maar ik kan niet alles tegelijk. Ik kwam hem tegen op het testevent en vroeg of hij nog van plan was om nog eens naar Nederland te komen. Hij gaf aan dat hij daar nog geen ideeën over had, maar als ik minstens vier mensen kon vinden die geïnteresseerd zijn, dat hij dan naar Amsterdam komt om een workshop te geven.

Via Linkedin en Twitter heb ik inderdaad een aantal mensen gevonden die interesse hebben in zijn workshop, dus ik hoef niet zelf naar Engeland te reizen, hij komt hierheen. Nu dat redelijk zeker is (mensen moeten straks nog wel gaan boeken natuurlijk), ga ik toch maar even door met het promoten van deze workshop.

Één van de meest ervaren testers en trainers in Exploratory Testing

Als je het over Exploratory testen hebt denken veel mensen aan James Bach en Michael Bolton, maar er zijn natuurlijk veel meer mensen actief en bezig met deze aanpak. Er worden echter nog niet zoveel trainingen gegeven op dit gebied. Zelfs als je de training van Michael of James hebt gehad is dit een aan te bevelen toevoeging op je kennis en kunde. Zijn workshops bestaan uit veel oefenen en oefenen en proberen. De workshop is daarmee dan ook geen monoloog van James, maar een samenwerking tussen James en de deelnemers. Wat mij betreft de beste manier om je kennis op te doen binnen een 'leslokaal principe'

30% korting als je nu al je interesse toont!!

Ik ben blij dat ik niet hoef af te reizen naar Engeland of ergens anders en misschien zie ik jou ook volgend jaar in de workshop?
James heeft een pagina gemaakt waar hij meer informatie op heeft gezet en als je je aanmeldt als geïnteresseerde dan krijg je 30% korting op de workshop, als je die dan later ook boekt. Dus zeker vrijblijvend, voor James om te peilen met hoeveel mensen hij rekening moet houden en zich voor te bereiden. Hij zal dan ook zelf de verdere communicatie doen.

Meld je snel aan op http://unbouncepages.com/exploratory-testing-in-amsterdam-2013/

Mocht je denken aan een in-house training, want die doet James ook, neem dan even contact op met James, of met mij, dan geef ik je contact informatie door.

Tot ziens op de workshop,
Rob van Steenbergen