xBTM: Harnessing the Power of Exploratory Testing

Previously posted on the PQA blog.

xBTM is a topic that I keep coming back to, and I think that is all in good order. Nothing is perfect from day one – we need to go back and rethink and revise our ideas continuously.

I’m a big fan of exploratory testing – it provides both flexibility and speed, characteristics that have become increasingly important, especially with the quick pace of short agile iterations.  But, how do you retain traceability in exploratory testing without losing your creativity? How do you, as a manager, actually manage testing that is unscripted and improvised? One answer is to use a combination of session-based test management (SBTM) and thread-based test management (TBTM) called xBTM.

Session-based Test Management (SBTM)

SBTM (Jonathan Bach, 2000) is the reply to the common misconception that exploratory testing is by its very nature always unplanned, undocumented and unmeasurable. Using SBTM, exploratory testing is done in time-boxed sessions. Each session has a mission that is specified in the test charter, for example, the sessions are actually planned ahead (but not scripted). A session results in a session report that provides a reviewable record of the testing that was done. Based on the session report, the test manager can derive various metrics to track the progress of the testing. SBTM doesn’t have much in common with ”ad hoc” testing, in fact – SBTM is quite strict, sometimes even too strict.

Thread-based Test Management (TBTM)

There are environmens too chaotic or hectic for SBTM to work properly, and this led to the introduction of TBTM (James Bach, 2010). TBTM is a generalization of SBTM that allows for more freedom and flexibility. The core of TBTM is threads and, similarily to sessions, threads are test activities intended to reach a certain test goal. The big difference is that whereas a session is a committment to complete a specific task in a given amount of time, threads come with no such committments. A thread can be interrupted, resumed, canceled or go on indefinitely. SBTM defines both where to start and where to end, but TBTM only definies where to start. In TBTM, activities are allowed to change over time. TBTM is the free-spirited sister of SBTM.

SBTM -> TBTM -> xBTM

SBTM and TBTM both have their strengths and weaknesses, and rather than having to choose one over the other, xBTM was created. xBTM unites the two exploratory techniques to get the full advantage of both, focusing on increased test efficiency and creation only of artifacts that actually add value. The name xBTM highlights that it is combination of SBTM and TBTM (x = S or T).

The xBTM workflow is centered around a mind map:

  1. List all test ideas and activities – these are your threads.
  2. Arrange all threads in a mind map grouped by function area or test technique. This constitutes the test plan.
  3. Prioritize the threads.
  4. Use SBTM when possible. Group threads to create suitable charters and create session reports once the charters are executed.
  5. Use TBTM when SBTM is not an option. Follow the threads where they take you and add new threads when needed.
  6. Update the mind map continuously, showing the progress and state of the threads. The mind map is the status report.

Using a mind map, rather than a traditional text document, makes conveying the test plan easier, and it helps communication — not only within the test team but across the whole project team. Figure 1 shows a simplified example of what the xBTM mind map for testing an e-shop might look like.

Image

Figure 1: Example of an xBTM mind map. Here, an e-shop is being tested, and the mind map shows some of the threads. For example, “Create user account” is being tested in a session, whereas “Add item to shopping cart” is being tested as a thread.

Want more?

Want to know more about xBTM? A detailed walk-through of applying xBTM on a real project is given on my old blog.

Also, look for our workshop ”Exploratory Testing on Agile Projects” at STAREAST in 2013.

Links

There are many different mind map tools to choose from and, in most cases, the core features are very similar. Which one you end up picking will mainly be a question of personal taste. The author has had very good experiences using the following three tools:

  • —XMind: A powerful tool with a good selection of icons and markers. The base version is free and there is a Pro version available for purchase. XMind was used to create the example in this article.
  • —Mindmeister: Collaborative tool that stores mind maps online and allows multiple users to edit the same mind map simultaneously. The free version limits the number of mind maps you can create and store.
  • —FreeMind: The simpler of the mind mapping tools, but still very useful and completely free.

Additionally there are a number of tools that help you manage your session-based testing. Most of these tools focus on recording the session reports. Three tools which the author has used in the past are:

  • —Rapid Reporter: A note-taking tool for exploratory testing sessions.
  • —Session Tester: A tool for recording and managing exploratory testing session.
  • —SBTExecute (scroll down to bottom of page for English): A tool that produces summary reports and calculates metrics from an Excel session report template.

Stay tuned for the next episode…

 

About these ads

4 thoughts on “xBTM: Harnessing the Power of Exploratory Testing

  1. Thanks for writing about this, Christin – TBTM is a new concept to me. Certainly an interesting path to follow, and the two previous examples you’ve given in your old blog post give a good illustration (also, like the screenshots of how you use xMind, something I’ve only just started to play with).

    Have you used SBTExecute to manage a xBTM project? If so, would be interested to hear your thoughts. Managing the session-based test reports is something I’m still experimenting with, and haven’t found a way I’m comfortable doing so yet.

    • Christin says:

      Hi Dean,

      Thanks for reading and commenting!

      Yes, I have used SBTExecute on an xBTM project, and I was quite happy with it. A lot of other session testing tools only help you with the note taking, but what I was missing was something that would derive metrics and produce fancy graphs from my session reports – much appreciated features if you need to communicate your test results.

      What tools have you tried? I’m eager to hear about other people’s experiences and preferred choices.

      Christin

      • I’ve only used a few you mentioned in your post – at the moment using Rapid Reporter (and then just manually creating spread sheet reports for external consumption) and XMind for test planning / reporting purposes. XMind is fantastic, I’m only just starting to really take full advantage of its features. I didn’t mind Session Tester, but feel more comfortable with RR.

        Good to hear SBTExecute is handy with metrics and graphs, would be nice to be able to do that sort of thing on projects that are exploratory-based. I’ll have to check it out!

      • Christin says:

        Hi Dean,

        XMind is really my favourite among mind map tools. Easy to use, nice looking, powerful features etc. I find the mind maps very helpful when it comes to planning and communication.

        Give SBTExecute a try – I really like how it auomatically generates nice-looking PDF reports. Let me know how it works out for you, and if you find any other nice tools.

        Christin

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: