Monday, June 12, 2017

A Test Manager?

CEWT #4 was about test management and test managers. One of the things that became apparent during the day was how much of a moveable feast the role associated with this title is. And that reflects my own experience.

A few months ago, when discussing courses for the line managers in the Test team, a trainer outlined what his course would cover and asked whether I'd got any heuristics for management. I gave him these, none of which were included in his synopsis:
  • Clear and present. (Say what you think and why, and what you are committed to; encourage and answer any question; be approachable, available and responsive, or say when you can be)
  • It’s all about MOI. (Motivation: explain why we are doing what we’re doing; Organisation: set things up to facilitate work, opportunities; Innovation: be ready with ideas when they’re needed)
  • Congruency in all decisions. (Consider the other person, the context, yourself)

In advance of CEWT, one of my team asked me what I felt my responsibilities as a Test Manager are. Off the top of my head, I suggested they included following:
  • Provide appropriate testing resource to the business.
  • Assist in the personal development of my staff.
  • Develop relationships in my teams, with my teams, across teams.

At the pub after CEWT last night I was asked what I did as a Test Manager. I replied that it's changed a lot over time, but has encompassed situations where:
  • I was the sole tester. (And also learning how to be a tester.)
  • I was planning and scheduling the testing for a small test team, working on a single product. (And also learning about planning and scheduling for others.)
  • I was planning assignments of testers to projects and teams across products. (And also learning about how to work without knowing so much about some of the work my team are doing.)
  • I was managing larger and larger teams. (And learning how to be a better manager.)
  • I was delegating larger and larger projects to other testers. (And learning how to help others to manage larger projects.)
  • I was keeping track of more and more projects across the company, as we grew. (And learning about finding ways to get the right information at the right costs.)
  • I was delegating line management responsibility to other testers. (And learning about how to help others find and express themselves in line management roles.)

Ask a slightly different question, or a different test manager, or in a different context, or about a different time ...

Get a different answer.

Thursday, June 8, 2017

But is it Automation?

Recently, I needed to quickly explore an aspect of the behaviour of an application that takes a couple of text file inputs and produces standard output.

To get a feel for the task I set up one console with an editor open on two files (1.txt and 2.txt) and another console in which I ran the application this way:
$ more 1.txt; more 2.txt; diff -b 1.txt 2.txt
a b c d e f
a b c
d e f
< a b c d e f
> a b c
> d e f

$ more 1.txt; more 2.txt; diff -b 1.txt 2.txt
a b c d e f
a b c  d e f

$ more 1.txt; more 2.txt; diff -b 1.txt 2.txt
a b c d e f
a b cd e f
< a b c d e f
> a b cd e f
As you can see I have a single command line that dumps both the inputs and the outputs. (And diff was not the actual application I was testing!)

After each run I changed some aspect of the inputs in the first console, pressed up and enter in the second console.

What am I achieving here? I have a simple runner and record of my experiments and an easy visual comparison across the whole set. It's quick to set up and in each iteration I'm in the experiment rather than the infrastructure of the experiment.

I could have, for example, created a ton of files and run them in some kind of scripted harness or laboriously by hand. But I was short of time and I wanted to spend the time I had on exploring - on responding to what I'd observed - and not on managing data or investing in stuff I wasn't sure would be valuable yet.

I still hear and see too much about manual and automated testing for my comfort. Is what I did here manual testing? Is it automation? Could a "manual tester" really not get their head around something like this? Could an "automation tester" really not stoop so low as to use something this unsophisticated?

Bottom line for me: there's a tool that is at my disposal to serve my needs at appropriate cost, with appropriate trade-offs, and in appropriate situations. Why wouldn't I use it?
Syntax highlighting:

Friday, June 2, 2017

Taking Note

In Something of Note, a post about Karo Stoltzenburg and Neil Younger's recent workshop on note-taking, I wrote:
I am especially inspired to see whether I can distil any conventions from my own note-taking ...  I favour plain text for note-taking on the computer and I have established conventions that suit me for that. I wonder are any conventions present in multiple of the approaches that I use?
Since then I've been collecting fieldstones as I observe myself at work, talking to colleagues about how they see my note-taking and how it differs from theirs, and looking for patterns and lack of patterns in that data.


I already knew that I'd been revising and refining how I take notes on the computer for years. Looking back I can see that I first blogged about it in The Power of Fancy Plain Text in 2011 but I'd long since been crafting my conventions and had settled on something close to Mediawiki markup for pretty much everything.  And Mediawiki's format still forms the basis for much of my note-taking, although that's strongly influenced by my work context.

These are my current conventions for typed notes:
  • * bullet lists. Lots of my notes are bullets because (I find) it forces me to get to "the thing"
  • ... as a way to carry on thoughts across bullets while preserving the structure
  • > for my side of a conversation (where that is the context), or commentary (in other contexts)
  • / emphasis
  • " for direct quotes
  • ---- at start line and end line for longer quoted examples, code snippets, command line trace etc
  • ==, ====, ==== etc for section headers
  • +,-,? as variant bullet points for positive, negative, questionable
  • !,? as annotations for important and need-to-ask

These are quick to enter,  being single characters or repeated single characters. They favour readability in the editor over strict adherence to Mediawiki, e.g. I use a slash rather than  repeated single quotes for emphasis because it looks better in email and can be search-replaced easily.

I am less likely to force a particular convention on paper and I realise that I haven't put much time into thinking about the way I want to take notes in that medium. Here's what I've come up with by observation:
  • whole sentences or at least phrases
  • quotation marks around actual quotes
  • questions to me and others annotated with a name
  • starring for emphasis
  • arrows to link thoughts, with writing on the arrows sometimes
  • boxes and circles (for emphasis, but no obvious rhyme or reason to them)
  • structure diagrams; occasional mind map
  • to-do lists - I rarely keep these in files
  • ... and I cross out what I've done
  • ... and I put a big star next to things I crossed out that I didn't mean to

Why don't I care to think so hard about hand-written notes? Good question. I think it's a combination of these factors: I don't need to, I write less on paper these days, the conventions I've evolved intuitively serve me well enough, it is a free-form medium and so inventing on the fly is natural, information lodges on paper for a short time - I'll type up anything I want to keep later.

Similarities and Differences

I want to get something of that natural, intuitive spirit when typing too, although I'm not expecting the same kind of freedom as a pen on paper. What I can aim for is less mediation between my brain and the content I'm creating. To facilitate this I have, for example:
  • practised typing faster and more accurately, and without looking at my fingers
  • learned more keyboard shortcuts, e.g. for navigating between applications, managing tabs within applications, placing the cursor in the URL bar in browsers, and moving around within documents
  • pinned a set of convenient applications to the Windows taskbar in the same order on all of the computers I use regularly
  • set up the Quick Access Toolbar in Office products, and made it the same across all Office products that I use
  • made more use of MRU (most recently used) lists in applications, including increasing their size and pinning files where I can

With these, for example, I can type Windows-7, Alt-5 to open Excel and show a list of recently-used and pinned files. Jerry Weinberg aims to record his fieldstones within five seconds of thinking of them. I don't have such strict goals for myself, but I do want to make entering my data as convenient as possible, and as much like simply picking up a notepad and turning to the page I was last working on as I can.

That's one way I'm trying to bring my hand and typed note-taking closer together in spirit, at least. There are also some content similarities. For instance, I tend to write whole sentences, or at least phrases. Interestingly, I now see that I didn't record that in my list of conventions for typed notes above. Those conventions concentrate solely on syntax and I wonder if that is significant.

I don't recall an experiment where I tried hard not to write in sentences. The closest I can think of is my various attempts to use mind maps, where I find myself frustrated at the lack of verbal resolution that the size of the nodes encourages - single words for the most part. Again, I wonder whether I don't trust myself enough to remember the points that I had in mind from the shorter cues.

In both hand and typed notes, I overload some of the conventions and trust context to distinguish them. For example, on paper I can use stars for emphasis or specifically to note that something needs to be considered undeleted. On screen I'll use ? for questions and also uncertainty. I also find that I rarely start numbered lists because I don't want the overhead of going back and renumbering if I want to insert am item into the list,

Something else that I do in both cases is "layering". In Something of Note I mentioned that I'd shown my notes to another tester and we'd observed that I take what I've written and add "layers" of emphasis, connections, sub-thoughts, and new ideas on top of them. (Usually I'll do this with annotations, or perhaps sidebars linked to content with arrows.)

Similarly, one of my colleagues watched me taking notes on the computer during a phone call and commented on how I will (mostly unconsciously) take down points and then go back and refine or add to them as more information is delivered, or I have commentary on the points I've recorded.

There are some differences between the two modes of note-taking. One thing that I notice immediately is that there is no equivalent to doodling in my computer-based notes where my hand-written notes are covered in doodles. I don't know what to conclude from that.

Also, I will use different textual orientations in my written notes, to squeeze material into spaces which mean it is physically co-located with text that is related to it in some way. I don't have that freedom on screen and so any relationships have to be flagged in other ways, or rely on e.g. dynamically resizing lists to add data - something that's less easy on paper.

Where I am aggregating content into a single file over time - as I do with my notes in 1-1 meetings - I almost always work top-down so that the latest material is at the bottom and I can quickly scroll up to get recent context. (I find this intuitive, but I know others prefer latest material at the top.)

Because I don't aggregate content over time in the same way on paper, I don't have quite the same option. I write all of my notes into the same notebook, regardless of context (though I may start a new page for a new topic) so I don't have lots of places to look for a particular note that I made.

Within a notebook, I can flick back through pages to look for related material. I date-stamp my notebooks with a sticker on the front so that I can in principle go back to earlier books, but I rarely do either over periods anything longer than a handful of days.

One other major difference - a side-effect, but a significant one - is that I can easily search my computer notes.


I found that there are situations where I'll tend to use one or other of the note-taking techniques, given free choice. I prefer hand-written notes for:
  • technical meetings
  • meetings where it's less important that I maintain a record
  • meetings where typing would be intrusive or colleagues have said they find it distracting
  • informal presentations, our Team Eating brown bag lunches, local meetups
  • face-to-face job interviews
  • team meetings
  • to-do lists
  • when I need to make diagrams
  • when I don't have access to my computer

Whereas computer-based notes tend to be used for:
  • 1-1 (whether I'm the manager or the report)
  • writing reports
  • writing testing notes (including during sessions)
  • writing blogs
  • where I'm trying to think through an idea
  • when I want to copy-paste data from elsewhere or use hyperlinks
  • when I want to not have to write up later
  • when I want to be able to continue adding content over an extended period of time 

And there are occasions where I use both in tandem. For example, when engaged in testing I'll often record evidence in screenshots and drop the file location into my notes.

I might sketch a mind map on paper to help me to explore a space, then write it up in an editor because that helps me to explore the nature of the relationships.  This is probably a special case of a more general approach where I'll start on paper and switch to screen when I feel I have enough idea - or sometimes when I don't - because editing is cheaper on the computer. From Tools: Take Your Pick:
Most of my writing starts as plain text. Blog posts usually start in Notepad++ because I like the ease of editing in a real editor, because I save drafts to disk, because I work offline ... When writing in text files I also have heuristics about switching to a richer format. For instance, if I find that I'm using a set of multiply-indented bullets that are essentially representing two-dimensional data it's a sign that the data I am describing is richer than the format I'm using. 
In particular, I will aggressively move to Excel for tabular data. (And I have been refining the way I use Excel for quick one-off projects too; I love tables.)


I am an inveterate note-taker and I think I'll always prefer to record more rather than less. But when it comes to the formatting, I'll always prefer less over more. For me, the form should serve the content and nothing else, and a simpler format is (all other things being equal) a more portable format.

It appears that I'm happy to exploit differences where it serves me well, or doesn't disadvantage me too much - I clearly am not trying to go to only hand-written or only computer-based notes. But I do want to reduce variation where it doesn't have value because it means I can switch contexts without having to switch technique and that means a lower cost of switching, because I might already be switching domain, task, type of reasoning etc. In a similar spirit, I am interested in consolidating content. I want related notes in the same place by default.

But I'm not a slave to my formatting conventions: something recorded somehow now is better than nothing recorded perfectly later. I will tend to do the expedient over the consistent, and then go back and fix it if that's merited. I very deliberately default to sticking to my conventions but notice when I find myself regularly going against them, because that indicates that I probably need to change something.

Right now I am in the process of considering whether to change from ---- at the start and end of blocks to using three dashes and four dashes at start and end respectively. Why? Because sometimes I need to replace the blocks with <pre> and </pre> tags for the wiki. Marking up the start and end with the same syntax doesn't aid me in search-replacing.

When I am trying to introduce some new behaviour, I will force myself to do it. If I fail, I'll go back and redo it to help to build up muscle memory. I think of this as very loosely like a kata. For example, I was slower at typing for a while when I started to type in a more traditional way, but I put up with that cost in the belief that I would ultimately end up in a better place. (And I did.)

I think that my computer note-taking is influencing the way that I write non-note content. To give one illustration: over the years I have evolved my written communications (particularly email) to have a more note-like structure. I am now likely to write multiple one-sentence paragraphs, pared back to the minimum I think is necessary to get across the point or chain of reasoning that I want to deliver.

Likewise, I try to write more, shorter paragraphs in my blog, because research I've read, and my own experience, is that this is a more consumable format on screen.  (After seeing how much content I'd aggregated for this blog post, I considered splitting it too.)

I use text files as repositories of related information, but I also sometimes have a level of organisation above the file I'm working in. I'm recruiting as I write this. If, after I review a CV, I want to talk to the candidate, I start a text file in the folder I'm maintaining for this round of recruitment. My notes on the CV go there, as do questions I'll ask when we speak. On the phone I'll type directly into the file, recording their answers, my thoughts on their answers, new questions I want to ask and so on. At the end of the interview, I'll briefly review and note down my conclusions in the file too.

The same technique applies to my team. I have weekly 1-1 with my team and an annual review cycle. I make a folder per person, inside that a folder per cycle and, inside that I have a text file, called Notes.txt. In 1-1 I will enter notes while we talk. Outside of 1-1 I'll drop thoughts, questions, suggestions and so on into the file in preparation for our next meeting. Over time, this becomes an historical record too, so I can provide longitudinal context to discussions.

This stuff works for me - or at least, is working for me right now better than anything else I've tried recently and given the kinds of assessments I've made of it - but none of it is set in stone. My overarching goal is to be efficient and effective and I'm always interested in other people's conventions in case I can learn something that helps me to improve my own.

Sunday, May 28, 2017

Best Practise

I've said many times on here that writing for me is a kind of internal dialogue: compose a position, propose it in writing, challenge it in thought, and repeat.

I get enormous value from this approach, and have done for a long time. But in two discussions last week (Lean Coffee and a testing workshop with one of the other teams at Linguamatics) I found additional nuances that I hadn't considered previously.

First: in some sense, the approach I take is like pairing with myself. Externalising my ideas sets up, for me, the opportunity to take an alternative perspective that doesn't exist to the same extent when I'm only working in my head. It's often about the way I'm thinking as much as the content of my thoughts, and I speculate that this is a good grounding for being criticised by others when we're working together.

Second: writing and re-reading makes my position clear to me, and forces me to work out a way in which I can put it across. Since I started blogging there are numerous times in discussions that I've realised I am paraphrasing from something I've written. In the past I've tended to be a bit embarrassed by that but now I can see that, in fact, it's largely because I spent the time working it out before that I have it available to me now.

These are both things that are useful to me and that I want to get more benefit from. And, while I might agree that outside of a specific context there are no best practices, I also know that if I want to get those outcomes from my writing, I'd best practise.

Wednesday, May 24, 2017

Cambridge Lean Coffee

This month's Lean Coffee was hosted by Redgate. Here's some brief, aggregated comments and questions  on topics covered by the group I was in.

What benefit would pair testing give me?

  • I want to get my team away from scripted test cases and I think that pairing could help.
  • What do testers get out of it? How does it improve the product?
  • It encourages a different approach.
  • It lets your mind run free.
  • It can bring your team closer together.
  • It can increase the skills across the test group.
  • It can spread knowledge between teams.
  • You could use the cases as jumping-off points.
  • I am currently pairing with a senior tester on two approaches at the same time: functional and performance.
  • For pairing to work well, you need to know each other, to have a relationship.
  • There are different pairing approaches.
  • How long should you pair for?
  • We turned three hour solo sessions into 40 minute pair sessions.
  • You can learn a lot, e.g. new perspectives, short-cuts, tips.
  • Why not pair with developers?

Do you have a default first test? What it is? Why?

  • Ask what's in the build, ask what the expectation is.
  • A meta test: check that what you have in front of you is the right thing to test.
  • It changes over time; often you might be biased by recent bugs, events, reading etc to do a particular thing.
  • Make a mind map.
  • A meta test: inspect the context; what does it make sense to do here?
  • A pathetic test: just explore the software without challenging it. Allow it to demonstrate itself to you.
  • Check that the problem that is fixed in this build can be reproduced in an earlier build.

How do you tell your testing story to your team?

  • Is it a report, at the whiteboard, slides, a diagram, ...?
  • Great to hear it called a story, many people talk about a report, an output etc.
  • Some people just want a yes or no; a ship or not.
  • I like the RST approach to the content: what you did, what you found, the values and risks.
  • Start writing your story early; it helps to keep you on track and review what you've done
  • Writing is like pairing with yourself!
  • In TDD, the tests are the story.

One thing that would turn you off a job advert? One thing that would make you interested?

  • Off: a list of skills (I prefer a story around the role).
  • Off: needing a degree.
  • Interested: the impression that there's challenge in the role and unknowns in the tasks.
  • The advert is never like the job!
  • Interested: describes what you would be working on.
  • Off: "you will help guarantee quality".
  • Interested: learning opportunities.
  • Interested: that it's just outside of my comfort zone.

Saturday, May 20, 2017

The A-Z of XP

After I blathered on and on about how much I'd enjoyed Ron Jeffries' Extreme Programming Adventures in C# the Dev Manager offered to lend me his copy of Extreme Programming Explained by Kent Beck.

Some background from Wikipedia:
Extreme programming was created by Kent Beck during his work on the Chrysler Comprehensive Compensation System (C3) payroll project. Beck became the C3 project leader in March 1996 and began to refine the development methodology used in the project and wrote a book on the methodology (in October 1999, Extreme Programming Explained was published).
So I took the book (it's the first edition) and I enjoyed it too, but differently. I might say that if Adventures is a road trip, Explained is a road atlas.

One of the things I liked about Explained (that it shares with Adventures) is the suggestion that only you can really decide whether XP can work in your context, and how. Also that Beck is prepared to offer you suggestions about when it might not.

But the world probably doesn't need any more reviews of this book so instead I'll note that I was a little surprised at the degree of upfront formality (which isn't to say that I don't think formality can license freedom to express yourself); sufficiently surprised that I mapped it to help navigate the rest. (And, yes, that's a map from an atlas.)

Image: Amazon

Monday, May 15, 2017

The Daily Grind

Houghton Mill is an 18th-century water mill, full of impressive machinery and, last weekend, actually grinding flour by the power of the river Great Ouse. Although I am not knowledgeable about these kinds of buildings or this technology I found myself spellbound by a small, but crucial, component in the milling process, the slipper.

The slipper is a kind of hopper that feeds grain into the millstones for grinding, Here's a short film I took of it in operation when I was there with some friends and our kids:

It has a system of its own, and also it is intimately connected to other systems.

It has inputs: a gravity feed brings grain into the top of the slipper; energy is supplied by the vertical axle which is in turn driven indirectly from the water wheel.

It has outputs: grain is dropped into the centre of the millstones immediately below it.

It is self-regulating: as the flow of the river varies, the speed of the wheel varies, the rotation of the axle varies, and the extent to which the slipper is agitated varies. Slower water means less grain supplied to the millstones, which is what is required, as they are also grinding more slowly. A second form of self-regulation occurs with the flow of grain into the slipper.

It has balance: there is a cam mechanism on the axle which pushes the slipper to the left, and a taut string which pulls it back to the right, providing the motion that encourages grain to move.

It can be tuned: the strings that you can see at the front provide ways to alter the angle of the slipper, and the tension of the string to the right can be adjusted to change the balance.

Tuning is important. If properties of the grain change (such as size, or stickiness, or texture, ...) then the action of the slipper may need to change in step. If the properties of the millstones change (e.g. they are adjusted to grind more coarsely or finely, or they are replaced for cleaning, or the surface roughness alters as they age, ...) then the rate of delivery of grain will need to adjust too.

Although the system is self-regulating, these are examples of variables that it does not self-control for. It has no inherent feedback mechanism for them, and so requires external action to change its behaviour.

Further, beyond the skilled eye and ear (and fingers, which are used to judge the quality of the flour) of the miller, I could see no means of alerting that a change might even be required. In a mill running at full tilt, with multiple sets of stones grinding in parallel, with the noise, and dust, and cramped conditions, this must have been a challenge.

Another challenge would be in setting the system up at optimum balance for the conditions that existed at that point. I found no evidence of gauges, or markers, or scales that might indicate standard settings. I noted that the tuning is analogue, there are infinite fine variations that can be made, and the ways in which the system can be tuned no doubt interact.

The simplicity of the self-regulation is beautiful to me. But I wondered why not regulate the power coming from the water wheel instead and so potentially simplify all other systems that rely on it. There are technologies designed to implement this kind of behaviour, such as governors and flywheels.

I wondered also about the operational range of the self-regulation. At what speeds would it become untenable: too slow to shake any grain, or too fast to be stable? There didn't seem to be scope for an automatic cut-out.

So that was an enjoyable ten minutes - while the kids were playing with a model mill - to practice observation, and thought experiments, and reasoning in a world unfamiliar to me.

I doubt you'll find it difficult to make an analogy to systems that you operate within and with, so I won't labour any points here. But I will mention the continual delight I find in those trains of thought in one domain that provoke trains of thought in another.