My first non-work presentation

This past Thursday evening I gave my first public presentation. I gave it at the monthly SASQAG meeting. I talked about some of my past experience with tools-assisted testing. For not having much experience, I thought it went well.

SASQAG is a great group to be a part of. They are friendly and supportive. I got a green mug (big enough to hold a can of soup, only way to get it is by speaking at SASQAG) and they took me out to dinner afterwards. One part of a great week.

Learning from #RSTORCAS: Keep calm and understand the context

One of the things that I learned from my recent attendance at #RSTOrcas class was that many testers (myself included) have a tendency to get started on testing assignments before we understand everything that we might about what we are being asked to do. That was something that came out in many of the testing exercises that were a part of the class. Those exercises were well crafted to show us our tendencies in that regard. After about the third time I saw it happen I started paying attention to it when I was in an observer role and I saw it happen again and again. When I was on the hot seat, I wasn’t entirely successful in keeping that idea in mind, though.

I think that when the pressure is on there’s some part of the brain that reverts to some kind of automatic behavior. The challenge is to try to implement new behavior that is more effective. One of the tools that I learned about at #RSTOcas was the Heuristic Test Strategy Model. In particular, the Project Environment part. Running through the list of Project Environment elements before starting a new testing assignment is a habit I’m trying to build now.

#RSTORCAS notes from day 3

Here are my notes from the third and final day of the RST class I recently attended:

RST – Liberation, Power, Mobility
Aware of the possibilities
Sees the system
Realizes there’s no one way
Personal to each person
Invariants – skill, motivation, heuristics
Work on  strengths, ignore/outsource weaknesses
Use weaknesses to convince people to help you
Take a personal inventory
Poke at weaknesses a bit – you may not have explored/developed enough
Be OK with the fact that it may not be for you
If you’re a tuna, don’t fight on land
Assert (gently) your natural rhythm
You have power
Impedance mismatches block energy
Building credibility
Find what your good at
Make sure people know it
Working with developers
Try this: give them respect
Exploratory Testing
An approach, not a technique
Applicable to any technique
Professional ad hoc testing (look up the real definition of “ad hoc”)
Not sloppy – done for a particular purpose
The explicit part of exploratory testing is simple; the tacit part is massive
All good testing is both exploratory and scripted
They are opposites but not mutually exclusive
Sometimes it’s good and important
Exploratory comes first
Learning how to self-manage also comes first
Formal scripted testing is an advanced topic not in this class
Moolya in India
No test scripts – mind maps
Invest in mentoring, training
Won’t do fake testing
All testing is structured unless it’s random
Self-management
Have some disposable time
Novice testers less – expert testers more
Take notes
Manage your mental engagement
Each project slightly different process
Rapid testers are in charge of their process
General defocusing heuristic
If you are frustrated, defocus
If you are confused, focus
Don’t have an inappropriate attachment to test data
Test documentation could be lists, mindmaps, grids on a spreadsheet
Should be concise
Documentation is like having children – think about maintenance
Plunge in and quit heuristic
If you don’t know how to do something, try it before giving up
If you want to find unanticipated bugs you need distractible people
Long leash heuristic
Give yourself a fixed amount of time to be distracted
Use an alarm clock to remind you to come back
Practice this for one day – a muscle you can develop
Development skills can be important

#RSTORCAS notes from day 2

The RST course is over and I’m going through my notes. Here are my notes from day 2 (which was Wednesday) This post is a little less pithy than my notes from day 1. Another aspect of the course not covered in my notes are the testing activities and post-activity debriefs. These are the real meat of the course but it’s hard to capture by writing about them and I suspect that everyone gets something different out of them.

My notes:

Learn the things that you have to test – maybe don’t just jump in and do stuff with the product as your first move if you have time. Some ideas:

  • Look at competing products
  • Look at past versions of the same product
  • Look at the spec if you have one

If time is tight, you may have to jump right in

  • It will feel more disorganized
  • It’s like an eating contest – messy

Eventually both approaches end up in the same place

Coverage
Frequently misunderstood to only mean code coverage.
There are other kinds: product coverage, risk coverage, requirements coverage, etc.
The mnemonic SFDIPOT helps us remember aspects of the product to consider (“The big 7″)

  • Structure (developers most concerned)
  • Function (novice testers most concerned)
  • Data (experienced testers will consider)
  • Interfaces (system, user, cloud)
  • Platform (dependencies outside the scope of your project; things you can’t fix yourself)
  • Operations (usage patterns
  • Time elements (similar to data, experienced people will consider)

Subtitle heuristic: “No user would do that” == No user I can think of, who I like, would do that on purpose

Other product aspects:
Testability (about controllability and observability; helps developers too when they have to debug)
Availability
Simplicity
Stability

Heuristic Test Strategy Model
Developing the model had a huge effect on James’ career
Came from thinking about Guide words in the safety engineering field (HazOps)
Helps you have an agenda in your head when you start a new testing project
At a high level, you:

  • Examine the product
  • Against Quality Criteria
  • Within a project environment

You chase risk – where is the fire likely to be burning?

Product relates to coverage
Quality criteria relates to oracles
Project environment relates to procedures
First two are concepts, last one relates to the real world

4 pillars:

  • Diversification
  • Cost vs. Value
  • Skill (like muscle)
  • Heuristics (like bones)

Skill without heuristics lacks reliability
Heuristics are mooring points for tacit knowledge

Rapid testing is about solving problems while minimizing waste
Not “best practices” thinking – about problems and solutions
#1 issue: finding the problem
No one style; You can have your own style
A tester needs to sweat the details

Sometimes we pick oracles that are too probably (Prior probability heuristic)
Disconnect from the automatic response
Debugging your mental model – one way is to restate things
Skepticism: refusal of certainty (not refusal of belief)
Coherence-based reasoning (pattern matching), the basis of stereotypes

Two types of thinking – System 1 and System 2 (Thinking, Fast and Slow; Kahneman)
System 1: reflexive, never asks for more data, always ready to go
System 2: reflective, rational, takes time and energy
We want both systems working and cooperating

Scientific skills
Feeling of ignorance: you need it
Makes room for learning
3 questions to ask:

  • Huh? (you don’t understand)
  • Really? (what you understand may not be true)
  • So? (it may not matter or may matter less than you think)

Requirements statements
Make sure you know what every word means – take it apart
No one person needs to supply all the ideas (pairing is good, but may be exhausting)
Make use of visualization
Realize there are tacit requirements too

Inattentional blindness
Sometimes you have to see tests run more than one time to understand
Video can help
Know the limitations of our observational skills

Oracles are how you know there is a problem
Expected results are happening all the time (at every test step)
Machines can’t observer anything, can’t interrupt a test

Assumption: you treat something as a fact when you have no evidence that it is a fact
Assumptions can be dangerous
How to assess: How likely is it that the assumption is wrong? What is the impact if it is?
Facts rest on a bed of assumptions
Testers are allowed to make reasonable assumptions
It’s hard to make a list of all our expectations
We are always in the process of making sense of the world
It is a mental process not a list
We do memorize lists]

It’s not possible to write down a test
It is possible to write down things that are useful
Humans are always looking at more than you can write down
Testing depends on tacit knowledge and shared models
Also, conversations about them. Also socialization

Rapid testing is rapid learning
Don’t judge things by the first 15 minutes
Don’t be intimidated
Review by rewrite, review by testing, review by enactment
Description vs. Actual vs. Intent
Notice unintended consequences