As both a consultant specializing in software quality and a maker of test automation tooling, I speak to dozens of software teams every month about their challenges in delivering consistently high-quality software at an ever-increasing pace.
More and more teams recognize the need for an effective full-cycle test strategy and have selected great tools and techniques, which is awesome! Even so, a few seemingly small misconceptions about how to apply these tools and techniques often prevents teams from gaining the benefits from them, and that’s not so awesome!
It’s kind of like buying a great new jacket, only to then try and wear it back-to-front. It’s not only awkward, but you also can’t see where you are going.
As silly as this analogy may seem, I often see software teams miss out on the potential benefits of quality tools and practices by applying them back-to-front.
When test automation is back-to-front
To understand the front and the back of testing, you have to take a moment to consider what it means to be the right way round. And that means starting with considering what the role of automated testing really is.
The role of automated testing is to ensure that your intended value is being delivered at the intended quality.
Let’s take a pause to think that through before we dive into more detail…
What are Value and Quality?
I’ve written in detail about these concepts before, and as they are extremely difficult to define, I am always evolving my thinking and definitions, which are currently as follows:
- Value is the reason you are building your software in the first place. It is typically expressed as a problem it solves for your customers or users. These customers or users are the consumers of a solution which ultimately benefits them. And because these benefits are subjective, it means value is based on the perception of the consumer. Therefore I define:
Value is a benefit, as perceived by the consumer.
- All value starts as “Potential Value”, meaning that it has no benefit unless a person actually receives it and derives a benefit from it. To achieve the potential of the value, you apply quality functions as you design and build the value. The more effort towards quality that you apply, the closer you get to achieving the full potential of the value. And since quality is extremely subjective, it is also based on the perception of the consumer. Therefore I define:
Quality is the extent of the potential value realized, as perceived by the consumer.
- The mission of the team to deliver the potential value requires constant readjustments that are continually applied throughout the product development life cycle. These must not be left until the end or owned by any single person or department.
That last point is the most critical. Only if quality functions are applied from the very beginning, throughout the life cycle and by everyone, can they extend the potential of your realized value.
To only apply quality control at the end of the process is backwards, and here is why:
As many of you may know, the cost of a bug is exponentially greater the later you discover it, and it also takes longer to fix it. Therefore only applying quality functions at the end is to apply them in the most expensive way.
But what you might not know is that more than half of all software bugs are actually created before any software is written at all — due to missing or incorrect specifications.
In other words, most teams skip focusing on quality at the point that it is most important and most cost-effective, and they instead concentrate efforts when it is too late and exponentially more expensive.
To some readers, this might sound strange, but to some others, it might sound strangely familiar!
Don’t let the four phases on the diagram fool you into thinking this is only a problem in waterfall delivery models. The same is true for any iterative approach. Whether you’re writing a story or a project plan, applying quality at the end is back-to-front!
Here are some signs that your team might be putting your quality on back-to-front:
- You have a QA role or department that “owns” quality
- You have little or no test automation and rely on manual/human regression testing (including Rainforest QA or an army of testers)
- You hired a test automation engineer, but they are the only ones writing any automated tests that run after developers commit code
- Your engineers don’t do their own automated testing and rely on QA to find bugs
- Your overall automated test coverage is low or is comprised entirely of unit tests
While all of these are much much better than having no quality measures at all, it is like wearing your raincoat back-to-front. It does still provide protection from rain, but it quickly becomes cumbersome and stops you from being able to continually adjust your path to get to where you’re going.
How to apply quality the right way round
The process of actualizing the value of your product is a constant process of adaption and readjustment. Think of it like doing the Tango- gracefully dancing with your partner, staying closely in step with one other as you glide across the dance floor.
I call these steps Quality Functions, and I’ve compared them to the actions of care you apply to your product. Now, in response to popular demand, I have produced educational content that explains each of these, and will shortly be available in my forthcoming book, “Quality, Faster.”
Check out these synopses that explain how you can be sure to apply quality from the front:
- Ensure quality is everyone’s responsibility
Given that the most critical point to apply quality is early in the process when the design and business teams are involved, the entire team must buy in to their responsibility to quality. This is not always easy to achieve, especially when there has been a legacy of quality being left to the QA department. So, in this section I share practical advice for how to ensure full team ownership. - Ensure quality starts early in the definition phase
There are only three types of bugs, and two of the three types are caused by poor specification. If I could recommend you take only one action to increase the quality of your software, it would be to unite your Business, Product, UX, Development and QA teams around creating a shared communication of the expected value and quality. Whether it be delivered as a whiteboard session or collaborative executable specification sessions, the communication between parties is in itself the most valuable quality function of all. - Bake value into your specifications
Given that the role of automated testing is to ensure that your intended value is being delivered as specified, a clear definition of the value must be present within your specifications. This means communicating not just the solution, but also the consumer and the benefit they receive. The following story narrative is great at doing this and takes on this structure: “In order to <benefit>, As a <consumer>, I want <a solution that provides the benefit>”. And by adopting a “Given, When, Then” scenario description instead of just bullet-point acceptance criteria that tie right back to the story narrative, you focus the whole team on ensuring real value is always delivered, while also freeing your developers to recommend the best solution to achieve this. - Generate constant feedback from the core
Many development teams have already experienced the benefit of real-time feedback from unit tests while they create code. What is less well-known is how to receive real-time feedback on core value and end-to-end value tests as well. When you do this, it ensures that your product never deviates from delivering the intended core value, and your team is always able to course correct early and cheaply. - Free your QA team to play to their strengths
Test automation is great for efficiently testing what is expected to happen and not happen. But what about the bugs that occur because nobody could foresee certain scenarios? Your QA engineers have a special superpower for anticipating the unexpected, and they should be enabled to spend the majority of their time doing so. If you successfully use test automation to automate the expected, your QA engineers can save you from the unexpected.
To summarize
- At this time in 2016, there is a plethora of tools, techniques and processes that can help deliver higher quality software faster than ever before. But to do so, we need to accept that delivering high quality value requires a constant ongoing tango of quality functions applied from the very outset of the idea, through to the end delivery of the value to the consumer.
- The most important and cost-effective place to start applying quality measures is from the very beginning and then throughout the process. To only apply quality at the end of the process is to be completely back-to-front.
Thanks, and I look forward to helping you deliver higher quality software, faster.
Sam
Let me know if you have any questions or thoughts in the comments below.
