Jane is the Head of Engineering at Mirata, a 300-people FinTech startup that just raised a 40 million dollar series C. Following the fundraising, Mirata’s CTO has asked Jane to come up with a hiring plan for the next 12 months.
While Jane has a clear vision of how many developers are needed to ship the roadmap, she is undecided about how many headcount would be required in the QA team. Her QA manager, who is super invested, would like to increase his team’s headcount to match the development team’s growth.
On the one hand, the QA manager is in his own right, especially since his team is already successful at discovering a significant amount of bugs in each release. On the other hand, the product objectives are so ambitious that Jane’s budget is barely enough to cover all the developers needed in the delivery teams.
What should Jane do?
Jane, like a lot of engineering leaders, considers Quality Assurance as a function instead of an activity. In most technology organisations, QA is usually the last team (or human beings) involved before releasing features in production. They write tests, try out new features and verify that everything that used to work, still does.
There are three problems with considering QA as a function. The first one is that QA can quickly become a bottleneck if the team doesn’t grow as fast as the development throughput like advised by Jane’s QA manager. The second problem is that it shifts responsibilities to the right. Since there’s a QA team, product owners and developers know that someone has “their back”, and are not necessarily as cautious as they should when building new features. The third and last problem is a hammer and nail one. Since the QA team’s existence and status are related to finding bugs, they will find as many as they can, sometimes uncovering problems that could have very well been found later on.
In the agile business novel Rolling Rocks Downhill, the main character, Steve, finds a parallel between the company’s cafeteria and his team’s development process. During a visit to the cafeteria, Steve discovers that the staff initially “cooked food from scratch, following supposedly 'foolproof' recipes and menus provided by the Head Office. They followed the recipes to the letter, but no one tested them to see how they tasted!”
To improve, they first started by tasting the meals before being served. Sometimes they could fix things with a little salt or some pepper. Sometimes it was a little trickier, and they had to throw away the food. After a change in process, the cooks “moved from late inspections (testing or tasting at the end of the process when it's very expensive to fix stuff) to in-process inspections, where you build quality into your product during the process."
If you apply the same logic to a development process, QA should be built in the delivery process, effectively shifting responsibilities to the left. So instead of increasing the QA’s team with the same ratio as the development teams, Jane should instead shift their role from gatekeepers to enablers. This is what N26 started doing recently.
One of my primary responsibilities as Software Quality Engineer is to enable team members to test. This can be done by various means and on different levels, starting from basic information like how to generate test data, where to find test devices or test cases, to broader topics like the release process, different types of testing and what to automate. We try to find the best way to convey information on a given topic, in the form of concise documentation, presentations, workshops or 1-on-1 pairing.
Martyna Wojna - Senior Software Quality Engineer @N26
Basecamp, the company behind the eponymous project management software, also applies the same principle. At their current size (millions of users and about a dozen people on the product team), they have one QA person who comes in toward the end of the cycle and hunts for edge cases outside the core functionality. They consider “QA as a level-up, not a gate or a check-point”.
With this mindset, QA engineers are more architects than makers, designing processes, implementing tools and automating tasks to allow others to test at various levels. So instead of increasing the QA team, maybe Jane could add one or two senior engineers and work with the manager on changing how quality assurance is implemented throughout the development process. A good ratio here would be 1 QA engineer for 10 developers.