A company’s goal is to generate profit; for a software company, profit always relates to additional product value to users. But, as startups expand, the generation of product value becomes more complex, often because product and engineering evolve into distinct departments, each chasing their unique goals. The product department typically prioritises roadmap completion, while engineering focuses on the velocity and quality of feature development.
Yet, especially when a startup fails to grow as fast as expected, it’s difficult for a CEO to assess the actual performance of their product engineering “machine” and its ability to provide high value to users at high velocity and quality. While the literature doesn’t lack performance metrics, they’re too often focused on either product or engineering, not the product engineering duo and how it generates user-perceived value.
Without a unified measure, it’s almost impossible to gauge the true impact of the product engineering synergy on delivering expected value. After many years of working on performance improvement with CTOs and CPOs, I’ve developed a metric of my own, the Time to Positive User Feedback (TPUF), a potential silver metric to bridge the product engineering gap and align the company-wide vision towards tangible user value.
The TPUF Metric Explained
Time to Positive User Feedback (TPUF) is a metric rooted in systems thinking—a theory popularised by Donella Meadows that encourages viewing a product or organisation as a cohesive conglomeration of interrelated components rather than as isolated parts. In the context of a software startup, systems thinking involves understanding how various elements of the product engineering system interact to transform an initial idea to solve a user problem into a fully functional feature that users actively engage with.
I divided the product engineering system into three critical sub-systems, each with a distinct function within the TPUF framework.
The first sub-system, Product Management 1, is the idea generation machine, where product managers formulate potential solutions and refine those with merit into detailed specifications for software engineering. The Software Engineering sub-system then takes the baton, translating these specifications into high-quality, production-ready features. The handoff then reaches the Product Management 2 sub-system, where product managers and developers analyse user engagement to confirm if they use the feature as intended. Should the actual usage deviate from the expected, this sub-system initiates the iteration process.
By leveraging TPUF, technology leaders can apply systems thinking to pinpoint how effectively these sub-systems perform and interact. A low TPUF suggests that the transition from idea to user satisfaction is seamless and swift, indicating a healthy, responsive product engineering system. Conversely, a higher TPUF may reveal inefficiencies or misalignments for the leadership to address, underscoring the need for iterative refinements to align the final product with user needs and expectations. Ultimately, TPUF serves as a lens through which company leadership can evaluate the entire product journey, ensuring product and engineering teams achieve their end goal—user satisfaction—with systemic efficiency.
Product Management Sub-system 1
Product Management Sub-system 1 is the inception point within the product engineering lifecycle. It's where ideas are born and vetted—addressing presumed user issues through creative brainstorming.
The main challenge here is to avoid “idea flooding”. With ideas unrestrained by physical limitations, there is a high risk of accumulating a backlog of unrealised concepts, mainly as other sub-systems are busy making other ideas happen, leading to a perpetual cycle of incompletion. To mitigate this challenge, savvy product leaders should select only the best ideas relentlessly and reject over 90% of the rest. That’s why it’s essential to have an efficient and fair green light mechanism that rejects many ideas, independent of who initially shared them.
If given the green light, product managers craft ideas into detailed specifications, ideally in cooperation with software engineering. This joint effort ensures that abstract concepts are translated into actionable blueprints with a shared understanding, minimising the back-and-forth that can occur when developers encounter unanticipated feasibility issues or have questions. It's a critical phase where market analysis, user research, and strategic planning converge to define the functionality, scope, and objectives of a product feature, setting the stage for its development and eventual user engagement.
Software Engineering Sub-system
The Software Engineering Sub-system is the engine room, where developers transform product specifications into tangible features. This phase encompasses development, testing, and the final push to production. Engineers must precisely interpret and execute the product specifications, coding robust and reliable features.
Engineering leaders' main challenge is moving from an approved specification to high-quality code into production as fast as possible. Challenges in this sub-system are manifold, including technical debt accumulation, which can bog down future development with maintenance overheads, and integration issues, where new code conflicts with the existing codebase.
Continuous integration and deployment practices are crucial to navigating these hurdles. The DORA (DevOps Research and Assessment) metrics are pivotal in enhancing the performance of the Software Engineering Sub-system. These metrics—Deployment Frequency, Lead Time for Changes, Mean Time to Recovery, and Change Failure Rate—offer a quantifiable means to assess the effectiveness and efficiency of software development and deployment processes.
By frequently merging code changes into a central repository, testing becomes automated and consistent, reducing integration issues. Prioritising technical debt repayment is vital, allotting time to refactor and improve the codebase alongside new feature development. Moreover, fostering a culture of code reviews can enhance code quality and knowledge sharing, preventing silos that can lead to delays and bugs.
Ultimately, Software Engineering measures its success by efficiently delivering high-quality features into production, setting the stage for user feedback and the continuous improvement cycle.
Product Management Sub-system 2
Product Management Sub-system 2 serves as the evaluative and iterative arm of the product engineering process, stepping in post-deployment. Its fundamental role is to scrutinise user interaction with newly released features, determining whether users use these additions as intended and if they genuinely solve their problems.
The iterative nature of this sub-system is critical; even if software engineering effectively deploys features, the job isn't done until those features achieve their intended usage. This requires diligent user feedback and usage data analysis to verify efficacy and value. The main challenge encountered here is the 'usage gap,' where there's a disparity between feature availability and user adoption or satisfaction.
To mitigate such challenges, product and engineering leaders must foster a culture of continuous feedback and readiness to iterate. This involves building robust monitoring systems that track user behaviour and feedback loops that quickly capture user sentiments. Furthermore, cross-functional teams should convene regularly to review this data and be ready to pivot or iterate on features as necessary.
The system’s success lies in its agility and willingness to embrace change. This may mean returning to the drawing board to refine or overhaul features that do not meet user expectations. By closing the loop between feature release and user satisfaction, this sub-system ensures that the company’s outputs are not just shipped but also sharpened and shaped by real-world use and feedback.
Conclusion
In conclusion, Time to Positive User Feedback (TPUF) is a compelling metric for startups seeking to synchronise their product and engineering efforts with market needs. But Goodhart's law warns that “when a measure becomes a target, it ceases to be a good measure”. If leaders start pursuing TPUF as an end goal, teams may cut corners, leading to subpar features and user experiences.
The pursuit of a low TPUF should not eclipse the need for a sustainable work pace, nor should it stifle the innovation that arises from allowing a team the space to explore and iterate. TPUF can guide startups judiciously towards greater efficiency and user satisfaction. Still, it should always be balanced with the human elements of creativity and resilience that drive long-term growth and innovation.