Key Success Factor 1: Ambitious and clearly defined validation outcome
Often, prototypes are built and tested without a clear view on what the validation really should prove. Adding to this validation teams stay within their comfort zone and do not reach far enough. A plethora of assumptions can be tested during validation but resources are limited. For example, you might want to prove that a great user experience can be developed, that customers continuously use the product, or that the product is technically feasible. Therefore, these priorities should be defined and clear metrics need to be established that allow for objective quantification of the results. The consequence of not doing this is a validation result that is not really addressing the right aspects in the right way or missing to evaluate some important questions at all. What we often see is that a prototype validation is too focused on one aspect (technology, user frontend, customer acceptance, …) but not the comprehensive entrepreneurial perspective. When done right validation goals are also set as stretch goals, questioning the status quo and really trying to make the hypotheses fall. A good way to start is to clearly agree (internally and with external stakeholders) on a set of validation hypotheses which can also become the inspiring north star for your validation project and even excite other stakeholders.
Ambitious and clearly defined validation outcome: How to make it work in practice
- Expert knowledge generation: Start validation by generating more knowledge on the validation object and its context by interviewing industry or target group experts. This will give you a deeper understanding what counts most for your validation project and what you should prioritize.
- Be bold with your validation focus/goals: Of course it is key to define realistic goals. However, as project experience shows setting ambitious goals improves your effectiveness even when not meeting them 100%. Further, stretch goals generate management attention, motivate the team and help to deeply scrutinize the whole business idea. Always set a goal, consider what is doable and than slightly adjust the goal to the upper side.
- Hypotheses as north star: Use the stated hypotheses as the project’s north star. It is extremely important to share them with your development team and designers to use the hypotheses
Key Success Factor 2: High-quality test user generation
As user tests have usually smaller sample sizes it is paramount to select test users that give a good representation of your target group. Wrong samples in quality and quantity might produce misleading results in the end confirming or rejecting a business idea by mistake. For example, prevent to test prototypes with a large group of ‘family and friends’ participants. Either they only participate to do you a favor and don’t like the business idea at all, or they tend to avoid giving negative feedback due to personal relationships – or even give too tough feedback because you asked them not to be too positive. Moreover, when target groups are hard to access, the process of acquiring test users is to be set up as efficiently as possible to save resources and lower bias.
High-quality test user generation: How to make it work in practice
- Early user acquisition: Treat user acquisition as a substantial task and start early. You can use expert resources to inform yourself about the target group. This helps you when communicating with the target group, setting up the test design in a user-oriented way and might generate further channels to reach out. Go and map all your available channels and key stakeholders for acquisition. Use the ones that allow for the best cost-quality ratio. Specialized agencies allow for high-quality samples.
- Target group selection with right incentives: Be careful how you motivate and pick your test users. Try to balance the trade-off between a large sample generated from your private network as well as incentives and a smaller sample of high-quality test users. Note that friends & family might be far off the target group or will have a bias in their ratings, so at least monitor these effects closely and ensure that they are intrinsically motivated. Financial incentives might be useful to acquire the right sample but be careful to not attract people that are only after the reward.
Key Success Factor 3: Tailored & prototype-aligned test design
Validation is primarily about running experiments to test hypotheses about your product, so test design is the centerpiece. A tailored test design that takes all essential variables into account will assure that the user feedback really helps to check the hypotheses and it will allow for flexibility when you need to adjust the setting. Many validation projects lack such a tailored test design for various reasons: Often design is not aligned to the validation hypotheses or there is a lack of understanding of the test users and their needs. We also regularly see that the test design is not aligned with the development of the prototype itself – foster exchange between developers & test planning to make sure that the prototype has the tech stack and features to test well within the validation setup and vice versa! When it comes to preparation of the tests, often team members which run the tests are not familiar with the test design – or the test design is lacking a plan B if something does not work as planned.
Tailored & prototype-aligned test design: How to make it work in practice
Alignment with hypotheses: Ensure testing design closely reflects the hypotheses. Mapping the hypotheses with your testing design will help you not to miss an aspect. A deliberate mix of qualitative and quantitative data is often helpful to cover different aspects and triangulate findings. If feasible, try to use the test design to also frontload and generate data on hypotheses that need to be answered later (e.g., pricing).
Inclusion of designers: Emphasize deeply with your team and align your testing approach with your designers. Ensuring this, you achieve that the test itself does not affect the users experience negatively. Personas can be a useful tool to create a test design with your user in mind.
Thorough internal test preparation: Run dedicated briefing sessions with the involved team members. Make sure that everything is in place by overcommunicating towards testing team and test users alike to prevent any negative experience from the testing procedure itself. Try also to secure key resources for testing as early as possible.
Proactive risk management: Plan for different scenarios and have fallback solutions in place. For this purpose, conduct a decent risk analysis when planning the test design.