User Research and Validating Hypothesis – Top Takeaways From The Lean Startup Conference 2013

Which Hypotheses to Test?
The Lean Startup practitioners list all the assumptions in their business model. There was a debate in the conference about which assumptions to validate first. One set of speakers felt the “Leap of faith” assumption should be validated first. But not all product ideas are as ground-breaking as “Sony Walkman” which had a “Leap of faith” assumption that people might enjoy music while walking. Alline Watkins advises startups to rely on intuition and pick any hypothesis without belaboring the choice of the riskiest hypothesis. Sean Murphy said that risks have different probabilities and are connected in a way that they need to be validated in a particular sequence. Startups have technical risks and market risks. We should rank them in the “Degree of Surprise” and validate the one whose outcome is likely to be most surprising.

Qualitative Research
Hypotheses are validated by interviewing users. There are three stages of the lean startup.
Stage 1 – Problem/Solution fit. In this stage we validate whether users really are hurting from the problem we are trying to solve and whether they perceive the proposed solution to really solve the problem. This research is qualitative and characterized by “High Touch” interviews with a handful of target users. We should avoid asking direct and leading questions. The idea here is to build empathy, seek insights and see patterns. Talk to them in terms of what they understand. E.g. the power of first cars before which horses were used for transportation was called Horsepower and applying similar logic power of electric bulbs was measured in Candle Power.
You should also identify potential competitors by checking how the problem is being solved currently. B2B interviews – start at lowest level in the hierarchy and navigate by asking who else is involved. Ask what s/he likes or dislikes about the current workflow.
Quantitative Research
Stage 2- Product/Market fit. In this stage we do more metrics based quantitative research by exposing a minimum viable product to determine a set of features that delivers value to a set of customers. Early adopters are those enthusiastic customers who act as champions or change agents – who often have a proven track record of convincing others in a company. Early adopters consist of technologists who like to play and visionaries who like to expand their business.
Stage 3- Growth Hypothesis Validation. There are 3 types of growth. 1-Paid growth which happens by getting new users via advertising. 2- Sticky Growth – Which happens by having old users return and existing users forming a habit of using your product. 3- Viral growth – This happens by users getting more users through word of mouth publicity. Validation is done by studying conversion metrics.
Modeling an experiment needs more attention than conducting it. David Bland presented experiment mapping– which is a technique to get the team involved in modeling an experiment right from the beginning. The team writes down on a board Assumptions- Hypotheses- Experiments- Outcomes leading to new assumptions and starting the next cycle. This improves the team’s common understanding.
We should definitely test how much the customer ready to pay. Generally its 10 to 40 percent of hard savings depending on perception of risk of the saving not materializing. Never ask customers what they are willing to pay. We should use anchors. The price shouldn’t be easily acceptable- you must face some resistance. We must charge the negotiated price and get paid.
Concierge Tests
You don’t have to completely build your product to test the product market fit. One low risk way is running concierge tests. Concierge test can be best explained by examples. When Instagram wanted to test the feature that allows users to apply special effects and filters; instead of building out the feature Instagram employed artists who would manually produce the pictures and email those to the beta customers. They could get valuable information on user preferences that helped them build the solution.
Similarly before Four Square built their product they could run a concierge test. They could give gift cards to popular places like Starbucks. They would motivate beta customers to text the venue of their visit by putting $1 in the gift card every time they do it. Most active patrons to a particular venue would get reward dollars.
Converting a successful concierge test to a product needs to be done thoughtfully. There are many aspects of a concierge service; its important for us to discover which part of the service is really valuable to the customer. You can start gradually automating the repetitive chores thus creating some product features. Finally you will probably automate all manual tasks. Moreover you might decide to continue to provide the concierge service as a premium add-on. As you automate you will start collecting metadata and analytic. You might consider providing the information you collect as a byproduct.
Tools for The Lean Startup
Landing pages that advertise the product and have a call to action serve as great Minimum Viable Products (MVP). Appery.io, Launchrock and WordPress are great tools to build MVP. IFTTT (If This Then That) is a useful tool for prototyping mobile apps. There are many products being built on platforms like appAnnie or Github . You should look those up for competitive apps. Kickstarter is a good platform to get your project crowdfunded. Leanstack can be used to build your business model in the form of one page “Lean Canvas”.