Review

Lean Analytics is an incredibly insightful, data-driven approach specifically designed to rapidly assist startups in building superior products. Expertly co-authored by Benjamin Yoskovitz and Alistair Croll, this book serves as an invaluable guide to understanding the most effective way to measure progress. It empowers businesses to ask the most impactful questions and discern what vital information can be gleaned from the data to make swift and informed decisions in our ever-changing market. This book is absolutely essential for entrepreneurs, data analysts, and product managers who are determined to leverage data to drive their product strategy.

Personally, I believe this book could have easily been divided into two compelling volumes. The book ambitiously delves into six distinct business models and five different stages, spanning over 400 pages and making it quite an extensive read. I would suggest that it is best read when you are situated within a company where you can directly apply the knowledge and simply bypass what's not relevant.

You Might Also Like:

Lean UX

Even though it seems positioned more toward designers, this is an incredibly important book for product managers. It does a great job of integrating the concepts from Lean Startup with existing methodologies (Design Thinking, Agile, UX design). It’s packed full of principles and practical implementation advice for teams. There’s little in this book I disagree with.

andrewclark.co.uk

The Right It

Marty Cagan acknowledges in "Inspired" that most of our products and features won't meet expectations, and the ones that do require a lot of iteration to reach that point. I believe this is THE most important observation about product development. This book does a great job of articulating that phenomenon and proposes a robust way to navigate it. This is the best book on validating product ideas that I’ve read. I will forever be sharing this book.

andrewclark.co.uk

The Lean Startup

This is a hugely influential book in the product management community. The Lean Startup codified how to take a scientific approach to create and manage startups. The author also explains why traditional firms have such trouble navigating uncertainty. The Lean Startup emphasises the importance of learning, and validating assumptions. This book popularised a number of concepts that are now considered foundational in product management (e.g. build-measure-learn, pivot, minimum-viable-product). It’s a must-read for anyone building digital products.

andrewclark.co.uk

Key Takeaways

The 20% that gave me 80% of the value.

• The Lean Startup focuses on quickly learning about your greatest risks in pursuit of a scalable and repeatable business model. Lean Analytics aids in tracking progress and deciding what to concentrate on next, as part of the build-measure-learn loop.
• Entrepreneurs have to be relentlessly optimistic. They create a reality distortion field of positivity to push through the odds stacked against them. Analytics is the necessary counterweight of realism.
• Analytics can help you quickly find your way to the right product and the right market. Good metrics should inform our next move and change our behaviour. Ratios and rates are great as they’re comparative and easier to act on.
• The Lean Analytics Cycle follows these steps:
1. Choose a crucial metric to improve.
2. Set a realistic target, using industry benchmarks as a guide.
3. Determine how to enhance the metric.
1. Find out what good users do, make it easier
2. Brainstorm ideas, test them, implement the best
4. Measure the impact of the change on the metric.
1. If the change fails, pivot, try a different approach, or conduct more discovery.
2. If the change succeeds, select a new metric to improve.

• Avoid these common data analysis pitfalls:
• Assuming clean data: Always clean your data to reveal patterns and prevent errors.
• Not normalising data: Compare data within context to avoid misleading results.
• Excluding/including outliers: Outliers can skew results. Include for insights, exclude for general models.
• Ignoring seasonality: Consider seasonal trends to accurately interpret data.
• Ignoring size in growth reports: A high percentage increase from a small user base may not be significant.
• Data vomit: Avoid overwhelming with too much data without actionable insights.
• Metrics that cry wolf: Be cautious with overly sensitive metrics that trigger false alarms.
• The ‘Not Collected Here’ syndrome: Don't dismiss external data that could offer valuable insights.
• Focusing on noise: Focus on the significant trends rather than getting distracted by minor fluctuations.
• The Lean Startup is the process you use to move toward and achieve your vision.Early-stage founders aren’t building a product, they’re building a tool to learn what product to build.

• The Lean Analytics stages suggest an order to the metrics you should focus on. The stages closely mirror what other Lean Startup advocates advise:
1. Empathy - Gate: I've found a real, poorly met need in a reachable market
• Focus on understanding the customer's needs and problems.
• You need to validate that there is a real need for your product or service in a market that you can reach.
2. Stickiness - Gate: I've figured out how to solve the problem in a way that they will accept and pay for
• Focus on developing a product or service that solves the customer's problem in a way that they are willing to pay for.
• You need to validate that your product or service is sticky, meaning that customers will continue to use it.
3. Virality - Gate: The users and features fuel growth organically and artificially
• Focus on making your product or service viral, meaning that it grows organically through word-of-mouth marketing.
• You need to validate that your product or service has the potential to grow virally.
4. Revenue - Gate: I've found a sustainable, scalable business with the right margins in a healthy ecosystem
• Focus on developing a sustainable and scalable business model.
• You need to validate that you can generate enough revenue to cover your costs and make a profit.
5. Scale - Gate: I can achieve a successful exit for the right terms
• Focus on scaling your business to reach a larger market.
• You need to validate that your business can be scaled to achieve your desired outcome.

• Choose the One Metric That Matters (OMTM). The one that’s crucial given your current stage. Success lies in focus and discipline. Optimising your OMTM squeezes that metric so you get the most out of it, but it also reveals the next place you need to focus your efforts, which often happens at an inflection point.
• Draw lines in the sand. Establish a clear target number for success metrics, and be ready to reassess if it's not met. Success can be defined by what your business model needs or by comparing to industry norms when the business model is not yet clear.

• To decide which metrics you should track, you need to be able to describe your business model simply and just think about the really big components.
• Business growth comes from improving one of these five knobs: selling more stuff, to more people, more often, for more money, more efficiently.
• Not all customers are beneficial. While some are valuable, others can be distracting, resource-consuming or harmful. It's essential to distinguish valuable users from detrimental ones, then implement changes to maximise the real users and minimise the harmful ones.
• A business model is a combination of things:
• The acquisition channel is how people find out about you.
• Selling tactics persuade visitors to become users or customers, often through scarcity, exclusivity, or additional benefits.
• Revenue sources, direct or indirect, may include transactions, subscriptions, ad revenue, data resale, donations, among others.
• The product type is what value your business offers in return for the revenue.
• The delivery model is how you get your product to the customer.

• The Problem-Solution Canvas helps you maintain discipline and focus on a weekly basis. What’s the problem, how do you propose to fix it, and how will you know if you succeeded? Use it to home in on the key problems you’re facing. Agree on and prioritise your problems.
• Current status: key metrics you’re tracking compared to previous period
• Last week’s lessons learned: What did you learn, what was accomplished?
• Top problems: List and prioritise the top problems
• For each problem:
• Hypothesised solutions: list the possible solutions. Rank them. Explain how you think they’ll solve the problem
• Metrics / Proof / Goals: List the metrics you’ll use to measure the solutions, list any qualitative proof you’ll need. Define the goals for the metric.

• There is a normal or ideal for most metrics, and that normal will change significantly as a particular business model goes from being novel to being mainstream.
• How you feel about a metric changes when you know what’s normal or best in class for your type of business. “When I first saw our churn, which was around 2%, I was very concerned, but when I found out that 2% is pretty much the lowest churn you’ll get in the hosting business, it changed my perspective a great deal.”
• Before investing time and effort into a metric, understand your position against competitors and industry averages. Benchmarks guide whether to continue working on a metric or tackle the next challenge.
• That said, realise there’s a reason most startups fail: average is nowhere near good enough.
• Strive to reach established target metrics, not adjust targets to current performance levels.
• Optimisation efforts usually have diminishing returns, indicating when it's time to shift focus to a different metric.
• Achieving local maxima and seeing diminishing returns on improvements can serve as a baseline and indicate when to shift focus to other areas.

A selection of insights:

• The MVP is a process, not a product. An MVP isn’t a product it’s a tool for figuring out what product to build. As you iterate, your goal is to improve on the core metrics that you’re tracking. If a new feature doesn’t significantly improve the One Metric That Matters, remove it.
• Correlation is nice, but spotting a leading indicator that causes a change later on is a superpower. You can now change the future. You just have to make that leading indicator happen more often.
• Identify common traits among users hooked on your product, focus on their needs, and expand. Claiming your beachhead will enable faster iteration on a highly engaged market segment.
• Treat your paying users as a separate customer base and track their behavior, churn, and revenue separately from your nonpaying ones
• Knowing when users churn gives you an indication of why they’re churning and what you can do about it.
• In a media site business model you get paid based on who your visitors are rather than what your site contains.
• UGC sites have a power curve of content creation, where a small number of people create the vast majority of content.
• If you can’t find 15 people to talk to, imagine how hard it’s going to be to sell to them.
• Sketch your business model and key metrics:

Deep Summary

Longer form notes, typically condensed, reworded and de-duplicated.

Preface:

• Lean Analytics builds upon the Lean Startup methodology. The Lean Startup focuses on quickly learning about your greatest risks in pursuit of a scalable and repeatable business model. Lean Analytics aids in tracking progress and deciding what to concentrate on next, as part of the build-measure-learn loop.

Part 1: Stop Lying to Yourself

• Entrepreneurs have to be relentlessly optimistic. Create a reality distortion field of positivity to push through the odds stacked against them. Analytics is the necessary counterweight.
• Analytics can help you quickly find your way to the right product and the right market.
• Good metrics should inform our next move and change our behaviour. Ratios and rates are great as they’re comparative and easier to act on.
• Metrics often come in pairs. The viral coefficient (# of people a user successfully invites to your service) and viral cycle time (how long it takes them to invite others) drive your adoption rate.
• Things to keep in mind when choosing metrics:
• Qualitative metrics are anecdotal, quantitative metrics provide hard numbers.
• Vanity metrics only feel good, actionable metrics guide behaviour.
• Exploratory metrics find insights, reporting metrics track normality.
• Leading metrics predict future, lagging metrics explain past.
• Correlated metrics change together, causal metrics influence each other.
• Quantitative data answers “what” and “how much,” qualitative data answers “why.”
• Leading metrics predict future trends while lagging metrics like churn only identify problems after the fact. To get leading metrics you need to be able to do cohort analysis and compare groups of customers over periods of time.
• Correlated metrics predict outcomes. Identifying causalities can help you change them. Causal relationships are rarely simple and often involve multiple independent metrics, each accounting for some of the dependent metric's behaviour.
• You need to engage with your customer directly to get the full picture. Metrics can’t explain why something is happening.
• Segmentation: splitting data into groups that share a common characteristic or behaviour.
• Cohort analysis: comparing similar groups over time. As your product changes, users that join later on will have a different experience. Cohort analysis allows you to see patterns clearly agains the lifecycle of the customer. Cohort analysis can be done for any metric you care about. Essentially it answers, how is the latest version of the product performing vs previous versions of the product. Known as ‘longitudinal’ studies.
• A/B and Multivariate Testing involve providing different groups with different experiences simultaneously. Multivariate testing enables you to conduct multiple tests concurrently. As a type of cross-sectional study, A/B testing requires a significant amount of data or users to yield meaningful results.
• The Lean Analytics Cycle
• Choose a metric that’s vital to improve
• Set a sensible target (industry benchmark)
• Figure out how to improve the metric
• Find out what good users do, make it easier
• Brainstorm ideas, test them, implement the best
• Measure the effect of the change on the metric
• If the change fails → pivot, try something else, or do more discovery
• If the change succeeds → choose a new metric

• Lean analytics can help make sure you don’t waste your life building something nobody wants.
• Bud Caddell: Spend your time on something you’re good at, that you want to do, and that you can make money doing.

Markets that don’t exist don’t care how smart you are. Mark Andressen

• The Lean Canvas is a great way of consistently articulating your hypotheses and verifying it with real customers.
• Problem: Identify real problems that people know they have.
• Customer Segments: Define your target market and the messaging that will help you reach them.
• Unique Value Proposition: What is the clear, distinctive, and memorable way to explain how you're better or different?
• Solution: Can you solve the problems effectively?
• Channels: Determine how you will deliver your product/service to customers and how you will receive their payments.
• Revenue Streams: Are they one-time or recurring? Is it a direct transaction or something indirect?
• Cost Structure: Identify the direct, variable, and indirect costs.
• Metrics: Do you know which numbers to track to understand if you're making progress?
• Unfair Advantage: What is the "force multiplier" that gives your efforts a greater impact than your competitors'?
• To survive you have to find the intersection of demand (for your product), ability (for you to make it), and desire (for you to care about it).

• Optimising part of your business using data without considering the big picture could be fatal.
• Humans are skilled in creativity and discovering potential, while machines optimize and validate ideas. Machine-only optimisation may miss larger opportunities while focusing on local maxima. Quantitative data is great for testing hypotheses, but lousy for generating new ones unless combined with human introspection.

Avoid these common data analysis pitfalls:

• Assuming clean data: Always clean your data to reveal patterns and prevent errors.
• Not normalising data: Compare data within context to avoid misleading results.
• Excluding/including outliers: Outliers can skew results. Include for insights, exclude for general models.
• Ignoring seasonality: Consider seasonal trends to accurately interpret data.
• Ignoring size in growth reports: A high percentage increase from a small user base may not be significant.
• Data vomit: Avoid overwhelming with too much data without actionable insights.
• Metrics that cry wolf: Be cautious with overly sensitive metrics that trigger false alarms.
• The ‘Not Collected Here’ syndrome: Don't dismiss external data that could offer valuable insights.
• Focusing on noise: Focus on the significant trends rather than getting distracted by minor fluctuations.

• The Lean Startup is the process you use to move toward and achieve your vision.
• Early-stage founders aren’t building a product, they’re building a tool to learn what product to build.

Part II. Finding the Right Metric for Right Now

Analytics Frameworks:

• Common metric Frameworks group metrics into sets to provide areas of focus.
• AARRR was coined by McClure: acquisition, activation, retention, revenue, and referral
• Acquisition: Users discover the service or product through SEO, SEM, social media, emails, public relations, marketing campaigns, and blogs.
• Activation: Visitors become active users by subscribing and using the product, influenced by its features, design, tone, incentives, and positive reinforcement.
• Retention: Engagement is fostered in one-time users through regular notifications, alerts, reminders, emails, and updates to keep them coming back.
• Revenue: Monetization is achieved through user transactions, ad clicks, subscriptions, downloadable content, and data analytics.
• Referral: Users are encouraged to promote the product through email sharing, social media widgets, campaigns, likes, retweets, and affiliate programs.
• The three engines of growth by Eric Reis:
• Sticky Engine: Focuses on user retention and frequent usage, emphasizing the importance of metrics like user stickiness, frequency of use, and returning visit actions. The goal is to create a product that users will continually return to, reducing churn and increasing engagement.
• Virality Engine: Centres on user referrals and the spread of the product through networks, with the viral coefficient being a critical measure. A viral coefficient greater than one signifies growth, but it's also important to consider the churn and loss rate to understand the true impact of virality on the product's expansion.
• Paid Engine: Involves making money from user activity, which becomes viable when the product is sticky and viral. It's crucial to balance the customer lifetime value (CLV) against the customer acquisition cost (CAC) and measure the time to customer breakeven to ensure sustainable growth and a profitable business model.
• The Lean Canvas by Ash Maurya. Use and update the Lean Canvas continuously. Think of each box as a pass/fail experiment. If your experiment fails, you don’t get to go to the next box. The exception is the “Key metrics” box, which is meant to keep a record of the most important metrics you’re tracking.
• Problem: Measure by the number of respondents who have and are aware of this need.
• Solution: Measure by the number of respondents who try the MVP, engagement, churn, most-used/least-used features, and people willing to pay.
• Unique Value Proposition: Measure by feedback scores, independent ratings, sentiment analysis, customer-worded descriptions, surveys, search, and competitive analysis.
• Customer Segments: Measure by how easy it is to find groups of prospects, unique keyword segments, and targeted funnel traffic from a particular source.
• Channels: Measure by leads and customers per channel, viral coefficient and cycle time, net promoter score, open rate, affiliate margins, click-through rate, PageRank, and message reach.
• Unfair Advantage: Measure by respondents’ understanding of the Unique Value Proposition, patents, brand equity, barriers to entry, number of new entrants, and exclusivity of relationships.
• Revenue Streams: Measure by lifetime customer value, average revenue per user, conversion rate, shopping cart size, and click-through rate.
• Cost Structure: Measure by fixed costs, cost of customer acquisition, cost of servicing the nth customer, support costs, and keyword costs.
• Sean Ellis’s (Growth Hacker) Startup Growth Pyramid.
• A pyramid framework for startup growth:
• The base layer is achieving product/market fit, confirmed through customer feedback.
• The middle layer is stacking the odds, focusing on identifying and refining a unique competitive advantage.
• The pyramid's pinnacle is scaling growth, including expanding into new markets and launching new products, based on a proven product and a sustainable competitive edge.
• The Long Funnel represents a customer journey that extends beyond a website, across various platforms and influences. It involves multiple visits before a conversion.
• The Lean Analytics Stage is an amalgamation of all of these frameworks:
1. Empathy - Gate: I've found a real, poorly met need in a reachable market
• Focus on understanding the customer's needs and problems.
• You need to validate that there is a real need for your product or service in a market that you can reach.
2. Stickiness - Gate: I've figured out how to solve the problem in a way that they will accept and pay for
• Focus on developing a product or service that solves the customer's problem in a way that they are willing to pay for.
• You need to validate that your product or service is sticky, meaning that customers will continue to use it.
3. Virality - Gate: The users and features fuel growth organically and artificially
• Foucs on making your product or service viral, meaning that it grows organically through word-of-mouth marketing.
• You need to validate that your product or service has the potential to grow virally.
4. Revenue - Gate: I've found a sustainable, scalable business with the right margins in a healthy ecosystem
• Focus on developing a sustainable and scalable business model.
• You need to validate that you can generate enough revenue to cover your costs and make a profit.
5. Scale - Gate: I can achieve a successful exit for the right terms
• Focus on scaling your business to reach a larger market.
• You need to validate that your business can be scaled to achieve your desired outcome.

• Choose the One Metric That Matters (OMTM). The one that’s crucial given your current stage. Success lies in focus and discipline. The Lean Startup encourages focusing on the most vital thing, at the right time with the right mindset.
• Four Reasons to Use the One Metric That Matters (OMTM)
• It helps answer the most crucial busiess question and identify risky areas.
• It sets clear goals and measures success, providing a definitive objective.
• It focuses the entire company, preventing an overload of information.
• It fosters a culture of experimentation, encouraging learning from successes and failures.
• Draw lines in the sand.
• Establish a clear target number for success metrics, and be ready to reassess if it's not met.
• Choosing the right target number is challenging, and startups often struggle with it, leading to uncertainty after experiments.
• Many experiments yield moderate success, neither failing nor excelling significantly.
• Success can be defined by your business model's requirements or by comparing to industry norms when the business model is not yet clear.
• Optimising your OMTM squeezes that metric so you get the most out of it, but it also reveals the next place you need to focus your efforts, which often happens at an inflection point.
• Ask could your entire company work exclusively on improving that one metric? What might break if you did?

What Business Are You In?

• To decide which metrics you should track, you need to be able to describe your business model simply and just think about the really big components.
• Business growth comes from improving one of these five knobs: selling more stuff, to more people, more often, for more money, more efficiently.
• More stuff: add desired products or services
• More people: increase users via virality, word of mouth, or paid ads
• More often: enhance stickiness, reduce churn, and promote repeated use
• More money: upsell and maximise user pricing
• More efficiently: cut service delivery/support costs or lower customer acquisition cost
• Not all customers are beneficial. While some are valuable, others can be distracting, resource-consuming or harmful. It's essential to distinguish valuable users from detrimental ones, then implement changes to maximise the real users and minimise the harmful ones.
• Choose your payment and incentive models to match the kind of segmentation you’re doing, the time it takes for a user to become a paying customer, how easy it is to use your service, and how costly an additional drive-by user is to the business. Not all customers are good.
• A business model is a combination of things:
• The acquisition channel is how people find out about you.
• Selling tactics persuade visitors to become users or customers, often through scarcity, exclusivity, or additional benefits.
• Revenue sources, direct or indirect, may include transactions, subscriptions, ad revenue, data resale, donations, among others.
• The product type is what value your business offers in return for the revenue.
• The delivery model is how you get your product to the customer.

• The book covers the following six business models:
1. E-commerce: Sales to customers.
2. SaaS: Subscription-based software.
3. Mobile Apps: Revenue from in-app purchases.
4. Media Sites: Earnings from ads.
5. User Content: Platforms like Twitter, Facebook, Reddit.
6. Two-sided Markets: Platforms for buyer-seller transactions.

E-commerce:

• On-site funnels are somewhat outdated, you need to understand what happens beforehand.
• Recommendations engines are becoming increasingly more important.
• What type of e-commerce business you have drives everything from marketing strategy to shopping cart size. The key factor is the annual repurchase rate: what percentage of people who bought something from you last year will do so this year? Based on that you can determine what mode you’re in:
• Acquisition mode: Focused on new customer acquisition if less than 40% of previous year's buyers return. Loyalty programs are not suitable. This is typical of mature e-commerce businesses.
• Hybrid mode: If 40-60% of last year's buyers return, growth comes from new and returning customers. The focus is on acquisition and increasing purchase frequency.
• Loyalty mode: If over 60% of previous year's buyers return, the focus should be on loyalty and increasing purchase frequency. Loyalty programs are effective in this case, which is typical for about 10% of mature e-commerce businesses, such as Amazon.
• Even before a year has elapsed, an e-commerce company can look at 90-day repurchase rates and get a sense of which model it’s in.
• A 90-day repurchase rate of 1% to 15% means you’re in acquisition mode.
• A 90-day repurchase rate of 15% to 30% means you’re in hybrid mode.
• A 90-day repurchase rate of over 30% means you’re in loyalty mode.
• Too many leaders try to increase loyalty. If you're in acquisition mode, you probably can't — and shouldn't try to — increase loyalty.
• It’s difficult to move the annual repurchase rate by more than 10%, despite a company’s best efforts.
• Online retailers should identify which messages and platforms attract buying visitors and, once on the site, focus on maximising their purchases.
• Metrics of interest in e-commerce:
• Conversion rate: The number of visitors who buy something.
• Purchases per year: The number of purchases made by each customer per year.
• Average shopping cart size: The amount of money spent on a purchase.
• Abandonment: The percentage of people who begin to make a purchase, and then don’t.
• Cost of customer acquisition: The money spent to get someone to buy something.
• Revenue per customer: The lifetime value of each customer.
• Top keywords driving traffic to the site: Those terms that people are looking for, and associate with you — a clue to adjacent products or markets.
• Top search terms: Both those that lead to revenue, and those that don’t have any results.
• Effectiveness of recommendation engines: How likely a visitor is to add a recommended product to the shopping cart.
• Virality: Word of mouth, and sharing per visitor.
• Mailing list effectiveness: Click-through rates and ability to make buyers return and buy.
• Page optimisation is important. Balance a high conversion rate with a high revenue per visitor, or high customer lifetime value (CLV), because that’s what’s really driving your business model.
• Capture the contribution (or net revenue) of each email campaign, considering the added income, campaign cost, and loss from unsubscribes.
• Subscription services regularly bill customers, making churn measurable. It's crucial to monitor expired payment information, renewal campaign effectiveness, and factors affecting renewals. As loyal user base grows, renewal revenue becomes a significant part of total revenue.

Software as a Service (SaaS):

• Metrics of interest in SaaS:
• Attention: How effectively the business attracts visitors.
• Enrollment: How many visitors become free or trial users, if you’re relying on one of these models to market the service.
• Stickiness: How much the customers use the product.
• Conversion: How many of the users become paying customers, and how many of those switch to a higher-paying tier.
• Revenue per customer: How much money a customer brings in within a given time period.
• Customer acquisition cost: How much it costs to get a paying user.
• Virality: How likely customers are to invite others and spread the word, and how long it takes them to do so.
• Upselling: What makes customers increase their spending, and how often that happens.
• Uptime and reliability: How many complaints, problem escalations, or outages the company has.
• Churn: How many users and customers leave in a given time period.
• Lifetime value: How much customers are worth from cradle to grave.
• The customer lifecycle starts with acquisition through viral or paid marketing, leading to usage and payment for services, with the hope of referral and upgrades.
• The ratio of customer lifetime value (CLV) to customer acquisition cost (CAC) is crucial to gauge the health and sustainability of a subscription business model.
• A 5–6x ratio means that for every dollar the company invests in finding a customer, it makes back $5 to $6.
• MRR growth is often a key metric. Companies watch churn, but are often more focused on customer acquisition payback in months.
• Focusing on the right metrics sequentially is essential: initially on whether the product meets a need, followed by the ability to scale.
• Stickiness and daily engagement are vital, with frequent users like Evernote's often becoming paying customers over time.
• The key metric for engagement is daily usage of your product. For products not used daily, establishing engagement takes longer. Quick, impactful value demonstration is crucial to prevent user drop-off. Creating new user habits is challenging but vital.
• Early adopters can guide the product development, but their needs may not represent the larger mainstream market.
• Identify common traits among users hooked on your product, focus on their needs, and expand. Claiming your beachhead will enable faster iteration on a highly engaged market segment.
• A data-driven approach is key for measuring engagement, understanding customer behaviour, and recognising which features or changes improve stickiness.
• Churn rate, the percentage of users or customers who stop using the service over time, is a critical metric that should be measured distinctively for paid and unpaid users.
• Churn is the percentage of people who stop using your service over a specific period.
• Track churn separately for users (not paid) and customers (paid).
$$Churn = \dfrac {Number\ of\ churns\ during\ period} { Number\ of\ customers\ at\ beginning\ of\ period}$$
• An inactive user, someone who hasn't logged in within 90 days, is considered to have churned.
• Churn helps calculate the average customer's lifetime value (average stay duration × average monthly revenue per user).
• If growing fast use a average numbers for the period analysed.
• Freemium is a sales tactic that needs careful usage in SaaS; building a group of loyal users faster than they erode is crucial.
• User engagement must be measured long before users become customers, and customer activity must be monitored long before they vanish.
• It’s important to measure engagement and activity before users become customers and anticipate potential churn to mitigate it effectively.
• While SaaS is typically associated with subscriptions, there are various ways to monetize on-demand software, which may be more suited to the nature of the product.
• To quantify and understand business risks, focus on optimizing business metrics in the correct order, ensuring that efforts are effectively improving the weakest points.
• To find ways you might improve things, segment users who do what you want from those who don’t, and identify ways in which they’re different.
• To decide whether a change worked, test the change on a subset of your users and compare that subset’s results to others.
• If you can’t test features in this way without fallout then compare the cohort of users who joined after the feature was added to those who came before.

Free Mobile Apps:

• Metrics relevant to Mobile Apps
• Downloads: Number of people who have downloaded the application.
• Customer Acquisition Cost (CAC): Cost to acquire a user or paying customer.
• Launch Rate: Percentage of people who download and launch the app, creating an account.
• Active Users: Percentage of users who’ve launched the application and use it daily or monthly.
• Paying Users: Percentage of users who make a purchase.
• Time to First Purchase: Duration from activation to first purchase.
• Monthly Average Revenue Per User (ARPU): Revenue from purchases and ads, includes application-specific information.
• Ratings Click-through: Percentage of users who rate or review in an app store.
• Virality: Average number of other users a user invites.
• Churn: Number of customers who uninstall the application, or don’t launch it within a certain time period.
• ARPU, calculated monthly, is your revenue divided by the number of active users. Inflating user numbers reduces ARPU, necessitating a realistic definition of "engaged".
• To measure customer lifetime value (CLV) in mobile games, calculate the average spend per player post-churn. Given players may stay for months or years.
• The company aims to boost downloads, enhance engagement, increase ARPU, reduce churn, and boost virality to lower customer acquisition costs. These goals can be in tension: like improving game enjoyment to decrease churn and maximising profit for high ARPU.
• Treat your paying users as a separate customer base and track their behavior, churn, and revenue separately from your nonpaying ones
• Monitor user churn at 1 day, 1 week, and 1 month, as reasons for leaving vary. Day-one churn may indicate a poor tutorial, week-long churn could suggest lack of game depth, and month-long churn may reflect inadequate update planning. Knowing when users churn gives you an indication of why they’re churning and what you can do about it.
• Often most of the money comes from a small number of users; segment them and treat them as a distinct group.

Media Sites:

• Your primary focus is sharing advertisers’ messages with viewers, and getting paid for impressions, click-throughs, or sales.
• Media sites care about:
• Audience and churn The number of people who visit the site and how loyal they are.
• Ad inventory: The number of impressions that can be monetised.
• Ad rates: The earnings from those impressions based on the content it covers and the people who visit.
• Click-through rates: The percentage of impressions that actually turn into money.
• Content/advertising balance: The balance between ad inventory rates and content that maximises overall performance.
• Engagement is much more important than traffic, so knowing how many visitors you’re losing, as well as adding, is critical.
• Ad inventory also depends on page layout and how many advertising elements are on each page.
• You get paid based on who your visitors are rather than what your site contains.
• Blank ads with a click-through rate of ~0.08% are comparable to some paid campaigns. If your ads' revenues are barely above this, investigate the cause.

Sketching out your business model and key metrics like this is a great idea.

User-Generated Content:

• Growing an active community that creates content is key to UGC businesses.
• Metrics that matter for UGC companies:
• Engaged visitors: Measure how often people return and their duration on site.
• Content creation: Percentage of visitors interacting with content, from creation to voting.
• Engagement funnel changes: Effectiveness of site in progressing people to more engaged content levels.
• Value of created content: Business benefit of content, from donations to media clicks.
• Content sharing and virality: Measure how content sharing drives growth.
• Notification effectiveness: Percentage of users taking action on notifications (push, email, etc.).
• UGC site success hinges on visitor retention.
• Recency: the last visit date.
• The day-to-week ratio, how many of today's visitors were here earlier in the week.
• Average days since last visit (exclude lapsed users 30 day cut off).
• For users who have accounts and take actions, you can measure engagement in other ways: days since last post, number of votes per day, and so on.
• User participation rates by content type:
• Reading web discussions and sharing photos are the most common activities, with a significant percentage of users engaging in these activities at least once a month.
• Activities like writing comments, posting questions, and writing reviews also see moderate engagement.
• Users are less frequently involved in answering questions, updating social sites, sharing videos, and posting to their own weblog, with the majority participating less than once a month or never.
• Content consumption is more frequent than content creation (like posting weblogs).
• UGC sites need to solve a chicken-and-egg problem. They need content to draw in users, and users to create content. Sometimes, this content can be seeded from elsewhere, but then pivoted to a community-edited model.
• The rate of content creation and enrollment matter significantly at the outset. Later, the question becomes whether quality content is rising to the top and if people are commenting on it.
• There are often tiers of engagement (lurking, voting, commenting, subscribing, submitting links, and creating). Each tier represents a degree of involvement and content generation by a user, and each type of user represents a different business value to the company.
• UGC sites have a power curve of content creation, where a small number of people create the vast majority of content.
• User-generated content has value, measurable it by cohort or traffic segment to decide on how to invest in visitor acquisition, identify which referring sites attract valuable users.
• Track how content is shared:
• to measure if virality is strong enough to sustain your business.
• to see how content is shared and with whom.
• to understand whether you should consider a paywall-style monetization strategy.
• Sustaining engagement relies on drawing users back with notifications. Gauge their effectiveness like email delivery rates: the outcome matches the number of messages sent.
• Track visitor engagement in UGC using an "engagement funnel". Expect an 80/20 split between lurkers and contributors, with a few becoming dedicated creators. Notify users of activity via email to maintain engagement. Be aware, fraud prevention is crucial for a UGC site.

Two-Sided Marketplaces:

• The company makes money when a buyer and seller come together to complete a transaction. Two-sided marketplaces have to attract both buyers and sellers.
• Early on do things that don’t scale: grow inventory by hand. Ultimately, volume of sales, and the resulting revenue, is the only metric that matters.
• Begin with a small marketplace demonstrating demand, supply, and transaction interest. Success depends on transaction size, frequency, and business specifics, the basic principle remains the same: revenue from transactions.
• Uber subsidised commission for drivers until their commission was sustainable.
• Step 1: Measure your ability to create an inventory (supply) or an audience (demand). Monitor the attraction, engagement, and growth of the seed group.
• A buyer is engaged if she’s made at least one purchase, and that a buyer is engaged if she’s searched for something in the last 30 days.
• Sellers are disengaged if they haven’t added a listing in the last 30 days, and listings are inactive if they don’t show up in buyers’ search results at least five times a week.
• Metrics to include:
• Buyer and Seller Growth: Measures rate of new buyer and seller addition.
• Inventory Growth: Rate at which new listings are added.
• Search Effectiveness: Matches between what buyers search for and inventory.
• Conversion Funnels: Measures conversion rates for sold items.
• Ratings and Signs of Fraud: Assesses buyer, seller ratings and potential fraud.
• Pricing metrics: in auction pricing are prices too high
• Buyer searches are the primary way in which buyers find sellers.
• Track the number of searches that return no results
• Look at search terms themselves. Dominant terms are opportunities.
• Allow user-regulated fraud monitoring by allowing them to flag violations of terms of service. Track the percentage of listings that are flagged, and whether this number is increasing or decreasing.
• A major issue is keeping transactions inside the network.
• Two-sided markets require solving the initial challenge of attracting enough buyers and sellers. It's beneficial to focus on money-spending individuals first. As sellers are inventory, it's essential to monitor their growth and its alignment with buyer demands. While transaction percentages are common earnings in marketplaces, other methods like aiding sellers in product promotion or charging listing fees can also be profitable.

What Stage Are You At?

• The Lean Analytics stages suggest an order to the metrics you should focus on.
• The stages closely mirror what other Lean Startup advocates advise:
1. Empathy: Understand and solve a problem that customers are willing to pay for.
2. Stickiness: Build a product that isn't just good, but one that customers keep coming back to.
3. Virality: Use word-of-mouth to acquire and onboard new users once you have a sticky product.
4. Revenue: Focus on monetiswation once your product is sticky and growing.
5. Scale: Expand your customer base into new markets and regions once revenue is steady.

Stage One: Empathy

• At this stage the focus should be on discovering and validating a problem and then finding out whether your proposed solution to that problem is likely to work. Get out of the building and gather qualitative feedback, primarily through problem and solution interviews.
• Speak to 15 people at each interviewing stage, don’t rushing ahead.
• Problem (or idea) discovery often starts with listening. Treat your idea is simply a starting point.
• You have to decide whether your problem is painful enough for enough people to take action.
• Then you have to be confident there are enough of them.
• You have to understand your target market:
• What’s homogenous within (commonalities you can appeal to)?
• What’s heterogeneous between (target each segment with a tailored message)?
• What does it take to make them aware of the problem?
• If you can’t find 15 people to talk to, imagine how hard it’s going to be to sell to them.
• Positive signs:
• Ready to pay immediately
• Actively seeking solution
• Engages with numerous questions
• Shows passion for problem-solving
• Positive body language
• Negative signs:
• Easily distracted
• Rambling unrelated topics
• Slumped or slouching posture
• Running Lean (O’Reilly). It’s a good complement to this book.
• The goal of problem interviews is to find a problem worth solving. They are open-ended.
• Test the customer segment.
• Set the problem context.
• Test the problem.
• Test the solution.
• Ask for a commitment (e.g. $100 pre-commitment).
• Qualitative metrics although they are easy to bias, they don’t lie.
• Ask why three times to force a respondent to explain the reasoning behind a statement.
• Ask a ‘Columbo’ question for the very end, after you’ve said your goodbyes.
• Problem validation can actually happen in two distinct stages (Location 3579)
• Ash Maurya suggests using a storytelling approach to provide context for problems in interviews. These can be convergent, where specific problems are ranked to quantify urgency and prevalence, or divergent, where the interviewee is allowed to identify problems within a larger problem space. Convergent interviews may risk focusing too narrowly, while divergent interviews may lead to a lack of focus or clarity.
• How to know if the problem is painful enough?
• Did the interviewee successfully rank the problems you presented?
• Was the interviewee engaged and focused throughout the interview?
• Did the interviewee agree to a follow-up meeting/interview (where you’ll present your solution)?
• Did the interviewee offer to refer others to you for interviews?
• Did the interviewee offer to pay you immediately for the solution?
• Everything is an experiment you can learn from.
• How are people currently solving the problem? If they aren’t, proceed with caution. You may first need to educate them of the problem (which adds another layer of complexity).
• Many startups fail as they only slightly solve a problem. To succeed, startups must surpass the standards set by market leaders.
• Estimate market size using both top-down and bottom-up analysis for comparison and accuracy. The top-down approach begins with a large figure which is then segmented, the other is the opposite.
• Understand your customers on a deeper level. Describe a detailed, real-life scenario for your solution, going beyond just defining target markets. Knowing how to reach and engage your customers in their daily lives is crucial for delivering your solution when they need it most.-
• Mapping customers' behaviours and aligning your activities can reveal opportunities to improve engagement and influence buyers.
• Tactics like surveys and landing pages enable large-scale communication, reaching a wider audience and strengthening the data-driven case based on interview feedback.
• Use Facebook's ad platform to reach targeted groups for feedback.
• Test different taglines and emotional appeals for messaging.
• Use surveys to inform strategic decisions.
• Surveys should include demographics, quantifiable questions, and open-ended questions.
• Use ads linking to surveys targeted at specific audiences.
• Assess market interest through ad clicks and link interactions. Use collected data to guide decisions. Assess the willingness of respondents to engage with your solution/product. Calculate the mean and standard deviation of quantifiable questions to determine a clear winner.
• Having validated the problem, begin validating the solution. Conduct interviews, surveys, and landing pages to gather qualitative feedback and build a minimum viable product (MVP). Test your hypotheses through a proxy.
• Remember, the goal is not to build a product but to de-risk a business model. As you build your MVP, continue gathering feedback and acquiring early adopters.
• Define what the MVP should deliver. Ensure it's not too shallow or too bloated to avoid failure. MVP is a process, not a product. It will undergo numerous iterations based on the build→measure→learn cycle of the Lean Startup.
• Define the One Metric That Matters (OMTM). Every feature in the MVP should relate to and impact the OMTM. Metrics around user acquisition at this stage are irrelevant. Instead, focus on engagement.
• Every feature should have a corresponding metric on usage and engagement. Be ready to remove features that are not being used or not creating value.
• Finally, ask yourself: Have you conducted enough customer interviews? Do you understand the customer? Do you believe your solution will meet customer needs? If all checks out, move to the next stage.

Stage Two: Stickiness

• The big question now is whether or not what you’ve built is sticky, so that when you throw users at it, they’ll engage.

In the stickiest situation since Sticky the stick insect got stuck on a sticky bun Rowan Atkinson’s Blackadder

• Focus on retention and engagement, examining user activity and feature interaction. Assess these metrics by cohorts to evaluate the impact of your changes. Aim for proof of your product's indispensability to users. Don't aim for quick growth, rather test user retention. If you can't retain a hundred users now, you're unlikely to retain a million later.
• The priority is to develop a core feature set, regularly used by a small, targeted initial user group generating meaningful results. You need to confirm user behaviour aligns with expectations and users derive significant value from the product before expanding. Until you can convert attention into engagement, refrain from driving new traffic.
• The MVP is a process, not a product.
• As you iterate, your goal is to improve on the core metrics that you’re tracking. If a new feature doesn’t significantly improve the One Metric That Matters, remove it.
• The MVP should be the simplest path to the user's "aha!" moment. Everything is negotiable. Don't reinvent familiar concepts like enrolment, but feel free to ignore them for testing. Focusing on one key metric, like survey response rate, allows tweaking all other business aspects, from sign-up to platform.
• An MVP isn’t a product it’s a tool for figuring out what product to build.
• Having lots of users isn’t traction unless those users are engaged and sticking around.
• Premature scaling can be disastrous, investing all your time and money into user acquisition only to watch the users churn through the business too quickly.
• The Goal Is Retention.
• The more engaged that people are with your product the more likely they’ll stay. By ignoring growth from virality (for now), you can simplify how you decide what to build next into your MVP.
• Do we believe that the feature we want to build (or the feature we want to change) will improve stickiness?
• 7 things to consider when planning a new feature:
1. The potential of the feature to improve the current situation.
2. The ability to measure the impact of the feature.
3. The time required to develop and implement the feature.
4. The complexity the feature might add to the product.
5. The associated risks, including technical, user, and future development risks, of the new feature.
6. The level of innovation the new feature brings to the table.
7. User feedback and requests, while considering the risks of solely relying on them.
• User actions speak louder than words.
• Rally uses an approach called ORID (Objective, Reflective, Interpretive, Decisional) from The Art of Focused Conversation by R. Brian Stanfield. This process encourages insights from all employees, fostering structured learning through reflections, aligning everyone with a compelling vision. The company cycles through build, measure, learn stages at multiple levels before actual feature development.
• Every experiment starts with a series of questions:
• What do we want to learn and why?
• What’s the underlying problem we are trying to solve, and who is feeling the pain?
• What’s our hypothesis?
• How will we run the experiment, and what will we build to support it?
• Is the experiment safe to run?
• How will we conclude the experiment, and what steps will be taken to mitigate issues that result from the experiment’s conclusion?
• What measures will we use to invalidate our hypothesis with data?
• What measures will indicate the experiment isn’t safe to continue?
• Data-driven product direction is a top-down, iterative process. Everything, even with an established product and loyal customers, is an experiment. While it requires extra engineering to manage features and measure user behavior changes, it pays off in reduced cycle time and improved learning.
• How to interpret feedback:
• Plan and organise tests: According to Laura, organising feedback based on a specific topic simplifies its interpretation.
• Target specific personas: Feedback should be grouped by similar personas for consistency, as different types of people will provide diverse responses.
• Review data promptly: Laura recommends quick data review and having another person in each session for immediate debriefing and summarisation.
• The Minimum Viable Vision. A MMV captivates, scales, has potential, is audacious and compelling. Signs you might have one:
• You’re building a platform.
• You have recurring ways to make money.
• You’ve got naturally tiered pricing.
• You’re tied to a disruptive change.
• Adopters automatically become advocates.
• You can create a bidding war.
• You’re riding an environmental change.
• You’ve got a sustainable unfair advantage.
• Your marginal costs trend to zero.
• There are inherent network effects in the model.
• You have several ways to monetise.
• You make money when your customers make money.
• An ecosystem will form around you.

The Problem-Solution Canvas

• The Problem-Solution Canvas helps you maintain discipline and focus on a weekly basis.
• Use it to home in on the key problems you’re facing. Agree on and prioritise your problems.
• Canvas Boxes:
• Current status: key metrics you’re tracking compared to previous period
• Last week’s lessons learned: What did you learn, what was accomplished?
• Top problems: List and prioritise the top problems
• For each problem:
• Hypothesised solutions: list the possible solutions. Rank them. Explain how you think they’ll solve the problem
• Metrics / Proof / Goals: List the metrics you’ll use to measure the solutions, list any qualitative proof you’ll need. Define the goals for the metric.
• Essentially: What’s the problem, how do you propose to fix it, and how will you know if you succeeded?

Stage Three: Virality

• Frank Bass in his 1969 paper, “A New Product Growth Model for Consumer Durables,” explained how messages trickle out into a market through word of mouth. This model is represented by a characteristic S-shape known as the Bass diffusion curve.

• In the Virality stage you should focus on user acquisition and growth.
• Premature growth burns money and time, and will quickly kill your startup (so fix stickiness first).
• Three types of Virality:
• Inherent virality is built into the product, and happens as a function of use.
• Artificial virality is forced, and often built into a reward system (e.g. sign up a friend for perks).
• Word-of-mouth virality is the conversations generated by satisfied users, independent of your product or service. Hard to track but super effective.
• Metrics for the Viral Phase
• Viral Coefficient (the number of new customers an existing customer is able to convert):
• How to calculate it:
• Calculate the invitation rate as invites sent divided by user count.
• Find the acceptance rate as signups divided by invites.
• Multiply both rates.
• So, existing customers: 2,000. Total invitations sent: 5,000. Invitation rate: 2.5. Number that get clicked: 500. Acceptance rate: 10%. Viral coefficient: 25%.
• A viral coefficient above 1 means the product is self-sustaining. With a viral coefficient above 1, every single user is inviting at least another user, and that new user invites another user in turn.
• How to influence it: Boost acceptance rate.Extend customer lifetime for more invitations.Shorten invitation cycle time for rapid growth. Encourage customers to invite more.
• Cycle time: Time between first use and invitations. Shorter cycle times have viral growth.
• Focus on actions driving virality in your product, ensure they're measured correctly and have appropriate targets.
• Growth Hacking
• Identify an early measurable metric (e.g., user invites), understand its correlation to a business goal (e.g., long-term engagement), predict future outcomes based on current metric, and modify the user experience to improve the goal.
• The early metric (or leading indicator) is key to growth hacking. Despite its apparent simplicity, finding and experimenting with a good leading indicator to influence the company's future is challenging.
• Example leading indicators:
• Facebook: connecting with seven friends within 10 days of account creation.
• Dropbox: putting at least one file in one folder on one of their devices.
• Characteristics of good leading indicators:
• Leading indicators often relate to social engagement, content creation, or return frequency.
• The leading indicator should be closely tied to a component of the business model.
• The indicator should come early in the user's lifecycle or conversion funnel.
• It should also provide an early extrapolation for quicker predictions.
• Correlation is nice, but spotting a leading indicator that causes a change later on is a superpower. You can now change the future. You just have to make that leading indicator happen more often.
• Compare engaged to not-engaged users, look for differences.
• Growth hacks come in all shapes and sizes. The point is to experiment in a disciplined manner.
• Product-focused growth hacks or “aha moments” need to happen early in the user’s lifecycle in order to have an impact on the greatest number of possible users.
• In the Virality and Revenue stages, you're searching for early metrics in a user's lifecycle that can predict or control future growth.
• Should You Move On to the Revenue Stage? Ask yourself these questions:
• Are you utilising any form of virality?
• If virality is currently weak, list three to five strategies to enhance it.
• What is your viral coefficient?
• Is your current viral coefficient sufficient for growth and reducing customer acquisition costs?
• What is your viral cycle time and how could it be reduced?
• Who are the user segments or cohorts that align with your business model?
• What commonalities do these users share?
• What changes could be made to your product, market, or pricing to address these users earlier in their customer lifecycle?

Stage Four: Revenue

• Time to switch focus to making money and using some of that to power growth. Customer lifetime value and customer acquisition cost drive growth. Run experiments to try capture more loyal users for less, tweaking how you charge, when you charge, and what you charge for.
• Your focus moves from proving your idea to proving you can make money in a scalable, consistent, self-sustaining way.
• Revenue per customer. Click-through rates, ad revenue, conversion rates and basket size, subscriptions and customer lifetime value. Should be compared to the cost of acquiring new users.
• You need to understand your business model and be able to articulate it easily. Think of it as a machine that generates more money than you put into it. Measuring the ratio of inputs to outputs tells you how good it is.
• To measure the health of the machine, divide how much you changed the annual recurring revenue in the past quarter by what it cost you to do so. You need three numbers to do this calculation:
• Your quarterly recurring revenue for quarter x (QRR[x])
• Your quarterly recurring revenue for the quarter before x (QRR[x – 1])
• Your sales and marketing expense for the quarter before x (QExpSM[x – 1])
• At 0.75 you have a problem. When you pump money into the machine, less money comes out.
• Better than 1 you’re doing well. You can fund your growth with the proceeds.
• You need to figure out which “more” increases your revenues per engaged customer the most (more stuff to more people for more money more often more efficiently):
• If your business model has physical, per-transaction costs, focus on efficiency.
• If you have a high viral coefficient, focus on customer acquisition to reach more people.
• If you have a loyal customer base with recurring purchases, focus on customer retention and encourage frequent purchases.
• If your business model is based on one-time, big-ticket transactions, focus on maximising the revenue from each transaction.
• If you’re a subscription model, and you’re fighting churn, then upselling customers to higher-capacity packages with broader features is your best way of growing existing revenues, so you’ll spend a lot of time on more stuff
• Customer Lifetime Value > Customer Acquisition Cost
• You need to spend less money acquiring customers than you get from them.
• The CLV-CAC math also needs to reflect the fact that there’s a delay between paying to acquire customers and those customers paying you back.
• Balancing acquisition, revenue and cashflow is the art of running a business: The key metrics are:
• The money in the bank you start with
• The amount of money spent on customer acquisition each month
• The revenue you bring in from users
• The rate of churn from users
• Be prepared to radically change, or even shut down, parts of your company in your quest for revenue.
• Market/Product Fit
• Resist the urge to build more features when things aren't going well.
• Consider pivoting into a new market if the product isn't the problem, but the target customer is.
• Freemium models require a large customer base.
• SMB’s is not a target market. Narrow down and focus on niche markets considering factors like industry, geography, recent purchases, budgets, industry growth, seasonality, legislative constraints, and decision-makers.
• It can be a good idea to fire a segment of your customers because of the drain they represent on the business
• The Breakeven Lines in the Sand
• You want to be breakeven, your revenues exceed your costs on a regular basis.
• Breakeven on Variable Costs: The money you make from a customer exceeds the cost of acquiring that customer and delivering the service. Each customer isn’t costing you anything.
• Time to Customer Breakeven: Assuming CAC < CLV, the next question is how quick is the payback period. This helps inform cash requirements of your business.
• EBITDA Breakeven: Earnings before income tax, depreciation, and amortisation. Pay-as-you-go costs like cloud computing is making this a valid measure again. Hibernation Breakeven: Could the company survive by just maintaining operations and servicing customers? This is referred to as "ramen profitability."

Stage Five: Scale

• Scale involves expanding your audience, entering new markets, and establishing predictability and sustainability. It's about proving the market.
• Michael Porter describes how companies compete. Firms can focus on a niche market (a segmentation strategy), they can focus on being efficient (a cost strategy), or they can try to be unique (a differentiation strategy).
• Porter noted that both large market share firms like Apple, Costco, Amazon, and small ones like coffee shops are profitable. The challenge lies with midsize firms, termed as the "hole in the middle" – too big for niche strategies, yet too small for cost or scale competition. Such firms must differentiate to survive the midsize gap and achieve efficiency.
• Getting attention at scale means your product or service can stand on its own, without your constant love and feeding.
• Expect competitors, to thrive, you need to claim your place in this market and establish the kinds of barriers to entry that maintain margins.
• Scaling is valuable for incremental revenue, but monitor for decreased engagement and rising acquisition costs. Evaluating churn by channel, alongside revenue and engagement, indicates if your product is scalable. Once confirmed, you can begin to scale acquisition.
• The Three-Threes Model
• Three tiers of management:
• Board & Founders (Strategy: meeting monthly/quarterly)
• Exec Team (Tactics and oversight: meeting weekly)
• Rank-and-file (Execution: meeting daily)
• Three Big Assumptions
• Each assumption has a metric and a line in the sand. This is your bet.
• Shouldn’t change more than once a month.
• Should be evident on your Lean Canvas
• Communicated to the whole company.
• Three Actions to take
• The tactics that will make the big assumptions happen, moving the metrics in the right direction.
• Exec team should break down each of them into 3 actions that can happen this week.
• Actions change regularly. Test and iterate, kill what doesn’t work.
• Three Experiments to Run
• Each day performing tasks to complete the actions
• Anyone can run a test (speaking with customers to tweaking features) provided it’s documented beforehand and the results contribute to the week’s actions.

Model + Stage Drives the Metric You Track

• Understanding your business type and current stage enables you to track and optimise the One Metric That Matters most at this moment.
• Once you’ve identified the metrics you should worry about, your next question is clear: what should I be trying for, and what’s normal?

Part III Lines in the Sand.

• There is a normal or ideal for most metrics, and that normal will change significantly as a particular business model goes from being novel to being mainstream.
• How you feel about a metric changes when you know what’s normal or best in class for your type of business. “When I first saw our churn, which was around 2%, I was very concerned, but when I found out that 2% is pretty much the lowest churn you’ll get in the hosting business, it changed my perspective a great deal.”
• Before investing time and effort into a metric, understand your position against competitors and industry averages. Benchmarks guide whether to continue working on a metric or tackle the next challenge.
• That said, realise there’s a reason most startups fail: average is nowhere near good enough.

Universal Metrics:

• Pre-revenue active user growth: 5% a week
• Post-revenue revenue growth: 5% a week
• Engagement: 30% of registered users visit once a month, 10% come daily
• Experiment with price points to plot your price elasticity and find the optimal price

One of the biggest misconceptions around pricing is that what you charge for your product or service is not directly related to how much it costs you to build or run it. Price is related to what your customers are prepared to pay. Neil Davidson, author of Don’t Just Roll the Dice

1. Research on price elasticity suggests that it applies most in young, growing markets.
2. Patrick Campbell found that most respondents compared themselves to the competition when setting pricing.
3. Testing is key. Once you find your revenue “sweet spot,” aim about 10% lower to encourage growth of your user base.
• Cost of Customer Acquisition: Don’t spend more than a third of the money you expect to gain from a customer on acquiring that customer.
• Virality: If virality is below 1, it reduces customer acquisition cost. Above 1, it fosters growth. Above 0.75 is pretty good.
• Mailing List Effectiveness: A well-run campaign should hit a 20–30% open rate and over 5% click-through. Focus on the subject line.
• Uptime and Reliability: 99.5% uptime
• Site Engagement: An average engaged time on a page of one minute is normal.
• Web Performance: Aim for a page load of 5 seconds or less. After 10, you’ll suffer.

ECommerce Metrics:

• Conversion Rates: Expect 1–3% maximum, to get to get to 10% you need seriously loyal users, lots of SKUs, and repeat customers. Average conversion rate across the Web is 2.13%.
• If visitors arrive with a strong intent to buy, you’ll do better - but you’ll have to invest elsewhere to get them into that mindset.
• Shopping Cart Abandonment: if 65% of people abandon their purchase before paying you’re doing well.
• Search Effectiveness: 90% of mobile searches lead to action, 50% to a purchase.
• Don't just think "mobile first." Think "search first."

• Freemium models are usually unsustainable. It’s hard to create something good enough that people want to use it, but bad enough they want to upgrade to premium.
• It also takes a long time to convert people.
• Expect 1% of users to upgrade to a paid plan in their first month, and 12% after 24 months.
• Freemium models work for products that have:
• A low marginal cost to serve
• A good viral coefficient
• Easy to learn
• Something that feels right to be free
• Increases in value to the user with time
• Err on the side of charging too little: make 10% less money but have 20% more customers.
• Upselling and Growing Revenue: Increase revenue from each customer by 20% a year. Get 2% of your paying subscribers to increase what they pay each month.
• Churn: The best have 1.5% - 3% a month. Get below a 5% monthly churn rate before you scale, then aim to get to 2%.
• Up to 50% will churn at their first billing event.

Mobile App Metrics:

• Mobile Downloads: No ‘typical’ here.”
• Mobile Download Size: Aim for less than 50 MB
• Mobile Customer Acquisition Cost: Pay around $0.50 for a paid install, and around $2.50 for a legitimate, organic one, but make sure that your overall acquisition cost is less than $0.75 per user.
• Application Launch Rate: 83% of downloads are launched
• % Active Mobile Users/Players: Most initial app users won't return, leading to a steep decline, followed by a slower drop in engagement.
• % of mobile users who pay: 1.5% of users will buy something
• Average revenue per paying user:
• Whales 10% of players, ARPPU of $20
• Dolphins 40% of players, ARPPU of $5
• Minnows 50% of players, ARPPU of $1
• Mobile app rating click through: less than 1.5% for paid, less than 1% for free apps.

Media Site Metrics:

• Click-Through Rates: Ads will get 0.5 to 2% CTR. Below 0.08%, you’re doing something wrong.
• Sessions-to-Clicks Ratio: Lose around 5% of clicks before the visitor ever gets to your site.
• Engaged Time: Aim for 90 seconds or more of engaged time on content pages
• Sharing with Others: Typically, a story is viewed by only nine people for each time it's shared, indicating that sharing primarily occurs within small, close groups. The ratio is 5 to 1 on Twitter and 36 to 1 on Reddit.

User-Generated Content Metrics:

• Time on Site: You’re sticky when visitors are spending 17 minutes a day with you.
• Content Upload Success:
• Jakob Nielsen suggests in an online population, 90% of people lurk, 9% contribute intermittently, and 1% are heavy contributors.
• 80% of respondents consume content passively, 62% share content, 43% comment, and 36% produce content.
• The difference between countries is notable: the Chinese share more than the English
• 23% of internet users only consume content. 16% react by voting, commenting, or flagging. 44% initiate by posting content or starting threads. 17% contribute, e.g. reviewing items on e-commerce sites.
• Reddit’s user contribution followed the 80/20 rule seen on many UGC sites; that is, 20% of users were logged in and voting, and 20% of those were commenting.
• Expect 25% of your visitors to lurk, 60–70% of your visitors to do things that are easy and central to the purpose of your product or service, and 5–15% of your users to engage and create content for you. Among those engaged users, expect 80% of your content to come from a small, hyperactive group of users, and expect 2.5% of users to interact casually with content and less than 1% to put some effort into interaction.

Two-Sided Marketplaces

• Use top 10 lists to understand your marketplace.
• Review KPIs like revenue and transaction numbers by product segments.
• Determine top 10 buyers and sellers.
• Identify key revenue-generating products or categories.
• Assess peak sales in terms of price ranges, times, and days.
• Top 10 lists offer insights into marketplace health.

What to Do When You Don’t Have a Baseline

• Aiming for a churn rate below 2.5% and a user time spent on site of 17 minutes for media or User-Generated Content (UGC) platforms is considered normal.
• Only a small percentage (2.5%) of people will interact with content, and most (65%) will stop using a mobile app within 90 days.
• Strive to reach established target metrics, not adjust targets to current performance levels.
• Optimisation efforts usually have diminishing returns, indicating when it's time to shift focus to a different metric.
• Achieving local maxima and seeing diminishing returns on improvements can serve as a baseline and indicate when to shift focus to other areas.

Part IV. Putting Lean Analytics to Work

1. Intrapreneurs need to have the responsibility and authority for change, ideally with an executive sponsor's support.
2. They must have access to both internal resources and real customers.
3. The team should be small, agile, and composed of high performers who are comfortable with risk.
4. Utilise on-demand technologies for flexibility and to handle rapid changes.
5. Maintain disciplined, simple, and consistent reporting without hiding anything from the organisation.
6. Consider the total cost of innovation and be open to choosing new suppliers when necessary.
7. Streamline the testing process and use reliable components for the new product.
8. Engage directly with end users and "eat your own dog food".
9. Establish clear goals and success criteria before initiating the project.
10. Limit access to the team by outsiders and reward performance based on results.

• Intrepreneurs need to get executive buy-in.