Review

Ex-Amazonians I’ve met speak highly of how the company operated. Sometimes it’s evident that they miss working in that way. It is not uncommon for them to join elsewhere and quickly lobby for the adoption of some of the methods in tis book. When a new executive who used to work at Amazon joined a company I was working for, water cooler conversations confirmed he was asking everyone to write narratives, and read them in silence at the start of meetings. That concept was completely alien to us.

This book does an OK job of introducing some of the key working practices that set Amazon apart from others. It also gives you an insight into some of the biggest moments in the companies history.

You Might Also Like:

Book Review and Summary: Hacking Growth by Sean Ellis and Morgan Brown

Get a comprehensive review and summary of the best-selling book "Hacking Growth" by Sean Ellis and Morgan Brown, the growth experts and authors. Learn about the key principles and practices for driving rapid and sustainable growth in your business

andrewclark.co.uk

Book Review and Summary: The Mythical Man Month by Fred Brooks

Get a comprehensive review and summary of the classic book "The Mythical Man Month" by Fred Brooks, the software engineering and management expert. Learn about the key principles and practices for managing software development projects and teams effectively.

andrewclark.co.uk

Product Operations

Prior to this book, there wasn’t a definitive Product Operations text. Product Operations was only loosely defined in a few blog posts and job descriptions. Finally, we have a stake in the ground. Something to reference or disagree with.

andrewclark.co.uk

Key Takeaways

The 20% that gave me 80% of the value.

Leadership Principles and Mechanisms

• Underpromise and overdeliver to ensure customer expectations are exceeded
• Jeff set challenging standards, demanding people work long, hard and smart
• Jeff role modelled customer obsession, innovation, frugality, personal ownership, bias for action and high standards to his early leadership team. But soon that wouldn’t scale.

• If you don’t change the underlying condition that created a problem, you should expect the problem to recur.
• The Annual Planning Process
• Autonomous single threaded teams paired with precise goal-setting to align to company goals.
• Planning takes 4-8 weeks.
• S-team creates high-level expectations or objectives for the entire company
• Each group then works on its bottom up Operating Plan (known as OP1). It includes:
• Assessment of past performance (goals achieved, missed and lessons learned),
• Key initiatives for the following year
• An income statement
• Resource requests (hires, marketing, equipment, fixed assets)
• OP1s are presented to a senior panel that reconcile the top down with the bottom up and reconcile gaps. This process is called OP2 and lays out the plan for the year.
• OP2 aligns the groups to the company goals. OP2 makes it clear what each group has committed to, how they’ll achieve it and what resources it will take.
• Any change to OP2 requires formal S-Team approval
• The S-Team goals process: (S-team: SVPs and Direct Reports to Jeff Bezos)
• During OP1 the most important initiatives and goals from each team are selected and labeled ‘S-Team goals’.
• For Example: The music team might set 23 goals, but the S-Team would adopt say 6 of those. The music team will then treat them as a priority.
• S-Team goals are input-focused metrics that measure specific activities teams need to perform → if they do → Amazon will achieve the desired business results.
• Amazon’s aligns compensation to what’s best for customers and the company over the long term. The bulk of compensation is in longterm stock options.

Hiring: Bar Raising Process

• Stakes are high when you hire somebody, traditional companies don’t put enough rigour into it
• The goal of the bar raisers is to create a scalable, repeatable, formal process for consistently making successful hiring decisions
• Amazon Bar Raisers get special training. One participates in every interview loop.
• As the name implies, each new hire should raise the bar, in one or more important ways
• The Bar Raisers can veto any hire and override the hiring manager
• Bar Raisers are picked for interviewing skills and ability to asses talent, adherence to high standards, credibility and leadership capability
• Managers put effort into finding better candidates not changing the process
• The interview process is designed to assess how past behaviour of candidates and their ways of working map to the leadership principles
• The Amazon hiring process has a flywheel effect, greater dividends the longer its used.

Organising, Separable, Single-Threaded Leadership

• Get clear accountability and leadership of each initiative in place
• Velocity (a measure of speed and direction) matters in business
• Innovation is a function of experiments per unit of time.
• Amazon struggled with dependencies (something one team needs but can’t supply for itself). To handle dependencies you need lots of coordination which slows things down. Soon you’re communicating more than building.
• Better coordination isn’t the answer. Don’t refine cross-team communication, eliminate it
• Single-Threaded Leadership: Each initiative has a single person responsible. That person can't have any other responsibilities. They operate an autonomous team (ideally free from dependencies).
• You have to untangle monolithic software and organisational structures to do this
• To reduce dependencies Amazon introduced an API mandate for all systems/services
• Two pizza teams are:
• Small, no more than 10 people
• Autonomous. No need to coordinate with other teams
• Evaluated by a well-defined “fitness function.” A number that’s a combination of a series of weighted metrics of a weighted series of metrics.
• Monitored in real time
• Own and responsible for all aspects of its area of focus
• Led by a multi-disciplined top-flight leader
• Self-funding, a team’s work must pay for the team
• Approved in advance by the S-Team
• Team size isn’t as important as having a leader whose sole focus was to get the job done (provided they had the appropriate skills, authority and experience)
• Initially it requires major investment. Each team starts out with dependencies, they must resist the urge to build new things and instead remove their dependencies and learn to instrument their success metrics. But expect improved agility, developer productivity, scalability, and a better ability to resolve and recover from outages and failures.
• The real benefit of microservices is that they enable you to establish small, autonomous teams that can assume a level of ownership of their code that isn’t possible with a monolithic approach
• Single-Threaded-Leader / Single-Threaded-Teams are the answer:
• Separable means almost as separable organisationally as APIs are for software. Single-threaded means they don’t work on anything else
• Clear unambiguous ownership of specific features or functionality and can drive innovations with a minimum of reliance or impact upon others
• single-threaded teams have far fewer organisational dependencies than conventional teams
• A team has sufficient autonomy if it can build and roll out changes without coupling, coordination, and approvals from other teams
• Single Threaded Teams unlock velocity through innovation, clarity in accountability and keep teams with a narrow focus.

The best way to fail at inventing something is by making it somebody’s part-time job Dave Limp. Amazon.

Communicating: Narratives and the Six-Pager

• The 6 pager: used to describe, review, or propose just about any type of idea, process, or business.
• Decision making meetings that use PowerPoint aren’t efficient or effective. Proposals at Amazon were necessarily complex, interconnected, required lots of information to explore, and had greater and greater consequences.It’s hard when to know how to give feedback in a PowerPoint presentation. Sometimes a question sets a group off in another direction, rendering the rest of presentation useless.
• Good slide decks follow the 6x6 rule. A maximum of 6 bullet points per slide, each with no more than 6 words. You need to limit information density, which is the problem. It requires that a proficient speaker adds the detail back in.

For serious presentations… it will be useful to replace PowerPoint slides with paper handouts showing words, numbers, data graphics, images together. High-resolution handouts allow viewers to contextualise, compare, narrate, and recast evidence. In contrast, data-thin, forgetful displays tend to make audiences ignorant and passive, and also to diminish the credibility of the presenter… Edward Tufte · The Cognitive Style of PowerPoint

• Standard format:
• Max Length 6 pages (no formatting tricks). An Appendix for supporting detail.
• You can cover so much more ground in the same time:
• Documents have 9x the information per page vs a slide
• People read 3x faster than people can present information
• Narratives enable executives to cover more ground in a day
• A good narrative anticipates objections and concerns of the readers.
• There’s no one format, headings, subheadings, graphs and data tables are all specific to the individual narrative.
• Two optional sections:
1. Call out one or more key tenets → the foundational reasoning that led you to this point. Establishing these helps anchor the reader. It’s best to dispute the tenants first if you disagree (as the rest of the proposal may not be relevant)
2. The FAQ. Anticipate counterarguments, points of contention, or statements that might be easily misinterpreted. Address these in the FAQ.
• Meeting Structure: Start meetings with silent reading of the doc. 60 minute meeting → 20 mins of reading, 40 of discussion (6 page). Leave two-thirds of the meeting time for discussing what we’ve read.Resist the temptation to read or present the doc.
• Expect audience members to ask questions, seek clarification, probe intentions, offer insights, and suggest refinements or alternatives
• Jeff arrived at unique insights by reading the same documents as others in the room. His approach was to assume each sentence is wrong until he can prove otherwise.
• The audience must objectively and thoroughly evaluate the idea and suggest ways to improve it.
• You should leave a meeting with an idea everyone stand behinds
• Silence is agreement, and is as much of a signal as loud critique.

Working Backwards: Start with the Desired Customer Experience

• Working Backwards is a systematic way to vet ideas and create new products.
• You start by defining the customer experience, then iteratively work backwards from that point until the team achieves clarity of thought around what to build.
• The tool that brings working backwards to life is the PR/FAQ (Press release / Frequently Asked Questions)
• Two key elements of the Working Backwards process:
• starting from the customer experience
• working backwards from that by writing a press release (announces the product and anticipates tough questions)
• The focus was less on the P&L and projected market segment share, but more about the service itself, the experience, and which products and services we thought would appeal most to the customer
• Thinking of product concepts as a press release became the process at Amazon
• Typically companies think up products or businesses that will be great for the company, then shoehorn those into unmet customer needs at the end, when they come to announce it. Amazon was trying to do the opposite.
• The PR gives the reader the highlights of the customer experience. The FAQ provides all the salient details of the customer experience.
• FAQs should include internal and external questions.
• External: anything you’d expect from the press and customers
• Internal: anything you’d expect from your team or leadership
• The product should be meaningfully better (faster, easier, cheaper) than what’s out there or it’s not worth building
• The press release (PR) portion should be a few paragraphs, and always less than one page.
• The frequently asked questions (FAQ) should be five pages or less. There are no awards for extra pages or more words
• A press release is about boiling things down the essence, keeping it short is the forcing function that helps us work out what’s important
• Book a one hour meeting with key stakeholders to review. Take onboard feedback, and make changes before going to leadership.
• Over time, leadership become skilled evaluators. This creates a flywheel of good decision making.
• Press Release Components:

• FAQ Components

• Product Managers will spend a lot of time working on ideas that won’t go to market. That’s better than building things nobody wants.

Metrics

• Manage Your Inputs, Not Your Outputs
• Focus on controllable input metrics. Focus on the activities which affect output metrics.
• Output Metrics are metrics you have little ability to control directly (like share price)
• Example Amazon metric: The ‘backlog’ is a tally of the orders taken minus shipments made.
• The Weekly Business Review purpose was to provide a more comprehensive lens through which to see the business
• The life cycle of the metric follows the 6Sigma DMAIC process. Go through the lifecycle in order will save you time.

• When you encounter a problem, the probability you’re actually looking at the actual root cause of the problem in the initial 24 hours is pretty close to zero

The Weekly Business Review

• The Weekly Business Review (WBR) is where metrics are put into action.
• The deck represents a data-driven, end-to-end view of the business.
• Over time modify, add, and remove metrics based on the strength and quality of the signal each emits
• Emerging patterns are a key point of focus. The goal is to highlight challenges as they emerge (rather than waiting for the yearly results)
• WBR meetings have an intense customer focus, deep dives into complex challenges, and insistence on high standards and operational excellence.
• Keep operational and strategic discussions separate. The WBR is a tactical operational meeting to analyse performance trends of the prior week.
• On Presentation of data:
• Graphs plot results against comparable prior periods.
• Anecdotes and exception reporting are woven into the deck.
• Use consistent and familiar formatting and sequence of data to speed interpretation. Graph design, time periods covered, colour palette, etc. The team gets faster at spotting trends, anomalies stand out more distinctly
• Focus on variances and don’t waste time on the expected. “Nothing to see here, move along”
• Business owners should be prepared to explain variances in their metrics.
• Dig into a meaningful variation that needs more attention
• Mistakes should be acknowledged as a chance to take ownership, understand the root cause, and learn from the experience
• Keep transitions between presenters slick
• Adding growth rates can help you spot trends.
• Input metrics are needed to explain output metrics. For example revenue is less informative than new customers, new customer revenue and existing customer revenue. Now we can tell better what’s going on
• Combine data with anecdote to tell the whole story.
• Keep an eye on the number of metrics and attendees, they only grow without oversight
• Variation in data is normal. And unavoidable. It’s therefore critical to differentiate normal variation (noise) from some fundamental change or defect in a process (signal).
• Attaching meaning to variations within normal bounds is misleading and wastes effort
• Understanding what’s normal is the responsibility of the metrics owner
• Input metrics often describe things customers care about, such as low prices, lots of available products, fast shipping, few customer service contacts, and a speedy website or app
• It’s important to openly discuss failures, learn from them, and look for inventions that will delight customers even more

Deep Summary

Longer form notes, typically condensed, reworded and de-duplicated.

Building Blocks: Leadership Principles and Mechanisms

• Jeff Bezos setup Amazon to capitalise on the amazing annual growth in internet usage 2300%
• Books were well suited to an online store:
• Easy to store and ship
• Physical stores were limited in range
• Two large publishers had electronic catalogues that could be leveraged
• Personalised recommendations could be powerful
• Initially Jeff could oversee everything, passionate about making every customer interaction perfect.
• Underpromise and overdeliver to ensure customer expectations were exceeded
• Jeff set challenging standards, demanding people work long, hard and smart
• In the early years, Jeff could transmit his message directly to a small leadership team. Principles of customer obsession, innovation, frugality, personal ownership, bias for action and high standards.
• A founder transmitted ethos doesn’t scale beyond 500 people. After that you need a system for hiring and value transmission
• Amazon’s leadership principles are deeply ingrained in every significant process and function at the company
• Amazon’s 14 Leadership Prinicples
1. Customer Obsession. Leaders start with the customer and work backwards. They work vigorously to earn and keep customer trust. Although leaders pay attention to competitors, they obsess over customers.
2. Ownership. Leaders are owners. They think long term and don’t sacrifice long-term value for short-term results. They act on behalf of the entire company, beyond just their own team. They never say “that’s not my job.”
3. Invent and Simplify. Leaders expect and require innovation and invention from their teams and always find ways to simplify. They are externally aware, look for new ideas from everywhere, and are not limited by “not invented here.” As we do new things, we accept that we may be misunderstood for long periods of time.
4. Are Right, A Lot. Leaders are right a lot. They have strong judgment and good instincts. They seek diverse perspectives and work to disconfirm their beliefs.
5. Learn and Be Curious. Leaders are never done learning and always seek to improve themselves. They are curious about new possibilities and act to explore them.
6. Hire and Develop the Best. Leaders raise the performance bar with every hire and promotion. They recognize exceptional talent, and willingly move them throughout the organization. Leaders develop leaders and take seriously their role in coaching others. We work on behalf of our people to invent mechanisms for development like Career Choice.
7. Insist on the Highest Standards. Leaders have relentlessly high standards — many people may think these standards are unreasonably high. Leaders are continually raising the bar and drive their teams to deliver high quality products, services, and processes. Leaders ensure that defects do not get sent down the line and that problems are fixed so they stay fixed.
8. Think Big. Thinking small is a self-fulfilling prophecy. Leaders create and communicate a bold direction that inspires results. They think differently and look around corners for ways to serve customers.
9. Bias for Action. Speed matters in business. Many decisions and actions are reversible and do not need extensive study. We value calculated risk taking.
10. Frugality. Accomplish more with less. Constraints breed resourcefulness, self-sufficiency, and invention. There are no extra points for growing headcount, budget size, or fixed expense.
11. Earn Trust. Leaders listen attentively, speak candidly, and treat others respectfully. They are vocally self-critical, even when doing so is awkward or embarrassing. Leaders do not believe their or their team’s body odor smells of perfume. They benchmark themselves and their teams against the best.
12. Dive Deep. Leaders operate at all levels, stay connected to the details, audit frequently, and are skeptical when metrics and anecdote differ. No task is beneath them.
13. Have Backbone; Disagree and Commit. Leaders are obligated to respectfully challenge decisions when they disagree, even when doing so is uncomfortable or exhausting. Leaders have conviction and are tenacious. They do not compromise for the sake of social cohesion. Once a decision is determined, they commit wholly.
14. Deliver Results. Leaders focus on the key inputs for their business and deliver them with the right quality and in a timely fashion. Despite setbacks, they rise to the occasion and never settle.
15. Strive to be Earth’s Best Employer. Leaders work every day to create a safer, more productive, higher performing, more diverse, and more just work environment. They lead with empathy, have fun at work, and make it easy for others to have fun. Leaders ask themselves: Are my fellow employees growing? Are they empowered? Are they ready for what’s next? Leaders have a vision for and commitment to their employees’ personal success, whether that be at Amazon or elsewhere.
16. Success and Scale Bring Broad Responsibility. We started in a garage, but we’re not there anymore. We are big, we impact the world, and we are far from perfect. We must be humble and thoughtful about even the secondary effects of our actions. Our local communities, planet, and future generations need us to be better every day. We must begin each day with a determination to make better, do better, and be better for our customers, our employees, our partners, and the world at large. And we must end every day knowing we can do even more tomorrow. Leaders create more than they consume and always leave things better than how they found them.
• Good intentions don’t work mechanisms do.
• If you don’t change the underlying condition that created a problem, you should expect the problem to recur.
• Foundational mechanisms:
• The annual planning process
• Autonomous single threaded teams paired with precise goal-setting to align to company goals
• Planning takes 4-8 weeks.
• S-team creates high-level expectations or objectives for the entire company
• Each group then works on its Operating Plan (known as OP1). This is a bottom up proposal.
• Assessment of past performance (goals achieved, missed and lessons learned)
• Key initiatives for the following year
• A detailed income statement
• Resource requests (hires, marketing, equipment, fixed assets)
• The OP1s are presented to a senior panel that reconcile the top down with the bottom up and reconcile gaps. This process is called OP2 and lays out the plan for the year. OP2 aligns the groups to the company goals. OP2 makes it clear what each group has committed to, how they’ll achieve it and what resources it will take.
• Any change to OP2 requires formal S-Team approval
• The S-Team goals process (S-team: SVPs and Direct Reports to Jeff Bezos)
• During OP1 the most important initiatives and goals from each team are selected and labeled ‘S-Team goals’.
• For Example: The music team might set 23 goals, but the S-Team would adopt say 6 of those. The music team will then treat them as a priority.
• Amazon’s compensation plan, which aligns incentives with what’s best for customers and the company over the long term.
• The process generates a large number of goals, with high detail and aggressiveness.
• S-Team goals are input-focused metrics that measure specific activities teams need to perform → if they do → Amazon will achieve the desired business results.
• Expect just 75% to be fully achieved.
• Progress is reported and tracked centrally by finance, thorough reviews are conducted quarterly on a rolling basis throughout the year (not all at the same time).
• Amazon leaders toil over execution details and regularly embody the deep dive leadership principle.
• The OP process aligns the entire company on what’s truly important to accomplish for the year
• The bulk of compensation is in longterm stock options, with salaries capped at $160k, creating a longterm mindset that aligns with customers and shareholders
• They might lose some short-term thinking people to higher offers from elsewhere

Hiring: Bar Raising Process

• We want missionaries not mercenaries
• The stakes are high when you hire somebody, traditional companies don’t put enough rigour into hiring
• Urgency bias: can apply to the whole process
• Confirmation bias, social pressure and group think: can affect the ultimate decision
• Personal bias: people want to surround themselves with people like them
• A lack of a formal process and training can really impact your hiring process.
• The ‘bar raiser’ process is one of Amazon’s first and most successful scalable, repeatable and teachable operational practices
• There becomes a point when you have new people, hiring new people, hiring new people. You need a process
• The Bar Raiser Solution
• Goal: scalable, repeatable, formal process for consistently making successful hiring decisions
• Designed to provide a hiring framework, minimise variability of ad hoc hiring processes and improve results
• Amazon Bar Raisers get special training. One participates in every interview loop.
• As the name implies, each new hire should raise the bar, in one or more important ways
• The Bar Raisers can veto any hire and override the hiring manager
• Bar Raisers are picked for interviewing skills and ability to asses talent, adherence to high standards, credibility and leadership capability
• People asked for exceptions, but none were allowed. Hiring always feels urgent.
• Managers put effort into finding better candidates not changing the process
• 8 Step Process:

• The Amazon hiring process has a flywheel effect, greater dividends the longer its used.

Organising, Separable, Single-Threaded Leadership

• Get clear accountability and leadership of each initiative in place
• Velocity (a measure of speed and direction) matters in business
• Innovation is a function of experiments per unit of time.
• As companies scale, they can suffer from bureaucratic drag.
• Single-Threaded Leadership: Each initiative has a single person responsible. That person can't have any other responsibilities. They operate an autonomous team (ideally free from dependencies).
• To do this requires untangle our monolithic software and organisational structures. You need to replace them step by step and optimise for speed of innovation
• Amazon struggled with dependencies (something one team needs but can’t supply for itself).
• To handle dependencies you need lots of coordination which slows things down. Soon you’re communicating more than building.
• Dependencies disempower teams and reduce their ability to make an impact.
• Amazon identified its monolithic tech stack as a bottle neck (monolith: functionality residing in a massive executable program).
• Teams move faster in areas where they don’t have dependencies.
• Organisational dependencies can be just as debilitating as technical ones.
• The dispiriting second-order effect of dependencies is disempowered teams.
• Better coordination isn’t the answer. Don’t refine cross-team communication, eliminate it
• When you think about eliminating communication your problems look different
• To reduce dependencies Amazon introduced an API mandate for all systems/services:
• API: a set of routines, protocols, and tools for building software applications and defining how software components should interact.
• Aim for loosely coupled interaction via machines through well-defined APIs ( vs humans through emails and meetings)
• Amazon tried something called New Project Initiatives (global prioritisation). It didn’t work.
• Teams submitted lengthy proposals for projects. They include teams impacted, summary of idea, consumer adoption model, P&L and why it was important
• A small leadership group screened all the submissions. They immediately rule out some.
• More promising ideas go through detailed technical and financial scoping. 30-40 people no hand to answer any questions.
• The leadership group would then decide what projects would go forward. Each team leader got an email saying what projects were approved, rejected and what other projects were dependent on them
• Projects had wide development estimates, and consumer response was always hard to predict. Assumptions were logged and revisited later to keep teams accountable for estimates and forecasts.
• Then they coined the two pizza rule
• Don’t manage dependencies, remove them
• Reorganise engineers into smaller teams that can become autonomous
• Coordinate less and build more
• Initially it was hoped teams would be no larger than the number of people that could be adequately fed by two large pizzas
• Two pizza teams are:
• Small, no more than 10 people
• Autonomous. No need to coordinate with other teams
• Evaluated by a well-defined “fitness function.” A number that’s a combination of a series of weighted metrics of a weighted series of metrics.
• Monitored in real time
• Own and responsible for all aspects of its area of focus
• Led by a multi-disciplined top-flight leader
• Self-funding, a team’s work must pay for the team
• Approved in advance by the S-Team
• Amazon concluded a service-oriented architecture would give a level of isolation that would allow teams to build many software components rapidly and independently. If multiple teams have direct access to a shared block of software code or some part of a database, they slow each other down.
• Ownership of code and databases are given to a team. If you want to another teams’, you use the API or submit a well-documented request to extend the API
• Initially it requires major investment. But expect improved agility, developer productivity, scalability, and a better ability to resolve and recover from outages and failures.
• The real benefit of microservices is that they enable you to establish small, autonomous teams that can assume a level of ownership of their code that isn’t possible with a monolithic approach
• Autonomous teams need to be pointed in the right direction and quickly course-correct when warranted
• Each team starts out with dependencies, they must resist the urge to build new things and instead remove their dependencies and learn to instrument their success metrics
• Make decisions with about 70% of the information you wish you had, waiting for 90% slows you down. You’re better off executing at 70% and correcting bad decisions.
• Think of cross-team projects as a kind of tax. Minimise them if they can’t be avoided altogether.
• The problems with two pizza teams:
• They only work in product engineering teams (not in finance in legal)
• Don’t use fitness functions: they’re worse that the underlying metrics. You spend too much time debating them, changing the weights breaks historical comparisons, and they obscure what’s causing them to modulate
• Most of the value is in the up-front work to agree on success metrics for a team
• Team size isn’t as important as having a leader whose sole focus was to get the job done (provided they had the appropriate skills, authority and experience)
• Two Pizza Teams → Single-Threaded-Leader / Single-Threaded-Teams

The best way to fail at inventing something is by making it somebody’s part-time job Dave Limp. Amazon.

• Separable means almost as separable organisationally as APIs are for software. Single-threaded means they don’t work on anything else
• Clear unambiguous ownership of specific features or functionality and can drive innovations with a minimum of reliance or impact upon others
• single-threaded teams have far fewer organisational dependencies than conventional teams
• A team has sufficient autonomy if it can build and roll out changes without coupling, coordination, and approvals from other teams
• Single Threaded Teams unlock velocity through innovation, clarity in accountability and keep teams with a narrow focus.

Communicating: Narratives and the Six-Pager

• There are two types of ‘narratives’ at Amazon:
1. The 6 pager: used to describe, review, or propose just about any type of idea, process, or business.
2. The PR/FAQ: used in the Working Backwards process for product development
• This chapter is about the 6 pager.
• Decision making meetings that use PowerPoint aren’t efficient or effective:
• The format made deep dives frustrating, inefficient, and error prone for both the presenter and the audience.
• An Edward Tufte (the father of data visualisation) essay influenced their thinking.
• The Cognitive Style of PowerPoint: Pitching Out Corrupts Within

As analysis becomes more causal, multivariate, comparative, evidence based, and resolution-intense…..the more damaging the bullet list becomes. Edward Tufte · The Cognitive Style of PowerPoint

• Proposals at Amazon were necessarily complex, interconnected, required lots of information to explore, and had greater and greater consequences.
• The linear piecemeal nature of PowerPoint makes referencing connected ideas hard, you can’t include enough text to fully express an idea
• PowerPoint strips the discussion of important nuance
• It’s hard when to know how to give feedback in a PowerPoint presentation. Sometimes a question sets a group off in another direction, rendering the rest of presentation useless.

For serious presentations… it will be useful to replace PowerPoint slides with paper handouts showing words, numbers, data graphics, images together. High-resolution handouts allow viewers to contextualise, compare, narrate, and recast evidence. In contrast, data-thin, forgetful displays tend to make audiences ignorant and passive, and also to diminish the credibility of the presenter… Making this transition in large organisations requires a straightforward executive order: From now on your presentation software is Microsoft Word, not PowerPoint. Get used to it Edward Tufte · The Cognitive Style of PowerPoint

• Removing PowerPoint and introducing 6 page narratives was not well received. Especially by those that were due to present soon.
• The format puts more emphasis on the idea, and reduces the emphasis on a dynamic presenter
• Good slide decks follow the 6x6 rule. A maximum of 6 bullet points per slide, each with no more than 6 words. You need to limit information density, which is the problem. It requires that a proficient speaker adds the detail back in.
• It’s harder to write a good 4 page memo than a 20 page powerpoint. The structure of a good memo requires you to think through what’s important and what’s related.
• Standard format:
• Max Length 6 pages (no formatting tricks). An Appendix for supporting detail.
• You can cover so much more ground in the same time:
• Documents have 9x the information per page vs a slide
• People read 3x faster than people can present information
• Narratives enable executives to cover more ground in a day
• The narrative forces greater clarity of thought from the proposer. The format allows you to show how you’ve carefully weighed and thoroughly analysed.
• A good narrative anticipates objections and concerns of the readers.
• Try the writing technique: Stage, support and conclude.
• Connect the dots for the reader - don’t just dump a list of bullet points

• Two optional sections:
1. Call out one or more key tenets → the foundational reasoning that led you to this point. Establishing these helps anchor the reader. It’s best to dispute the tenants first if you disagree (as the rest of the proposal may not be relevant)
2. The FAQ. Anticipate counterarguments, points of contention, or statements that might be easily misinterpreted. Address these in the FAQ.
• There’s no one format, headings, subheadings, graphs and data tables are all specific to the individual narrative.
‣
Example headings from An Amazon quarterly business review:
• Introduction
• Tenets
• Accomplishments
• Misses
• Proposals for Next Period
• Headcount
• P&L
• FAQ
• Appendices (includes things like supporting data in the form of spreadsheets, tables and charts, mock-ups)
• A 6 pager can explore any argument or idea you want to present to a group of people
• Learn by reviewing and emulating successful examples
• Meeting Structure:
• Start meetings with silent reading of the doc.
• 60 minute meeting → 20 mins of reading, 40 of discussion (6 page)
• 30 minute meeting → 10 mins of reading, 20 of discussion (3 page)
• Leave two-thirds of the meeting time for discussing what we’ve read
• It takes 3 minutes to read a page, so shorter meetings require shorter narrative docs
• Resist the temptation to read or present the doc
• Two discussion structures:
1. Collect high-level feedback first, then then pore over the document line by line.
2. Ask single individual to give all their feedback on the entire document, then ask the next person in the audience to do the same
• Expect audience members to ask questions, seek clarification, probe intentions, offer insights, and suggest refinements or alternatives
• The goal is to seek the truth about the proposed idea or topic.
• Allocate a knowledgable note taker
• Jeff arrived at unique insights by reading the same documents as others in the room. His approach was to assume each sentence is wrong until he can prove otherwise.
• This thinking ended up changing the return/refund policy. Amazon processes refunds before goods are received
• Narratives increase the quantity and quality of effective communication
• Good narratives take many days to write. Circulate for review, iterate and repeat before exposing your best effort to peers
• The audience must objectively and thoroughly evaluate the idea and suggest ways to improve it.
• You should leave a meeting with an idea everyone stand behinds
• Silence is agreement, and is as much of a signal as loud critique.

Working Backwards: Start with the Desired Customer Experience

• Working Backwards is a systematic way to vet ideas and create new products.
• You start by defining the customer experience, then iteratively work backwards from that point until the team achieves clarity of thought around what to build.
• The tool that brings working backwards to life is the PR/FAQ (Press release / Frequently Asked Questions)
• Execs at Amazon had to do a lot of context switching. They quickly need to get up to speed, understand the most important issues, then map out a trajectory before the next meeting
• Amazon iterated on how to do this, but the principle of customer obsession and became the right thing to focus on
• Two key elements of the Working Backwards process:
• starting from the customer experience
• working backwards from that by writing a press release (announces the product and anticipates tough questions)
• Jeff used to ask for mock-ups early in a project. If provided he would probe teams on the detail. Jeff didn’t want you to ask approval until you knew what the customer experience would be and how it would be better for customers than the competition.
• half-baked mock-ups were evidence of half-baked thinking
• Writing up ideas is hard work, but meetings changed. There was more meat and more detail to discuss, so the sessions were livelier and longer.
• The focus was less on the P&L and projected market segment share, but more about the service itself, the experience, and which products and services we thought would appeal most to the customer
• Thinking of product concepts as a press release became the process at Amazon
• Typically companies think up products or businesses that will be great for the company, then shoehorn those into unmet customer needs at the end, when they come to announce it. Amazon was trying to do the opposite.
• The Kindle originated from a PR/FAQ doc. They wrote the press release and started working backwards, the shape of the project changed. The focus became about what would be great for customers, and they had to invent multiple solutions to customer problems
• Prior to PR/FAQs exec conversations were focused on technology challenges, business constraints, sales and financial projections, and marketing opportunities.
• FAQs should include internal and external questions.
• External: anything you’d expect from the press and customers
• Internal: anything you’d expect from your team or leadership
• The FAQ section is where the writer shares the details of the plan from a consumer point of view and addresses the various risks and challenges from internal operations, technical, product, marketing, legal, business development, and financial points of view
• When executives review features in the PR expect them to ask … “so what?”
• The product should be meaningfully better (faster, easier, cheaper) than what’s out there or it’s not worth building
• The PR gives the reader the highlights of the customer experience. The FAQ provides all the salient details of the customer experience.
• Expect to write many drafts. The PR/FAQ helps create a framework for getting and taking onboard feedback
• PR/FAQ boundaries:
• The press release (PR) portion should be a few paragraphs, and always less than one page.
• The frequently asked questions (FAQ) should be five pages or less. There are no awards for extra pages or more words
• A press release is about boiling things down the essence, keeping it short is the forcing function that helps us work out what’s important
• Book a one hour meeting with key stakeholders to review
• Everyone reads it
• The writer asks for general feedback
• The most senior attendees tend to speak last
• After high-level responses, the writer asks for specific comments, line by line, paragraph by paragraph.
• Everyone engages in intense debate and discussion of the key ideas and the way they are expressed.
• The writer takes onboard feedback, and makes changes before going to leadership. The more smart revisions you make the more chance you’ll get funded
• Over time, leadership become skilled evaluators. This creates a flywheel of good decision making.
• Press Release Components:

• FAQ Components

• Product Managers will spend a lot of time working on ideas that won’t go to market
• Only the best get prioritised and resourced. And that’s the whole point. A lot of time and money is saved by filtering out the bad ideas early.
• The PR/FAQ helps teams understand the constraints and problems that would prevent a new product idea from being viable
• Bringing clarity to why you aren’t doing something is important too
• To counterbalance, you still need to take on hard problems if solving them would unlock substantial value.
• The PR/FAQ should be a living document and be updated once the project is underway

Metrics

• Manage Your Inputs, Not Your Outputs
• Focus on controllable input metrics. Focus on the activities which affect output metrics.
• Output Metrics are metrics you have little ability to control directly (like share price)
• It is common for companies to pay attention to the wrong signals.
• Amazon executives would review 3 pages of metrics when struggling through a holiday season:
• The ‘backlog’ is a tally of the orders taken minus shipments made.
• The Weekly Business Review purpose was to provide a more comprehensive lens through which to see the business
• The life cycle of the metric follows the 6Sigma DMAIC process. Go through the lifecycle in order will save you time.

• Don’t confuse the lack of emphasis on output metrics to be a lack of interest. Amazon cares about hits profit and revenue, but it knows it can’t affect them by talking about them alone.

‣

The famous Amazon flywheel, shows the power of focusing on input metrics:

• When you encounter a problem, the probability you’re actually looking at the actual root cause of the problem in the initial 24 hours is pretty close to zero

The WBR: Metrics at Work

• The Weekly Business Review (WBR) is where metrics are put into action.
• Each meeting begins with the virtual or printed distribution of the data package (up-to-date snapshot of graphs, tables, and occasional explanatory notes to all metrics)
• The deck represents a data-driven, end-to-end view of the business.
• Follow the customer experience.
• Flow from topic to topic can reveal the interconnectedness of activities.
• Over time modify, add, and remove metrics based on the strength and quality of the signal each emits
• Emerging patterns are a key point of focus.
• The goal is to highlight challenges as they emerge (rather than waiting for the yearly results)
• WBR meetings have an intense customer focus, deep dives into complex challenges, and insistence on high standards and operational excellence.
• Most senior executives review the full WBR deck of metrics.
• Keep operational and strategic discussions separate. The WBR is a tactical operational meeting to analyse performance trends of the prior week.
• On Presentation of data:
• Graphs plot results against comparable prior periods.
• Anecdotes and exception reporting are woven into the deck.
• Use consistent and familiar formatting and sequence of data to speed interpretation. Graph design, time periods covered, colour palette, etc. The team gets faster at spotting trends, anomalies stand out more distinctly
• Focus on variances and don’t waste time on the expected. “Nothing to see here, move along”
• Business owners should be prepared to explain variances in their metrics. If you can’t provide a crisp explanation, say ‘I don’t know but I will find out’
• It’s not the Inquisition BUT it’s okay to dig into a meaningful variation that needs more attention, and to point out when high standards have not been met.
• Mistakes should be acknowledged as a chance to take ownership, understand the root cause, and learn from the experience
• Keep transitions between presenters slick
• Anatomy of a Metrics Chart
• Zoom in: Place trailing 6 weeks and trailing 12 months side by side on the same x-axis.
‣
This adds a zoom function to a static graph

‣
Watch YOY trends by plotting growth rates.

• Remember output metrics show results, but input metrics provide guidance and insight
• Input metrics are needed to explain output metrics. For example revenue is less informative than new customers, new customer revenue and existing customer revenue. Now we can tell better what’s going on
• Most charts show goals for metrics.
• Combine data with anecdote to tell the whole story. Customer service collects and summarises customer feedback. Often the anecdotes tell us things the data can’t.
• Jeff B enabled customer service to stop products from being sold if they generated a high number of returns
• In the WBR keep an eye on the number of metrics and attendees, they only grow without oversight. Make sure teams are finding the signal in the noise with their metrics.
• Make sure the session doesn’t become about point scoring, make sure it’s OK to talk about mistakes.
• Variation in data is normal. And unavoidable. It’s therefore critical to differentiate normal variation (noise) from some fundamental change or defect in a process (signal).
• Attaching meaning to variations within normal bounds is misleading and wastes effort
• Understanding what’s normal is the responsibility of the metrics owner
• Statistical methods like XMR control charts can help
• Input metrics often describe things customers care about, such as low prices, lots of available products, fast shipping, few customer service contacts, and a speedy website or app
• Customers often don’t care about output metrics (revenue and free cash flow)
• It’s important to openly discuss failures, learn from them, and look for inventions that will delight customers even more