Use a smaller tuition number (say $1,000)
Here’s what this tweaked xyz1 would look like:
xyz1B: At least 40% of Google engineers commuting from San Francisco to Mountain View who hear about BusU’s $1,000 (not eligible for company reimbursement) courses will visit the BusU4Google.com website and will submit their google.com email address to be informed of upcoming classes.
Or we could move on to pretotyping xyz2 or xyz3. We have several available options open to us. But while we are pondering the next steps something happens . . .
A Lucky Break
I get the following email from a Google employee named Bob:
Hi Alberto, I heard about your BusU idea from my friend Emily with whom I often commute to work. I love the concept, and I’d love to teach a class. I have a PhD in Artificial Intelligence from Berkeley, and I already have a ten-hour “Intro to Machine Learning” minicourse that I’ve taught several times at both Berkeley and Google (to great reviews, average 4.8/5.0). This would be a lot of fun for me, so you don’t even have to pay me. When can we start?
I am thrilled by Bob’s email and by his offer to teach an intro to AI class for free. In fact, it’s almost too good to be true. Remember xyz3?
xyz3: At least 10% of Google engineers commuting from San Francisco to Mountain View will pay $300 for a one-week “Introduction to Artificial Intelligence” class on the bus taught by a Stanford AI professor.
Bob is not a Stanford professor, and Machine Learning is a specific subset of AI, but close enough. This is a major lucky break! But I am not too surprised by it. I have learned to expect things like this to happen when I get out of Thoughtland and begin to pretotype my ideas in the real world.
After I review the data from the first pretotypes and consider this new development, I decide on a new course of action. I meet with Bob and tweak xyz3 as follows:
xyz3A: At least 10% of Google engineers commuting from San Francisco to Mountain View will pay $300 for a one-week “Introduction to Machine Learning” class on the bus taught by a fellow Google employee.
Next, I call several local transportation companies and learn that chartering a suitable 40-person bus (including driver) from San Francisco to Mountain View will cost approximately $1,000 per day, or $5,000 per week. I also learn that some of the buses already have television screens that the instructors can use to project slides or as electronic blackboards. Great.
I calculate that if I can get at least twenty people to sign up (at $300 each) for the one-week AI class, I can cover the bus rental cost, gain some valuable firsthand experience running such a service, and even pay Bob for his time and commitment. Based on YODA from the previous experiment, I estimate that if we email 200 Google employees, we should get at least 10% of them to sign up for such a class. And if we get more, even better—the bus can accommodate up to forty people.
I tell Beth about Bob’s offer and explain our new plan. She’s impressed and even more enthusiastic than before: “I love the idea of employees teaching other employees.” After confirming available dates with professor Bob, I list our first real course on the website and add a registration and an online payment page.
The next day, Beth sends an email to two hundred Google employees on the list describing the class, the $300 cost, and the schedule. The message makes it clear that this is not a Google-sanctioned service, but a pilot program for a new company, and that it does not qualify as a reimbursable educational expense.
In less than two days forty-eight people register and pay! We sell out our first class and even have eight people on the waiting list. That’s a 24% (48 out 200) response rate—considerably better than our 10% estimate for xyz3A. Things are looking really good.
We now have $14,400 in the BusU bank account—that’s 14,400 skin-in-the-game points (300 points for each $300 payment times 48 registered students). That’s a lot of the kind of YODA I like. The initial set of customers are also committing many hours of their time (even more skin in the game), but since they’d be spending that time on the commute anyway, I play it conservatively and decide not to include that in the skin-in-the-game calculation. It’s time to board the bus!
I reserve the charter bus and, two weeks later, at 8:30 a.m. on a rainy Monday morning, the first BusU class leaves San Francisco with thirty-five students on board. Why only thirty-five people? Three people could not make it because of a change of plans—and ask for a refund. And two of them arrive late and miss the bus. Unfortunately, it is too late to get the people on the waiting list to fill in the empty seats. I issue the refunds, and I still have plenty of money to cover the cost of the pretotype. However, if I go forward with the idea, I will have to account for these kinds of situations by coming up with a refund and missed-the-bus policy. Perhaps I can do something similar to what airlines do and overbook the class by 5% or 10%. These are the kinds of invaluable real-world lessons you learn and data you collect by running pretotypes.
On the plus side, the rest of the week goes very smoothly. Thirty-five students finish the one-week course and get the inaugural batch of BusU diplomas.
What’s next? We had a successful first experiment, but we need more data. Specifically, we’d love to know if the initial level of interest in BusU will translate into an ongoing level of interest. Most businesses need repeat customers to be successful and stay profitable, and BusU is no exception. What percentage of our initial customers will sign up for another course?
From New Data to New Decision
It is customary to have students fill out a questionnaire after a course to rate the material and the instructor. In addition to that, such surveys also ask questions like, “Would you take another course from BusU?” and “Would you recommend BusU to your colleagues?” By now, you should realize that the answers to such “Would you . . . ?” questions may provide you with some interesting insights, but they are not data—because there’s no skin in the game. So rather than asking skinless hypothetical questions, Bob and I decide to send an email offer to the first group of students with two skin-in-the-game options for the next class:
Option 1: A $3,000 full ten-week AI course (as in the original plan)
Option 2: A $300 one-week follow-up course, “Machine Learning 201”
We also ask them to share with us any feedback or suggestions they might have.
Two days later, we look at our new data:
Twenty-one people signed up for the follow-up course. That’s an awesome result: more than half the students want to return.
Nobody signed up for the $3,000 ten-week class.
A majority of the students commented that although they enjoyed the morning class and got a lot out of it, the evening session was much tougher to follow. They said that after a full day of work they were tired and all they wanted to do on the ride home was relax. (Bob also confessed that teaching the evening class was much harder, because he was also tired.)
This is a realistic example of the invaluable YODA that you can collect when you take an idea out of Thoughtland and pretotype it. This data speaks loud and clear: $300 one-week classes will be more popular—and much easier to schedule, plan, and sell—than the original idea of a $3,000 ten-week course. Furthermore, we’ve learned that although the morning lectures worked well, the evening ones were a challenge for the students and the teacher, because everyone was tired at the end of the day.
Over the next six weeks we run two more pretotyping experiments with one-week courses, one with Google employees and one with LinkedIn employees. The results from these new experiments align with and confirm xyz3A. This means that our hypothesis can now be promoted to data and that we have our first set of YODA:
YODA1: Out of a sample of 600, 132 (22%) of Google and LinkedIn engineers commuting from San Francisco to Mountain View paid $300 for a one-week BusU course on artificial intelligence.
This YODA comes with almost $40,000 of skin in the game ($300 times 132).
Furthermore, in the process of pretotyping, w
e’ve also acquired additional valuable YODA, including the following:
YODA2: Zero out of 132 (0%) students who completed a one-week $300 BusU course on AI signed up for a $3,000 ten-week course.
YODA3: 48% of students who completed a one-week $300 BusU course on AI signed up for another one-week course at the same price.
YODA4: About 12% of students who sign up will miss the first class and ask for a refund.
YODA5: 89% of the students prefer a morning-only class.
Instead of wasting time writing a BusU business plan based on OPD, opinions, and all sorts of untested assumptions, we invested the time to collect data to demonstrate that there is indeed a business. Before writing a business plan, make sure that there is a business.
What’s Next?
The YODA spoke loud and clear. A significant number of high-tech professionals are interested in BusU’s one-week courses, but not in the longer (and much more expensive) classes. So, at least for now, we abandon the idea of offering ten-week classes for $3,000 (i.e., Idea 1). We decide instead to focus on one-week $300 classes (Idea 2), but with a tweak to the original plan: we will limit the formal lectures to the morning commute, and use the evening commute for more relaxed and informal office hours and Q&A with the instructor.
We update the XYZ and xyz hypotheses and run a few more pretotyping experiments to validate this new model. In the process we collect even more YODA and learn additional valuable lessons (e.g., we can earn an additional $60 per student by selling coffee and snacks on the bus). Here’s what the TRI Meter for the BusU business model for $300 classes (Idea 2) looks like after five experiments:
It looks as if the new version of BusU has a good chance of being The Right It, wouldn’t you agree?
A Few Notes About the BusU Example
Before we close this chapter, I’d like to point out a few important things about this example.
Note 1: In the process of validating our idea, we’ve collected the kind of firsthand data (YODA) with skin in the game that will help us make a very strong business case if we decide to seek VC funding.
Instead of a Thoughtland-based business plan filled with hope, hype, and fantasy five-year financial projections, we can show actual costs, actual revenue, actual profit, and the kind of skin-in-the-game market feedback that is hard to ignore (e.g., 48% of the students who took one class signed up for at least another class).
With several successfully completed courses, we have also demonstrated that we know how to run such a business (i.e., that we can execute the idea competently—build It right).
By investing our own skin in the game (our time and money) to test the idea and by working through the numerous challenges and obstacles, we’ve shown commitment and resiliency.
Finally, by tweaking our original vision and business model to reflect the data we collected, we’ve demonstrated flexibility and agility in responding to the market—two must-have characteristics to succeed in today’s rapidly changing markets.
A team bringing all of the above data and evidence to a potential investor will not only significantly increase their chances of getting funded, but also be in a strong position to ask for a higher valuation.
Note 2: This was a fictional scenario meant to illustrate our methods, but it’s based on a true story. I did have the idea for BusU while stuck in commute traffic and for a period of time I seriously considered starting such a business. Furthermore, everything I’ve described is both plausible and doable: Google does offer free commuter buses to its employees, Google does reimburse some tuition expenses, charter buses can be rented for about $1,000 per day, and I am sure that I would be able to find some AI experts willing to teach the classes.
Note 3: This example points toward a happy ending—but not for the first version of our idea. If we had gone full speed ahead with the original plan based on $3,000 ten-week accredited classes, our BusU business would have crashed! The original BusU idea was The Wrong It; we found The Right It version of BusU through testing and tweaking.
Note 4: My goal for the BusU example was to show you one possible way the tools and tactics we’ve learned might be combined and a possible scenario of how things might work out. The specific sequence of events and results will vary from case to case. But, generally speaking, your goal is to go from rough idea, to testable hypotheses, to experiments, to data from those experiments, and then to use that data to make an informed decision on the next steps. Depending on the nature and quality of the data you get, those next steps can range anywhere from making minor tweaks to the idea or hypotheses (and running more tests), to scrapping the idea altogether, to—if you are lucky—determining that your idea is likely to be The Right It and going forward with it.
9
Final Words
I began this book with the following ominous warning:
It waits. Patient.
Confident that it will soon get its prey—it always does.
Few escape its bite, none its tentacles.
One way or another, the Beast of Failure gets us all.
I wrote those somber words in one of my notebooks in the days right after my startup failed. They came out of my pen as I was trying to figure out how the hell this could have happened. As you can probably tell, I wasn’t exactly in a cheery mood at the time. It was a dark and bitter period for me for two reasons, one obvious and one not so obvious.
The obvious reason for my gloomy mood was that our company, after an incredibly promising start, almost $25 million of investment from three of the most successful VC firms in the world, and five years of really hard work by dozens of top-notch people, had to be shut down and sold off. As much as that hurt and embarrassed me and many of the people associated with the company, I knew we’d all recover and move on.
The final meeting with the investors, the board of directors, and the lawyers was not exactly fun, but it was not as terrible as I—a novice at failure on this scale—thought it would be. No one yelled, no one got angry, no one pointed blaming fingers. Instead, the tone of the meeting was a combination of disappointment, understanding, and—to my surprise—a lack of surprise. The investors, the other board members, and the lawyers had been through the same scenario (brilliant idea + good plan + plenty of funding + strong and experienced team + competent execution = failure) many times already. For them it was par for the course. Most startups fail, even those that hold much promise and seem like a sure thing. Remember Webvan?
I knew the stats about startup failure too, but somehow I didn’t think that those dismal odds had any relevance to our company. They couldn’t possibly apply to us. For starters, we had gone through exhaustive technical due diligence, done our market research, wrote a great business plan, and gotten funding from great investors. Then, after putting together an amazing team, we built the exact product we said we were going to build and our target market told us they wanted, needed, and would definitely buy. In other words, we had checked all the boxes.
So WTF!? Why The Failure!? It made no sense to me.
That was the root of the second, less obvious (but deeper and longer-lasting) reason for my gloom. What made it so pitch dark, so bitter, and so pervasive was that several of my deeply held beliefs about the way the world is supposed to work had been shattered. The formula for success that I witnessed, learned, believed in, and emulated had failed me. To my internal logic it was as if, all of a sudden, I could no longer count on 2 plus 2 adding up to 4.
At first, I felt lost, hopeless, betrayed. Then, slowly but steadily, the shock and pain of my first big failure morphed into something healthier: a desire to understand how and why something like this could happen—not just to our company, but to anyone who tries to bring a new idea into the world. I wanted to use that understanding to prevent it from happening again. As I mentioned in the Prelude, the Beast of Failure had bitten me, and I decided to bite back, or at least learn how to armor and defend myself against the worst of its bite.
I would figure out where and
how we had gone wrong, discover ways to prevent those errors in the future, and share those findings with the world. That’s what this book is all about.
In this concluding chapter I review and summarize those findings, repeat and highlight some of the most important points, and close with some important final words of advice and encouragement.
The Right It: A Recap
A Review of the Hard Facts
Part I began with the following imperative, the guiding principle of this book:
Make sure you are building The Right It before you build It right.
To reach The Right It, you must come to terms with a number of hard facts, hard because they are hard to take, hard to avoid, and unlikely to change. The mother of all hard facts is the Law of Market Failure:
Most new products will fail in the market, even if competently executed.
Most new products that make it to market fail because they are The Wrong It, defined as an idea for a new product that, even if competently executed, will fail in the market.
Next hard fact:
No amount of design brilliance, engineering prowess, or marketing fireworks can save a product that is The Wrong It from the jaws of the Beast of Failure.
This second fact explains why even the world’s most successful companies (e.g., Coca-Cola, Disney, Google) will often launch new products that fail in the market. These expertly executed new products fail because they are based on the wrong Premise—the market is not interested in them regardless of how well (or by whom) they are designed, built, or marketed.
I then introduced the hero of our book: The Right It is an idea for a new product that, if competently executed, will succeed in the market. Which brought us to our third hard fact:
Your only chance for success is to combine competent execution with a product that is The Right It.
But how can we know if a product is The Right It before we have actually built It? How can we tell if the market will want It if It does not even exist yet? The seemingly logical—but catastrophically wrong—answer to that question is: “We’ll ask people if they want it or will buy it.”
The Right It Page 18
