The Phoenix Project

by Gene Kim

  It has been particularly gratifying to see how this book has been used across technology organizations, often in a book club format to discuss how the current system can pit Development, Operations, and Information Security against each other, making it virtually impossible to achieve the most important organizational objectives, and more importantly, how this led to completely new types of interactions to explore a better way of working, with much improved outcomes.

  I’m always amazed at stories where people have used The Phoenix Project to find each other: often a change agent will recommend or give out copies of The Phoenix Project to scores of people, paying careful attention to who comes back saying, “Wow, what’s happening to Parts Unlimited is happening to us, isn’t it?” Sometimes seeing a copy of the book prominently on someone’s desk or bookshelf serves as a signal that this person is a fellow traveler, one who sees a common problem that spans functional domains and is potentially a fellow conspirator, a fellow risk-taker, who is willing to work together to create a coalition to overcome powerful, entrenched incumbent systems.

  The Phoenix Project is ultimately a book about transformation, and so it is incredibly gratifying to see it being used as an instrument to create transformations in real life as well.

  Surprises on the Journey

  There are so many surprises and learnings on this journey, and I wanted to share a couple that seemed fitting for this afterword.

  One of the most delightful and startling blog posts I’ve read about The Phoenix Project was by Dave Lutz, famous for many things, including the server-room inspired cover song of “Imagine” by The Beatles that he performed at DevOpsDays Mountain View 2011. In this post, he ponders the role of Brent, an all too familiar character in Operations. He writes, “I find myself wondering what the outcome of the project would have been if Bill’s first action was to fire Brent. Would the project have been finished earlier? (I don’t have a moral problem firing a fictitious character in a thought experiment. I wouldn’t do this in real life of course!)”

  Mr. Lutz writes about how there are two types of Brents: hoarders and sharers. He writes:

  I’ve also come across otherwise smart [people] who are of the mistaken belief that if they hold on to a task, something only they know how to do, it’ll ensure job security. These people are knowledge Hoarders.

  This doesn’t work. Everyone is replaceable. No matter how talented they are. Sure it may take longer at first to find out how to do that special task, but it will happen without them.

  For me, what Lutz wrote was fascinating for so many reason, like how The Phoenix Project has become shorthand to describe certain categories of problems and a safe way to discuss and conduct thought experiments, not just on the effects of processes, but on people.

  And by the way, Mr. Lutz, I can state for the record that during the writing of this book we always knew that Brent was a sharer, not a hoarder; in fact, Brent is the only character whose name we didn’t change. And without doubt, as you speculated, our real-life Brent always had the best interests of the organization at heart and was merely a victim of the system.

  Another genuine surprise for me is how some people have written that we (the authors) must hate Information Security, and more specifically, that we hate Information Security people. One of the best examples actually came from a friend of mine, Paul Love, who was a co-author with me on Visible Ops Security.

  In a blog post, I published an email that he wrote me: “When I first read The Phoenix Project: A Novel About IT DevOps, and Helping Your Business Win, John [the CISO character] made me angry. As a 20 year [sic] security veteran, John’s totally selfish ‘my way or the highway’ attitude actually made me physically mad. Who did this guy think he was anyway? Why was Gene painting the infosec practitioner in such an unflattering light?”

  He continued:

  After finishing the book, I took a moment to look back on my career. Thinking of all of the people like John who I’d run into and worked with over the year I realized, with a little bit of terror, why I hated him so much.

  Before I studied Visible Ops and DevOps, I was John.

  I’ve sometimes received the same sort of reaction from people who don’t know that I spent the majority of my career in the Information Security field as co-inventor of Tripwire, spending thirteen years as the founder and CTO of a company focusing on automating security and compliance.

  As the saying goes, “We tease who we love.” In many ways, John the CISO is my favorite character, and in many ways, his journey most closely mirrors my own. As my friend Jez Humble, co-author of The DevOps Handbook, observed, “[John] is a phoenix, too.” Well put, Jez!

  Whether we are a John, a Brent, a Wes, a Patty, or a Bill, when we’re trapped in a system that prevents us from succeeding, our job becomes thankless, reinforces a feeling of powerlessness, and we feel like we are trapped in a system that preordains failure. And worse, the nature of technical debt that is not paid down ensures that the system gets worse over time, regardless of how hard we try.

  We now know that DevOps principles and patterns are what allow us to turn this downward spiral into a virtuous spiral, through a combination of cultural norms, architecture, and technical practices.

  As The Phoenix Project found it’s way into the world, another group of us were working on The DevOps Handbook (Jez Humble, John Willis, Patrick DuBois, and me). One of the amazing moments during the creation of this book was when our editor, Anna, asked us each to describe our DevOps “aha” moment.

  The amazing thing was that all of our answers were uncannily similar: We each described the incredible frustration of how difficult it was to do our work, whether it was the toil or the suffering. And we all shared the exhilaration of discovering the better way, under this broad umbrella of practices we call DevOps.

  You can read about all of our “aha: moments in the first pages of the extended excerpt of The DevOps Handbook included in the back of this 5th Anniversary Edition of The Phoenix Project.

  Looking into the Future

  The problems that DevOps solve are at the center of what every modern organization is facing. Now more than ever, technology is not just the nervous system of an organization—it actually composes the majority of the muscle mass.

  As Jeff Immelt, former CEO of GE, wrote, “Every industry and company that is not bringing software to the core of their business will be disrupted.” Or as Jeffrey Snover, Technical Fellow at Microsoft, paraphrasing Dr. Nicholas Negroponte, wrote, “In previous economic eras, businesses created value by moving atoms. Now they create value by moving bits.”

  When The Phoenix Project was first published in 2013, DevOps was primarily used in Internet companies, commonly known as the FAANGs (Facebook, Amazon, Apple, Netflix, and Google). Of course, also in this category are Flickr, LinkedIn, Microsoft, Yahoo, Twitter, GitHub, and countless more.

  Now, five years later, it has been amazing to see these principles and practices in large, complex organizations across every industry vertical. This is incredibly exciting, because this is undoubtedly where the majority of the economic value of DevOps will be created.

  IDC, the analyst firm, says that there are about eleven million developers on the planet and seven million operations people on the planet. At the time of this writing, it would be wildly optimistic to project that one million of these engineers are already using DevOps principles and practices.

  If that’s the case, DevOps has 6.5% market share, leaving 93.5% of the market to go. The majority of these engineers are in large, complex organizations—these are the most well-known brands across every industry vertical or supporting our largest government agencies or military services.

  The mission at hand is how we can elevate their productivity so that they’re as productive as the high performers. We know through more than four years of State of DevOps Reports conducted by Puppet that high perfomers are two to three orders of magnitude more productive than their peers. In my mind, helping everyone reach t
his level of high performace will create trillions of dollars of economic value per year and is where the next surge of productivity will come from.

  In 2016, I was talking with my friend Rob England, often known as his moniker The IT Skeptic. We were fellow travelers in the ITIL space ten years ago. We talked about how he famously and visibly changed his mind about DevOps. Initially, he believed, as many do, that anything that increases deployment frequency and enables developers more freedom inevitably leads to disaster. But through many interactions, he eventually realized that DevOps can lead to far better outcomes. If you want to explore his journey more, you can read my full interview with him on CA.com, “Face-to-Face DevOps: To Protect and Serve.”

  In our conversations together, we talked about how DevOps is inevitable, inexorable, and remorseless, and how DevOps is incredibly disruptive to the technology sector, to the technology field, and for anyone in technology.

  There’s no doubt that DevOps is radically changing and transforming how we work in technology. Organizations that cannot adopt DevOps practices will be at a massive competitive disadvantage. As Dr. W. Edwards Deming is famously paraphrased, “Learning is not compulsory...neither is survival.”

  Without a doubt, the best times for technology are ahead of us, not behind us. There’s never been a better time to be in the technology field, and to be a lifelong learner.

  On behalf of my co-authors, thanks to everyone who has made this journey so amazing and worthwhile!

  —Gene Kim

  Portland, OR

  December 5, 2017

  ACKNOWLEDGEMENTS

  First and foremost, I want to acknowledge all the support from my loving wife, who put up with far more than I promised, Margueritte, and my sons, Reid, Parker, and Grant.

  I want to thank Todd Sattersten, Tim Grahl, Merridawn Duckler, and Kate Sage for their incredible help and support throughout the development process of this book. Also, my profound thanks to the tireless contributions and scrutiny from Paul Muller from hp, Paul Proctor from Gartner, Branden Williams from rsa, Dr. Tom Longstaff at Johns Hopkins University, Julia Allen from sei/cmu, Adrian Cockcroft from Netflix, Christopher Little from bmc, Bob McCarthy, Lisa Schwartz from itsm Academy, Jennifer Bayuk, Ben Rockwood from Joyent, Josh Corman from Akamai, James Turnbull from Puppet Labs, Charlie Betz from Enterprise Management Associates, Dr. Gene Spafford from cerias at Purdue University, Dwayne Melancon from Tripwire, and Michael Krigsman from Asuret.

  I also want to attribute the contributions of my fellow coauthors of The DevOps Handbook, Jez Humble, Patrick DeBois, and John Willis. Among others, they helped crystallize the practices that became The Three Ways that Erik talked about.

  I want to acknowledge John Allspaw, Paul Hammond, and Jez Humble for their groundbreaking and seminal contributions of showing how fast flow in the it value stream is really done.

  And thank you to all the other reviewers who helped shape the manuscript: David Allen, David Bills, Kip Boyle, Shane Carlson, Carlos Casanova, Scott Crawford, Iris Culpepper, Mike Dahn, Chris Eng, Paul Farrall, Daniel Francisco, Kevin Hood, Matt Hooper, Tom Howarth, Kevin Kenan, Paul Love, Norman Marks, Tom McAndrew, Ally Miller, David Mortman, Wendy Nather, Michael Nygard, John Pierce, Dennis Ravenelle, Sasha Romanosky, Susan Ryan, Fred Scholl, Lawrence “Butch” Sheets, Bill Shinn, Adam Shostack, Ariel Silverstone, Dan Swanson, Joe “Feech” Telafici, Jan Vromant, and Lenny Zeltser.

  The methodology used to create, link, and compute Dick’s organizational kpis to it activities is based on the Risk-Adjusted Value Managementtm methodology, developed by Paul Proctor and Michael Smith at Gartner, Inc.

  The tool used to scope the specific audit internal control objectives to specific it controls is called gait, developed by the Institute of Internal Auditors.

  And my heartiest thank you to my assistant, Hannah Concannon, who made it possible for me to focus on writing and finishing the book, as well as helping me do all the final edits.

  I want to also acknowledge Tim Ferriss and the help of the other alumni of the Kimono group, who helped me understand the theory and practice of book launches.

  Gene Kim

  Portland, or, June 10, 2012

  I would like to thank my wife, Erica, and my daughters, Emily and Rachel, for their patience and understanding with my chosen profession, which requires so much travel. Special thanks to my joyfully subversive serial coconspirators, Gene Kim and George Spafford, for being highly adaptable and tolerating my loquacious rants.

  I have been ridiculously fortunate to work with some of the most creative and brilliant cxos in my practice over the years, such as Will “Prefontaine” Weider, cio of Ministry Healthcare; Robert Slepin, cio of John C. Lincoln Health Network; Oliver Eckel, ceo of Cognosec; Rob Leahy, cfo of Transdermal Corporation; Jeff Hughes, vp of Radiant Systems; Paul O’Neil, ceo of Kerzner International; and Nana Palmer, coo of Kerzner International—you all have taught me so much about courage in experimentation and radically improving it throughput.

  Lastly, I would like to thank my friend and partner in crime for many of these improvement learnings, John Dennin, Senior Engagement Manager at Assemblage Pointe, Inc.

  Kevin Behr

  Lancaster, pa, June 1, 2012

  The journey from Visible Ops to When IT Fails further cemented my utmost respect and appreciation for Gene and Kevin. The challenges and exchanges we’ve had in the course of writing this book tested our collective abilities to put in writing what we have encountered in reality in the it industry.

  Gentlemen, thank you very much!

  Most importantly, thank you for the unwavering love, motivation, support, and patience of my better half, Rowena. Thank you to my children, Paolo, Alyssa, and Erika, who all unselfishly put up with my chaotic and time-consuming schedule, even when on vacation. To my parents, Carroll and Alpha, thank you for instilling in me a love of learning. You have been an instrumental part in my continued quest to keep improving in all aspects of my life.

  George Spafford

  Saint Joseph, mi, June 1, 2012

  Table of Contents

  The Three Ways

  Preface

  Foreword

  Imagine a World Where Dev and Ops Become DevOps:

  An Introduction to The DevOps Handbook

  Part I Introduction

  01 Agile, Continuous Delivery, and the Three Ways

  02 The First Way: The Principles of Flow

  03 The Second Way: The Principles of Feedback

  04 The Third Way: The Principles of Continual Learning and Experimentation

  Endnotes

  Preface

  Aha!

  The journey to complete The DevOps Handbook has been a long one—it started with weekly working Skype calls between the co-authors in February of 2011, with the vision of creating a prescriptive guide that would serve as a companion to the as-yet unfinished book The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.

  More than five years later, with over two thousand hours of work, The DevOps Handbook is finally here. Completing this book has been an extremely long process, although one that has been highly rewarding and full of incredible learning, with a scope that is much broader than we originally envisioned. Throughout the project, all the co-authors shared a belief that DevOps is genuinely important, formed in a personal “aha” moment much earlier in each of our professional careers, which I suspect many of our readers will resonate with.

  Gene Kim

  I’ve had the privilege of studying high-performing technology organizations since 1999, and one of the earliest findings was that boundary-spanning between the different functional groups of IT Operations, Information Security, and Development was critical to success. But I still remember the first time I saw the magnitude of the downward spiral that would result when these functions worked toward opposing goals.

  It was 2006, and I had the opportunity to spend a week with the group who managed the outsourced IT Operations of a large a
irline reservation service. They described the downstream consequences of their large, annual software releases: each release would cause immense chaos and disruption for the outsourcer, as well as customers; there would be SLA (service level agreement) penalties, because of the customer-impacting outages; there would be layoffs of the most talented and experienced staff, because of the resulting profit shortfalls; there would be much unplanned work and firefighting so that the remaining staff couldn’t work on the ever-growing service request backlogs coming from customers; the contract would be held together by the heroics of middle management; and everyone felt that the contract would be doomed to be put out for re-bid in three years.

  The sense of hopelessness and futility that resulted created for me the beginnings of a moral crusade. Development seemed to always be viewed as strategic, but IT Operations was viewed as tactical, often delegated away or outsourced entirely, only to return in five years in worse shape than it was first handed over.

  For many years, many of us knew that there must be a better way. I remember seeing the talks coming out of the 2009 Velocity Conference, describing amazing outcomes enabled by architecture, technical practices, and cultural norms that we now know as DevOps. I was so excited, because it clearly pointed to the better way that we had all been searching for. And helping spread that word was one of my personal motivations to co-author The Phoenix Project. You can imagine how incredibly rewarding it was to see the broader community react to that book, describing how it helped them achieve their own “aha” moments.

  Jez Humble

  My DevOps “aha” moment was at a start-up in 2000—my first job after graduating. For some time, I was one of two technical staff. I did everything: networking, programming, support, systems administration. We deployed software to production by FTP directly from our workstations.

  Then in 2004 I got a job at ThoughtWorks, a consultancy where my first gig was working on a project involving about seventy people. I was on a team of eight engineers whose full-time job was to deploy our software into a production-like environment. In the beginning, it was really stressful. But over a few months we went from manual deployments that took two weeks to an automated deployment that took one hour, where we could roll forward and back in milliseconds using the blue-green deployment pattern during normal business hours.