The Phoenix Project

by Gene Kim

  Instead, I see only Patty sitting at the conference table, typing away on her laptop.

  “Welcome to the cab, Bill. I hope you can find an empty chair,” she says.

  “Where is everybody?” I ask.

  I’m baffled. When I ran the midrange group, my team would never miss our change management meetings. It was where we coordinated and organized all our work to make sure we didn’t shoot ourselves in the foot.

  “I told you yesterday that change management around here is hit-or-miss,” Patty says, sighing. “Some groups have their own local change-management process, like yours. But most groups do nothing at all. Yesterday’s outage is just proof that we need to have something at the enterprise level. Right now, the left hand rarely knows what the right hand is doing.”

  “So, what’s the problem?” I ask.

  She purses her lip. “I don’t know. We sent a bunch of people to itil training, so they could get up to speed on all the best practices. We brought in some consultants, who helped us replace our ticketing system with an itil-compliant change management tool. People were supposed to put change requests into it, where it would get routed for approvals. But, even after two years, all we have is a great process on paper that no one follows and a tool that no one uses. When I pester people to use them, all I get are complaints and excuses.”

  I nod. itil stands for it Infrastructure Library, which documents many it best practices and processes, and the itil program has had a reputation of spending years merely walking in circles.

  I’m bothered that Wes isn’t here. I know he’s busy, but if he’s not here, why would any of his people bother to show up? Efforts like this must start and be continually maintained from the top.

  “Well, they can bring their complaints and excuses to me,” I say adamantly. “We’re rebooting the change management process. With my total support. Steve’s told me to make sure people can stay focused on Phoenix. Screwups like the san failure made us miss a Phoenix deliverable, and now we’re paying for it. If someone wants to skip a change management meeting, they obviously are in need of some special compassionate coaching. From me.”

  At Patty’s puzzled expression at my Phoenix reference, I tell her about how Wes and I spent our morning being run over by the bus. Sarah and Chris were at the wheel, but Steve was in back, cheering them on to floor it.

  “Not good,” she says, disapprovingly. “They even ran over Kirsten, huh?”

  I nod silently but refuse to say more. I always liked that phrase in Saving Private Ryan: “There’s a chain of command: gripes go up, not down.”

  Instead, I ask her to walk me through the current change process and the way it’s been automated in the tools. It all sounds good. But there’s only one way to see if the process works.

  I say, “Schedule another cab meeting for the same time Friday. I’ll send out an e-mail to all the cab members letting them know that this is mandatory.”

  When I get back to my cubicle, Ellen is at my desk, bending over my laptop, writing a note.

  “Everything working, I hope?” I ask.

  She startles at the sound of my voice. “Oh, my God. You scared me,” she says laughing. “Support left you a replacement laptop because they couldn’t get your laptop to boot, even after a half hour of trying.”

  She points at the far side of my desk, and I do a double take.

  My replacement laptop appears to be almost ten years old—it’s twice as large as my old one and looks three times as heavy. The battery has been taped on, and half the keyboard lettering is worn off from heavy use.

  For a moment, I wonder if this is a practical joke.

  I sit down and bring up my e-mail, but everything is so slow that several times I thought it had locked up.

  Ellen has a sympathetic expression on her face. “The support guy said that this is all they have available today. Over two hundred people are having similar problems, and many aren’t getting replacements. Apparently, people with your laptop model also have had their’s break because of some security patch.”

  I forgot. It’s Patch Tuesday, when John and his team roll out all their security patches from our major vendors. Once again, John is causing huge issues and disruptions for my team and me.

  I merely nod and thank her for the help. After she’s gone, I sit down and type out an e-mail to all the cab members, my keystrokes often taking ten seconds to show up on the screen.

  From: Bill Palmer

  To: Wes Davis, Patty McKee, IT Operations Management

  Date: September 3, 2:43 PM

  Priority: Highest

  Subject: Mandatory CAB meeting Friday, 2 PM

  Today, I attended the weekly CAB meeting. I was extremely disappointed that I was the only one there, besides Patty, especially given the totally avoidable, change-related failure yesterday.

  Effective immediately, managers (or their assigned delegates) are required to attend all scheduled CAB meetings and to perform their assigned duties. We are resurrecting the Parts Unlimited change management process and it will be followed to the letter.

  Any person(s) caught circumventing change management will be subject to disciplinary action.

  There will be a mandatory CAB meeting Friday at 2 PM. See you there.

  Call me if you have any questions or concerns.

  Thanks for your support,

  Bill

  I hit send, waiting fifteen seconds for the e-mail to finally leave my outbox. Almost immediately, my cell phone rings.

  It’s Wes. I say, “I was just about to call you about the laptops. We’ve got to get replacements to our managers and employees so they can do their jobs, you hear?”

  “Yeah, we’re on it. But I’m not calling about that. And I’m not calling about Phoenix, either,” he says, sounding irritated. “Look, about your memo on change management: I know you’re the boss, but you better know that the last time we did one of these change management kumbayas, we ran it straight into the ground. No one, and I mean absolutely no one, could get a single thing done. Patty insisted on having everyone take a number and wait for her pointy-heads to authorize and schedule our changes. It was absolutely ridiculous and a total waste of time.”

  He’s unstoppable: “That software application she made us use is a total piece of crap. It takes twenty minutes to fill out all those fields for a simple five-minute change! I don’t know who designed the process, but I think they assumed that we all get paid by the hour and want to talk about doing work instead of actually doing work.

  “Eventually, the Networking and Server Team staged a rebellion, refusing to use Patty’s tool,” he continues heatedly. “But John waved an audit finding around and went to Luke, our old cio. And just like you did, Luke said that following policies was a condition of employment, threatening to fire anybody who didn’t follow them.

  “My guys were spending half their time doing paperwork and sitting in that damned cab meeting,” he continues. “Luckily, the effort finally died, and John was too clueless to catch on that no one was actually going to the meetings anymore. Even John hasn’t gone to one of those meetings in over a year!”

  Interesting.

  “I hear you,” I say. “We can’t repeat that, but we also can’t have another payroll disaster. Wes, I need you there, and I need you to help create the solution. Otherwise, you’re part of the problem. Can I count on you?”

  I hear him sigh loudly. “Yeah, sure. But you can also count on me calling ‘bullshit’ if I see Patty trying to create some sort of bureaucracy that sucks out everybody’s will to live.”

  I sigh.

  Before, I was merely worried that it Operations was under attack by Development, Information Security, Audit, and the business. Now, I’m starting to realize that my primary managers seem to be at war with each other, as well.

  What will it take for us to all get along?

  CHAPTER 5

  • Thursday, September 4

  I wake up with a jolt when the alarm clock goes off at
6:15 a.m.. My jaw still hurts from clenching it all night. The dismal prospects of the upcoming Phoenix launch were never far from my mind.

  As usual, before climbing out of bed, I quickly scan my phone for any bad news. Usually, I would spend about ten minutes replying to e-mails—it always feels good to lob a couple of balls off my side of the court.

  I see something that makes me bolt upright so abruptly that I wake up Paige. “Oh, my God. What, what?” she asks frantically, not fully awake.

  “It’s another e-mail from Steve. Hang on, darling…” I say to her, while I squint to read it.

  From: Steve Masters

  To: Bill Palmer

  Cc: Nancy Mailer, Dick Landry

  Date: September 4, 6:05 AM

  Priority: Highest

  Subject: URGENT: SOX-404 IT Audit Findings Review

  Bill, please look into this ASAP. I don’t need to tell you how critical it is to have a clean SOX-404 audit.

  Nancy, please work with Bill Palmer, who is now in charge of IT Operations.

  Steve

  >>> Begin forwarded message:

  We just concluded our Q3 internal audit in preparation for the upcoming SOX-404 external audit. We discovered some very concerning deficiencies that we need to discuss with you. Due to the severity and urgency of the findings, we need to meet with IT this morning.

  Nancy

  Indeed, there’s a two-hour meeting scheduled for 8 a.m. on my calendar, set up by Nancy Mailer, Chief Audit Executive.

  Holy crap. She is incredibly smart and formidable. Years ago during the retail acquisition integration, I watched her grill a manager from the business we were acquiring. He was presenting their financial performance, when she started a rapid-fire interrogation, like a cross between Columbo, Matlock, and Scarface.

  He quickly broke, admitting that he was exaggerating his division’s performance.

  Recalling that meeting, my armpits feel damp. I haven’t done anything wrong. But given the tone of the e-mail, she is clearly hot on the trail of something important, and Steve just threw me in her path.

  I’ve always run a very tight ship in my Midrange Technology group. This kept Audit from interfering too much. Sure, there would still be a lot of questions and documentation requests, requiring us to spend a few weeks collecting data and preparing responses. Occasionally, they would find something, but we would quickly fix it.

  I like to think that we built a mutually respectful working relationship. However, this e-mail portends something more ominous.

  I look at my watch. The meeting is in ninety minutes, and I don’t have a clue about what she wants to talk about.

  “Shit!” I exclaim, as I jostle Paige’s shoulder. “Honey, can you drive the kids into school today? Something really bad just came up involving the Chief Audit Executive and Steve. I need to make some phone calls and get to the office right away.”

  Annoyed, she says, “For two years you’ve always taken the kids on Thursdays! I have an early start today, too!”

  “I’m sorry, honey. This is really important. The ceo of the company asked me to handle this. Steve Masters. You know, the guy on tv and who gives the big speeches at the company holiday party? I can’t drop another ball after a day like yesterday. And the newspaper headline the night before that—”

  Without a word, she storms down the stairs.

  When I finally find the conference room for the 8 a.m. meeting, I immediately notice how silent it is, devoid of the usual small talk that fills the time while attendees trickle in.

  Nancy sits at the head of the table, with four other people sitting around her. Sitting next to her is John along with his ever-present, black three-ring binder. As always, I’m surprised by how young he is. He’s probably in his mid-thirties with thick, curly black hair.

  John has a haggard look about him, and like many college students, has continually gained weight in the three years he’s been here at Parts Unlimited. Most likely from all the stress associated with his failing moral crusade.

  John actually reminds me more of Brent than anyone else in the room. However, unlike Brent who normally wears a Linux T-shirt, John wears a starched, collared shirt that’s slightly too large.

  Wes is conspicuously underdressed compared to everyone in the room, but he obviously doesn’t care. The last person in the room is a young man who I don’t recognize, presumably the it auditor.

  Nancy begins, “We have just concluded our Q3 internal audit in preparation for the upcoming external sox-404 audits. We have a grave situation. Tim, our it auditor, found an eye-opening number of it control issues. Worse, many are repeat findings going into the third year. Left unresolved, these findings may force us to conclude that the company no longer has sufficient controls to assert the accuracy of its financial statements. This could result in an adverse footnote from the external auditors in the company 10-k filings with the US Securities and Exchange Commission.

  “Although these are only preliminary findings, due to the gravity of the situation, I have already verbally informed the audit committee.”

  I blanch. Although I don’t understand all the audit jargon, I know enough that this could ruin Dick’s day and mean potentially more bad front-page news.

  Satisfied that I understand the severity of the situation, Nancy nods. “Tim, please walk us through your conclusions.”

  He takes out a huge stack of stapled papers, handing one out to everyone assembled. “We have just concluded our audit of the it general controls at Parts Unlimited for all of the critical financial systems. It took a team of four people over eight weeks to create this consolidated report.”

  Holy crap. I lift the two-inch thick stack of papers in my hand. Where did they find a stapler this big?

  It’s a printed Excel spreadsheet, with twenty rows per page in tiny eight-point type. The last page is numbered page 189. “There must be a thousand issues here!” I say in disbelief.

  “Unfortunately, yes,” he responds, not entirely able to hide his smug satisfaction. “We found 952 it general control deficiencies, of which sixteen are significant deficiencies and two are potential material weaknesses. Obviously, we’re very alarmed. Given how soon the external audit starts, we need your remediation plan as soon as possible.”

  Wes is hunched over the table, one hand on his forehead, the other hand flipping through page after page. “What kind of horseshit is this?”

  He holds up one page. “‘Issue 127. Insecure Windows operating system max_syn_cookie setting’? Is this a joke? In case you haven’t heard, we’ve got a real business to run. Sorry if that interferes with this full-time audit employment racket you’ve got going on here.”

  Trust Wes to say what people are thinking but are too smart to actually say aloud.

  Nancy responds gravely, “Unfortunately, at this point, the phase of control review and testing is over. What we require from you now is the ‘management response letter.’ You need to investigate each of these findings, confirm them, and then create a remediation plan. We’ll review it and then present to the audit committee and the board of directors.

  “Normally, you would have months to prepare your response letter and execute your remediation plan,” she continues, suddenly looking apologetic. “Unfortunately, the way the audit testing calendar worked out, we only have three weeks until the external auditors arrive. That’s regrettable. We’ll make sure to give it more time in the next audit cycle. But this time around, we require your response by…”

  She looks at her calendar. “One week from Monday, at the very latest. Do you think you can make it?”

  Oh, shit.

  That’s just six working days away. We’ll need half that time just to read the entire document.

  Our auditors, who I’ve long believed are a force for justice and objectivity, are crapping on me, too?

  I pick up the huge stack of papers again and look at a couple of random pages. There are many entries like Wes read, but others have references to inadequate s
ecurity settings, presence of ghost login accounts, change control issues, and segregation of duties issues.

  John flips his three-ring binder open and says officiously, “Bill, I brought up many of the same issues with Wes and your predecessor. They convinced the cio to sign a management waiver, stating that he accepted the risk, and do nothing. Given that some of these are now repeat audit findings, I don’t think we’ll be able to talk our way out of it this time.”

  He turns to Nancy. “During the previous management regime, it controls clearly weren’t a priority, but now that all the security chickens are coming home to roost, I’m sure Bill will be more prudent.”

  Wes looks at John with contempt. I can’t believe John is grandstanding in front of the auditors. It’s times like this that make me wonder whose side he’s really on.

  Oblivious to Wes and me, John says to Nancy, “My department has been remediating some other controls, which I think we should be given credit for. For starters, we’ve completed the tokenization of the pii on our critical financial systems, so at least we dodged that bullet. That finding is now closed.”

  Nancy says dryly, “Interesting. The presence of pii is not in the scope of the sox-404 audit, so from that perspective, focusing on the it general controls might have been a better use of time.”

  Wait. John’s urgent tokenization change was for nothing?

  If that’s true, John and I need to talk. Later.

  I say slowly, “Nancy, I genuinely don’t know what we can get to you by Friday. We’re buried in recovery work and are scrambling to support the upcoming Phoenix rollout. Which of these findings are the most important for us to respond to?”

  Nancy nods to Tim, who says, “Certainly. The first issue is the potential material weakness, which is outlined on page seven. This finding states that an unauthorized or untested change to an application supporting financial reporting could have been put into production. This could potentially result in an undetected material error, due to fraud or otherwise. Management does not have any control that would prevent or detect such a change.