by Gene Kim
I nod. I might be in the technology backwaters, but my team has been involved with Phoenix for years. Everyone knows how important it is.
“We’re years late delivering,” he continues. “Our investors and Wall Street are howling. And now, my board is losing confidence in our ability to hit our commitments.
“I’ll be blunt,” he says. “The way things are going, I’ll be out of a job in six months. As of last week, Bob Strauss, my old boss, is the new chairman of the company. There’s a vocal group of shareholders trying to split up the company, and I don’t know how much longer we can fend them off. What’s at stake here is not just my job, but the nearly four thousand employees who work here at Parts Unlimited.”
Suddenly, Steve looks much older than the early fifties I had guessed him to be. Looking right at me, he says, “As acting cio, Chris Allers, our vp of Application Development, will report to me. And so will you.”
He stands up and starts to pace, “I need you to keep all the things that are supposed to be up, well, up. I need someone reliable, who isn’t afraid to tell me bad news. Above all, I need someone I can trust to do the right thing. That integration project had many challenges, but you always kept a cool head. You’ve built a reputation as someone who is dependable, pragmatic, and willing to say what you really think.”
He’s been candid with me, so I reply with the same. “Sir, with all due respect, it seems very difficult for senior it leadership to succeed here. Any request for budget or staff is always shot down, and executives are replaced so quickly, some never even get a chance to fully unpack.”
With finality, I say, “Midrange Operations is critical to getting Phoenix done, too. I need to stay there to see those things through to completion. I appreciate you thinking of me, but I can’t accept. However, I promise I’ll keep my eyes open for any good candidates.”
Steve looks at me appraisingly, his expression surprisingly grave. “We’ve had to cut budgets across the entire company. That edict came straight from my board. My hands were tied. I won’t make promises I can’t keep, but I can promise you I’ll do whatever it takes to support you and your mission.
“Bill, I know you didn’t ask for this job, but the company’s survival is at stake here. I need you to help me save this great company. Can I count on you?”
Oh, come on.
Before I can politely decline again, I suddenly hear myself saying, “Yes, sir, you can count on me.”
I panic, realizing that Steve somehow used some Jedi mind trick on me. I force myself to stop talking, before I make more dumb promises.
“Congratulations,” Steve says, standing up and shaking my hand firmly. He clasps my shoulder. “I knew you’d do the right thing. On behalf of the entire executive team, we’re grateful for you stepping up.”
I look at his hand grasping mine, wondering if I can backpeddle my way out.
Not a chance in hell, I decide.
Swearing to myself, I say, “I’ll do my best, sir. And could you at least explain why no one who accepts this position lasts very long? What do you want most from me? And what don’t you want?”
With a resigned half smile, I add, “If I fail, I’ll try to make sure it’s in a new and novel way.”
“I like that!” Steve says, laughing loudly. “What I want is for it to keep the lights on. It should be like using the toilet. I use the toilet and, hell, I don’t ever worry about it not working. What I don’t want is to have the toilets back up and flood the entire building.” He smiles broadly at his own joke.
Great. In his mind, I’m a glorified janitor.
He continues, “You have a reputation of running the tightest ship in the it organization. So I’m giving you the entire fleet. I expect you to make them all run the same way.
“I need Chris focused on Phoenix execution. Anything in your area of responsibility that takes focus away from Phoenix is unacceptable. That applies not just to you and Chris, but everyone else in this company. Is that clear?”
“Absolutely,” I say, nodding. “You want the it systems to be reliable and available, and for the business to be able to depend upon them. You want disruptions to normal operations kept to an absolute minimum so that the business can focus on getting Phoenix done.”
Looking surprised, Steve nods. “Exactly. Yes, well put. Whatever you said, that’s exactly what I want.”
He hands me an e-mail printout from Dick Landry, the cfo.
From: Dick Landry
To: Steve Masters
Date: September 2, 8:27 AM
Priority: Highest
Subject: ACTION NEEDED: payroll run is failing
Hey, Steve. We’ve got serious issues with this week’s payroll. We’re trying to figure out if the problem is with the numbers or in the payroll system. Either way, thousands of employees have paychecks stuck in system & are at risk of not getting paid. Seriously bad news.
We must fix this before payroll window closes at 5 PM today. Please advise on how to escalate this, given the new IT org.
Dick
I wince. Employees not getting paychecks means families not being able to pay their mortgages or put food on the table.
Suddenly, I realize that my family’s mortgage payment is due in four days, and we could be one of the families affected. A late mortgage payment could screw up our credit rating even more, which we spent years repairing after we put Paige’s student loans on my credit card.
“You want me to jump on this and manage the incident to conclusion?”
Steve nods, giving me the thumbs-up. “Keep me posted on the progress, please.” His expression turns grave. “Every responsible company takes care of its employees. Many of our factory workers live from paycheck to paycheck. Do not create hardship for their families, you hear? This could get us in trouble with the union, maybe even triggering a work-stoppage, creating some very bad press for us.”
I nod automatically. “Restore critical business operations and keep us out of the front-page news. Got it. Thanks.”
Why, exactly, I’m thanking him is not clear.
CHAPTER 2
• Tuesday, September 2
“How’d it go in there?” Stacy asks kindly, looking up from her keyboard.
I just shake my head. “I can’t believe it. He just talked me into taking a new job I don’t want. How did that happen?”
“He can be very persuasive,” she says. “For what it’s worth, he’s one-of-a-kind. I’ve worked for him for nearly ten years, and I’ll follow him anywhere. Anything I can help with to make your job easier?”
Thinking for a moment, I ask, “There’s an urgent payroll issue that needs to be fixed. Dick Landry is on floor three, right?”
“Here you go,” she says, before I’ve finished asking my question, handing me a Post-it note with all of Dick’s contact information. Office location, phone numbers, and everything.
Grateful, I smile at her. “Thanks a lot—you are fantastic!”
I dial Dick’s cell phone on my way to the elevator. “Dick here,” he answers gruffly, still typing in the background.
“This is Bill Palmer. Steve just made me the new vp of it Operations, and he asked me to—”
“Congratulations,” he interrupts. “Now look, my people found a huge payroll irregularity. When can you get to my office?”
“Right away,” I reply. I hear the click of him ending the call. I’ve had warmer welcomes.
On the third floor, I walk through Finance and Accounting, surrounded by pinstriped shirts and starched collars. I find Dick at his desk, still on the phone with someone. When he sees me, he puts his hand over the mouthpiece. “You from it?” he asks gruffly.
When I nod, he says into the phone, “Look, I gotta run. Someone who’s supposedly going to help is finally here. I’ll call you back.” Without waiting for an answer, he hangs up the phone.
I’ve never actually seen someone who routinely hangs up on people. I brace myself for a conversation that is likely to be short on a
ny comforting “let’s get to know each other” foreplay.
As if in a hostage situation, I slowly raise my hands, showing Dick the printed e-mail. “Steve just told me about the payroll outage. What’s the best way for me to get some situational awareness here?”
“We’re in deep kimchi,” Dick responds. “In yesterday’s payroll run, all of the records for the hourly employees went missing. We’re pretty sure it’s an it issue. This screwup is preventing us from paying our employees, violating countless state labor laws, and, no doubt, the union is going to scream bloody murder.”
He mutters under his breath for a moment. “Let’s go see Ann, my Operations Manager. She’s been pulling her hair out since yesterday afternoon.”
Walking quickly to keep up, I nearly run into him when he stops and peers through a conference room window. He opens the door. “How’s it going in here, Ann?”
There are two well-dressed women in the room: one, around forty-five years old, studies the whiteboard, filled with flowcharts and a lot of tabulated numbers, and the other, in her early thirties, types on a laptop. Spreadsheets are strewn all over the large conference room table. The older woman gestures with an open marker at what appears to be a list of potential failure causes.
Something about the way they dress, and their concerned and irritated expressions, makes me think they were recruited from a local accounting firm. Ex-auditors. Good to have them on our side, I suppose.
Ann shakes her head in exhausted frustration. “Not much progress, I’m afraid. We’re almost certain this is an it systems failure in one of the upstream timekeeping systems. All of the hourly factory worker records got screwed up in the last upload—”
Dick interrupts her. “This is Bill from it. He’s been assigned to fix this mess or die trying, is what I think he said.”
I say, “Hi, guys. I’ve just been made the new head of it Operations. Can you start from the beginning and tell me what you know about the problem?”
Ann walks over to the flowchart on the whiteboard. “Let’s start with the information flow. Our financial system gets payroll data from all our various divisions in different ways. We roll up all the numbers for salaried and hourly personnel, which includes wages and taxes. Sounds easy, but it’s extremely complex, because each state has different tax tables, labor laws, and so forth.
“To make sure something doesn’t get screwed up,” she continues, “we make sure the summarized numbers match the detailed numbers from each division.”
As I hurriedly jot down some notes, she continues, “It’s a pretty clunky and manual process. It works most of the time, but yesterday we discovered that the general ledger upload for hourly production staff didn’t come through. All of the hourlies had zeroes for their hours worked and amount due.
“We’ve had so many problems with this particular upload,” she says, obviously frustrated, “that it gave us a program that we use to do manual corrections, so we don’t have to bother them anymore.”
I wince. I don’t like finance personnel manually changing payroll data outside the payroll application. It’s error-prone and dangerous. Someone could copy that data onto a usb drive or e-mail it outside of the organization, which is how organizations lose sensitive data.
“Did you say all the numbers for salaried employees are okay?” I ask.
“That’s right,” she replies.
“But hourly employees are all zeroes,” I confirm.
“Yep,” she again replies.
Interesting. I ask, “Why do you think the payroll run failed when it was working before? Have you had problems like this in the past?”
She shrugs. “Nothing like this has happened before. I have no idea what could have caused it—no major changes were scheduled for this pay period. I’ve been asking the same questions, but until we hear from the it guys, we’re stuck dead in the water.”
“What is our backup plan,” I ask, “if things are so hosed that we can’t get the hourly employee data in time?”
“For crying out loud,” Dick says. “It’s in that e-mail you’re holding. The deadline for electronic payments is 5 p.m., today. If we can’t hit that window, we may have to FedEx bales of paper checks to each of our facilities for them to distribute to employees!”
I frown at this scenario and so does the rest of the finance team.
“That won’t work,” Ann says, clicking a marker on her teeth. “We’ve outsourced our payroll processing. Each pay period, we upload the payroll data to them, which they then process. In the worst case, maybe we download the previous payroll run, modify it in a spreadsheet, and then re-upload it?
“But because we don’t know how many hours each employee worked, we don’t know how much to pay them!” she continues. “We don’t want to overpay anyone, but that’s better than accidentally underpaying them.”
It’s obvious that plan B is fraught with problems. We’d basically be guessing at people’s paychecks, as well as paying people who were terminated, and not paying people who were newly hired.
To get Finance the data they need, we may have to cobble together some custom reports, which means bringing in the application developers or database people.
But that’s like throwing gasoline on the fire. Developers are even worse than networking people. Show me a developer who isn’t crashing production systems, and I’ll show you one who can’t fog a mirror. Or more likely, is on vacation.
Dick says, “These are two lousy options. We could delay our payroll run until we have the correct data. But we can’t do this—even if we’re only a day late, we’ll have the union stepping in. So, that leaves Ann’s proposal of paying our employees something, even if it’s the incorrect amount. We’d have to adjust everyone’s paycheck in the next pay period. But now we have a financial reporting error that we’ve got to go back and fix.”
He pinches the bridge of his nose and continues to ramble. “We’ll have a bunch of odd journal entries in our general ledger, just when our auditors are here for our sox-404 audits. When they see this, they’ll never leave.
“Oh, man. A financial reporting error?” Dick mutters. “We’ll need approval from Steve. We’re going to have auditors camped out here until the cows come home. No one’ll ever get any real work done again.”
sox-404 is short for the Sarbanes-Oxley Act of 2002, which Congress enacted in response to the accounting failures at Enron, WorldCom, and Tyco. It means the ceo and cfo have to personally sign their names, attesting that their company’s financial statements are accurate.
Everyone longs for the days when we didn’t spend half our time talking to auditors, complying with each new regulatory requirement du jour.
I look at my notes and then at the clock. Time is running out.
“Dick, based on what I’ve heard, I recommend that you continue to plan for the worst and we fully document plan B, so we can pull it off without further complications. Furthermore, I request that we wait until 3 p.m. before making a decision. We may be still able to get all the systems and data back.”
When Ann nods, Dick says, “Okay, you’ve got four hours.”
I say, “Rest assured that we understand the urgency of the situation and that you’ll be apprised of how it’s going as soon as I find out myself.”
“Thanks, Bill,” Ann says. Dick remains silent as I turn around and walk out the door.
I feel better, now that I’ve seen the problem from the business perspective. It’s now time to get under the covers and find out what broke the complex payroll machinery.
While walking down the stairs, I dig out my phone and scan my e-mails. My feeling of calm focus disappears when I see that Steve hasn’t sent out an announcement of my promotion. Wes Davis and Patty McKee, who until today were my peers, still have no idea that I’m now their new boss.
Thanks, Steve.
When I enter Building 7, it hits me. Our building is the ghetto of the entire Parts Unlimited campus.
It was built in the 1950s, and last re
modeled in the 1970s, obviously built for utility, not aesthetics. Building 7 used to be our large brake-pad manufacturing factory until it was converted into data center and office space. It looks old and neglected.
The security guard says cheerfully, “Hello, Mr. Palmer. How is the morning going so far?”
For a moment, I’m tempted to ask him to wish me luck, so he can get paid the correct amount this week. Of course, I merely return his friendly greeting.
I’m headed toward the Network Operations Center, or as we call it, the noc, where Wes and Patty are most likely to be. They’re now my two primary managers.
Wes is Director of Distributed Technology Operations. He has technical responsibility for over a thousand Windows servers, as well as the database and networking teams. Patty is the Director of it Service Support. She owns all the level 1 and 2 help desk technicians who answer the phones around the clock, handling break-fix issues and support requests from the business. She also owns some of the key processes and tools that the entire it Operations organization relies upon, like the trouble ticketing system, monitoring, and running the change management meetings.
I walk past rows upon rows of cubicles, the same as every other building. However, unlike Buildings 2 and 5, I see peeling paint and dark stains seeping through the carpet.
This part of the building was built on top of what used to be the main assembly floor. When they converted it, they couldn’t get all the machine oil cleaned up. No matter how much sealant we put down to coat the floors, oil still has a tendency to seep through the carpet.
I make a note to put in a budget request to replace the carpets and paint the walls. In the Marines, keeping the barracks neat and tidy was not only for aesthetics but also for safety.
Old habits die hard.
I hear the noc before I see it. It’s a large bullpen area, with long tables set up along one wall, displaying the status of all the various it services on large monitors. The level 1 and 2 help desk people sit at the three rows of workstations.
