“We had a problem with shore power. We violated a red tag.”
Ugh. My heart sank. Anything but shore power, I thought. Santa Fe was under scrutiny for previous maintenance and procedural mistakes, some of which had been with shore power. Continued problems with shore power would indicate that we hadn’t moved beyond poor past practices.
In this case, a sailor had energized breakers on the pier after the conditions for energizing them had been met (so there was no hazard), but he hadn’t cleared the red tag before doing so (indicating we were just lucky). You don’t want to be accidentally safe.
While I was directly and immediately accountable to Submarine Squadron Seven, and Commodore Mark Kenny, for the performance of Santa Fe, I was also accountable to Naval Reactors for the safe operation of the reactor plant. Naval Reactors is the organization set up by Admiral Hyman Rickover to build, maintain, manage, man, and certify naval nuclear power operations. It has an incredible record of success as a result of well-thought-out management processes. One of the reasons for success is that each port has an independent Naval Reactors team that reports back through a special chain of command directly to the director, a four-star admiral.
To understand the importance of this, recall the Enron–Arthur Andersen scandal. When Enron imploded in 2001, Arthur Andersen, the auditor, was earning $25 million annually from audit fees and an additional $25 million from consulting fees. They were inspectors and performers. Human instinct gets in the way of adequate inspection and enforcement when an individual or a group is also responsible for correcting deficiencies in performance. The Naval Reactors local field representatives are structured in a way that would make such conflicts of interest impossible. They are chartered to ensure safe reactor plant operations, period. They are freed of the burden of worrying about how hard things are, what the effect will be on retention of another Saturday training session, or how a delayed underway will impact the operational commander. This independence frequently aggravates ship drivers like me because Naval Reactors appears to be obstructionist, but they play a critically important role. It’s one of the reasons for the long-term success of the program.
No one was hurt. Nevertheless, the engineer said he would report the problem up the chain, to both Squadron Seven and Naval Reactors. Ugh again. There was guidance on what kind of problems should be reported to which organization. This seemed to fall on the border, and I was tempted to handle it “in-house.” Why did we need all this outside attention just as things were starting to go well? My instincts were to somehow protect my people from the scrutiny of these outside organizations. We could have not reported it; they would likely have never known. On the other hand, reporting it would invite additional monitor watches, possibly additional periodic and one-time reports, skepticism about the competency of Santa Fe’s leadership, and a lot of management time.
Rick was adamant, and he was right. We set up a critique for the next day, Saturday, and he called his counterparts at Squadron Seven and the Naval Reactors office and invited them to the critique. I called Commodore Kenny and told him as well. I fought off any thought of trying to let our problem slide by and openly welcomed the oversight organizations into our tent.
We called this idea of being open and inviting outside criticism “Embrace the inspectors.”
Even so, Saturday was going to be a long day.
Mechanism: Embrace the Inspectors
We applied “embrace the inspectors” not only to one-time critiques and problems such as the shore power mishap, but also to entire inspections. We would utilize the inspectors to disseminate our ideas throughout the squadron, to learn from others, and to document issues to improve the ship.
This mechanism sends the signal that we are in charge of our destiny, not controlled by some force. It runs counter to the instincts expressed by many of my officers and chiefs to minimize the ship’s visibility to the outside, especially when problems were involved. EMBRACE THE INSPECTORS is a mechanism for CONTROL, organizational control. In other words, the crew of Santa Fe are responsible for Santa Fe. We found we needed this parallelism with internal control. Later, we’d hand out T-shirts that jokingly read, “DON’T BE A VICTIM.”
Concerning areas where we were doing something exceptionally innovative or expertly, we viewed the inspectors as advocates to share our good practices with. Concerning areas where we were doing things poorly and needed help, we viewed them as sources of information and solutions. This created an atmosphere of learning and curiosity among the crew, as opposed to an attitude of defensiveness.
Later on in my command, Santa Fe had a material inspection by a group of officers from the Board of Inspection and Survey (INSURV). Their reports carry significant weight and expose the submarine force to “big Navy” observers. Officers have lost their commands over bad INSURV inspections. When the INSURV team reported to our submarine, I handed them a list of known deficiencies. These were things that were so fundamental to the design or so difficult to repair that we had been unsuccessful. By getting them documented in the INSURV report, we ensured that the Navy would apply resources to fixing the problems, thus making all submarines more effective warships.
Embrace the inspectors turned out to be an incredibly powerful vehicle for learning. Whenever an inspection team was on board, I would hear crew members saying things like, “I’ve been having a problem with this. What have you seen other ships do to solve it?” Most inspection teams found this attitude remarkable.
As a result, Santa Fe was getting superior grades on inspections. Over time our sailors learned a lot and became incredibly good at their jobs; they also continued to evince a hunger for learning.
• • •
Embrace the inspectors can be viewed as a mechanism to enhance competence, but I think it fits even better in the discussion of control because it allowed us not only to be better submariners but also to maintain control of our destiny.
QUESTIONS TO CONSIDER
How do you use outside groups, the public, social media comments, and government audits to improve your organization?
What is the cost of being open about problems in your organization and what are the benefits?
How can you leverage the knowledge of those inspectors to make your team smarter?
How can you improve your team’s cooperation with those inspectors?
How can you “use” the inspectors to help your organization?
PART III
COMPETENCE
One of the two pillars that support control is competence. Competence means that people are technically competent to make the decisions they make. On a submarine, it means having a specific technical understanding of physics, electricity, sound in water, metallurgy, and so on.
The emphasis in the book thus far has been on pushing decision making and control to lower and lower levels in the organization. We found, however, that control by itself wasn’t enough. The chapters in this part will focus on the mechanisms we employed to strengthen technical competence. They are:
Take deliberate action.
We learn (everywhere, all the time).
Don’t brief, certify.
Continually and consistently repeat the message.
Specify goals, not methods.
“Mistakes Just Happen!”
Are you content with the reason “Well, mistakes just happen” when it comes to managing your business? We rejected the inevitability of mistakes and came up with a way to reduce them.
January 30, 1999: In Port, Pearl Harbor (150 days to deployment)
Saturday morning and the wardroom of the USS Santa Fe was packed. The petty officer who had caused the red tag violation, members of our watch team, the engineering officer of the watch, the engineer (Lieutenant Commander Rick Panlilio), the XO, the division officer, the COB, and the senior nuclear chief (Chief Brad Jensen) were sitting around the table. In addition, we had the observers from Squadron Seven and Naval Reactors.
I sat at the head of the table w
ith the flashlight in front of me, thinking about how to approach this critique. It wasn’t going to be good enough to just have a bunch of empowered people; we needed actually to be better.
The petty officer involved was a well-intentioned sailor who’d never been in trouble. I was sympathetic to the crew, who had worked incredibly hard over the past two weeks to get the ship under way, conduct our training, do the inspection, and accommodate all the changes. This was something I would wrestle with my entire command tour—balancing the courage to hold people accountable for their actions with my compassion for their honest efforts. We would need to understand what had happened, and I didn’t want to take the easy way out and blame the petty officer who had moved the tag in error.
One measure of discipline in a military unit is the number of captain’s mast cases. Captain’s mast, also known as nonjudicial punishment (NJP), is a form of military justice that allows the captain to invoke near-immediate punishment without a trial by court-martial. Punishments are classified as administrative and are limited generally to forfeiture of pay, reduction in rank, or restriction to the boat. On board Santa Fe, there had been a couple captain’s masts a month and that was too many.
It was widely assumed that if you violated a red tag you would go to captain’s mast. The idea was to convey that this was important business and you had to pay attention. While that was true, I didn’t believe in invoking a captain’s mast automatically.
Eventually my department heads and chiefs would lead critiques, but I needed to lead this one. When I opened the meeting, no one—least of all me—expected to be there for eight hours.
“Let me start by welcoming the squadron and Naval Reactors representatives.”
Several documents lay before us on the table: the procedure, a watch bill, and the tags themselves, among others. Later on during my command we would end up with a finely honed approach to conducting these critiques, but at this point, it was a bit ad hoc. We were developing the methodology as we went along. (To see where we ended up, and for a more detailed process for conducting critiques, visit davidmarquet.com to read “How we learn from our mistakes on nuclear submarines: A seven-step process.”)
I opened the proceedings.
“Petty Officer M, can you tell me what happened?”
“Well, I knew we met conditions to shut the breaker, and I was just thinking that was the next step in the procedure. We had the procedure out and had reviewed it. I knew the red tags were hanging but just moved them aside to shut the breaker. Not sure what I was thinking.”
Gasps.
“You moved a red tag aside?”
“Yes, it was hanging right in front of the breaker. There was one on each of the three pier breakers, three across, right there.”
Murmuring.
I’m sure he was expecting to go to captain’s mast and be fined. Yet, he was willing to tell us the truth quite bluntly without any attempt at obfuscation. This needed to be rewarded.
“Thank you very much for your candor. You and the rest of the watch team can go home. Supervisors stay behind.”
This caused a stir. What, no recriminations? No captain’s mast? No yelling?
I was taking a risk. If we later discovered that someone’s actions were sufficiently neglectful to warrant punishment I would have painted myself into a box. However, I felt the candor and honesty of Petty Officer M were more important than continuing the current process of inquisition, fear, and punishment.
“Now, gentlemen, how are we going to prevent this from happening again?”
And that’s what we spent the next seven and a half hours talking about.
Mechanism: Take Deliberate Action
We ran through all the usual suspects. First, it was suggested that we do some refresher training, a commonly proposed solution.
“Let me ask you this. Training implies a knowledge deficiency. I should be able to identify that with a test. So what question on a test do you think any of these guys would have gotten wrong?” No one could think of one. It wasn’t a knowledge deficiency, and training wasn’t the solution.
“We need to add supervision.” This is another favorite solution, like adding the XO to the chart review process. We discussed what a supervisor would do, where he would stand, and how he could have prevented this mistake. Grudgingly, it was agreed that adding a supervisor might have prevented shutting down the second and third breakers, but not the first. Anyway, we already had significant supervision of the event through the Chief in Charge, the watch officer, the electrical division officer, and the engineer. If all those supervisors hadn’t prevented it from happening, how would adding another one help? No one could think of the mechanism by which an additional supervisor would have prevented the mistake.
I pushed the team to come up with something that would have prevented the mistake in the first place. Exasperated with my unwillingness to accept any of the rote answers, someone blurted out, “Captain, mistakes just happen!”
Now we were getting somewhere. We discussed what it would take to reduce mistakes made at the deck plate level, at the interface between the operators and the equipment, not simply discover them afterward. These were mistakes such as turning the wrong valve, opening the wrong breaker, and moving red tags—actions no one consciously meant to do.
“Sir, it’s attention to detail.” This was a commonly used phrase as well, but telling the men to pay more attention didn’t seem likely to make a difference in the long run in the number of mistakes. We’d tried that before.
“How so?”
“Well, he was just in auto. He didn’t engage his brain before he did what he did; he was just executing a procedure.”
I thought that was perceptive. We discussed a mechanism for engaging your brain before acting. We decided that when operating a nuclear-powered submarine we wanted people to act deliberately, and we decided on “take deliberate action” as our mechanism. This meant that prior to any action, the operator paused and vocalized and gestured toward what he was about to do, and only after taking a deliberate pause would he execute the action. Our intent was to eliminate those “automatic” mistakes. Since the goal of “take deliberate action” was to introduce deliberateness in the mind of the operator, it didn’t matter whether anyone was around or not. Deliberate actions were not performed for the benefit of an observer or an inspector. They weren’t for show.
Our mechanism to prevent recurrence of the problem was to implement the taking of deliberate actions on board Santa Fe. I would take no punitive action against the honest petty officer who had pushed aside the red tag. The Squadron Seven and Naval Reactors observers would go back and brief their supervisors on our plan, and they would make an assessment of Santa Fe and me. Since deliberate action seemed like a useful concept and I was a new captain, I figured they would withhold judgment and just see how it played out. That’s what I was banking on, at any rate, because we needed more time to implement the changes that would make the sub and its crew excellent.
On Monday we had quarters on the pier to discuss the concept “take deliberate action” with the crew. I first explained what had happened with the red tag and the critique of the incident, and then I described what thinking deliberately meant and why we were going to do it. Even though it wasn’t presented as a bargain, I think that the crew, knowing their shipmate had been spared captain’s mast, were more receptive to the alternative—take deliberate action.
Deliberate action was accepted by the nuclear-trained personnel fairly readily because it built on a concept they had been exposed to at nuclear power school called “point and shoot.” Unfortunately, deliberate action was a tough sell with much of the rest of the crew, and we would ultimately pay for that.
Deliberate Action Is Not for Show
I believe “take deliberate action” was the single most powerful mechanism that we implemented for reducing mistakes and making Santa Fe operationally excellent. It worked at the interface between man and machine: where petty officers
were touching the valves, pumps, and switches that made the submarine and its weapons systems work. TAKE DELIBERATE ACTION is a mechanism for COMPETENCE. But selling the crew on this mechanism’s value was hard going.
One problem in getting the crew to perform deliberately was the perception that deliberate action was for someone else’s (a supervisor’s, an inspector’s) benefit. Even though we continually talked about how deliberate action was to prevent the individual from making silly mistakes, I would overhear sailors discussing deliberate action among themselves in this misperceived way.
The second problem was overcoming the perception that deliberate action was something you did as a training exercise, but in a “real situation,” you would just move your hands as fast as possible. I used the following thought experiment to dispel this error: Suppose we are conducting a training drill around Pearl Harbor and the ship loses all propulsion due to errors. What happens? We would surface and call for help, which is nearby. We’d critique the event and write the appropriate reports. No one would die. What happens, however, if we lose all propulsion in a “real situation” in the face of the enemy due to errors? Now people might die. The key is that as the importance of doing things right increases, so does the need to act deliberately.
How Can You Implement Deliberate Action?
Turn the Ship Around!: A True Story of Turning Followers into Leaders Page 12