Other data points included the status of the life-support system, fuel readings for the ascent stage and communications. But soon, Mission Control advised Armstrong and Aldrin—in NASA parlance—they were “go” to stay.
But just after the four-minute ascent window had passed, a problem arose. A pressure reading in the fuel line leading to the descent engine was rising rapidly, with the temperature rising as well. Some of the fuel had frozen into a “plug” due to a surge of cold helium after the engine shut down, blocking the fuel line. Grumman engineers looked at the problem and knew if the temperature reached 400°F (204°C), the fuel could become unstable and explode. A procedure was quickly readied to instruct the crew to “burp” the line to release the pressure. But just as suddenly, the pressure and temperature dropped. The frozen plug had likely melted due to the rising temperature of the fuel. Everyone in Mission Control and at Grumman breathed a sigh of relief.
AFTER THE EAGLE HAD LANDED, EVERYONE involved with the Apollo Guidance and Navigation Computer collectively drew their breaths. Those in Boston at the Instrumentation Lab and the team members on-site at MSC needed to confer—quickly. Hardly anyone had heard of the 1202 and 1201 alarms, much less knew what they meant. Of course, Jack Garman knew the cause was an overload of data for the computer, but what caused the alarm to sound during Apollo 11’s landing?
“As Eagle sat on the lunar surface the evening of July 20, you can imagine how all of us on the ground felt, as we had just seen our software puke up its guts during the landing,” said Ken Goodwin, a Massachusetts Institute of Technology (MIT) system engineer who was stationed in the MER during Apollo 11. “We knew we had been saved by Hal Lanning’s priority-driven executive and waitlist operating system, coupled with NASA’s requirement to incorporate a restart capability into the landing software. But foremost in our minds, we knew the ascent program for leaving the moon was just as demanding computer-wise as the descent programs. We had just barely made it down to the lunar surface. How were we going to get back up again?”
A view of the Mission Evaluation Room during Apollo 11. Ken Goodwin is seated on the far right. Credit: NASA, image courtesy of Ken Goodwin.
The Guidance, Navigation and Control (GNC) team worked through the night to determine the problem. It turned out that the overload came from a combination of reasons, not just one: a late change to the configuration of the software and an underlying known issue with the computer’s electronics that was thought to have less than a 1 percent chance of ever happening.
Over the years, blame has been placed on a “checklist error,” the assumption being that Buzz Aldrin turned on the rendezvous radar when it wasn’t needed. The rendezvous radar’s job was to scan for the orbiting Command Module (CM) for a potential rendezvous. For landing, it normally wouldn’t be required. But in the event of a landing abort, having the radar on and ready to go could have been beneficial in terms of saved time and greater efficiency.
Having the rendezvous radar switched to the Lunar Module guidance computer (LGC) was a change that Aldrin had asked for. This change had been analyzed, studied and approved by several people involved with the software, and it seemed innocuous at the time. It was added to the crew’s checklist and timeline, so turning the radar on was not a crew mistake; Aldrin did exactly what the instructions told him. The problem came in that the change was made shortly before Apollo 11’s flight and the procedure wasn’t fully tested.
MIT computer team at the Instrumentation Laboratory during the Apollo 11 mission. Facing the camera are Eldon Hall and Dick Battin. Credit: Draper.
“What Buzz Aldrin thought—and a small group in the [Instrumentation] Lab concurred—is that it would save a little procedural time during a landing abort if the rendezvous radar was switched to the LGC versus the fully tested operational procedure of having it in the AUTO/SLEW switch configuration,” said Goodwin.
This setting meant the radar had to be manually positioned by an astronaut and that the radar wouldn’t send data to the computer. Unfortunately, what no one knew at the time was that the radar was inundating the computer with data, due to a combination of the procedural change in settings—which was made after most of the formal testing on the software had been completed—and the underlying problem with the computer’s electronics.
“The classic software belief is ‘don’t change your software at the last minute.’ Anyone in the software business knows that as a rule not to be violated, but that’s exactly what we did,” said Goodwin.
The underlying problem had to do with the phasing of two power supplies in the computer. The frequency between the two power supplies should have been frequency-locked and phase-synchronized. However, the original designs called for only frequency locking between the two. MIT member George Silver, who usually worked at Cape Canaveral, identified this as a potential problem before the Apollo 11 flight, but the Instrumentation Lab’s software engineers determined the chance of it causing a substantial issue was extremely small. With a short timeline and a low probability of failure, the software engineers decided to fly Apollo 11’s power supplies as they were, and the fix would be instituted for subsequent flights.
Unfortunately, the rendezvous radar was turned on at the exact moment where the phase synchronization would cause a problem. The computer couldn’t make sense of the radar settings, causing constant interruptions to the computer.
“Only in the LGC switch configuration would the underlying problem of the power supplies’ phase synchronization become a problem that had a 1 percent probability of occurring, based on when the computer was turned on,” said Goodwin.
As luck would have it, the power “on” sequence hit that 1 percent sweet spot that caused the rendezvous radar to overburden the computer. “When the computer dropped from a two-second cycle to a one-second cycle,” Garman explained, “suddenly there was this extra involuntary load on the computer, meaninglessly adding data to the cycles. It needed to run at over 100 percent capacity, meaning there wasn’t enough time to do everything—hence the problem.”
“George Silver realized what had happened,” Goodwin said, “and he got through to us in Mission Control, allowing us to get to the root cause. We were able to come up with a few workarounds to avoid the computer being overloaded on ascent and we had them instituted about eight hours before the astronauts would lift off from the Moon. We didn’t get any sleep, but knew we would avoid the problem during Apollo 11’s lift-off from the Moon.”
Solving this problem for the subsequent Apollo missions would require changes to the hardware. But could something else be hidden among the components that might cause the computer to overload on the next flight? “That started a search for problems that went on for years,” Garman said.
The serendipity of the computer alarm simulation that included Garman and SimSup Dick Koos can hardly be overstated. That essential and fortuitous training just fifteen days prior to the Apollo 11 Moon landing very likely changed the course of history.
At the Honeysuckle Creek Tracking Station during the Apollo 11 mission. Standing is Mike Dinn, deputy station director. Image courtesy of Colin Mackeller.
WHEN NASA OFFICIALS MADE THE decision to move up the timeline for the extravehicular activity (EVA)—the moonwalk—Mike Dinn was ready. Dinn was the deputy station director of the Honeysuckle Creek tracking station near Canberra, Australia, part of NASA’s Manned Space Flight Network of radio dishes that made communications for Apollo possible. Three 85-foot (26-m) antennas equally spaced around the world were located at Goldstone, California; Fresnedillas, Spain; and Honeysuckle Creek.
With the timing of the moonwalk, Honeysuckle Creek was the prime station assigned to receive the initial TV pictures from the Moon. Dinn had trained his team at the station with simulations, similar to how NASA trained astronauts.
“I had been in Houston for a meeting,” Dinn said, “and had the chance to take part in the simulations in Mission Control, where they were simulating communications and switching to
the various tracking stations. I was so impressed that when I returned to Australia, I instituted our own simulations. We trained and practiced and gave a lot of thought to our contingency plans for any possible failures.”
Dinn strived to make his team’s simulations as realistic as possible and even tried to find someone who spoke with an American accent to play the role of the astronauts; but alas, one of their own team had to play the part.
The televised view of astronaut Neil Armstrong descending the ladder of the Apollo 11 Lunar Module (LM) prior to making his first step on the Moon. Credit: NASA.
For television footage to be broadcast live from the Moon took several levels of coordination. NASA determined a heavy-duty camera used by the Department of Defense was capable of withstanding the rigors of space travel. The camera was stowed in just the right spot inside the LM’s Modularized Equipment Stowage Assembly (MESA), which held the equipment and experiments the astronauts needed for their EVA. As Armstrong stepped onto the first step of the LM, he pulled a release to open the MESA, allowing the camera to peek out from its insulated perch, just to the left of the LM’s ladder. Aldrin activated a circuit breaker inside the LM, turning the camera on. Armstrong’s first steps could now be visible to the estimated six hundred million people around the world watching this historic event on their televisions.
Back in Houston, Tom Moser—who had constructed the assembly for the US flag—watched tensely as Armstrong stepped on each rung. When Armstrong leaped from the last step, in a flash, Moser visualized that the ladder had failed, and a jagged edge of metal might penetrate the astronaut’s pressurized spacesuit—all because of the last-minute decision to add the flag to the flight. But all was well. Armstrong just needed to jump down to the Moon. He had landed Eagle so gently that its shock absorbers didn’t compress as designed, making the ladder about 3 feet (1 m) higher above the lunar surface than expected.
At 9:56 p.m. Central time on Sunday, July 20, Armstrong first set foot on the Moon, saying, “That’s one small step for a man, one giant leap for mankind.”
One of the only images of Neil Armstrong on the Moon during the historic first EVA on the Moon. Armstrong is standing at the modular equipment storage assembly (MESA) of the Lunar Module. Credit: NASA.
Immediately Armstrong began to share what he was experiencing: The nature of the lunar surface was a fine, powdery material. He noted he sunk in only ¼ inch (0.5 cm) or less and that the LM footpads had penetrated only a few inches. He observed that the exhaust of the descent engine had not cratered the area directly below the LM engine nozzle. Like a tourist, Armstrong took pictures and—after some prodding from Mission Control—collected the first lunar samples, which he stowed in a spacesuit pocket.
Aldrin soon followed him out, and the camera was positioned on a tripod about 30 feet (9 m) from the LM. People on Earth watched in wonder as the two astronauts quickly learned how to skip around the landing site in the one-sixth lunar gravity, collecting rocks and soil samples and setting up their experiments, the Apollo Scientific Experiments Package. When they prepared to place the US flag on the lunar surface, the telescoping top support bar would not fully deploy. This gave the partially extended flag the appearance of waving in the breeze. The astronauts talked with President Nixon, then placed commemorative medallions on the Moon’s surface that bore the names of the three Apollo 1 astronauts and two cosmonauts who gave their lives in the pursuit of space exploration. A 1½-inch (4-cm) silicon disk containing miniaturized goodwill messages from seventy-three countries also stayed behind.
Buzz Aldrin egresses the Lunar Module Eagle and begins to descend the steps of the LM ladder as he prepares to walk on the Moon. This photograph was taken by astronaut Neil Armstrong during the Apollo 11 extravehicular activity (EVA). Credit: NASA.
A view of Mission Control in Houston during the Apollo 11 moonwalk. Credit: NASA.
The entire EVA lasted more than two and a half hours. Norman Chaffee watched from home with his family, Jerry Woodfill took his shift in the MER, Frank Hughes watched from Mission Control, Jerry Bell and Ken Young stayed at MSC even though their and shift had ended. No one wanted to miss the moment they worked so hard to make possible.
Armstrong and Aldrin returned to the LM and tried to sleep. Overnight, engineers worked in Houston to solve the two problems that weighed heavy on everyone’s minds: Where was Eagle on the lunar surface, and would the broken switch for the ascent engine work?
The view of Earth from lunar orbit, as seen by the astronauts of Apollo 11. Credit: NASA.
There was only one way to figure out their location. The Capcom woke Aldrin early to perform a rendezvous radar check. The vectors between the LM and the CM in orbit allowed engineers in Mission Control to determine the landing site. It was about 5 miles (8 km) away from any spot they had pinpointed earlier.
As Armstrong, Aldrin and Collins prepared for lift-off and rendezvous, engineers at MSC devised a potential workaround for the broken ascent engine switch. But Aldrin came up with his own solution. He used his felt-tip Duro Pen marker and pushed in the breaker. The circuit became live, and the crew was now ready to proceed with the countdown to leave the Moon.
In what many considered one of the riskiest moments of the entire mission, Armstrong and Aldrin activated the ascent engine, while explosive bolts separated the top stage from the bottom descent stage. It all worked perfectly, and the ascent stage zoomed to orbit to meet with Columbia for docking. The astronauts transferred the lunar samples and film canisters to Columbia, performed the required maneuvers to prepare for leaving the Moon and headed for home.
The view out the window of the Lunar Module after the first EVA on the Moon. The flag and the astronauts’ footprints are visible on the lunar surface. Credit: NASA.
The view of the Apollo 11 Service Module during reentry of Earth’s atmosphere, as seen from a NASA KC-135 aircraft. Credit: NASA/JSC, high-resolution version courtesy of Colin Mackeller
ABOUT FIFTEEN MINUTES BEFORE REENTRY into Earth’s atmosphere, Apollo 11’s service module (SM) was separated from the CM, with the SM jettisoned away to prevent further contact between the two vehicles. However, about five minutes after the two craft separated, the crew reported seeing the SM fly by:
Aldrin: “Houston, we got the Service Module going by. A little high and a little bit to the right.”
Capcom Ron Evans: “Roger. Thank you.”
Collins: “And it’s rotating just like it should be. Thrusters are firing.”
Evans: “Good. It’s got a lot of gas there to burn out too.”
Aldrin: “It’s coming across now from right to left.”
Evans: “Houston. Roger.”
Several minutes later, reentry began, with a blackout period of communications for about three minutes and forty-five seconds.
Just a few seconds after the loss of communications with the Apollo 11 crew, Captain Frank A. Brown, a Qantas Boeing 707 pilot flying about 450 miles (725 km) away from the reentry point, reported live on Australia’s ABC radio that he could see the reentry of the two vehicles.
“I see the two of them, one above the other. One is the Command Module; the other is the Service Module…. I see the trail behind them—what a spectacle! You can see the bits flying off. Notice that the top one is almost unchanged while the bottom one is shattering into pieces. That is the disintegrating Service Module.” Later, he said the breakup of the SM “lit up the darkness over the Pacific Ocean like daylight.” Also, Colonel Oakley Baron of the Apollo Range Instrumentation Aircraft communications fleet photographed the spectacular, fiery Apollo 11 reentry from his USAF aircraft flying at 43,000 feet.
Seemingly, what no one realized at the time—not the astronauts, the Capcom or others in Mission Control, or those who saw the disintegrating SM light show—was that this series of events should never have happened as they did. The SM and CM traveling close to each other after separation and during reentry was a significant and serious mission anomaly.
�
�The Service Module should have been nowhere close, absolutely nowhere close to the Command Module as they began reentry,” said Gary Johnson. “If things had gone bad, we could have lost the Apollo 11 crew. If the SM had contacted the CM in the entry corridor at their speeds of about 25,000 miles per hour, the result would have been catastrophic.”
Additionally, pieces of the disintegrating SM striking the CM could have been disastrous.
If the separation and reentry sequences had worked as intended, a series of thruster firings on the SM would have moved the two vehicles apart, with the SM shifting to a different trajectory, where it should have then skipped out of the atmosphere into a highly elliptical Earth orbit, to reenter the atmosphere again at a later time. This would have eliminated the risk of contact with the CM during reentry.
But that didn’t happen. And an investigation into this anomaly found the same near-catastrophe also took place on the two previous missions coming back from the Moon, Apollo 8 and Apollo 10. In those cases, the crews didn’t happen to see the SM after separation, so this problem didn’t become evident until after Apollo 11’s reentry anomaly.
Johnson isn’t sure who caught the problem, but during the crew technical debriefing session after the Apollo 11 astronauts’ return to Houston, the crew mentioned seeing the SM after separation.
The interviewer asked, “Did you see the Service Module?” Collins said, “Yes. It flew by us.” Aldrin added, “It flew by us. It flew by to the right and a little above us, straight ahead. It was spinning up. It was first visible in window number 4, then later in window number 2, really spinning.”
“People going through the crew debrief transcript saw this and said, ‘Whoa, wait a minute, that shouldn’t have happened,’“ said Johnson. “A formal investigation was conducted and they looked at radar data and pictures taken from airplanes of the reentry and all the data showed that, sure enough, the two vehicles were coming in on the same entry corridor. Then they went back and looked at Apollo 8 and 10 radar data and, lo and behold, they found the same thing.”
Eight Years to the Moon Page 31
