Eight Years to the Moon

by Nancy Atkinson

  Even though many design elements for the computer hardware were now falling into place, a nagging issue remained for the Instrumentation Lab: memory. The original design, based on the Mars Probe, had just 4 kilobytes words of fixed memory and 256 words of erasable (a word is a fixed-size piece of data handled by the computer’s processor). As NASA added more aspects to the Apollo program, the memory requirements kept rising: to 10, 12, 16, 24 and finally to 36 kilobytes of fixed memory and 2 kilobytes of erasable. The added memory needs meant adding more of the core-rope components to the computer. By the standards of the time, that was a relatively large amount of data.

  “The nice thing about the core-rope memory was that it was an electromechanical memory,” said Battin, “which really meant that you could not lose your memory. That is, if you had an electrical failure or a similar problem, it would still work. The only way you could lose this type of memory is if you destroyed it physically.”

  Battin called the process for constructing the core-rope memory the LOL method.

  “Little Old Ladies,” he said. “Women in the Raytheon factory would literally weave the software into this core-rope memory.” While women primarily performed the weaving, they weren’t necessarily elderly. Raytheon employed many former textile workers adept at weaving, who would follow detailed instructions from a computer tape to weave a nickel alloy wire through the tiny magnetic “doughnuts” to create the non-erasable memory. At first, the process for constructing the core-rope memories was quite labor-intensive: Two women would sit across from each other, each weaving by hand a stream of wires through tiny magnetic cores, pushing a probe with the wire attached from one side to the other.

  In the language of computer ones and zeros, if the wire was a one, it ran through the doughnut; if it was a zero, the wire ran around it. For one memory component, it took bundles of a half mile of wire woven through 512 cores. One module could store more than sixty-five thousand pieces of information. By 1965, a more mechanical method of weaving the wires was implemented, based on machines used in New England’s textile industry. But still the process was extremely tedious, and one program could take several weeks or even months to weave, with more time needed to test it. Any errors in the weaving meant it had to be redone. The CM computer contained six sets of core-rope modules, while the LM computer held seven.

  In total, there were approximately thirty-thousand parts in the computer. Each component would be put through an electrical test and a stress test. Any failure called for rejection of the component.

  “Even though the memory was reliable,” Battin said, “the thing NASA didn’t like about it is was the fact that very early on you needed to decide what the computer program was going to be. They asked us, ‘What if we had a last-minute change?’ And we said, ‘We can’t have last-minute changes, and anytime you want to change the memory means a six-week slippage, minimum.’ When NASA said that was intolerable, we told them, ‘Well, that’s the way this computer is, and there isn’t any other computer like it that you can use.’“

  The Instrumentation Lab also had to come up with a method to check whether the software actually worked. With no means of testing it in genuine spaceflight conditions, they needed to find a way to do it virtually with a computer. A young MIT graduate student named Bill Widnall was instrumental in developing this capability, researching the mathematical models it would take to create such a computer simulation.

  “We developed simulations that were entirely digital, no analog interfaces with astronauts or spacecraft,” said Widnall. “We could carry this out in a general-purpose ground computer and test out the proposed spacecraft computer program by simulating how every instruction worked. We could determine if there was lack of precision, the amount of time it took to execute each instruction and the interfaces between the information coming in and the commands going out.”

  The Instrumentation Lab’s groundbreaking work allowed them to simulate the space environment exactly, no approximations. The digital simulations let them test situations that would be difficult to test, even in the “real” world. The lab was able to answer multiple questions: How would fuel slosh in zero gravity? What is the lag in time for a thruster jet to turn on until it gets to full thrust? If the main engine was gimbled back and forth improperly, could the docking tunnel between the CM and LM be damaged or break?

  Widnall compared their concept to how—at about the same time in history—physical tests in wind tunnels were being phased out in favor of computational fluid dynamics.

  Testing equipment at the Instrumentation Laboratory. Credit: Draper

  “By having an all-digital simulation, we were able to explore many more situations in greater detail,” said Widnall. “We were at the forefront of realizing that you can do a lot through simulation rather than having to test everything with actual all-up hardware. Additionally, the simulations were critical in making sure there were no defects in the software once the missions were under way, since we couldn’t have the astronauts go test it ahead of time out in space to see if it worked.”

  These computer simulations did expose flaws in the software design, allowing modifications to be made before the actual flights in space.

  Widnall was also part of the team that developed the concept of a digital autopilot for the AGC. Apollo would need a more advanced control algorithm to automatically operate the large engines and the small reaction control jets.

  “This is analogous to figuring out how to drive your car from one red light to another red light in the minimum amount of time,” said Widnall. “If you were a drag racer, you would not gently use the accelerator; you’d floor it. But you’d need to know when to use the brake in just the right manner so you wouldn’t run through the red light but stop at the right place.”

  Of course, the dynamics in space are different, but with a digital autopilot, the computer would be able to determine how long an engine or thruster would need to fire for a specific operation—a big burn to enter orbit around the Moon or small thrusts to perform rendezvous.

  Astronaut Rusty Schweickart goofing off, with his helmet backwards, while testing the optics for the Apollo Guidance Computer at MIT. Credit: Draper.

  The computer’s real-time operating system allowed astronauts to enter simple commands, and the interface was called the Display and Keyboard (DSKY; pronounced like “diss-key”). The DSKY was also built by Raytheon and designed by Ramon Alonso at the Instrumentation Lab. The interface consisted of a simple numerical keyboard (no letters), a row of status lights and a set of lighted numerical indicators. Astronauts instructed the computer by keying in numerical codes in a “verb-noun” sequence; for example, if they wanted the computer to perform the translunar injection burn to send the spacecraft on a trajectory toward the Moon, they needed to run program 15. This meant punching the Verb button, then the numbers 3 and 7, and then punching the Enter button to change the program. Then the astronaut would push the Noun button and enter numbers 1 and 5 and punch Enter.

  All the verb and noun inputs were documented on the astronauts’ checklists and a set of “cue cards” held together by rings kept near the DSKY. The information included reference data, such as the list of verbs and nouns, alarm codes and program details. AGC operation may sound confusing, but the general consensus among engineers and astronauts alike was the Apollo computer was easy to operate, especially after repeated use in the simulators. After a while, operations became second nature, NASA computer engineer Jack Garman said. “It’s like playing the piano—you don’t have to see your fingers to know where they are.”

  Integrated into the computer was the guidance and navigation system, which would measure every change in speed and direction and keep track of the spacecraft’s position. In space, everything is moving, and the system would allow the Apollo astronauts to reach their target.

  While designing and building all the hardware posed challenges, as work progressed on the AGC through 1965 and into 1966, the magnitude and complexity of
one particular aspect stood out: programming the software. It became the defining problem of the computer in meeting both timelines and specifications.

  “Software development was not very mature in the mid-sixties,” said Garman. “The notion of modules and subroutines either didn’t exist, or they were very far-out concepts. All programming was basically done at the ones-and-zeros level, assembly language programming.”

  The Instrumentation Lab assigned what they called “rope mothers,” a nonsexist phrase referring to the lead person “parenting” the development of the software. They were accountable for making sure the software was programmed correctly. Since the programs for each flight would be different, each set of core-rope memory modules would be diverse. The programmers worked on several flights at once, attempting to keep up with the ambitious timeline of upcoming missions. Rope mothers helped keep everything in order.

  In designing the software to conduct complicated tasks, the software engineers needed to come up with ingenious ways to fit the code within the memory constraints. And of course, none of this had ever been done before—at least not to this level of scale and complexity. At any given time, the AGC might have to coordinate several tasks at once: taking readings from the radar, calculating trajectory, performing error corrections on the gyros and determining which thrusters should be fired, as well as transmitting data to NASA’s ground stations and taking new inputs from the astronauts.

  Hal Laning devised what he called an executive program, which assigned tasks different priorities and allowed high-priority tasks to be done before low-priority ones. The computer could allocate memory among different tasks and keep track of where a task had been interrupted.

  Astronauts visit the Instrumentation Lab, pictured with Draper employees. Astronauts in the front row are Gus Grissom, Roger Chaffee, Dave Scott, Jim McDivitt, Rusty Schweickart and Ed White. Draper employees include David Hoag, Russ Larson, Dick Battin, Jim Nevins, Jack Dunbar and Gerry Levine. Credit: Draper.

  “The software was designed so that it would memorize where it was so that if it was interrupted, it could jump back and restart,” said Garman. “These were called the ‘breakout points,’ and the computer could memorize data in such a way that if something was interrupted, it could go back to the prior step and restart. This was the way it could recover from transient failures too.”

  However, by the fall of 1965, it became evident to NASA the AGC was in serious trouble, as the development of the programs was significantly behind schedule. The fact that a relatively unknown quantity called software could delay the entire Apollo program was not well received by NASA hierarchy.

  And that’s when Bill Tindall stepped in as a troubleshooter. He was eventually named chief of Apollo data priority coordination, responsible for all guidance and navigation computer software development by the Instrumentation Lab. He maintained his wide-ranging responsibilities at MSC for mission planning with the Mission Planning and Analysis Division, but he now added weekly visits to MIT to his schedule, clamping down on the Lab’s ability to meet deadlines.

  “Bill was the person at NASA that would just dive right in wherever he saw a knotty problem,” said Catherine Osgood, who worked with Tindall on the rendezvous team. “And when he just barely got that under control, he was into the next one. When he found the onboard software was just stumbling along, he decided that that was his next project. He called it a big bag of worms.”

  A portion of a “Tindallgram” from March 7, 1968, a memorandum from NASA’s Bill Tindall.

  Tindall went through the Lab’s work with a fine-tooth comb and quickly grasped the key issues and clearly characterized the proposed fixes.

  “Tindall came down hard on the Lab,” said Ken Goodwin, who would join the MIT Instrumentation Lab in 1967. “He would come to MIT in what we called Black Friday Meetings. He would say, ‘Don’t do this, take this out. Why do you need that?’”

  Tindall’s oft-used motto was “Better is the enemy of good,” and he’d tell the people at MIT, “Look, we just want the basic thing to work, no fancy displays, bells and whistles.”

  Of course, that rubbed some at the Instrumentation Lab the wrong way. “They were these gifted people, doing things that had never been done before,” said Goodwin, “but here was Tindall saying, ‘Hey, if we are going to meet schedule, we need to cut out the riffraff and get down to business.’”

  Tindall documented all the issues and meetings in his notorious “Tindallgrams.”

  “Bill’s Tindallgrams were written just the way he talks,” said Osgood, “but he left out the swear words. For Bill, a swear word was an adjective. It just flowed with the rest of his conversation.”

  Most of Tindall’s memos were long and detailed but also included well-placed constructive criticism. In his first MIT Tindallgram, Tindall defined the work at the Instrumentation Lab as “sloshing through the mud.” Another early memo said, “Well, I just got back from MIT with my weekly quota of new ulcers, which I thought might interest you.” Next came, “This is another of my gripping reports on the status of the Apollo spacecraft computer programs.”

  “Even though he kicked our butt, we ended up being in awe of what Tindall was doing,” said Goodwin. “He found duplication in the software that made it extremely slow, he stopped the tendency for writing lengthy test programs to try out different things. He told us, ‘This is not a research project; you have to get serious.’”

  But as the Data Priority meetings progressed, Tindall’s criticism of the Instrumentation Lab’s effort softened as he came to a better understanding of the complexity of the tasks at hand. He did everything he could to provide additional manpower and equipment from MSC and other contractors, such as assigning NASA personnel to short-term residence at the Lab or arranging for the Lab to use some of the test facilities at the contractor sites.

  Tindall helped institute certain restrictions, such as the requirement that the software be designed to use only 85 percent of the computer’s total capability, always leaving 15 percent in reserve in case of an emergency or problem.

  “The 85 percent was a software design goal, a performance target on the computer’s hardware resources to execute the software,” said Goodwin. “Due to the interrupt-driven executive design of the software, the computer’s processing load was somewhat nondeterministic depending on what the computer was faced with during various points in the mission.”

  Another necessity of Tindall’s was the refining of the breakout points for how the computer interrupted tasks. Tindall also astutely realized the complexities of the computer interfaces between all the systems on board the spacecraft. He oversaw what he called an interface control document, which detailed how one system interfaced with another and listed any change to one system and how it might affect another. Since everything with Apollo continued to happen at breakneck speed, it was hard to keep everyone in the loop. But if engineers or astronauts asked for a specific change in software or procedures, the interface control document could record and share that change among all systems managers.

  Working toward landing on the Moon was “an incredible and audacious task,” wrote David Hoag, the program manager for the Instrumentation Lab, “and the guidance equipment for the mission was created out of primitive principles, prolific imagination and a lot of hard work.”

  GEMINI 7 AND 6

  This wasn’t the original plan, but Gemini 7 ended up launching before Gemini 6, and the two missions combined for the first true rendezvous in space. Gemini 6’s planned launch in October 1965 was thwarted when an unpiloted Agena spacecraft—one that Gemini 6 was supposed to conduct rendezvous maneuvers with—blew up during liftoff. NASA and the Gemini spacecraft contractor McDonnell Aircraft hatched the idea to launch Gemini 7 first, and to change Gemini 6’s mission to rendezvous with Gemini 7 instead, which was scheduled to conduct the longest flight of a US spacecraft to date, fourteen days.

  Gemini 7 launched on December 15, 1965, and astronauts Frank Borman and Jim Lovel
l demonstrated their ability to do experiments in a “shirt sleeve” environment and the lightweight pressure suits, and the capability of staying in space for a two-week period of time.

  Eight days later, on December 12, Wally Schirra and Tom Stafford sat in their spacecraft waiting for launch when the engines started but cut one second after engine ignition because an electrical umbilical separated prematurely. This was the first time an astronaut mission was aborted after ignition start. The mission launched successfully three days later on December 15. Gemini 6 caught up to Gemini 7 and the two Gemini spacecraft flew in zero relative motion with between 45 and 120 feet (14 and 37 m) between them. Station-keeping maneuvers involved the spacecraft circling each other and approaching and backing off, where all four astronauts took turns flying in formation for over five hours. Gemini 6 returned to Earth on December 16, while Gemini 7 remained in Earth orbit and reentered two days later.

  Top left: A photograph of the Gemini 7 spacecraft—nose toward camera—was taken from the Gemini 6 spacecraft during rendezvous and station-keeping maneuvers at an altitude of 163 nautical miles during orbit number 5, on December 15, 1965. Credit: NASA.

  Bottom left: Astronauts Frank Borman (right), command pilot, and Jim Lovell for the Gemini 7 mission. Credit: NASA.

  Top right: Frank Borman using the visual acuity device and a portable mouth thermometer during his experiment in space. Credit: NASA.

  Bottom right: Gemini 7 (left) and Gemini 6 spacecraft meet up once again, this time at Mayport Naval Station near Jacksonville (Florida) after unloading December 20, 1965, from the carrier USS Wasp. Credit: NASA.