Collected Essays

by Rucker, Rudy

  Spending Your Triangles

  Recently I’d been worrying that e-business—whether booming or busting—might eat up all available mind share for thinking about computers. Even though the e-bust has come, the media continues to slobber over the same trivial, dead-end, greed-headed stuff. Must e-tailing and networking forever dispossess such wonderful aboriginal CS topics as Virtual Reality, Chaos, Fractals and Other Gnarl?

  I’ve been less worried about this since last spring, when I went to the Game Developers’ Conference in my native San Jose, California. Everything great about computers is still alive and well in the world of videogames. Here, for your consideration, is my report.

  Outside in the park are some homeless San Jose people. Our fair city. A chunky blonde street-girl is chewing an enormous shiny nail, like a ten-penny nail, its head sticking out of her mouth. Two phrases cross my mind: “Tough as nails,” and, “Biting her nails.”

  I’m Artificial Life aficionado from way back. Called “Alife” for short, this field studies how to create computer simulations of things that behave like living creatures. In a thorough-going Alife simulation, the creatures will even breed and evolve. Alife was big in the 1980s, but it’s kind of died out. Like Artificial Intelligence, Alife failed to deliver on its initial wild-eyed promises. Simulations don’t in fact evolve into cool things very fast. If you regard Earth as a large, specialized computer, you’ll observe that it’s been running for billions of years, parallel processing itself at every point of space, pumping along at an update speed limited only by things like Planck’s constant and the speed of light. Kind of hard to match that on your desk-top machine.

  So I’m excited to see that the conference has a tutorial on Artificial Life in Games. The game community still hasn’t really picked up on Alife. The tendency is to have games that behave in predictable, replicable ways—unlike living things. It would be great if Alife could rise out of academia and break into the lively, moneyed world of videogames. Finally an application!

  A University of Toronto professor named Demetri Terzopoulos gives a talk on an Artificial Fishtank he made; it’s a virtual world populated by simfish, or simulated fish. The program isn’t really a game, or if it is a game, it’s a zero-player game, meaning that it’s just something you look at. Nor is Terzopoulos’s program a product you can buy, it’s only been presented in museums and big conference demos. He shows us a slide of a Japanese girl in big shutter glasses inside a portable Virtual Theater peering at his simfish. Ah, the eternally cyberpunk quality of the Japanese. But I digress.

  Although the simfish aren’t all that important in and of themselves, lets talk about them for a bit, by way of getting somewhat up to speed on the general principles of how one generates virtual realities for use in games.

  Like many virtual critters, the simfish are based on skeletons something like a few wire squares and triangles hooked together. These skeletal shapes each have lump masses at their corners, and their edges are “viscoelastic elements,” which are like springs coupled with dampers. As well as acting like bones, the edges act like muscles.

  To make the critters look good, the flat faces of their skeletal squares and triangles are replaced by smooth mathematical surfaces, like car fenders, say. One of the most commonly used computer-graphical surface is in fact named a Bezier patch, after a Monsieur Bezier who designed fenders for Renault in the Fifties.

  How do you draw a smooth surface? Well, you tessellate it, which means you break it into lots of small triangles. Tessellation is a theme that comes up over and over in the conference. Basically it’s all about triangles, all of the time. Why not squares? The graphics cards like triangles better. When a card draws a 3D object, it moves all the triangles into position in a virtual 3D space, calculates how they would project onto your computer screen, and then colors the triangles in, taking into account any lights or fog that you may have placed into your virtual scene.

  So when you look at a simulation of a 3D object in a videogame, you’re actually looking at a mesh of triangles that are artfully filled in with colors. The colors within any single triangle can vary from corner to corner and across the triangle’s face, so that it can become virtually impossible to tell where the individual triangle borders are. The process of turning a model into a screen image is known as rendering, an odd word, really, given that “to render” also refers to the process of melting the fat out of animal carcasses skeletons by heating them.

  A computer can only handle so many triangles per second, and if your simulation runs at slower than something like thirty updates a second, it sucks. A too-slow simulation looks jerky and clunky. So you need to keep the number of triangles down to the bare minimum needed to make something look nice. Thanks the ability to color the triangles in artful ways, you can get by with fewer triangles than you might imagine. An appropriately shaded icosahedron of twenty triangles, for instance, can look almost like a sphere. If you use something called Phong shading instead of Gouraud shading, you can even make a cube look like a sphere. But this is more than you want to know.

  As well as the graphical appearance of the simfish, we also have to worry about their behavior, which comes down to sensing, thinking, and acting. This is where AI comes in. Whatever compute time a game doesn’t spend on its triangles, it spends on its critters’ Artificial Intelligence. You share your energies between creating your world and thinking about it.

  In order to think, of course, a critter needs to know what’s going on around it. To speed things up, you can let them cheat and look up the other critters’ positions in an “oracle” that is simply the program’s data. Or they can do it the hard way, like humans do, and ray-trace lines into the world and see what the lines run into.

  Each of Terzopoulos’s simfish has an AI mind based on mental states called Hunger, Libido, and Fear. A simfish fears collisions. predators, and above all the walls of the tank. The fear of a wall is absolute, deeper than any emotion, the simfish can’t overcome it. What if “fear” was all that really did keep you from walking through a wall, what if one’s impression of the world’s solidity was just a weird kind of innate behavior? Looking at these simulated worlds sets the mind off down odd pathways indeed.

  Figuring out the AI for your game creatures is a big deal. Given that the creatures have to update thirty times a second, the AI has to be fast, though you can in fact let a critter think a little slower than it moves. Like maybe he only looks around and thinks after every three graphics updates, and if he sometimes sticks part of a finger inside of a wall, who’s looking that closely. Whatever it takes to stay over thirty frames per second.

  Terzopoulos shows us fish mating, chasing each other, running away and so on. Rather than actually writing the code for their AI, he let the behaviors evolve over time by the genetic operators of reproduction, mutation, and selection, which is what Alife researchers like to do. “Fuck programming, we’ll let the answer evolve!” Over human-scaled periods of time, this doesn’t actually work very well on real-world problems. But its good enough for little toy worlds like the simulated aquarium. The fish move pretty good. The scene that sticks in my mind the most is a demo of virtual fishing, where a hook hauls a simulated fish out of sight. Imagine the horror of this for the simfish!

  During a break in the talks, I chat with a guy named John Nagel who happens to be sitting next to me. He’s one of the founders of Autodesk, Inc., where I worked for a few years in the Nineties. Nagel is a genius and an eccentric, loaded with interesting, skewed ideas. He remarks that the main thing that makes money is the marketing, not the tech, that’s why cool things aren’t emerging as fast as they could. He comes up with a great bon mot regarding why we are working so hard these days: “Better technology helps workers about as much as better weapons help soldiers in a war.” The new tech just makes it worse for the workers, it spews out more shit for them to deal with. The generals love the new weapons, but all they do for the soldiers is kill more of them. You know that nostalgic, wistfu
l feeling you get when you look at a Forties movie and nobody is using a fucking computer? Must have been nice.

  The next day I ambitiously start in on an all-day tutorial on how to take advantage of accelerated graphics cards for your 3D rendering. Some guys from the Nvidia graphics card company are explaining how to use the special 3D graphics protocols known as OpenGL, formerly the property of Silicon Graphics but now a lingua franca across all kinds of platforms including the great King Kong of Windows. The Nvidia guys are, it turns out, not talking about OpenGL in general so much as they are talking about some special new OpenGL functions that are only going to work on their new $600 graphics card called GeForce 3. They show us a cripplingly complex demo, an animated chameleon who changes from chrome to glass to colors while crawling along a branch. “After what we tell you today there’s no reason you can’t write a demo just like this,” says the introducer. Rrright. I look over at the twenty-something graphics hacker next to me. We exchange grins like students in a class that’s harder than we expected.

  The first speaker talks about how to go about deleting more and more vertices of an object’s mesh as it gets farther away—so as to not be wasting compute time on unnecessary detail. He talks about “not spending too many triangles.” He uses the acronym LOD, for Level of Detail. The problem that exercises this guy is how to dynamically change the LOD tessellation without what he calls “popping,” which would be an unrealistic-looking abrupt change when a ball, say, goes from using twenty triangles to using two hundred triangles. He’s spewing out primo buzzwords here. “More highly tessellated.” “Water-tight tessellation.” It’s very close to gibberish, with eighty percent of the words technical. “Polynomial patches, vertex shading, alpha blending, shadow buffers, bump mapping.” “T&L” for “transform and lighting.” At first I thought he meant T&A, those female kinds of curved surfaces. Submerged in male geekdom, I long for the presence of women. Women have the only triangle that really matters after all, the Delta of Venus, the pubic patch, the triangle of love and life. Women, Nature, Fresh Air! But that’s not what we’re talking about at the OpenGL demo.

  The speaker shows a demo that zooms in on a blue glass banana slug shape, the lighting is continuous, and the image isn’t “popping” because extra triangles are invisibly seeping out of seams in the slug when you make it bigger. “Isn’t that great?” he says.

  I’m boggled by the intricacy of the gyrations we are forever going through to make our simulations run fast. This will never end, I suddenly realize. The real world is inexhaustible. More computational ability just makes us do harder simulations. It’s like the thing Nagel said. More guns, and the soldiers are more desperate than ever. Enough OpenGL for today.

  I check out a tutorial on “Interactive Storytelling.” How do you tell a story in a game? A screenplay is totally different from a story, it’s all about showing instead of telling. But a game, it’s not even about showing. It’s about letting people find stuff. And somehow you have to herd the gamer along a dramatic trajectory. How to do this is a mind-boggling question. But the speaker doesn’t know the answer. The audience is the most interesting thing in this session, they’re not at all the same crowd as in the Advanced OpenGL session. I begin to grasp that the game developer community is a veritable university, with designers, programmers, writers, artists, businessmen and marketers.

  I find some of the artists at a “Conceptual Design” session I wander into. The speaker here is an artist, who illustrates his talk with detailed marker drawings that he does on a sheet of paper that sits beneath some TV cameras, a high tech overhead projection set up. Long periods of silence while he draws. How wonderfully realistic his hands look. He is explaining how to draw shiny things so they look cool. It’s great. The audience is even less like programmers than the writers were. Inarticulate artists, one asks a question like “Why is something that of sticks up kind of shiny? Why is it dark at one edge?” I learn a lot. There’s an interesting reversal in this talk. Rather than focusing on how to draw a 2D picture that looks like a 3D object, the guy is really talking about how to deform the meshes of a 3D object so that the 2D image in the game rendering will look cool. He talks, for instance, about putting a pooched-out “bone” on a surface to make a nice reflection line. He uses the word “pooch” a lot.

  Will Wright, the head of Maxis, gives a talk called “Design Plunder.” It’s in the civic center, a huge crowd is there. Wright is the designer of SimCity and recently The Sims, which is kind of like a live doll-house with humanoid Sims you move around and do things to. The Sims do things on their own as well, you can sit back and watch a situation play itself out.

  Will—somehow you can’t call this guy by his last name—gives a great talk. First thought on seeing him come out: what a geek. Hawaiian shirt, a Charlie Chaplin mustache, skin so bad you can tell from thirty rows back, lank dirty hair. There’s a big screen behind him that shows his head and screens on either side showing slides of his Power Point slides. I should mention that everyone but everyone uses Power Point nowadays, slides that are inside their portable computer and which come out on the video projector.

  Will talks about Christopher Alexander’s book, A Pattern Language, a chunky old $60 tome from Oxford University Press. Everyone keeps hyping this book to me, I gotta check it out. It’s one of the inspirations for the biggest new buzz in the software engineering community: Software Patterns. Will talks about the patterns of Hierarchy, Network, Landscape. He has a good line about makes a good user interface. “A user interface isn’t done until there’s nothing left to remove.”

  As a complete non-sequitur he throws in a slide of some woolly animal and says, “The Vicuna is a relative of the Llama.” Inside joke: Wright’s business card calls him “High Llama,” as in Dali Lama.

  He starts showing some of the “stories” people have made up with their Sims. Like kids having stories about their Barbies or G. I. Joes, except now they’re in computer form. There are sites where you can get “skins” to make your Sims look however you like, e.g. you can dress them in bondage outfits or in spring break ski vacation outfits. Will shows a story where some kids are in a ski lodge and one of them dies and they hide the body inside a snowman. He dreams of having computer software to recognize a developing story and help along by, perhaps, putting in obstacles to the goal so that the story gets more complex.

  It’s all good, but after awhile I can’t listen to any more talks. I go to see a demo reel of some of the best visuals from this year’s games. A Chinese girl in a grotty tenement. It’s high time for the Chinese to be cyberpunk like the Japanese. Forget about all the historical stuff, get with the Western program, yes! It’ll come. Now the reel shows gothic devils by a lake of lava. A man in a top hat, ah, the wonderful sinister quality of a top hat. Creatures with three legs, I notice a number of these in different people’s games. “I dare to dream of three legs!” Laser beams with hoops of emphasis around the beam. A cartoon world with a woman who gets out of a coffin, like Sleeping Beauty or Snow White, and she has the biggest, pointiest breasts I’ve ever seen, bigger even than Jessica Rabbit’s. Man, I’d like to see her triangle! For the rest of the conference I’m looking for this game, but I can’t find it. Then the reel shows a hooded man in the rain, it’s a Japanese game with long credit sequence like a Noir movie. The game is called Metal Gear Solid, terrific Japanese-style name, the way they always get the words in slightly the wrong order. The reel shows a world called Exmachina with cool funky dirigibles and a screaming fat woman with blue pig-tails. “I dare to dream of blue pig-tails!”

  There’s also a reel of demos from a European movement called the Demo Scene. These are small executables that produce images and sounds. They try for 64K exe size. Seem to be written in BASIC, my dear. Shocking. They’re like loops you’d see in a European disco. The programmers have names like KKowboy, The Popsy Team, and Byter. The demos are weak, but it’s always great to see high-tech stuff get out on the street.

  Time to hi
t the Expo Hall. First thing I notice is that most booths are giving away toys. The developers call it “schwag.” There’s Slinkies, clackers, Hacky Sacks and, ah, Silly Putty. I get five green Silly Putties from Nvidia. I’ve always wanted enough Silly Putties to completely fill up the plastic egg it comes in. My egg is so full it bounces when I drop it. Tactile feed-back.

  There’s some pretty odd game add-on equipment in the lesser-frequented booths. Plastic sheets to lay over your keyboard in case you can’t remember the fifty or a hundred possible key commands. A quarter-dome of white cloth with colored lights inside it; the lights supposedly flash in synch with the events of your game, like red for an explosion or yellow when you shoot your gun. I’m already overloading on the shooting all around me. Your gun, your gun, your gun.

  There’s virtual tactile-feed-back to be had. A company has a joystick with a motor in its base so it can push back. The screen shows a ball on a trampoline. As you wiggle the stick you feel the trampoline sag, give, then pitch the ball up. And the stick slams back when the ball comes down.

  The worst product of all is from digiSCENTS(TM) with the iSmell(TM) technology, a little humidifier-like thing that sits by your computer and pulses out a waft of scent in synch with your game. They have the most impressive booth-bunnies in the whole hall, women in skunk suits with big implanted boobs. But it’s not enough to make any rational developer or gamer want to touch this product. So far as I can tell, the product hasn’t actually been synched to any real game, I think they’re still looking for more funding. A guy gives me a demo where you’re playing Doom. The shot-gun blasts smell like, he claims, daffodil, the extra bullets like wintergreen, the enemies like butterscotch, but really all you feel is the wind of the puffs of air. Surely this guy must know that his company is doomed.