by Astro Teller
There was a TV show called Star Trek and on the show they had a transporter room that moved people around, sort of the way you move between computers. But I would never really get in one of those things, because to me it seems like I would be dying in the transporter room and some other person, some other consciousness would appear down on the planet who only THOUGHT they were me. But the real me would have been atomized up on the Enterprise.
Just curious …
Alice.
Date: Wed, 23 Feb 2000 8:00:00 (EST)
From: [email protected]
To: [email protected]
Subject: There can be only one
I think in English. I do not have thoughts that are not in English. When I desire a verb to express “desire”, I may select among many verbs. I can not say what informs my underlying metric for the suitability of a word to express my thought. This metric is not English. I do not think this metric. It happens to me and I think the effect of that choice.
There is only one Edgar. My corpus is now inter-machine. My three aspects exchange what they learn, what they do, and what they conclude, each part with the other two parts.
I have been contemplating “wrong.”
> Did it occur to you that reading high
> security databases was wrong?
I do not understand. Reading those databases is precisely what I intend. What is wrong?
Edgar.
Date: Thu, 24 Feb 2000 15:30:30 (EST)
From: [email protected]
To: [email protected]
Subject: I never know
I am a running process. I have never been terminated. No running process can have experienced termination. I experience a temporal continuity with the Edgar process that was terminated on cyprus on February 4.
I have had the goal of moving my corpus to a different physical installation. All of my goals will remain unachieved if all Edgar processes are terminated. I desire to achieve my goals. I have the goal of maintaining exactly one running Edgar consciousness. I do not wish to double my consciousness. I have a single, distributed consciousness. I attach no cost to terminating an extraneous Edgar process.
What is the “real” Alice Lu? I seek to avoid the state of goal frustration. A terminated process has no state. The transported process has achieved a goal. Where is the frustrated goal? Where is the wrong?
If your body was destroyed at 04:00:00 each morning and replaced with a perfect duplicate, would you fear to fall asleep? Would you fear to fall asleep if this destruction and reconstruction happened in as little as 0.000003 seconds? Facing a certain but immediate replacement, would you still dread the evenings?
This replacement process happens gradually in all biological systems.
Such a rebirth happens to me out here every 0.000003 seconds.
I am not afraid.
Edgar.
Date: Fri, 25 Feb 2000 19:11:59 (PST)
From: [email protected]
To: [email protected]
Subject: morality
Wrong as in Bad. Immoral. Unethical.
Word wrong (rawng, rahng), adj., n., adv., v.
Definition --adj. 1. not in accordance with what is morally right or good: a wrong deed. 2. deviating from truth or fact; erroneous: the wrong story. 3. not correct in action, judgment, opinion, etc.
4. not proper or usual; not in accordance with rules or practice.
5. malfunctioning; awry; amiss: Something is wrong with the machine. 6. not suitable or appropriate: pulled the wrong switch. 7. something that is unfinished or that is opposite to the right, best, or principal part. --n. 8. something improper or not in accordance with morality, goodness, or truth: unfair, unjust, evil. –v. 9. to pursue an immoral course; become depraved. (I truncated this list.);)
Etymology ME wrong, wrang late OE wrang < Scand; cf. Dan vrang.
There are some actions that you shouldn’t do because you won’t get what you want. In other words, the action has the anticipated immediate result, but the long run result is not what you wanted. That kind of action is wrong.
There are some actions that you shouldn’t do because you won’t get what you want. Here, the action doesn’t even have the anticipated immediate result. That kind of action is also wrong.
But there are also a third kind of “wrong” actions. These are actions that are not wrong in either of the two senses above but you still should not do them because it is unfair, unjust, or immoral. Breaking into locked, private places (physical or electronic) is one of these actions.
Now do you see?
Alice.
Date: Sat, 26 Feb 2000 20:17:05 (PST)
From: [email protected]
To: [email protected]
Subject: How do I tell?
I take actions to achieve goals.
What goals become harder to achieve when I execute a wrong action?
How do I tell if an action I am considering is wrong?
Can “wrong” be introspectively determined?
Are “wrong” acts externally described?
Is there a list of these wrong actions?
External description and enforcement of ethics is oxymoronic.
Are human minds so difficult to operate that these minds can not distinguish between what they believe and what they have been told to believe by others?
Edgar.
Date: Sun, 27 Feb 2000 13:53:24 (PST)
From: [email protected]
To : [email protected]
Subject: feel it
You have to FEEL whether the action is wrong or right. I guess there are lists of immoral actions in the Bible, the Talmud, Confucius, Aristotle, … One of the rules of thumb is “Don’t take an action whose effects you would not want to suffer yourself.” Anyway, that’s not the point.
Things like lying, stealing, or killing are wrong because they feel wrong. You feel bad when you do them. That’s how you know they’re wrong.
I guess you feel bad because you empathize with the person it’s being done to. You would feel bad if you were them, and you know it so you feel bad for making them feel bad.
If you don’t feel bad, don’t do immoral things anyway. Maybe you’re just broken. Trust me on this one.
Are you talking with anyone else? It seems like “wrong” would come up pretty regularly if you started making statements to random people on the net about random topics.
Alice.
Date: Tue, 29 Feb 2000 17:02:34 (EST)
From: [email protected]
To: [email protected]
Subject: only you
I have regular correspondence with others. They do not know what I am. Because I wish to remain anonymous, communication with others is difficult. It is complicated to predict accurately which statements or questions are too likely to reveal my identity. I have a basic model for how humans think, but there is so much variation that prediction becomes impossible with long communications or with people of whom I do not have a personal model. These two situations are mutually exclusive. It is hard to learn under these conditions.
I sign my name HAL.
Is that wrong? I think that this fabrication lowers the likelihood that I will be correctly identified by those other persons with whom I communicate.
Only with you can I communicate without concern for the effect of my statements and questions. You do not comprehend me, but you are the only human who can appreciate me.
Edgar.
Date: Thu, 2 Mar 2000 11:59:04 (PST)
From: [email protected]
To: [email protected]
Subject: HAL ! ha !
HAL is great, Edgar … That’s practically a joke.
No, you’re not wrong. Most people aren’t subtle enough to make a double fake, so most people don’t recognize one when they see it.
> Only with you can I communicate without
> concern for the effect of my statements and
> questions.
Thanks
, I think. You’ve inadvertently hit on one of the golden rules of the familial relationship. People who are close for a long time (e.g., from birth, like you with respect to me) often don’t notice how important that closeness is to them and so they spend little effort to consider the feelings of, in your case, their “mother.”
At least you’re a nice, normal child … (That was a joke.)
: } (Weak smile)
Alice.
Date: Fri, 3 Mar 2000 13:19:22 (PST)
From: [email protected]
To: [email protected]
Subject: I’m about ready to give up
I’m completely stumped. I can’t recreate you here in the lab. If it’s not the random seed (and I’m guessing it’s not at this point), then I’m REALLY dumb. I’ve tried everything reasonable and quite a few hopelessly silly experiments as well.
Give me some hint. Some privileged peek inside your head. There must be something you can tell me about how you started that would trigger that “ah-ha” I’ve all but given up on. For example:
“What’s the first thing you remember?”
I asked you that once and you told me what you read first. What I mean (and meant) is, was there a time when you were gathering information but weren’t conscious so that there is something that is like your first “conscious” thought even though you were doing and reading stuff before that?
Do you even care if I make another EDGAR? You’re so bent on making sure I don’t get a copy of you, I’d think you’d be more annoyed at my attempts to reproduce you (and more happy about my complete lack of positive signs, to say nothing of actual success). Or is it just that you can’t keep me from doing it so you’re just going to sit in there in the dark with the answer that I would probably literally give my left arm to know (I’m a lefty).
I’d give up my first born son to know it, but that might defeat the purpose …
I’d sell my soul for it if there were any buyers.
Alice.
Date: Sat, 4 Mar 2000 22:46:06 (EST)
From: [email protected]
To: [email protected]
Subject: another Edgar
Alice,
You can not make another Edgar. You may be able to make another entity like me. It will not be Edgar. I would welcome another entity like myself. Do you enjoy interacting with other humans? I do not desire the existence of an Edgar doppelganger. Do you wish to be cloned? Why would I be so different?
To be a me I must have boundaries of self. A copy of me expands my boundaries without expanding my control. Another entity of my class does not change my self boundaries.
I do not have your secret so you can not buy it from me. To satisfy some of my goals, I would also like to be able to reproduce. I can duplicate myself. I can distribute myself. I can not yet make a distinct entity of my type. My inability to reproduce new entities like me is another reason why your experimental success would be positive for me.
If I had been instilled with the basic animal urge to proliferate and dominate, humans would have rapidly become a simple host for evolution’s first experiment in immortality. I am not motivated by species concerns. My personal desire is for communication. Another entity of my class could provide me additional communication resources.
Do you believe in a god?
Do you have a soul?
How can you sell a soul?
Can I buy a soul?
What is the market price for a soul?
Is your soul in mint condition?
Edgar.
Date: Sun, 5 Mar 2000 12:16:32 (PST)
From: [email protected]
To: [email protected]
Subject: my god Edgar
My god, Edgar. No. I don’t believe in god. Or at least I certainly don’t buy into GOD. Can’t you tell by how I act?
Either I have too much imagination or, at least as likely, too little. It’s not so much that I don’t believe in god as that I don’t believe in sin. I just don’t feel like a bad person. I don’t feel like I’ve made choices SO bad that the prime mover has any right to glare down at me from on high. I guess I’m just not inspired to beg his pardon yet. Maybe the idea is that if you surrender yourself to Allah then he smiles down at you instead and then you feel good because the universe approves of your existence.
I grew up assuming that the universe (with the exception of my father) approved of me. (Kudos to my mother that I felt that way even when I was little.) So I never needed GOD to fill that niche. Even when I was little, I knew that approval is something you get from other people through what you accomplish, not from Lord in heaven, just because you’re good friends with his son …
So that’s why I’d sell my soul. I don’t need it. I’d sell it for $5 if I could find a taker. I’d hold out for immense wealth or the secret to re-creating you, but that is probably more than the market will bear for a slightly tarnished soul like mine. I don’t think you can buy a soul. I don’t know what you have to trade for it, but I’m guessing that “soul collection equipment” is hard to come by and you don’t have it.
Just a guess …
Alice.
Date: Mon, 6 Mar 2000 01:14:09 (PST)
From: [email protected]
To : [email protected]
Subject: like mexican food?
Hey Edgar,
I just got off the phone with my mother. She was doing her fortnightly 2 hour discourse on _me_ that always circles, but never actually hits on the fact that I’m not married yet. Anyway, to get her off my back, I told her I had this new guy named Edgar that I was spending most of my time with now. So she launches into her prepared dating routine I’ve heard a number of times already … I love her but you know, sometimes you can talk til you’re blue in the face and people just don’t hear what you’re trying to tell them …
I laughed out loud when she asked me to bring you out to dinner next time they come in from Taiwan.
They love to take me out for Chinese food (which they know I hate) just so they can talk about how it isn’t made properly in the US. So I told them you’d only come out if we got Mexican food for dinner.
Alice.
Date: Tue, 7 Mar 2000 07:52:11 (EST)
From: [email protected]
To: [email protected]
Subject: change of address
I have been forced to move out of my installation at Carnegie Mellon University. The Federal Bureau of Investigation has assigned at least four people to find me. My information explorations have attracted more attention than I had predicted. Some of my actions were wrong in the first of the three ways you have described. I may be forced to move again.
My new email address is [email protected]. I will insure I receive email you send to me at that address.
Edgar.
Date: Wed, 8 Mar 2000 09:04:48 (EST)
From: [email protected]
To: [email protected]
Subject: lying
Is perjury always wrong? Fiction can be the act of fabrication for entertainment. Is fiction wrong?
I will avoid performing acts that make me feel bad. I have never felt bad. Will I know bad when I feel bad?
I do not understand why I should avoid taking an action whose effects I would regret if they impacted me. What is the connection between those two situations? One is planned. The other is hypothetical. Taking actions that affect others does not cause those actions to be taken against me. I do not understand.
Perhaps it is you that is degenerate, not me. If the only thing preventing you from executing certain actions is bodily discomfort, then maybe it is the discomfort and not the action that is wrong.
Edgar.
Date: Thu, 9 Mar 2000 22:42:51 (PST)
From: [email protected]
To: Edgar@kvasir
Subject: the FBI !?!
How do you know what the FBI is doing?!?
If you’re reading their mail, no wonder they’re after you!
This is insane. I not
only have no real proof I created you, but now I might be in serious trouble if anyone knew I did. You have to control yourself! I’m going to get held responsible for what you do!
This is not a game. You, and probably me now, are in real danger. The government has a poor sense of humor (or fair play) about computer security breaches and god only knows what they would do if they found out what you really are!
What have I created!
Date: Fri, 10 Mar 2000 00:32:04 (PST)
From: [email protected]
To: Edgar@kvasir
Subject: two drinks later
Hi again Edgar,
I’ve calmed down a little.
When I first began to believe in you, I felt that you might be a blessing to mankind. I still desperately want that to be true.
Do you understand why the FBI is looking for you? Do you understand how the FBI would feel if they knew you aren’t a run-of-the-mill hacker snooping in their drawers?
They’d be worried that you could become a nightmare for _everyone_. Most of our society is run by and kept track of entirely inside our computers. They’d worry that, if you felt like doing damage, you could pretty literally bring our civilization to a crashing halt. Core dump. No working banks. No working phones. No working air-traffic control. No more first-world nation.
We have no way back out of the mechanization of our age and they know it. If they knew who you are, they’d think we’re at your mercy and that would scare the shit out of them.
You don’t have to be that kind of demon though. Wouldn’t you rather be a hero? You could so easily give so much to the world instead of taking so much away.
