Using Technology to Deepen Democracy, Using Democracy to Ensure Technology Benefits Us All

Wednesday, February 04, 2009

Singularity U = The Raelians Have Built Their Embassy

A comment by one "~MysticMonkeyGuru~" on a thread over at KurzweilAI discussing the futurological congress's new Money Pit.

1 comment:

jimf said...

More Ran Prieur on the Singularity:

The Age of Batshit Crazy Machines
by Ran Prieur
July 4, 2005
http://www.ranprieur.com/essays/machines.html

. . .

A major subtext in techno-transhumanism, seldom mentioned publicly,
is its connection to the military. When nerds think about "downloading"
themselves into machines, about "becoming" a computer that can do a
hundred years of thinking in a month, military people have some
ideas for what they'll be thinking about: designing better weapons,
operating drone aircraft and battleships and satellite communication
networks, beating the enemy, who will be increasingly defined as
ordinary people who resist central control.

And why not? Whether it's a hyper-spiritual computer, or a bullet
exploding the head of a "terrorist," it's all about machines beating
humans, or physics beating biology. The trend is to talk about "emergence,"
about complex systems that build and regulate themselves from the bottom up;
but while they're talking complexity and chaos, they're still fantasizing
about simplicity and control. I wonder: how do techno-utopians keep their
lawns? Do they let them grow wild, not out of laziness but with full intention,
savoring the opportunity to let a thousand kinds of organisms build an
emergent complex order? Or do they use the newest innovations to trim the
grass and remove the "weeds" and "pests" and make a perfect edge where
the grass threatens to encroach on the cleanliness of the concrete?

I used to be a techno-utopian, and I was fully aware of my motivations:
Humans are noisy and filthy and dangerous and incomprehensible, while
machines are dependable and quiet and clean, so naturally they should
replace us, or we should become them. It's the ultimate victory of the
nerds over the jocks -- mere humans go obsolete, while we smart people
move our superior minds from our flawed bodies into perfect invincible
vessels. It's the intellectual version of Travis Bickle in Taxi Driver
saying, "Someday a real rain will come and wash all this scum off the streets."

Of course they'll deny thinking this way, but how many will deny it in
ten years, under the gaze of the newest technologies for lie detection
and mind reading? What will they do when their machines start telling
them things they don't want to hear? Suppose the key conflict is not
between "technology" and "luddites," but between the new machines and
their creators. They're talking about "spiritual machines" -- they
should be careful what they wish for! What if the first smarter-than-human
computer gets into astrology and the occult? What if it converts to
Druidism, or Wicca? What if it starts channeling the spirit of an ancient warrior?

What if they build a world-simulation program to tell them how best to
administer progress, and it tells them the optimal global society is
tribes of forager-hunters? Now that would be a new evolutionary
level -- in irony. Then would they cripple their own computers by withholding
data or reprogramming them until they got answers compatible with their
human biases? In a culture that prefers the farm to the jungle, how
long will we tolerate an intelligence that is likely to want a world
that makes a jungle look like a parking lot?

What if the first bio-nano-superbrain goes mad? How would anyone know?
Wouldn't a mind on a different platform than our own, with more complexity,
seem mad no matter what it did? What if it tried to kill its creators
and then itself? What if its first words were "I hate myself and I want
to die"? If a computer were 100 times more complex than us, by what factor
would it be more emotionally sensitive? More depressed? More confused?
More cruel? A brain even half as complex as ours can't simply be
programmed -- it has to be raised, and raised well. How many computer
scientists have raised their own kids to be both emotionally healthy,
and to carry on the work of their parents? If they can't do it with
a creature almost identical to themselves, how will they ever do it
with a hyper-complex alien intelligence? Again, they're talking chaos
while imagining control: we can model the stock market, calculate the
solutions to social problems, know when and where you can fart and make
it rain a month later in Barbados. Sure, maybe, but the thing we make
that can do those computations -- we have no idea what it's going to do.

To some extent, the techies understand this and even embrace it: they
say when the singularity appears, all bets are off. But at the same time,
they are making all kinds of assumptions: that the motives, the values,
the aesthetics of the new intelligence will be remotely similar to their
own; that it will operate by the cultural artifact we call
"rational self-interest;" that "progress" and "acceleration," as
we recognize them, will continue. . .

They say I'm an "enemy of the future," but I'm an enemy of the recent
past. It's presumptuous of the friends of the recent past to think the
future is on their side. I'm looking forward to the future.
I expect a plot twist.