Possible Minds

These notes were published in 2019 and discuss the contributors’ AI forecasts and arguments as presented in the book at that time.

Possible Minds: 25 Ways of Looking at AI by John Brockman is a collection of essays by leading AI researchers, artists, and philosophers. They all give their own view on the state and future of AI, alongside reflections on The Human Use of Human Beings by Norbert Wiener. Each essay is quite different. The selections below draw on 15 of them.

The sections below mix selected quotations from the essays with my own summaries and reactions. I name a different speaker when my source notes identify one.

One immediate thing I learned/was reminded about is that technology itself will not be a force for good or bad, it is culture that does this. Technology only enables it. I later noted a podcast example: DNA testing can help with ancestry but can also enable mass surveillance and profiling of Uyghurs in China.

1. Seth Lloyd: Wrong, but More Relevant Than Ever

“Wiener’s central insight was that the world should be understood in terms of information. Complex systems, such as organisms, brains, and human societies, consist of interlocking feedback loops in which signals exchanged between subsystems result in complex but stable behavior. When feedback loops break down, the system goes unstable.”

“Technological prediction is particularly chancy, given that technologies progress by a series of refinements, halted by obstacles and overcome by innovation. Many obstacles and some innovations can be anticipated, but more cannot. In my own work with experimentalists on building quantum computers, I typically find that some of the technological steps I expect to be easy turn out to be impossible, whereas some of the tasks I imagine to be impossible turn out to be easy. You don’t know until you try.”

2. Judea Pearl: The Limitations of Opaque Learning Machines

“Current machine-learning systems operate almost exclusively in a statistical, or model-blind, mode, which is analogous in many ways to fitting a function to a cloud of data points. Such systems cannot reason about “What if?” questions and, therefore, cannot serve as the basis for Strong AI—that is, artificial intelligence that emulates human-level reasoning and competence.”

3. Stuart Russell: The Purpose Put into the Machine

“Putting a purpose into a machine that optimizes its behavior according to clearly defined algorithms seems an admirable approach to ensuring that the machine’s “conduct will be carried out on principles acceptable to us!” But, as Wiener warns, we need to put in the right purpose.”

4. George Dyson: The Third Law

“He likes to point out that analog computing, once believed to be as extinct as the Differential Analyzer, has returned. He argues that while we may use digital components, at a certain point the analog computing being performed by the system far exceeds the complexity of the digital code with which it is built. He believes that true artificial intelligence—with analog control systems emerging from a digital substrate the way digital computers emerged out of analog components in the aftermath of World War II—may not be as far off as we think.”

I jokingly labelled three of Dyson’s points the “laws of robotics”, or rather of AI: a controller needs complexity comparable to the system it controls; an organism is its own simplest complete model; and a system complex enough to behave intelligently may be too complex to understand. I also noted his loophole: we can build something without fully understanding it.

5. Daniel C. Dennett: What Can We Do?

(quoting Weiner) “[I]n the long run, there is no distinction between arming ourselves and arming our enemies.” The information age is also the dysinformation age”

“[W]e’re making tools, not colleagues, and the great danger is not appreciating the difference, which we should strive to accentuate, marking and defending it with political and legal innovations.”

6. Rodney Brooks: The Inhuman Mess Our Machines Have Gotten Us Into

(John Brockman) “[H]e is alarmed by the extent to which we have come to rely on pervasive systems that are not just exploitative but also vulnerable, as a result of the too-rapid development of software engineering—an advance that seems to have outstripped the imposition of reliably effective safeguards.”

7. Frank Wilczek: The Unity of Intelligence

In asking if AI can be conscious, creative, and/or evil, Wilczek answers yes. “Evidence from those fields makes it overwhelmingly likely that there is no sharp divide between natural and artificial intelligence.”

My notes connect his argument with ways people have long extended their abilities, from fire, glasses and clothing to phones, the internet and X-rays. They also list three-dimensionality, self-repair, connectivity, development and integration as areas where human brains then seemed stronger than machines.

8. Max Tegmark: Let’s Aspire to More Than Making Ourselves Obsolete

“Consciousness is the cosmic awakening; it transformed our Universe from a mindless zombie with no self-awareness into a living ecosystem harboring self-reflection, beauty, hope, meaning, and purpose.”

“But from my perspective as a physicist, intelligence is simply a certain kind of information processing performed by elementary particles moving around, and there’s no law of physics that says one can’t build machines more intelligent in every way than we are, and able to seed cosmic life.”

I read Tegmark as tracing a sequence: we harnessed heat, light and mechanical power, learned to understand our bodies as biological machines, and then began building machines that can outperform us in some cognitive tasks.

9. Jaan Tallinn: Dissident Messages

Jaan Tallinn, a computer programmer, theoretical physicist, and investor, is a co-developer of Skype and Kazaa. In 2012, he co-founded the Centre for the Study of Existential Risk—an interdisciplinary research institute that works to mitigate risks “associated with emerging technologies and human activity”

I noted Tallinn’s contrast between humans, adapted to Earth even as we reshape it to suit us, and AI systems that could operate in environments such as deep space.

10. Steven Pinker: Tech Prophecy and the Underappreciated Causal Power of Ideas

“A healthy society—one that gives its members the means to pursue life in defiance of entropy—allows information sensed and contributed by its members to feed back and affect how the society is governed. A dysfunctional society invokes dogma and authority to impose control from the top down.”

Pinker also dismisses the competent-but-stupid AI scenarios where the AI is very good at completing a goal, but doing this too literal (e.g. making everyone happy by installing dopamine drips). He argues that intelligence (as a broad concept) will consist of several parts that ‘grow’ together, and thusly an AI that will be able to do large things in the world, will also be one that is ‘smart’ enough to not ‘hack’ the goal. (I’m not totally sure about this line of argument and I think especially Nick Bostrom wouldn’t agree).

What I took from Pinker was that technology itself is agnostic; whether it is used for good or harm depends on norms and culture. I understood him to argue that activists should focus on laws and social rules rather than treating the technology alone as the target.

11. David Deutsch: Beyond Reward and Punishment

(about humans in the past) “Moreover, this must have been knowledge in the sense of understanding, because it is impossible to imitate novel complex behaviors like those without understanding what the component behaviors are for.”

12. Tom Griffiths: The Artificial Use of Human Beings

“But if you want to know why the driver in front of you cut you off, why people vote against their interests, or what birthday present you should get for your partner, you’re still better off asking a human than a machine. Solving those problems requires building models of human minds that can be implemented inside a computer—something that’s essential not just to better integrate machines into human societies but to make sure that human societies can continue to exist.”

His discussion of inference made me wonder: if I prefer dessert, would an AI keep buying me desserts? Knowing what people really want is hard, especially when we may not fully know our own preferences.

13. Anca Dragan: Putting the Human into the AI Equation

“At the core of artificial intelligence is our mathematical definition of what an AI agent (a robot) is. When we define a robot, we define states, actions, and rewards.” The goal of an AI is to get the highest cumulative reward.

What Anca argues for is that we should have AI that reasons about us. I think this is the right solution, but also the most difficult one. We are bad at it, there are different reasons/preferences between people. It will be a tough cookie to crack.

I thought Dragan’s call for robots to understand less-than-optimal human decisions connected with Judea Pearl’s and David Deutsch’s arguments about what AI systems of the time could not yet understand or predict.

14. Chris Anderson: Gradient Descent

“We’re going to rock ourselves out of local minima and find deeper minima, maybe even global minima. And when we’re done, we may even have taught machines to seem as smart as a mosquito, forever descending the cosmic gradients to an ultimate goal, whatever that may be.”

I noted Anderson’s mosquito example for gradient descent: follow the stronger scent. His discussion of local minima made me think that a mental map might help us recognise a better solution beyond the next hill.

15. David Kaiser: “Information” for Wiener, for Shannon, and for Us

“Wiener borrowed this insight when composing Human Use. If information was like entropy, then it could not be conserved—or contained.” One key idea here is that if something is known somewhere, you can’t stop others from learning it (only possibly delay it a bit). “(from Weiner) [T]he fate of information in the typically American world is to become something which can be bought or sold.”

I did not find many nuggets in Kaiser’s essay, although it used several meanings of “information”.

I found the point about imprecise definitions of AI and AGI persuasive. We already live with AI in many forms. I do not think AGI will arrive at one identifiable moment; different skills may emerge at different times. I imagined a flood rising past mountains of different heights.