This is an article I came upon recently that I found absolutely fascinating. One, because it’s about a subject I write a lot about in the Transcendence Diaries, knowing full well that it may be the most unpopular and non-topical subject in the society we currently live in — [I am working on a piece presently related to just this, called The Death Of the Intellectual In the Modern Age], and two, because the author, Hoffman, attempts to explore consciousness from a scientific approach, resisting the popular trend of relegating the study of consciousness to the fields of philosophy or metaphysics. There’s hardcore science here. It may not be an easy read in some places; but it’s thoroughly refreshing to follow along the thoughts and thinking process of someone so ravenously intellectually curious and well thought out. A rare occurrence in our time. The piece was originally published in Edge on January 27, 2020. I am re-posting it here in the Diaries for informational purposes and as reference material. I believe it’s a must read for anyone interested in Ontology or Consciousness Studies. – Ed Hale

A Conversation with Donald D. Hoffman [1.27.20]

. . . I want to propose that realism is false, and what we’re seeing is more like a user interface or a virtual reality headset. Think about a virtual reality game of tennis. You’re playing VR tennis with a friend, you both have your headset and body suits on, you see your friend’s avatar on a tennis court and you start playing. Your friend hits the tennis ball to you, and you hit the same tennis ball back to your friend, but is your friend seeing exactly the same tennis ball that you’re seeing? Well, of course not. There’s no public tennis ball. You have some photons being sprayed to your eye by your headset, and those photons are causing your visual system to create your own perception of what you would call a green tennis ball. Your friend has a headset on, which is spraying photons to his eye, and his visual system is creating his own green tennis ball perception.

It turns out that both of those perceptions are coordinated by something else, namely a supercomputer that’s sending the photons to both headsets, causing both headsets to work in coordination. . . .

All the things that we would do to say that objects really exist even when they’re not perceived hold here in virtual reality. . . . That doesn’t mean that the tennis ball exists and has any physical properties when it’s not perceived; it just means that there is some objective reality.

DONALD D. HOFFMAN is a full professor of cognitive science at the University of California, Irvine. He is the author, most recently, of The Case Against Reality: Why Evolution Hid the Truth from Our Eyes. Donald D. Hoffman’s Edge Bio Page

---

REALISM IS FALSE

Some of the questions I’m asking myself are about the relationship between consciousness and the physical world. I’m trying to understand the classic mind-body problem—how consciousness is related to the physical brain or to physical systems more generally, perhaps computing systems. That’s been a conundrum for centuries. Gottfried Leibniz understood it, Thomas Huxley understood it, Francis Crick understood it and said we should really study it. So, I’ve been studying it.

What’s bothering me and many people in our field is that we have so far failed to get a scientific physicalist theory of consciousness that starts with neural activity, or starts with computer programs or some kind of abstract functional architecture and, without any further magic, gives us specific conscious experiences, like the taste of chocolate or the smell of garlic, arising in very specific mathematically precise ways from those physical or functional systems.

Right now, I’m trying to start with a theory of consciousness in which consciousness itself is fundamental. So again, it’s a mathematically precise theory. When we try to come up with a mathematically precise new theory, one of the things we have to do is think about the basic assumptions that we’re going to build into the theory. Every scientific theory starts with certain assumptions, certain axioms if you will, and then tries to build up an explanation of the other things. No theory in science can explain everything. We always have a few things that we assume, and then we try to explain everything else in terms of those few things.

In physical theory, for example, we’ve assumed space, time, matter, or quantum fields are fundamental, and then we can explain chemistry and biology. We’ve tried to use that kind of framework so that with those assumptions, we can try to boot up a theory of consciousness that explains exactly what physical systems or computational systems must be the taste of chocolate and could not be the taste of vanilla. There’s not a single theory that’s been proposed that can explain even one specific conscious experience.

So, what are the basic assumptions that we would need to build into a theory of consciousness? We don’t want to put too many assumptions on the table. We want the minimal number of assumptions that will give the maximum explanation. I’ve been playing with the idea of what I call a conscious agent, which has a set of conscious experiences and can act on those experiences. I have a mathematical formalism for it. Briefly, it’s measurable spaces of conscious experiences and Markovian kernels for decisions and actions based on those experiences.

One thing that comes out of this formalism is that it’s computationally universal. Anything about learning, memory, problem solving, intelligence, self, any of those things that we would think should ultimately be part of a theory of consciousness are not part of my assumptions; those are things that I will try to build out of networks of these conscious agents. The idea is that we’ll have these interacting social networks of conscious agents and, by the dynamics of the networks of conscious agents, we’ll build up theories of learning, memory, problem solving, intelligence, and the notion of a self.

I have a wonderful team of collaborators including Chetan Prakash, Manish Singh, Chris Fields, Robert Prentner, Federico Faggin, and Mauro D’Ariano working with me on the mathematics and the network dynamics and so forth. Ultimately, to solve the mind-body problem—how consciousness is related to the physical world—we’re going to have to start with this theory of consciousness and show how the physical world arises. We’re assuming consciousness is fundamental, not space, time, and matter. We’re going to have to get space, time, matter, and all of modern physics coming out from this network of conscious agents. The question is how to do that. Is that something that is at all compatible with some of the best views in modern physics?

Our team has been looking at some of the recent developments in physics, in particular the work of Nima Arkani-Hamed and his collaborators, in which they’re saying that spacetime has been the foundational idea in physics. In some sense, physics has been about what happens inside space and time for centuries. Spacetime has had a good run; it’s been a foundational assumption in physics. But there are lots of indications, especially from quantum theory and general relativity, that spacetime cannot be fundamental. As some of the physicists are putting it: spacetime is doomed. That’s not my quote, it’s theirs. There’s got to be something deeper that’s fundamental, outside of space and time, that gives rise to space and time. We’re not saying quantum mechanics is wrong or general relativity is wrong. They’re beautiful and powerful theories, but at some point, there are questions they can’t answer and problems that cannot be explained.

For example, spacetime itself. If you try to observe it at finer and finer scales with a bigger and bigger microscope, one problem is eventually the energies that are required to look at finer and finer resolution of spacetime, when you get down to the Planck scale, the energies create a black hole and you destroy the very thing that you’re trying to look at. And if you add more energy, the black hole just gets bigger. Physicists will say that if spacetime is not something we can measure with absolute precision, then it’s not a fundamental concept. We need something more fundamental.

Another idea they have is that in quantum theory you have an observer and a system, and the observer itself needs to be infinite to have infinite resolution in the measurements that it makes of a system. If you have a room in which you’re trying to do a measurement, to get more precise measurements, the observer has to be bigger with more mass. At some point, the observer itself collapses the room into a black hole. As they say, there are no local observables in quantum theory.

The question that I’m dealing with now is, how can I connect this idea of conscious agents and some of the new theories that physicists are coming up with that try to go beyond space and time?

There’s something called scattering amplitudes, the scattering behavior of particles in the Large Hadron Collider. So you smash protons together at near the speed of light. In many cases, you’ll have quarks and gluons hit each other and spray out, so you might have two gluons coming in and four gluons spraying out. You see these things in the detectors, and you can talk about the probabilities or what they call the amplitudes for these various scattering events. They’ve discovered that if you do the computations of the scattering amplitudes in space and time using Feynman diagrams, you get hundreds of pages of math. It’s ugly and you can’t do it in real time because you’re doing a billion of these collisions per second, roughly. They found that they could collapse these expressions to simple expressions, from hundreds of pages down to two or three terms, if they don’t do the computation in space and time.

One of the things they deal with is something called the amplituhedron. It’s a geometric object outside of space and time, and the volumes of various parts of the amplituhedron correspond to the probabilities of these scattering events. This amplituhedron has symmetries that cannot be expressed in space and time. The physicists are discovering that there’s this new realm behind space and time. They don’t know what it’s about. Right now, they’re following the math, which is telling us that there is this structure outside of space and time and it makes the computation simpler, gives us insight into symmetries that you can’t see in space and time.

Maybe this dynamic of conscious agents that we’re thinking about could be the realm behind space and time. My big project over the next couple of years, with the physicists on my team, is to try to understand how the dynamics of conscious agents might give rise to this amplituhedron.

One of the ideas I’m looking at has to do with the dynamics of conscious agents, the so-called Markovian dynamics. That just means that what you’re going to do at this moment depends pretty much on your current state. So, whatever your current state is, it governs all the probabilities of what you’re going to do at the next decision point. You have only a finite memory of what you’ve done in the past, and it’s only a finite memory of what you’ve done in the past that influences your future behavior.

When you look at these kinds of Markovian dynamics, you can look at their long-term behavior. We have a step-by-step behavior of what conscious agents are doing at each step of their interaction. Think of their interactions like a vast social network, like the Twitterverse. There’s a bunch of conscious agents, like a bunch of Twitter users, and they’re all interacting with each other. But what they’re doing is passing experiences back and forth between each other.

We can look at the dynamics of what’s happening at each step of this social network in this interaction, or we can look asymptotically. As the number of interactions goes to infinity, what kinds of patterns do you see there? That’s where I’m thinking we might get the connection to physics and the amplituhedron, not at the step-by-step dynamics of conscious agents. That’s too fine a grain. If we look at the infinite long-term asymptotic behavior of these social networks of conscious agents, that asymptotic behavior erases a lot of the detailed information about the social network and how it works. On the other hand, it’s capturing the long-term patterns. That’s going to be one of those central proposals. What physics has been doing is capturing just the long-term asymptotic behavior of these networks of conscious agents. That’s why it hasn’t looked conscious at all.

For example, if you’re looking at the freeways in Southern California from an airplane, you just see a bunch of little dots moving around. There’s not much evidence of any consciousness or intelligence. You’re looking at it from a high level and you’re erasing a lot of information. You don’t see all the conscious individuals inside the cars. You just see this pattern of flow, of little dots on streets. That’s what physics has been seeing. It’s not seeing the step-by-step dynamics of the conscious agents. It’s only seeing a top-level asymptotic description of the long-term behavior of these social networks of conscious agents. That’s why we haven’t seen things that look like they’re conscious, because we’re only seeing the long-term behavior.

Of course, there’s a lot of specific mathematical steps that we’ll have to take to prove that the asymptotic dynamics of these social networks precisely fits into the structure of the amplituhedron, which they have shown can give rise to the interesting features of quantum theory and relativity theory combined.

That’s one thing I’m trying to work on—flesh out this model of conscious agent networks, look at the asymptotic behavior of these dynamics, and then plug that into the amplituhedron. That whole process will help me with another big problem we’ve got, which is if consciousness is fundamental, there’s this social network of conscious agents out there and they’re interacting—why? The right answer is, I don’t know. I’m trying to first come up with some principled ideas that are at least plausible for what the dynamics of consciousness is fundamentally about.

One idea my team and I are playing with is Gödel’s incompleteness theorem. Gödel showed that if you have any sufficiently complex mathematical system, and that system has a set of axioms, there will be truths that are consistent with those axioms, but they cannot be proved from that set of axioms. There are unprovable truths. And if you add those new truths (that you couldn’t prove before) as axioms to a bigger system, then Gödel’s theorem says there will be yet new truths that can’t be proven within your bigger system of axioms.

Effectively, this means that the exploration of mathematical structure is, in principle, endless. There will be no end to the exploration of what we can do in mathematics. Why is that interesting in the context of a theory of consciousness and conscious agents? Well, it turns out that consciousness and mathematics are intimately linked.

There’s a field called psychophysics that has studied conscious experiences since 1860. One thing that we’ve discovered in our psychophysical studies in the lab and with the mathematical models is that conscious experiences are highly structured. We can write down mathematical models that predict not only judgments of similarity between various like colors, but also predict precisely what three-dimensional structures you will see and when you will see them. It’s mathematics through and through. I’m not saying that consciousness just is mathematics; it’s more like consciousness and mathematics are like a living organism and the bones. The bones are the mathematics and consciousness is the living organism. That’s one reason why we can hope to build a mathematical model of consciousness and conscious agents. The mathematics is a genuine insight into the structural aspects of consciousness, but of course there’s more to consciousness than just the mathematics.

This is where Gödel’s theorem comes in. It says the structures that consciousness can take and that these conscious agents can explore are endless. One idea is that the goal of consciousness and of these conscious agents is endless exploration of all the possible varieties of conscious experiences and their structures. It may or may not be true, but at least it seems deep enough that it’s a plausible candidate to answer the question of what the dynamics of consciousness is all about.

Suppose we hit a dead end there and that idea turns out to be wrong, that Gödel’s theorem, as interesting as it is, turns out not to be an adequate foundation for our dynamics of conscious agents. If we can take our theory of conscious agents, show how it plugs into, say, the amplituhedron, and then eventually into quantum field theory and general relativity, then what we may be able to do is reverse engineer things. Once we know how to map from conscious agent dynamics into modern physics, can we reverse that map? Can we take what we know about modern physics and its dynamics, pull it back into the realm of conscious agents, and say what kinds of dynamics would get pulled back? That may then focus our attention on certain kinds of conscious agent dynamics that may then help us to grope toward the answer to the question of what consciousness is all about.

~ ~ ~ ~

I got my BA in quantitative psychology from UCLA. While I was there, I took some classes on artificial intelligence and neuroscience of vision that caught my interest. One class pulled those together, a graduate class that I took in which we looked at the work of David Marr. He was bringing artificial intelligence ideas together with neuroscience ideas to study human vision. His idea was to be mathematically precise, to come up with mathematical theories that you could implement in a computer for things like seeing in 3D, object perception, and object recognition. As an undergraduate, I thought this was wonderful. This was someone who was using mathematics, computers, and artificial intelligence to solve problems in human vision, and eventually to build robotic vision systems.

I was very interested in the relationship of computing to humans. I was interested in questions like, are humans just computers or are we more than computers? And, what’s the relationship between human cognition and computation? David Marr was at MIT in the Artificial Intelligence Laboratory and what’s now the Brain and Cognitive Science Department, so I went there, and he and Whitman Richards became my co-advisors.

I worked with Marr for only about fourteen months because he died young, at age thirty-five of leukemia, unfortunately. It was a great loss personally and to the field. But I did have that chance to work with him and the wonderful team that he’d assembled around him. I got to jump in and see what artificial intelligence can do, how far it can go in understanding human vision.

I completed my PhD there, working on human vision. Then I went to UC Irvine as a professor of cognitive sciences in 1983, and I’ve been there ever since. Now my own research is focused on specific problems in human vision, because it’s good to take on specific problems if you’re trying to understand how human nature is related to computation. It’s good to jump in and try to build computational devices that model human nature and see how far you can go. It turns out you can go quite far. In fact, there’s almost no area of cognitive science—learning, memory, problem solving, sensory perception, language development—that isn’t beautifully treated by these functionalist computer kinds of models. There’s only one area that has been a problem, and that is conscious experience.

~ ~ ~ ~

There’s an attitude toward things that accepts the possibility that everything I believe is false. But if I’m right about anything, I’m right that I have experiences—that I’m having a headache right now, or that I’m experiencing a chair in front of me, or a table. As philosophers say, I’m having an experience “as of” a table, or an experience as of a chair, or as of a spoon. So, if I look ahead of myself and I see a table, I’m having an experience as of a table. If I close my eyes, then my experience changes and I no longer have an experience as of a table. Then when I open my eyes, I have once again an experience as of a table.

My physicalist colleagues will say that the table is what’s real; it’s there all the time. Even when my eyes are closed, there is a table that exists even if no perceiver were to look at it. The table not only exists, but it has roughly the shape, texture, color, and other properties that I see. That’s a pretty strong claim.

The physicalist is making the stronger and more tendentious claim, that physical objects have definite values of physical properties, like position, momentum, spin, even if no creature observes it. That’s a strong claim, and it might even sound like a non-scientific claim. That’s more than I’m claiming if I just take conscious experiences as fundamental. All I’m claiming is that when I open my eyes, I have an experience as of a table, and when I close my eyes, who knows what’s happening in objective reality. Of course, you could turn it around and say I’m claiming that if consciousness is fundamental and the physical world isn’t fundamental, there is no table when I don’t observe, no object with a definite position, momentum, and spin. That also seems to be a non-scientific claim. How can you claim something about a physical object and its properties when nothing is observing it? How can you possibly have an experiment to test that?

This kind of debate about whether physical objects exist and have definite properties when they’re not observed is one that Einstein was pushing back in the 1920s and 1930s. It seemed to Einstein that quantum mechanics was saying the moon doesn’t exist when no one observes it, at least in the interpretation of quantum mechanics that Niels Bohr and Werner Heisenberg, the Copenhagen interpretation, had.

Wolfgang Pauli was quite impatient with Einstein. He said the kinds of questions that Einstein was asking were like asking how many angels dance on the head of a pin. Who cares? This was a metaphysical thing that couldn’t be answered with experiment anyway, so why bother with it? That was Pauli’s attitude. Pauli was a towering genius, one of the greatest physicists of the 20th century. It turns out though that he was wrong—this is a question that we can ask and answer experimentally.

A physicist named John Bell, in 1963, found a series of experiments that could test whether something like an atom has a definite value of position, or momentum, or spin even when it’s not observed. It sounds impossible. How could you have a series of experiments that definitely tell you an answer to the question of whether something exists with definite values of properties even when you don’t look at it? Bell discovered that you could test something called local realism, to which there’s two parts.

Realism is the claim that physical objects have definite values of position, momentum, and spin when they’re not observed—that’s realism. Locality is the additional assumption that those definite values of the physical properties have influences that propagate no faster than the speed of light through space. Bell proposed this set of experiments, something called Bell’s inequalities—a beautiful theorem that he came up with. It took a couple of decades, but we got the technology roughly in the 1980s and then started doing the experiments. The experiment has been done many times.

People were blown away by the answer, which is that local realism is false. That has been established by experiment repeatedly. Local realism is absolutely false, but there’s two aspects to it. It could be that either realism is false—particles or objects don’t have definite values of their properties when they’re not observed—or it could be that locality is false—influences can propagate faster than the speed of light. Or it could be that both locality and realism are false.

Then there was another theorem in 1963 and 1964 that Bell and two physicists named Simon Kochen and Ernst Specker proved. It’s about realism and what they called “non-contextuality.” It’s not about local realism, it’s about non-contextual realism. The question here is, is non-contextual realism true? Non-contextual realism is the claim that physical objects, like an atom, have a definite position, or spin, or momentum when they’re not observed. Second, these definite values, their prior nature does not depend on how you choose to measure. The kind of measurement you make does not in any way alter these preexisting values. That’s non-contextual realism.

It turns out that our best theory, quantum theory, predicts quite clearly that non-contextual realism is false. Local realism is false, non-contextual realism is false, and that leaves it quite open that realism itself is false. If realism is false, that raises a couple of questions. Is that true only for microscopic objects—electrons, protons, neutrons, and photons—and not more macroscopic objects?

It’s turning out that this border between the microscopic and the macroscopic, first, is very suspicious. No one has ever been able to make a principled size or scale distinction. What size is microscopic and what scale is macroscopic? Recent experiments have been showing that we can put bigger and bigger systems of atoms—some getting pretty big now, thousands of atoms—and put them in quantum superpositions so that the quantum effects that the Kochen-Specker-Bell inequalities are true of these systems that involve thousands of atoms. These are huge molecules with thousands of atoms—getting close to the size of a virus. We suspect that as we continue to develop technology, we’ll find that this boundary between the microscopic and the macroscopic is not nearly so firm as you might think.

The bottom line is local realism is false and non-contextual realism is false. So, what does that mean about the notion of public physical objects? What do we mean in science by third-person science and public physical objects? Intuitively, what we talk about is the way science works and the way it’s in some sense objective. I can watch a ball rolling down an inclined plane, I can measure its acceleration, and I can compute the effects of gravity on it. Then you can look at that very same ball and make your own independent measurements of that public physical ball. If your measurements and my measurements agree, then we can start to have objective science.

There’s this notion of public physical objects and third-person science in the sense that independent observers can do scientific experiments on the same object and come to some kind of agreement. Sometimes the agreement isn’t absolute, like if we’re measuring the length of a meter stick. It turns out if you’re moving fast relative to me, you will get a different length for the meter stick than I will. There’s something called the Lorentz contraction that happens. We can take those kinds of things into account and have a dictionary between the distance you measure and the distance I measure. If they’re the same up to the Lorentz contraction, then we would still say that we agree. And even in special relativity, the spacetime interval is something that we would all agree with on the exact number. That’s the general notion that we have of public physical objects and third-person science.

The idea that local realism and non-contextual realism are false leads me to argue that in fact realism is false. I want to propose that realism is false, and what we’re seeing is more like a user interface or a virtual reality headset. Think about a virtual reality game of tennis. You’re playing VR tennis with a friend, you both have your headset and body suits on, you see your friend’s avatar on a tennis court and you start playing. Your friend hits the tennis ball to you, and you hit the same tennis ball back to your friend, but is your friend seeing exactly the same tennis ball that you’re seeing? Well, of course not. There’s no public tennis ball. You have some photons being sprayed to your eye by your headset, and those photons are causing your visual system to create your own perception of what you would call a green tennis ball. Your friend has a headset on, which is spraying photons to his eye, and his visual system is creating his own green tennis ball perception.

It turns out that both of those perceptions are coordinated by something else, namely a supercomputer that’s sending the photons to both headsets, causing both headsets to work in coordination. Notice in this example that it looks like there’s a public object, namely a green tennis ball, but there isn’t. There is your tennis ball that you perceive and that disappears when you close your eyes, and your friend’s tennis ball that he perceives and disappears when he closes his eyes. There’s no public tennis ball in this example.

All the things that we would do to say that objects really exist even when they’re not perceived hold here in virtual reality. We might say, I know that this table exists because I closed my eyes and my friend Joe can see the table even when I don’t look. Or I can close my eyes and touch the table and can feel it even when I’m not seeing it. Or I can take this spoon and close my eyes, drop it, and know exactly where to look when I open my eyes. You can do all those things in virtual reality. I can take my green tennis ball in virtual reality, close my eyes, drop the tennis ball and know where I’m going to see it. That doesn’t mean that the tennis ball exists and has any physical properties when it’s not perceived; it just means that there is some objective reality.

I’m not denying that there is an objective reality. There is some objective reality that exists independent of whether or not I perceive it, but that objective reality is not space and time or anything inside space and time. Those are just human forms of perception. That’s what quantum theory is telling us. It’s telling us local realism is false, non-contextual realism is false, and realism is false, at least what we call realism of objects in space and time. They don’t exist, except when they’re perceived. They don’t have their properties, except when they’re perceived because spacetime is not fundamental. That’s what the physicists are now telling us, like Nima Arkani-Hamed. Spacetime is doomed. There is an objective reality, but it’s not space and time. It’s a deeper reality outside of space and time. Spacetime is emergent and is not fundamental.

Here’s a cognitive neuroscientist talking about consciousness being fundamental reality, not space and time, and that’s surely treading on the turf of physics. So, what do physicists think about this? Do they just dismiss this out of hand? There’s an interesting history of physicists and their ideas about consciousness. Some of the early quantum physicists were very interested in consciousness. Erwin Schrödinger was interested in it, so were Eugene Wigner and John von Neumann. Wigner thought that consciousness was fundamental, and von Neumann said that as well. There are various interpretations as to whether he was serious about it or not, but he did talk about consciousness being fundamental.

There were a number of physicists who have said that, but among modern physicists, I would say that most simply do not take the idea that consciousness could be fundamental seriously. They would be dismissed pretty much out of hand. The idea that spacetime is doomed, that there’s something beyond space and time, doesn’t entail that that something is consciousness.

Some physicists are proposing that consciousness might be a state of matter. Max Tegmark, for example, has the notion of perceptronium, where certain states of matter could give rise to conscious experience. That idea is very different from the kind of idea that I’m proposing. I’m not proposing that consciousness is a special state of matter. I’m saying that consciousness is fundamental outside of space and time. Space and time itself, and what we call physical objects and their matter inside space and time, are interface descriptions of what’s going on in the dynamics of conscious agents.

Other physicists are proposing other models of what’s behind space and time; again, not consciousness, maybe quantum information—quantum bits and quantum gates. I certainly understand why a physicist would not feel inclined to jump all the way in and say consciousness is fundamental. The proof will be in what we can do. If we can get a mathematically precise theory of conscious agents and the network dynamics of those conscious agents, and we can show that it plugs in, say, to the amplituhedron that Nima Arkani-Hamed has been looking at, and it gives us new predictions, then and only then would I expect that physicists take this stuff seriously. I certainly understand them not taking it seriously until I make some new concrete prediction that affects physics.

I heard a talk recently by Nima Arkani-Hamed in which he said something he advised was just speculative on his part. He said that maybe one of the problems that they’re having in trying to get a deeper understanding of physics that resolves some of the paradoxes between quantum theory and gravity is the division between the subject and the object, between the observer and the observed. Somehow that division, which is required by quantum mechanics, is a real source of problems because the observer has to effectively be infinite if you’re going to have any precise measurements in quantum theory. That has to do with the idea that there are all these quantum fluctuations, and if you’re trying to measure something to infinite precision and you have a finite measuring device, then the quantum fluctuations will perturb the measuring device and give you the wrong answers by the time you get to the fiftieth decimal point, or the hundredth decimal point, or ten to the hundredth decimal point. He was saying maybe we’re going to have to figure out a way to either get rid of that division or multiple ways of doing that division. There’s something about the division between the observer and the observed that will have to be changed.

What’s interesting to me is that in this theory of conscious agents, that’s precisely what I do. The observer and the observed distinction goes away. All are the same mathematical structure, and all are conscious agents. In this dynamical theory, when agents interact, they form new agents. You can have simple agents with few conscious experiences, maybe only two. We might call that a one-bit agent; it only has two experiences, but they can interact to create two-bit agents and four-bit agents, all the way up to however big you want. What agents are really observing are other agents. So, the division between subject and object is not this fundamental distinction. The observer-observed are all the same kind of thing. The boundary between them is completely fluid.

~ ~ ~ ~

I’m collaborating with several mathematical physicists right now, working to get some predictions that will grab the interest of this community. One of the biggest influences on me, the person who got me into cognitive neuroscience was David Marr. His writing was powerful, his ideas were brilliant, and he grabbed my attention when I was in my early twenties. It was a great privilege to work with David Marr and then with Whitman Richards, who was my co-advisor while Marr was alive. After Marr died, Whitman Richards was my sole advisor. He was just a wonderful adviser. He gave me the freedom to pursue what I wanted to. He gave me feedback, treated me as an equal, and treated my ideas with respect. We were friends for decades afterwards until his death just a couple of years ago. Whitman had a long-term impact.

Another impact on me was a mathematician named Bruce Bennett, who was a professor of algebra and geometry here at UC Irvine. He took me under his wing when I first came here to UCI, and he and I collaborated for fifteen or twenty years. I’m not a mathematician, so he was very patient and taught me a lot of mathematics. Chetan Prakash, who is a mathematical physicist, also has had a big influence on me and has continued to collaborate with me.

More recently, Federico Faggin has been a big influence. Faggin is probably a name that most people haven’t heard but should know. He was the young genius at Intel who invented the microprocessor. He helped perfect the silicon gate technology. He went on to invent the Z80 and the 8080. He was the CEO of Zilog, also the CEO of Synaptics, where they developed the touch pad. Federico is a genius. He’s also very interested in consciousness.

He heard me give a talk six or seven years ago on my mathematical model of consciousness, struck up a conversation with me, and we’ve been friends ever since. We collaborate closely. His ideas are similar to mine. We’re on the same page, but different enough that it’s interesting. We have strong debates on the details, which is very good. Federico has helped to assemble a team that he’s funding. It would be difficult to get the National Science Foundation or the NIH to fund my research because it’s so far out there, but Federico Faggin is funding it from his own foundation, the Faggin Foundation, for which I’m exceedingly grateful. It’s not the funding that’s the primary thing, although it’s very helpful, but Federico’s ideas are extremely influential and helpful to me.

In terms of some other peers in philosophy of mind, I’m quite impressed with the work of Dave Chalmers. I like his thorough analysis, his mathematical sophistication, his philosophical sophistication, and his non-doctrinaire approach. I like how he surveys the various possibilities and looks at the pros and cons of the various possibilities. I never see him getting dragged into ad hominem debates. He always keeps it where it should be, which is not in personal attacks, but just focusing on the strengths and weaknesses of various ideas. I’ve been heavily influenced by Dave Chalmers and his writing.

There are definitely people who would disagree with me, as I believe Dan Dennett does. He is into conscious illusionism. I talk about conscious realism. I think conscious experiences are real and maybe the foundational reality. Dan Dennett says that space, time, and matter are fundamental. What we call consciousness, in particular, phenomenal conscious experiences—the “what is it like” aspect of consciousness—is merely an illusion that comes about when certain processes in our brain are monitoring the activity of other processes in our brain. The way they monitor and the language in which they couch what they’re monitoring is what leads to the illusion of consciousness. Keith Frankish, Dan Dennett, and others are spearheading this illusionism approach.

It’s not my approach and I disagree with it, but I’m glad they’re mapping out that part of the conceptual space. It’s important to have different points of views. Thinking about their ideas forces me to rethink certain aspects of my own approach. Yeah, we disagree, but it’s a profitable and useful kind of disagreement.

One other person that I should mention that was a big influence was Francis Crick. He was the one who gave permission for scientists to jump in and study consciousness. When I was a graduate student at MIT, I was interested in consciousness, but it wasn’t considered a proper subject of science. It was a little bit too woo woo. I studied it, but I didn’t call it that. I published a book with my collaborators, Bruce Bennett and Chetan Prakash called Observer Mechanics (1989). It’s effectively a mathematical model of dynamics of consciousness, but we just called it observer mechanics and left the consciousness out. Within a few years it was perfectly fine to talk about consciousness and that was largely due to the influence of Francis Crick.

Francis also was the intellectual leader of a group in Southern California that I was lucky to participate in called the Helmholtz club. The Helmholtz club brought together thinkers and professors from various universities in Southern California. We all met at the university club at UC Irvine for nearly twenty years on a roughly monthly basis, with some breaks. A group of a dozen or fifteen of us were the core group, and we would bring in guests and outside speakers. We were after understanding this hard problem of consciousness. Francis was looking at it from a hard-nosed neurobiological point of view—the neurocircuits and the activities that cause conscious experiences. He was hoping to demystify consciousness just like he’d demystified life when he and Watson discovered the structure of DNA. He was looking for the double helix of neuroscience that would demystify consciousness. It was a great pleasure to watch him at work, to see him grappling with the neuroscience data, questioning researchers about their latest findings, and then trying to come up with a model of how neuroactivity could create conscious experience.

I’m not going to be here forever. I need to help the next generation understand the ideas and carry on when I’m no longer able to do it. There is a balance that we all have to strike up between how much time we spend communicating the ideas and how much time we spend having fun exploring the ideas. That’s what it’s like climbing a mountain. You climb it because it’s there. We’re exploring these ideas because it’s incredibly fun to explore them, but then it’s time to stop having that fun. I do enjoy the communication process, but it’s different than the exploration of the ideas.

I try in my public communication, in podcasts and so forth, to communicate to a broad, intelligent but non-specialist audience. I would hope that intelligent, lower division undergraduates could understand what I’m saying. That’s my goal, because that’s often my audience. On the other hand, I’m hoping to catch the brilliant minds who know high-level physics or mathematics and could push this thing to heights that I can’t push it. I try to make it interesting to a broad audience, but also have enough beef that it is not dismissed by people who are talented and could find a real project in this.

I’m planning to officially retire from the UC Irvine this July. I’ll still be on the faculty, but I’ll be emeritus. I still plan to bring in grant money and do the research. As anybody who is a professor knows, you spend a lot of time teaching, and doing committee work, writing grants and reviewing grants, so all of the extraneous duties disappear. That’s one reason why I’m retiring. I’ll have more extended time to sink deeply into the ideas, especially when I’m trying to make this connection, which is my goal between the long-term behavior of conscious agent networks and perhaps the amplituhedron, or these interesting structures that physicists are finding that seem to be prior to spacetime and may give rise to spacetime.

My goal is to work with my team to get a mathematically rigorous theory over the next four or five years, and to get this far enough along that even if we don’t have the whole thing worked out, the ideas are promising enough that it’s worth writing a book that focuses on the idea that consciousness is fundamental. Even if I can’t bring that all the way home, I would like to bring it part of the way and then entice a new generation that’s mathematically sophisticated and sophisticated in physics to then bring it all the way home, and do it quickly enough that I can read it. I want to know the answer. That’s my real motivation. I want to know the answer to the hard problem of consciousness. Does the idea that consciousness is fundamental and could give rise to physics pan out? I am exceedingly interested in that. If I don’t get it, I need to get a book out there to have brighter people work on it so I can read their papers. That’s my goal