An Argument Against Reality

In our everyday lives, we tend to assume that our perceptions—of sights, sounds, textures, tastes—are adequate representations of the real world. Of course, when we pause and reflect on this—or when we find ourselves deceived by a visual illusion—we realize with surprise that what we perceive is never directly the world itself, but rather our brain's best guess about what the world is like, a kind of internal simulation of an external reality. Still, we rely on the fact that our simulation is reasonably adequate. If it were not, would not evolution have weeded us out by now? It is unlikely that true reality [in itself] is forever beyond our reach, but surely our senses at least provide us with a residue of true reality.

Donald D. Hoffman, a professor of cognitive science at the University of California, Irvine, says this is not the case. Hoffman has spent the past three decades studying sensory perception, artificial intelligence, evolutionary game theory, and the brain, and his conclusion is quite startling: the world presented to us through our sensory perceptions bears no resemblance to reality. Furthermore, Hoffman says we should be grateful to evolution for presenting us with such a magnificent illusion, because by driving truth to extinction, it maximizes evolutionary fitness.

Addressing questions about the nature of reality, and distinguishing the observer from the observed, is a project that crosses the boundaries of neuroscience and fundamental physics. On one side, we find perplexed researchers trying to understand how a three-pound lump of gray matter, obeying nothing but the ordinary laws of physics, can give rise to first-person conscious experience. This is what has been fittingly called the "hard problem."

On the other side are quantum physicists, who marvel at the strange fact that quantum systems do not appear to be definite, localized objects in space until we observe them. Experiment after experiment has shown—defying common sense—that if we assume the particles making up ordinary objects have an objective, observer-independent existence, we arrive at incorrect answers. The core lesson of quantum physics is clear: there is no public object sitting out there in some pre-existing space. As the physicist John Wheeler puts it, "Useful as it is under ordinary circumstances to say that the world exists 'out there' independent of us, that view can no longer be upheld."

So while neuroscientists struggle to understand how something like a first-person reality can exist, quantum physicists grapple with the mysterious problem of how anything other than first-person reality could exist. In short, all roads lead to the observer. And that is where you can find Hoffman—a boundary-crossing scientist, trying to provide a mathematical model of the observer, and attempting to reach the reality behind the illusion. Quanta Magazine met with him to try to understand more.

Gefter: People often use Darwinian evolution as an argument that our sensory perceptions adequately reflect reality. They say, "Obviously we must be connected to reality in some way, otherwise we would have been destroyed long ago. If I think I see a palm tree in front of me but it is actually a tiger, I am in trouble."

Hoffman: That's right. The classic argument is that those of our ancestors who saw more correctly [more accurately] had a competitive advantage over those who saw less correctly, and therefore were more likely to pass on the genes that contained the code for more accurate perception. As a result, after thousands of generations, we can be quite confident that we are the descendants of those who saw more correctly, and thus we too see correctly [everything as it really is]. But I think [such a view] is profoundly wrong. [This view] lacks a proper understanding of the fundamental reality of the [mechanism of] evolution, namely that evolution is based on fitness functions—mathematical functions that describe how and how well a strategy achieves the goals of survival and reproduction. [In this regard,] the mathematical physicist Chetan Prakash has proven a theorem I proposed: according to the [mechanism of] evolution by natural selection, an organism that sees reality as it [actually] is will never be fitter than an organism of equal complexity that perceives no [part of] reality but is instead tuned solely for fitness. Never.

Gefter: You have also done computer simulations to demonstrate this [theorem]. Can you give an example?

Hoffman: Suppose in reality there is a resource [a reserve] like water, and you can quantify its available amount in an objective order—very little water, a medium amount of water, a great deal of water. Now suppose your fitness function is linear, so a small amount of water yields a small amount of fitness, a medium amount yields medium fitness, and a great deal of water yields a great deal of fitness—in that case, the organism that sees the truth of the [amount of] water in the world will win, but only because the fitness function [in this case] happens to align with the true structure of [water resources] in reality. In general, [however,] in the real world, this will never be the case. A more realistic [scenario, what actually is the case,] is a bell curve—for instance, with too little water you die of thirst, but with too much water you drown, and only somewhere in between is suitable for survival. Now, the fitness function [of us organisms] does not match the structure in the real world. And this alone is enough to drive truth to extinction. For example, an organism tuned for fitness might see small and large amounts of an available resource as, say, red, to indicate low fitness, while it might see intermediate amounts as green, to indicate high fitness. The perceptions [of such an organism] will correspond to fitness, but not to truth. [This organism] will see no distinction between small and large—[since] it only sees the color red—even though such a distinction exists in reality.

Gefter: But how can seeing a false reality be beneficial for an organism's survival?

Hoffman: [Here] there is a metaphor that has only become available to us in the last thirty or forty years, and that is the desktop interface. Suppose there is a blue, rectangular icon in the lower right-hand corner of your computer desktop—does that mean that the file itself is blue and rectangular and lives in the lower right-hand corner of your computer? Of course not. But these [considerations] are the only things one can assert about anything on the desktop—[that] this thing has color, location, and shape. These are the only categories available to you, and yet none of them are true of the file itself or anything else [inside] the computer. They cannot possibly be true. This is an interesting thing. If your entire view of reality were limited to the desktop, you could not give a true description of the innards of the computer. And yet the desktop is useful. That blue, rectangular icon guides my behavior and hides from me the complex reality I do not need to know. That is the key idea. Evolution has shaped us with perceptions that allow us to survive. [These perceptions] guide adaptive behaviors. But part of this [guidance] involves hiding what we do not need to know. And this [hidden part] is almost all of reality, whatever reality [in itself] might be. If you had to spend all that time figuring out [the whole of reality], the tiger would eat you.

Gefter: So everything we see is one grand illusion?

Hoffman: We have been shaped to have [only] the perceptions that keep us alive, so we have to take them seriously. If I see something I think is a snake, I do not pick it up. If I see a train, I do not throw myself in front of it. I have evolved these symbols [or icons] to help me survive, so I must take them seriously. But it is a logical fallacy to think that if we must take [these symbols] seriously, we [likewise] must take them literally.

Gefter: If snakes aren't snakes and trains aren't trains, [then] what are they?

Hoffman: Snakes and trains, like physical particles, have no objective, mind-independent features. The snake I see is a description created by my sensory system to inform me of the fitness consequences of my actions. Evolution shapes not optimal solutions, but satisfactory ones. A snake is a satisfactory solution to the problem of how I should act in a particular situation. My snakes and trains are my mental representations; your snakes and trains are your mental representations.

Gefter: How did you first become interested in these matters?

Hoffman: As a teenager, I was very interested in the question, “Are we [humans] machines?” My studies in science seemed to indicate that yes, [we are machines]. But my father was a pastor, and in church they said we are not [machines]. So I decided to resolve [the issue] for myself. [This issue] is a kind of important personal question—if I am a machine, I would like to discover that [reality]! And if I am not [a machine], I would like to know, what is that special magic beyond the machine? That is how, in 1980, I went to the Artificial Intelligence Laboratory at MIT and began working on machine perception. The field of vision research [at that time] had achieved new successes in creating mathematical models for specific visual capabilities. I noticed that [these models] all shared a common mathematical structure, so I thought it might be possible to design a formal structure for [the category of] observation that encompasses all of them, [and] perhaps all possible modes of observation. I was somewhat inspired by Alan Turing. When he invented the Turing machine, he was trying to provide a notion of computation, and instead of putting [unnecessary] frills on [his invented machine], he said, let's get the simplest, most stripped-down mathematical description that might work. That simple formalism is [now] the foundation of the science of [computer] computation. So I asked myself, could I provide an equally simple formal foundation for the science of observation?

Gefter: A mathematical model of consciousness.

Hoffman: Exactly. My [initial] intuition was that conscious experiences exist. I have pains, tastes, smells, all my sensory experiences, moods, emotions, and so on. So I will say [I reasoned that]: one part of this structure of consciousness is a set of all possible experiences. When I am having an experience, based on this experience I might want to change what I am doing. So I need a set of actions I might take and [also] a decision strategy, which, given my experiences, allows me to change how I act. This is the core idea of the whole [project]. I have [defined] a space of experiences X, a space of actions G, and an algorithm D, which allows me to take a new action based on my experiences [in the moment]. Then I posited a W as [a symbol for] a world, [a world] which is also a probability space. The world somehow influences my perceptions, so there is a perception map P from the world to my experiences, and when I perform an action, I change the world, so there is [another] map A from the space of actions to the world. This is the entire structure of this [model]. Six elements. [My] claim is that: this [model] is the structure of consciousness. I put this [claim] out there for everyone so others would have something to attack [to refute].

Gefter: But if there is a W, are you saying there is an external world?

Hoffman: That is the surprising thing about this [model]. I can pull the [element] W out of the model and put a conscious agent in its place, thereby obtaining a circuit of conscious agents. In fact, you can have many networks of arbitrary complexity. And the world [in reality] is just that.

Gefter: [You mean] the world is only [composed of] other conscious agents?

Hoffman: I call it conscious realism: objective reality consists only of conscious agents, only of points of view. More interestingly, I can place two conscious agents in interaction with one another, such that the mathematical structure of this interaction also satisfies the requirements for the definition of one conscious agent. This mathematics reveals something to me. I can take two minds, and these two can create a single, unified new mind. Let me give a concrete example. We have two hemispheres in our brain. But when you perform split-brain surgery, a complete transection of the corpus callosum, you get clear evidence of two distinct consciousnesses. Before this separation occurred, it seemed as though there was only a single, unified consciousness. So it is not unlikely that there is a single conscious agent. And yet it is also the case that there are two conscious agents there, and you can see this when the two hemispheres are separated. I did not expect this; the mathematics compelled me to accept this conclusion. It showed that I can take separate observers, put them together, and create new observers, and continue this ad infinitum. And so on, all the way down to conscious agents.

Gefter: If it is conscious agents all the way down, all first-person points of view, what becomes of science? Science has always been a third-person description of the world.

Hoffman: The idea that what we are doing is measuring publicly accessible objects, the idea that objectivity results from the fact that you and I can measure the same object under precisely the same conditions and obtain the same results—it is now quite clear from quantum mechanics that this idea must be abandoned. Physics tells us that there are no public physical objects. So what is going on? I think about it this way: I can talk to you about my headache and believe that I am communicating effectively with you because you have had your own headaches. The same would hold true for talking about apples and the moon and the sun and the universe. Just as you have your own headache, you have your own moon. But I assume it is relevantly similar to mine. This is an assumption that could be false, but it is the source of my communication, and it is the best we can do regarding public physical objects and objective science.

Gefter: It does not seem that many people in neuroscience or the philosophy of mind think about fundamental physics. Do you think this has been an obstacle for those seeking to understand consciousness?

Hoffman: I think it has been. Not only do they ignore the advances in fundamental physics, they often state their neglect explicitly. They will openly say that quantum physics has nothing to do with the aspects of brain function that are causally involved in consciousness. They are certain that it must be the classical properties of neural activity—the spike rates, the strength of connections at synapses, and perhaps dynamical properties as well—properties that exist independently of any observer. These are all very classical concepts under Newtonian physics, where time is absolute and objects exist absolutely. It is no wonder that neuroscientists are perplexed as to why they are not making progress. They are depriving themselves of the insights and breakthroughs that physics has produced. These insights are available to us, and yet my colleagues say, "No thank you, we will stick with Newton. We will remain 300 years behind in our physics."

Gefter: I think they are reacting to models like those of Roger Penrose and Stuart Hameroff, [models according to which] there is still a physical brain, [a brain] that still resides in space, but apparently performs some astonishing feats. In contrast, you are saying, "Look, quantum mechanics tells us that we must question the very concept of 'physical objects' existing in 'space.'"

Hoffman: I think that's exactly right. Neuroscientists say, "We don't need to invoke those kinds of quantum processes, we don't need quantum wave functions collapsing inside neurons, we can simply use classical physics to describe processes in the brain." I [in contrast] emphasize the larger lesson of quantum mechanics: neurons, brains, space... these are just symbols we use, they are not real. It is not that there is a classical brain doing some quantum magic. [The point] is that there is no brain! Quantum mechanics says that classical objects—including brains—do not exist. So this is a much more radical claim about the nature of reality and does not involve a brain performing some astonishing quantum computations. So even Penrose has not gone far enough in his theory. But, you know, most of us are born realists. We are born physicalists, [and] giving this [realism] up is really, really hard.

Gefter: To return to the question you started with as a teenager: Are we machines?

Hoffman: The formal theory of conscious agents that I am developing is computationally universal—meaning, it is a machine theory. And [precisely] because the theory is computationally universal, I can derive all of cognitive science and neural networks from this theory. Nevertheless, at present I do not think we are machines—partly because I distinguish between the mathematical representation and what is being represented. As a conscious realist, I posit conscious experiences as ontological primitives, [as] the most fundamental constituents of the world. I claim that experiences are the real coin of this realm [the world]. The experiences of everyday life—my real feeling of a headache, my real feeling of the taste of chocolate—that is the ultimate nature of reality.

.

---

.

Translator's Addendum:

In an email to Dr. Hoffman, I inquired about his view regarding the antinomic consequences of the "conscious realism" theory—namely, how such a theory, which ultimately seeks to present a "theory of everything," would not be challenged when the theory is applied to itself?

Dear Professor Hoffman,

I was very excited to study the findings of your research on human perception; your research, if duly considered by the philosophical community, would have a catastrophic impact on our current epistemology—though I consider such a "catastrophic impact" a form of progress in philosophy.

However, I wanted to know whether you have considered the self-contradictory consequences of your proposition:

a) Proposition: "If our sensory perceptions have been shaped by evolution through natural selection, then the probability that we perceive reality as it [in itself] is, is zero"!

b) Conclusion: Then "space and time" along with that "red tomato" before my eyes are merely "useful fictions." In other words, they do not exist in reality.

But

If “time” does not exist in reality, then evolution, which by definition is predicated on the existence of “time,” must also be false! And this, in the final analysis, leads to the paradoxical claim that “if evolution is true, then evolution is false!”—in other words, to begin with, your “interface theory” must inevitably presuppose the truth of evolution/time, but at the end of the investigation, the theory leads to the negation of the truth or reality of time/evolution. In short, if “time” is a “useful fiction,” then evolution is a “useful fiction,” and if evolution is a “useful fiction,” then your own theory is also a “useful fiction!”

This seemingly absurd conclusion could itself be a great achievement in epistemology; as Hegel had prophesied, the final stage in the odyssey of Absolute Spirit is when Spirit (or soul) finally (at the end of history) gains awareness of the self-contradictory nature of its own knowledge, and it is then that humanity will realize that life itself is truly a contradiction.

Have you independently examined or addressed this issue somewhere?

I am currently translating your interview in Quanta Magazine for an Iranian philosophy website. If you would be kind enough to reply to this email and address the matter as you see fit, I will append your response to the translation of your interview.

With respect

.

---

.

Dear Ba’sat,

Thank you for your email and your insightful question.

You correctly conclude that the “Fitness-Beats-Truth” (FBT) theorem leads to a contradiction. Indeed, this was my hope when I began investigating the possible truth of such a theory. The theorem refutes a fundamental assumption of conventional evolutionary biology—that space and time, and physical objects such as DNA, exist even when unperceived.

But I see this [conflictual situation] as the standard way to make progress in science. We take our best scientific theories and push them to their absolute limits, trying to break [falsify] them. If we succeed in breaking a great theory, we are not disappointed. On the contrary, this is a great moment of discovery, and a grand opportunity to revise and improve our theories.

We also try to break theories by pitting them against each other and searching for contradictions. The situation with general relativity and quantum field theory is a case in point. These two theories contradict each other, and therefore one or both of them cannot, in their current form, be correct. Again, we are not disappointed. We are excited by the possibility that we can learn something deeper about the universe.

The “Fitness-Beats-Truth” (FBT) theorem, using the algorithmic core of evolutionary theory, namely the formalism of evolutionary game theory, shows that some of the peripheral physicalist assumptions of evolutionary theory are almost certainly false. This [situation] then compels us to search for new, non-physicalist foundations for a scientific theory of reality. One constraint on this new theory is that when we finally project it back onto the spacetime interface of Homo sapiens, it must [still] be able to yield conventional Darwinian evolution.

This is how scientists treat their theories: these theories are wonderful tools that help us refine our claims about reality. In this way, we have the chance to find out exactly where we have made a mistake. No theory and no scientist has authority. Every theory is a useful tool, and every scientist is a student of nature. Scientists are also human, and some scientists are dogmatic about their scientific claims. But science as a social project, by pitting one scientist against various claims and by requiring all theories to be precise and falsifiable, provides a solution to such dogmatism.

The greatest obstacle to scientific progress is the assumption that we already know everything. And the greatest aid is the conscious recognition of the limits of our current conceptions and the willingness to precisely and testably optimize our current theories.

I have elsewhere proposed a non-physicalist theory of reality. I have also attached several recent papers that develop [my] theory. My theory is probably incorrect. But I am trying to make it more precise, so that we can find out exactly at which point [the theory] leads to error, and so that we might perhaps have a relative chance to repair it, or to initiate an entirely new kind of theory.

Best wishes

Don.

.

---

.

Dear Prof. Hoffman,

I couldn't be more thrilled at reading the findings of your research into human perception; your research would, if duly noticed by the philosophical community, have no less than catastrophic impact on our epistemology at large--and I would consider such a "catastrophic impact" as philosophical progress actually.

Nonetheless, I was wondering if you'd considered one major self-refuting implication of your theorem:

1. A) "If our senses have developed through evolution by natural selection, then the probability that we perceive reality as it is, is zero"
2. B) Then "time and space" along with the proverbial "red tomato" in front of me are just "useful fictions" i.e. they do not exist in reality.

BUT

If "time" does not exist in reality, then evolution, which is by definition predicated on the existence of "time", has to be false! And this, in the final analysis, amounts to arguing that " if evolution is true, then evolution is false"!--in other words, to get off the ground, the "interface theory" has to inevitably presume the truth of evolution/time, yet the theory finds itself negating the truth of time/evolution by the end of the inquiry. More succinctly yet, if "time" is a "useful fiction", then evolution is a "useful fiction", and if evolution is a "useful fiction" then the "interface theory" is a "useful fiction"!

This seemingly absurd conclusion might well be the greatest achievement in epistemology; as Hegel prognosticated, the last stage in the odyssey of the Absolute Spirit is when the Spirit finally (at the end of History) gains consciousness of the self-contradictory character of its self-knowledge; then, man will come to realize that truly his life is a paradox.

I wonder if you have addressed this issue independently.

P.S.: Currently I'm translating your Quanta Magazine interview for an Iranian philosophy website; in case you are kind enough to reply to this very email, addressing the issue above as you see fit, I would annex your reply to the end of your interview.

Respectfully,

Bessat Elmie



 

 Dear Bessat,

Thank you for your email, and your insightful question.

You are right to conclude that the "Fitness-Beats-Truth" (FBT) theorem raises a paradox. Indeed, that was my hope when I began investigating the possibility that such a theorem might be true. The theorem does falsify a major assumption of standard evolutionary biology—that space and time, and physical objects such as DNA, exist even if they are not perceived.

But I view this as a standard way to make progress in science. We take our best scientific theories and push them to their limits to try to break them. If we succeed at breaking a great theory, we aren't dismayed. To the contrary, it is a great moment of discovery, and a great opportunity to revise and improve our theories.

We also try to break our theories by pitting them against each other, looking for contradictions. Such is the case for general relativity and quantum field theory. They contradict each other, and so one or both cannot, in their current form, be correct. Again, we aren't dismayed by this. We are excited at the possibility to learn something deeper about the world.

The FBT theorem uses the algorithmic core of evolutionary theory, namely the formalism of evolutionary game theory, to show that some of the peripheral physicalist assumptions of evolutionary theory are almost surely false. This then prompts us to look for new, non-physicalist, foundations for a scientific theory of reality. One constraint on this new theory must be that when we project it back into the space-time interface of Homo sapiens, we should get back standard Darwinian evolution.

This is the way that scientists treat their theories: they are wonderful tools that help us to be precise in our claims about reality. That way we have a chance to find out precisely where we are wrong. No theory and no scientist is an authority. Each theory is a helpful tool, and each scientist is a student of nature. Scientists are human beings, and some scientists are dogmatic about their scientific claims. But science as a social enterprise provides a remedy for such dogmatism, by pitting scientists with different claims against each other, and by requiring all theories to be precise and subject to possible falsification. The greatest obstacle to scientific progress is the assumption that we know it all already. The greatest help is a sober recognition of the limits of our current ideas, and a willingness to try to make precise and testable improvements to our current theories.

I have proposed a non-physicalist theory of reality here. I have also attached some recent papers that develop the theory. My theory is probably wrong. But I strive to make it precise, so that we can find out precisely where it is wrong, and perhaps have a chance to fix it, or launch an entirely new kind of theory.

Best wishes,

Don

.

---

.

A two-minute clip of Donald D. Hoffman (original language)

PDF file of the text on this page

.

---

.

An Argument Against Reality

Interview with Donald D. Hoffman

Translator: Be'sat Elmi

.

---

.