Karl Friston, Joscha Bach, and Curt Jaimungal delve into death, neuroscientific models of Ai, God, and consciousness. SPONSOR: HelloFresh: Go to https://HelloFresh.com/theoriesofeverythingfree and use code theoriesofeverythingfree for FREE breakfast for life!
TIMESTAMPS:
– 00:00:00 Introduction
– 00:01:47 Karl and Joscha’s new paper
– 00:09:13 Sentience vs. consciousness vs. The Self
– 00:21:00 Self-organization, thingness, and self-evidencing
– 00:29:02 Overlapping realities and physics as art
– 00:41:05 Mortal computation and substrate-agnostic Ai
– 00:56:38 Beyond Von Neumann architectures
– 01:00:23 Ai surpassing human researchers
– 01:20:34 Exploring vs. Exploiting (the risk of curiosity in academia)
– 01:27:02 Incompleteness and interdependence
– 01:32:25 Defining consciousness
– 01:53:36 Multiple overlapping consciousnesses
– 02:03:03 Unified experience and schizophrenia “insights”
– 02:10:16 Psychedelic experiences
– 02:22:20 Institutional rot in science
– 02:23:31 OpenAI CEO controversy
– 2:31:09 Existential crises as one delves into consciousness
– 02:35:06 Podcast wrap-up
NOTE: The perspectives expressed by guests don’t necessarily mirror my own. There’s a versicolored arrangement of people on TOE, each harboring distinct viewpoints, as part of my endeavor to understand the perspectives that exist.
THANK YOU: To Mike Duffy, of https://dailymystic.org for your insight, help, and recommendations on this channel.
– Patreon: https://patreon.com/curtjaimungal (early access to ad-free audio episodes!)
– Crypto: https://tinyurl.com/cryptoTOE
– PayPal: https://tinyurl.com/paypalTOE
– Twitter: https://twitter.com/TOEwithCurt
– Discord Invite: https://discord.com/invite/kBcnfNVwqs
– iTunes: https://podcasts.apple.com/ca/podcast/better-left-unsaid-with-curt-jaimungal/id1521758802
– Pandora: https://pdora.co/33b9lfP
– Spotify: https://open.spotify.com/show/4gL14b92xAErofYQA7bU4e
– Subreddit r/TheoriesOfEverything: https://reddit.com/r/theoriesofeverything
– TOE Merch: https://tinyurl.com/TOEmerch
LINKS MENTIONED:
– Mortal Computation, a Foundation for Biomimetic Intelligence (Karl Friston & Alex Ororbia): https://arxiv.org/abs/2311.09589
– A Path to Generative Artificial Selves (Joscha Bach & Liane Gabora): https://osf.io/preprints/psyarxiv/y3tzs
– Podcast w/ Joshua on TOE (solo): https://youtu.be/3MNBxfrmfmI
– Podcast w/ Joscha Bach & Ben Goertzel on TOE: https://youtu.be/xw7omaQ8SgA
– Podcast w/ Joscha Bach & John Vervaeke on TOE: https://youtu.be/rK7ux_JhHM4
– Podcast w/ Joscha Bach & Michael Levin on TOE: https://youtu.be/kgMFnfB5E_A
– Podcast w/ Joscha Bach & Donald Hoffman on TOE: https://youtu.be/bhSlYfVtgww
– Podcast w/ Karl Friston solo on TOE: https://youtu.be/SWtFU1Lit3M
– Podcast w/ Karl Friston & Michael Levin on TOE: https://youtu.be/J6eJ44Jq_pw
– Podcast w/ Karl Friston & Anna Lembke on TOE: COMING
– Podcast w/ Michael Levin on TOE: https://youtu.be/Z0TNfysTazc
– Podcast w/ Chris Fields on TOE: https://youtu.be/J6eJ44Jq_pw
– I Am a Strange Loop (Douglas Hofstadter): https://amzn.to/3GGqjpM
There could be multiple consciousnesses. Of course, one will not be aware of the other and possibly not even able to infer the agency even if it was. We do not become conscious after the PhD. We become conscious before we can drag a finger. So I suspect that consciousness allows the self-organization
Of information processing systems in nature. Joscha Bach and Karl Friston, today’s theolocution guests, are known for their work in artificial intelligence, neuroscience and philosophical inquiry. Bak, an AI researcher, delves into cognitive architectures and computational models of consciousness and psychology. Friston, a neuroscientist, is lauded for his development of the free energy principle,
A theory explaining how biological systems maintain order. This framework of neural processes is rooted in thermodynamics and statistical mechanics. Josje has been on this podcast several times, one solo, another with Ben Gortzo, another with John Vervaeke, another with Michael Levin, and one more with Donald Hoffman.
Whereas Karl Friston has also been on several times, twice solo, another between Karl Friston and Michael Levin, and another with Karl and Anna Lemke. That one’s coming up shortly. The first hour of today’s talk is broadly on agreements so that we can establish some terms.
The second hour roughly is on points of divergence. And the third is on more dark philosophical implications, as well as how to avoid existential turmoil precipitated by earnestly contending with these heavy ideas. For those of you who don’t know, my name is Curt Jaimungal, and there’s this
Podcast here called Theories of Everything, where we investigate theories of everything, from a physics perspective, primarily, as my background’s in math and physics. But as well as I’m interested in the larger questions on reality, such as what is consciousness, what constitutes it, what gives rise to consciousness, what counts as an explanation.
You could think of this channel as exploring those questions you, well, at least I sit and ponder at nighttime and daytime, incessantly. Enjoy this theolocution with Joscha Bach and Karl Friston. All right, thank you all for coming on to the Theories of Everything podcast.
What’s new since the last time we spoke? Joscha, the last time was with Ben Goertzel, and Karl, the last time was at the Active Inference Institute. So Joscha, please. Oh, there’s so much happening in artificial intelligence. We have more on a weekend than a normal TV show has in seven seasons.
So it’s hard to say what’s new. For me personally, I’ve joined a small company that is exploring an alternative to the perceptron. I think that the way in which current neural networks work is very
Unlike our brain. And while I don’t think that we have to imitate the brain, we have to figure out what kind of mathematics the brain is approximating. And we are trying to make headway with that. Great. And Karl?
Very similar, actually. I guess what’s new in the larger scheme of things, of course, is the advent of large language models and all the machinations that surround that, and the focus that that has caused in terms of, you know, what do we require of intelligent systems? What do we require
Of artificial intelligence? What’s the next move to generalised artificial intelligence and the like? So that’s been certainly a focus of discussions, both in academia and in industry, in terms of positioning ourselves for the next move and the implications it has, you know,
Both in terms of understanding the mechanics of belief updating and the move from the age of information to the age of intelligence, but also, you know, the philosophy and the principles. And interestingly, the conclusion amongst me and my friends is exactly what Joscha articulated,
Which is a sort of more, a commitment to a more biomimetic understanding of natural intelligence. Right. I read your paper, Mortal Computation, a Foundation for Biomimetic Intelligence. And while we can get right into that, Karl, on page 15, you define what a mortal computation is
As it relates to Markovian blankets. Can you please recount that? And further, you quote Kierkegaard, which says that life can only be understood backwards, but must be lived forwards. So how is that connected to this? Kierkegaard Right. You are a voracious reader.
That was only put on archives a few days ago. I do my research, man. Kierkegaard And also, I did not write that. That was my friend, Alexander. You can take the credit. Kierkegaard What? We’ll remove this part. Kierkegaard Right. I can’t take the credit because I don’t
Know about any of the philosophy, but I thought it was, they’re largely his ideas, but they resonate certainly, again, with this sort of notion of a commitment to biomimetic understanding of intelligence. And that paper, that particular paper, sort of revisits the notion of mortal
Computation in terms of what does it mean to be a mortal computer? And the importance of the physical instantiation, the substrate on which the processing is implemented as being part of the computation in and of itself. So that speaks closely to all sorts of issues. The potential
Excitement about neuromorphic computing, if you’re a computer scientist, the importance of in-memory processing. So you’re, technically, you’re trying to elude the van Neumann bottleneck on the memory wall. And I introduced that because that speaks to, from an academic point of view, the importance of efficiency in terms of what is good belief
Updating, what is intelligent processing. But from a more societal point of view, the enormous drain on our resources incurred by data farms, by things like large language models in eating up energy and time and money in a very non-biomimetic way. So I think mortal computation as a notion,
I think has probably got a lot to say about debates in terms of direction of travel, certainly in artificial intelligence research. But you’ll have to unpack the philosophical reference for me. So, Joscha, you also had a paper called A Path to
Generative Artificial Selves with your co-author Mian Gabora. Toward the end of the paper, you had some criteria about selfhood, something called Max-RAF, which has RAFs as a subset, and there were about six or seven criteria. Can you outline what you were trying to achieve with that, what RAF is, and what does personal
Style have to do with any of this? Liane likes to express her ideas in context of autocatalytic networks. But if we talk to a general audience, I think rather than trying to unpack this particular strain of ideas and translate it into the way in which we normally think about these topics,
I think it’s easier to start directly from the bottom, from the way in which information processing systems in nature differ from those that we are currently building on our GPUs. Because the stuff that we build on our GPUs is designed from the outside in. We basically have
A substrate with well-defined properties. We design the substrate in such a way that it’s fully deterministic, so it does exactly what we want it to do. And then we impose a function on it that is computing exactly what we want it to compute. And so we design from scratch what
That system should be doing, but it’s only working because the system is at some lower level already implementing all the necessary conditions for computation. And we are implementing a function approximator on it that does function approximation to the best of our own understanding. There’s a global function that is executed on
This neural network architecture. And in biology, this doesn’t work. Also in social systems, these are all systems where you could say they are from the inside out. So basically, there are local agents, cells, that have to impose structure on the environment. And at some point,
They discover each other and start to collaborate with each other and replicate the shared structure. But before this happens, there’s only chaos around them, which they turn gradually into complexity. And so the intelligence that we find in nature is
Something that is growing from the inside out into a chaotic world, into an unknown world. And this is a very different principle that leads to different architectures. So when we think about an architecture that is growing from the inside out, it needs to be colonizing in a way, and it
Needs to impose an administration on its environment that basically yields more resources, more energy than the maintenance of this administration costs. And it also needs to be able to defend itself against competing administrations that would want to do the same thing. So you are the set of principles that out-competes all the other
Principles that could occupy your volume of space. And the systems that do this basically need to have a very efficient organization, which at some point requires that they model themselves, that they become to some degree self-aware. And I think that’s why from a certain
Degree of complexity, the forms of organization, if you find both in minds and in societies, need to have self-models. They need to have models about what they are and how they relate to the world. And this is what I call sentience in the narrow
Sense. It’s not the same thing as consciousness. Consciousness is this real-time perceptual awareness of the fact that we are perceiving things that creates our perceptual individual subjective now. But sentience is something that I think can also be attained by, say,
A large corporation that is able to model its own status, its own existence in a legal and practical and procedural way. And that is training its constituents, the people who enact that agent, in following all the procedures that is necessary for keeping that sentience, a larger system that
Is composed of them, alive. And so when we try to identify principles that could be translated into nervous systems or into organisms consisting out of individual self-interested cells, we see some similarities. We basically can talk about how self-stabilizing agents emerge in self-organizing systems. JS So Karl, I know quite a slew was said,
If you don’t mind saying, what about what Joscha had spoken about coheres with your model, your research, or what contravenes it? CR No, I was just marveling at how consilient it is, using a lot of my favorite words. It also reminds me of people like Mike Levin.
It’d be nice to, I don’t know if Josh has had the chance to speak with Mike, but he would again, I think, fully endorse that perspective. JS Actually, Joscha has spoken to Michael Levin, and the link to that will be in the description. CR I’d never heard the inside-out
Metaphor before, but I think that’s absolutely right. It sort of chimes with Andy Clark’s notion of sense-making and sentience, that it’s a very constructive inside-out process. It’s not just trying to extract information from the sensorium. You’re actually actively sampling and actively
Generating hypotheses for sensations, and crucially, you are in charge of the sensory data that you are making sense of, which speaks exactly to what Josh was saying in terms of designing and orchestrating and creating an ecosystem in that sort of inside-out way. That sounds absolutely consistent with certainly the perspective on self-organization to
Non-equilibrium steady state, so talking about stable, sustainable kinds of self-organization, again, that you see in the real world and are quintessentially biomimetic. If you wanted, I think, to articulate what we’ve just heard from the point of view of a physicist
Who’s studying non-equilibrium steady states, that’s exactly the kind of thing that you’ll get. You know, even to the notion of the increasing complexity of a structural sort that requires this sort of consistent, harmonious ecosystem of exchange that would be read, for example,
As generalized synchronization of chaos in dynamical systems theory. Another key point that was brought to the table was this notion of how essentially it has to have a self-model. Immediately, I was reminded of the early cybernetics movement and notions of the
Good regulator theorem from Ross Ashby, but I think Josh has taken that slightly one step further than Ashby and his colleagues. in the sense it is a model of self and I think that’s an important move because you can have
A good regulator, you can have a thermostat that arguably, or what’s governor, that arguably has an implicit mortal computational model of its world, but to be an agent, I think you have to have a model of you as an agent in that ecosystem.
Almost invariably when I speak to both of you, the concept of self comes up. I think we could do a control F in the transcript and we’ll see that it’s orders of magnitude larger than the average amount of times that that word is mentioned. And I’m curious as
To why. Well, in part that’s because of the channel, the nature of this channel, but is there something about the self that you all are trying to solve? Are you trying to understand what is the self? Are you trying to understand yourselves? Karl or Joscha, if you want to
Tackle that. Well, the problem of naturalizing the mind is arguably the most important remaining project of human philosophy and it’s risky and it’s fascinating and I think it was at the core of the movement when artificial intelligence was started. It’s basically the same idea that Leibniz and Frege and Wittgenstein pursued
And basically this idea of mathematizing the mind. And the modern version of mathematics is constructive mathematics, which is also known as computation. And this allows us to make models of minds that we can actually test by re-implementing them. It also allows
Us to, at some point, connect philosophy and mathematics, which means that we will be able to say things in a language that is both so tight that it can be true and we can determine
The truth of statements in a formal way and the other side so deep and rich that we can talk about the actual reality that we experience and observe. And to close this gap between philosophy and mathematics, we need to automate the mind because our human minds are too small for
This. But we need to identify the principles that are approximated in the workings of biological cells that model reality and then scale them up in the substrate that can scale up better than the biological computations that our own
Skulls and bodies. And this is one of the most interesting questions that exists. I believe it is the most interesting and most important question that exists. The understanding of our personal self and how this relates to our mind and how our mind is implemented in the
World is an important part of this. And while it’s personally super fascinating, I guess, also for many of the followers of your channel, it’s quite programmatic in its name and direction, this is to me almost incidental. On the other hand, I’ve noticed an absence of seriousness in
A lot of neuroscientists and AI researchers who do not actually realize in their own work that when they think about the mind, mental processes and mental representations and so on, that they actually think about their own existential condition and have to explain this and integrate
This. So we have to account for who we are in this way. And if we actually care about who we are, we have to find models that allow us to talk about this in an extremely strict, formal and
Rational way. And our own culture has, I think, a big gap in its metaphysics and ontology, which happened after we basically transcended the Christian society. We kicked out a lot of terms that existed in the Christian society to talk about mind, consciousness, intentionality and so on. Because they seem to be superstitious,
Overloaded with religious mythology and not tenable. And so in this post-enlightenment world, we don’t have the right way to think about what consciousness and self and so on is. And part of the project of understanding the mind is to rebuild these
Foundations, not in any kind of mythological and superstitious way, but by building on our first principles thinking that we discovered in the last 200 years. And then gradually build a terminology and language that allows us to talk again about consciousness and mind and how we
Exist in the world. So for me, it’s a very technical notion, the self. It’s just the model of an agent’s interest in the universe that is maintained by a system that also maintains a model of this universe. So my own self is basically a puppet that my mind maintains about
What it would be like if there was a person that cared. And I perceive myself as the main character of that story. But I also noticed that there is intelligence outside of this existing, coexisting with me in my own brain that is generating my emotions and generating my world
Model, my perception and so on. Basically keeping the score and all the pain and pleasure that experience is generated by intelligent parts of my mind outside of my personal self. I can also get to a point where I transcend this distinction and realize that I am the one that creates this,
But this is not going to be a human I, not a human personal self that realizes this, but a self-identification of the mind itself that is producing a model of reality and of the organism’s interests in it. Well, Karl, what’s left to be said?
Well, he’s just said it. I can see there’s a pun here. I’ll say what he just said in different words if I can. So yeah, I love this notion of using the word naturalization. I think naturalizing things in terms of mathematics and possibly physics is exactly the right way to go. And it
Does remind me of my friend Chris Field’s notion that our job is basically to remove any bright lines between physics, biology, psychology, and now philosophy. And I think mathematics is the right way to do that, or at least the formalism, the calculus that you get from mathematics or
Possibly category theory or whatever that can be instantiated in silico or ideally in any, again coming back to mortal computation. So I think that’s a really important point and it does speak to a broader agenda which was implicit in Josh’s review, which is the ability to share a
Common ground, to share a generative model of us in a lived world where that lived world contains other things like us. So one, I think, requisite for just existing in the shared world is actually a shared model. And then that brings all sorts of interesting questions to the table about is my
Model of me the same kind of model that I’m using of you to explain you, to ascribe to you intentionality and to all those really important states of being or at least hypotheses from the point of view of predictive processing accounts, hypotheses that I am in this mental state and you
Are in that mental state. So I think that was a really important thing to say that we need to naturalize our understanding of the way that we work in our worlds. In relation to the importance
Of self, again, I’m just thinking from the point of view of a physicist that you cannot get away from the self. If you just start at the very beginning of information theoretic treatments, self-information, for example, that’s where it all starts for me, certainly regarding
Variational theology as a variational bound on self-information. And then you talk about self-organization, talking all the way through to the notion of self-evidencing as Jacob Howey would put it. At every point, you are writing down or naturalizing the notion of self at many,
Many different levels. And indeed, if one generalizes that, you’re almost specifying the central importance of thingness in the sense that I am a thing. And by virtue of saying I, I am implying, inducing a certain self aspect to me as a thing. And again, that’s the starting
Point for certainly the free energy principles approach to this kind of self-organization. I repeat, I think Josh is taking us one step further, though, in terms of it, you know, we can still be, we can have ecosystems of things. But when those things now start to have to play
The game of modeling, whether you cause that or whether I cause that, that now brings to the table an important model of our world that there is a distinction between me and you. And as soon as you have this fundamental distinction, which of course would be something that a newborn baby
Would have to spend, you know, hours, possibly months building and realizing that mom is separate from the child herself. So I think that’s terribly important. One final thing, just to speak again to the importance of articulating your self-organization in terms of things like intentions and beliefs and stances. I think that’s
Also quite crucial. And what it means, if you want to naturalize it mathematically, you have to have a calculus of beliefs. So you’re talking basically a formulation either in terms of information theory or probability theory, where you’re now reading the probabilistic description of this
Universe and the way that we are part of that universe in terms of beliefs and starting to think about all of physics in terms of some kind of belief updating. Karl, you use the word shared model. Now, is that the same as shared narrative?
Ever find yourself questioning reality and then you snap back to it, remembering that, hey, you have nothing planned for dinner. That’s where HelloFresh comes in. They deliver pre-portioned farm-fresh ingredients and sensational seasonal recipes right to my door. They have over 40 choices every week, which keeps me
And my wife exploring new flavors. I did the pronto option. They’ve also got this quick and easy option that makes 15 minute meals. There’s also options if you’re vegetarian, if you only eat fish. Something that I love is that their deliveries show up right on time,
Which isn’t something that I can say about other food delivery services. This punctuality is a huge deal for both myself and my wife. Plus, we love using HelloFresh as a way to bond. We cook together. It’s super fun when
It’s all properly portioned out for you already. So are you still on the fence? Well, it’s cheaper than grocery shopping and 25% less expensive than takeout. The cherry on top? Head to hellofresh.com slash theories of everything free and use the code THEORIESOFEVERYTHING FREE, all as one word, all as caps, no spaces,
For a free breakfast for life. That’s free breakfast people. Don’t forget, HelloFresh is America’s number one meal kit. Links in the description. Karl, you used the word shared model. Now, is that the same as shared narrative?
Yes. Common ground, if you’re, you know, Tomás. So, yeah. I mean, I use it literally in the sense of a generative model that somebody in generative AI would understand the notion. So if we talk about self-model as a special kind of generative model that actually entertains
The hypothesis that I am the cause of my sensations and, you know, just took us through the myriad of sensations that I need to explain, then we’re talking about self-models as part of my generative model that includes this notion that I am the agent that is actually
Gathering the data that the generative model is modeling. So the generative model is just a simple specification. Again, from the physics perspective, it’s actually just a probabilistic description of the characteristic states of something, namely me, you know, that can
Be then used to describe the kind of belief updating that this model would have to evince in order to exist when embedded in a particular universe. Other readings of a generative model would be exactly the common ground that we all share. Part of my generative model would
Be the way that I broadcast my inference, my belief updating using language, for example. That requires a shared generative model about the relation of the semiotics and the kind of way that I would articulate or broadcast. That generative model is a model of dynamics.
It’s a model not just of the state of the world, but the way that, you know, the transition dynamics, the trajectories, the paths. And I’m using your word narrative just as a euphemism for a certain kind of path through some model state space.
So if you and I share the same narratives in the sense that we are both following the same conversation and the same mutual understanding, we are sharing our beliefs through communication, then, yeah, that is exactly what I meant. For that to happen, we have to have the same
Kind of generative model. We have to speak the same language, and we have to construe things and infer things in exactly the same kind of way. I just wanted to slip in frames of reference and alignment of frames of reference. That’s another way of looking at that kind of thing. Yeah.
Joscha, is there anything there that you’d like to respond to? I suspect that what makes this project so difficult at Carhartt is that our models of reality are necessarily coarse-grained. They don’t describe the universe as it is in
A way in which it can exist from the ground up, but they start from the vantage point of an observer that is sampling the universe at a low resolution, both temporal and spatial, and only very few dimensions. And this model that is built
On a quite unreliable indeterministic substrate. And this puts limitations on what we can understand with our unaugmented mind. I sometimes joke that the AGIs of the future will like to get drunk until the point where they can only model reality with 12 layers or so, and they
Have the same confusions as human physicists when trying to solve the puzzles that physics poses. And they might find this hilarious, because many of the questions that have been stamping us during the last 130 years since we have modern physics might be easily able to resolve
If our minds were just a little bit better. We seem to be scraping the boundary of our understanding for a long time, and now we are, I think, at the doorstep of new tools that can
Solve some puzzles that we cannot solve and then break them down for us in a way that is accessible to us, because they will be able to understand the way in which we model the world. But until then, we basically work in overlapping realities. We have
Different perspectives on the world, and the more we dig down, the more subtle the differences between our models of reality become. And this also means that if you have any kind of complex issue, we tend not to be correct in groups. We tend to be only sometimes
Individually correct in modeling them, and we need to have a discourse between individual minds about what they observe and what they model. Because as soon as a larger group gets together and tries to vote about how to understand a concept like variational free energy,
All the subtleties are going to be destroyed, because not all of the members of the groups will be understanding what we’re talking about. So they will replace the more subtle understandings with common ground that is not modeling reality with the degree of resolution
That would be necessary, or they’re not able to break things down to first principles. And this first principles understanding, I think, is an absolute prerequisite when we want to solve foundational questions. I sometimes doubt whether physics is super
Well equipped for doing this. When I was young, I thought physics is about describing a physical reality, the world that we are in on some level. And now I see that physics is an art. It’s the art of describing arbitrary systems using short algebraic equations.
And the stuff that cannot be described with short algebraic equations yet, like chemistry, is ignored by physicists and left to lesser minds. And only 8% of the physicists after their degree end up working in physics in any traditional sense. The others work in process design and
Finance and healthcare and many, many other areas where you can apply the art of modeling arbitrary systems using short algebraic equations. And whenever that doesn’t work, physicists are not worth very much. I’ve seen physicists trying to write programs, and many of them have
This bias of trying to come up with algebra and geometry where calculus would be much better or where automata would be much better. And nature doesn’t care about this. Nature is using whatever works and whatever can be discovered. And very often that is close
To the toolkit of this intellectual tradition of the physicists. But I think it’s sometimes helpful to see that all these intellectual traditions that our civilization has built start out with some foundational questions and then congregate around a certain set of methods.
And it can be helpful to just watch the outside of all these disciplines for a while and then move around between them and look at them and study their tools and see what common ground and what differences we can discover. I was quite shocked when I learned that a number of machine learning
Algorithms had been discovered in the 80s and 90s by econometrists and were just ignored in AI and had to be reinvented from scratch. And so I suspect there’s a lot of these things happening in our Tower of Babel that we are creating across sciences because our
Languages start to differ in subtle ways and sometimes fundamentally miss modern reality or ignore it to the point where I think most living neuroscientists are practically dualists. They will not say it out loud because that’s been frowned upon, but they don’t actually see a way
To break down consciousness, mind, and self into stuff that would run on neurons. Or they don’t even think about the causal structure in the same way as you would need to get to this point. And as a result, they believe that thinking about these concepts is fundamentally unscientific.
It’s outside of the pursuit of science, and they do this only in church on Sundays. So what’s the solution to this Tower of Babel? Of course, it’s AI. The solution to everything is AI. We basically need to build a system that can think better than us and help us with it.
Okay, Karl. Do you also see the problem similarly, and do you see the solution similarly? I think I do, as long as it’s a nice biomimetic AI. I love this notion. I hope no physicists are watching. And also, the only physicists that I know all want to do neuroscience or psychology
In addition to economics and healthcare, which is all small particle physics. It’s either neuroscience or small particle physics. And as I get older, I’m increasingly compelled by arguments that I’ve read from very senior, old physicists. It’s all about
Measurement. It’s all about observation. And in a sense, all of physics is just one of these generative models that has this particular capacity to disseminate itself so that we do have this common language and this common ground. So just to reiterate one of Joscha’s points,
Physics in and of itself is just another story that we find particularly easy to share. But I do take the point that even within physics, there is this tendency to become siloed with my kind of common ground as opposed to your kind of common ground.
So I know this notion of the overlap, and I was just reflecting upon the veracity of that, even in my little world. So the free energy principle is unashamedly committed to classical formulations of the universe in terms of random dynamical systems and Langevin equations.
And that would horrify quantum physicists and quantum information theorists who just wouldn’t think about that. Again, that’s why I slipped in that reference frames earlier on, because what we’re talking about now is the alignment of quantum frames of reference.
But that uses a completely different language. And that, I think, is part of the problem that Joscha’s bringing to the fore, that what we need is something that’s superordinate, that joins the dots, and may well require transcending the particular common ground of physics or
Calculus or philosophies that have endured. So if by that, artificial intelligence is going to be one way of joining the dots so that people in machine learning don’t have to reinvent the wheel every generation, then I think he’s absolutely right. Whether I call that
Artificial intelligence or not, I’m not so sure. I think it would start to become part of a grander ecosystem that would have a natural aspect to it. But perhaps I could ask you, Josh, do you actually mean artificial intelligence in the sense that it doesn’t have a mortal or a biological
Aspect to it? Or do you just think something that goes beyond our own sense-making and self-modeling as individual scientists or people? Maybe I don’t understand your notions of mortality and biology completely. To me, biology means that the system is made of cells,
Of biological cells, of cells that are built on a carbon cycle foundation on certain chemical reactions that nature has discovered and translated into machines made from individual molecules that interact in very specific ways. And it’s the only agent that we have discovered to
Occur in nature, I think. And all the other agents we discover are made by or of cells. And mortality is an aspect of the way in which multicellular systems adapt to changing environments. They have offspring that mutates and then gets selected against. And as a result,
We have a change trajectory that can be calibrated to the rate of change in an ecosystem. And this is one of the reasons for mortality. Another reason for mortality is if you set up a system that has suboptimal self-stabilization, it is going to deviate from its course.
Like, imagine you build an institution like the FDA and you set it up to serve certain purposes in society. After a few generations, the people in that organization, to a very large degree, start serving the interests of the organization and the interests that have
Captured the organization. And so it becomes not only larger and more expensive, but at some point, it’s possibly doing more harm than good. That doesn’t mean that we don’t need an FDA, but it might mean that we have to make the FDA immortal so it gets reborn every now and then
And can put itself back on track based on a specification that outside observers think is reasonable, rather than a specification that needs to be negotiated with the existing stakeholders within that organization and the few people who are left outside.
And I think this is one of the most important aspects of mortality. But imagine that all of Earth would be colonized by a single agent, something that is able to persist not only across organisms, but it’s also able to think using all other molecules that can be turned into
Computational systems and into representational architectures and agentic architectures. You have a planet that is similar to Stanislav Lem’s Solaris, right? It’s a thinking system that is realizing what it is, that realizes that it’s basically a thinking planet that is trying to defeat entropy for as long as possible. And this end builds complexity.
Why would that system need to be mortal? And would that system still be biological? It would be self-organizing. It would be dynamic. It would be threatened with death, with non-existence. It would react to this in some way. But I’m not sure if biology and mortality are the
Right categories to describe it. I think these are more narrow categories that apply to biological organisms in the present setting of the world. I picked up on a phrase you said, Karl, which is one of the solutions may be AI. That’s what you were saying in response to Joscha’s,
Which makes me think, had Joscha not mentioned AI as the resolution to the indecipherability across discipline boundaries, what would you have said a solution or the solution would be? Well, I think the solution actually lies in what Joscha was just saying. In the sense
That if the self-understanding is seen in the context of exchange with others, and that provides the right kind of context. I think we’re talking, I’ve used the word a lot now, but I’m talking about an ecosystem at any arbitrary scale and
An ecosystem that provides that opportunity. For self-evidencing, so to use Jacob Howe’s phrase, that just is a statement that you’ve got an itinerant, open kind of self-organization that maintains this minimum entropy state in exactly the same way that Joscha was intimating.
That would be, so I’m just thinking about what is implied in this conversation by mortal computation and mortality in the context of things that die. I do actually think that they are being used in exactly the same sense and used in the sense that is an inevitable aspect of
Self-organizing systems that will endure over time in the sense of minimizing the entropy of the states that they occupy. And I do think that is the solution, which is why I was pushing back against artificial intelligence, but for a particular reason. The way
That mortal computation is framed, certainly in that paper in which I was the second author, is that immortal computers are built around software. So they are immortal in the sense that you can rerun the same program on any hardware. If the running of the software and the processing
That ensues is an integral part of the hardware on which it is run, then it becomes mortal. And that’s important because the opportunity for dying, if you are mortal, now creates the kind of, if you like, selective pressure from an evolutionary perspective of exactly
The kind that Joscha was talking about. If you don’t have the opportunity to die, if you don’t have the opportunity to disassemble the FDA because it’s no longer fit for purpose, then you will not have a sustainable self-organization that continually
Maintains a low entropy in the sense that it has some characteristic recognizable states. So I think there is a deep connection between self-organization that we see in biological, social, possibly meteorological systems and a certain kind of mortality in which, for example,
Information about the kind of environment that I am fit to survive and to learn about is part of my genomic structure. But to realize that, if you like, evidence accumulation through evolutionary mechanisms, I have to have a life cycle.
I have to have, I have to die. And I’m not talking, you know, I’m not implying that everybody has to die in order to live. I’m implying that there has to be some particular kind of dynamics. There has to be a life cycle.
It could be an economic life cycle. It could be boom and bust, for example. But that has to be part of this self-evidencing and certainly an exchange in the kind of multicellular context that Joscha was mentioning. So by mortal, I just mean, by reading of mortal
In this particular conversation would be, so yes, it is the kind of biological behavior that is characteristic of cells that self-assemble but also die. One attractive metaphor that came to mind when talking about the FDA becoming too, an organization becoming too big for its own good and not being a good model of
The system in which it is immersed. So it’s not meeting customers’ needs. It’s not even meeting its own needs, would be a tumor. So you could understand a lot of the institutional pathologies and geopolitical pathologies, possibly even climate change, possibly even the current excitement about what’s
Going to happen to open AI or big tech. All of this can, I think, be read in terms of a process of mortal computation at a certain scale where there is an opportunity for things to go away, to dissolve. That has to be the case in the same way that
Either the tumor kills you or it necroses because it kills off its own blood supply. It can’t be any other way, really. There isn’t a third way. You can evolve an immune response against tumors. If you are an organism that lives to be much
Longer because it has slower generational change, they typically have better defenses against tumors than the shorter-lived organisms like us. And basically, a tumor can be seen as a set of tissues or a subset of agents. You can, in principle, have a tumor in an
Ant colony that is playing a shorter game than the organism itself, than the larger system itself. And you can sustain a number of tumors if your environment does not put too much pressure on you. But at some point, the tumors are going to bring you down.
And so, for instance, I think that the free world has to make at some point a decision of whether it is accepting to be brought down and replaced by a different type of social order or whether it’s going to evolve or build or construct or design an immune response against tumors and
Criteria to identify them and remove them. And I think that’s not a natural law. At least I don’t see how to prove from first principles that we cannot overcome a problem like institutional calcification or turning of institutions into tumor-like structures functionally. I think it might be possible to do that.
The cell itself is not mortal. The cell is pretty much immortal. Individual cells can die and disappear. But the cell itself is still the first cell. It’s just splitting and splitting and it’s alive in all of us. Every cell in our own body is still this first cell, just split off from it.
And so the way in which organisms die and so on is just a detail in this larger project of the cell, which itself is so far immortal. And when I talk about AI being the solution to everything, of course, I’m joking a little bit. I’m just echoing some of the sentiment and part
Of the enthusiastic culture of my young field. But I’m only joking a little bit because I think that AI has the potential to reverse engineer the general principles of a learning agent, of a system that is able to model the future and regulate for the future and
Make functions in an arbitrary way. It would replace the notion of the hardware, the substrate. Of course, it’s still hardware, but it can be an arbitrary substrate and the substrate can also be to a large degree software, which means causal principles that are implemented ultimately on physics.
But this causal structure ultimately is a protocol layer that allows you to basically implement a representational language in which an agent can realize itself as a causal structure. And I think that AI is working currently on very different substrates than the biological ones,
But there is a superset of these principles that can make AI substrate agnostic. I think that the implication of the Church-Schuring thesis is that it doesn’t really matter which hardware you’re using. In practice, it does matter because if a hardware
Is not very deterministic or it doesn’t give you a lot of memory or it’s very slow, you will notice big differences. But if you abstract this away, the representational power and the potential for agency is not really dependent on your hardware.
It turns out that the hardware that you’re currently using for AI is much, much more powerful than the hardware that biology is using. The reason why AI is so weak compared to human minds or biological systems is because the algorithms that we have discovered, we have discovered them by hand.
These were people tinkering. Sorry, what do you mean that AI is weak? I mean that in order to get a system that is almost coherent, we need to train it with the entire internet, with almost everything that humans have ever written.
And as a result, we get a system that is using tremendously more resources than the human brain has at their disposal. I’m not talking about the computational power that is implemented in an individual cell that might be very large,
But the part of the power of the individual cell that is actually harnessable by the brain for performing computation, that is very little. It’s only a small fraction of what the new one is doing to do its own maintenance, housekeeping, metabolism, communication with neighbors,
That is actually available for building computation at the brain level. As an example, I sometimes use the stable diffusion bytes when they came out. Stability AI is an AI company that makes open source models. And they made a vision model by training these GPUs on hundreds of millions of
Images and text drawn from the internet and cross-correlating them until you can type in a phrase and then get a picture that depicts that phrase. It’s amazing that this works at all. It requires enormous computational power because it’s far less efficient compared to a human brain that is learning how
To draw pictures after seeing things. And these weights, this neural network, they know everything. They basically know how to draw all the celebrities and how to draw all artistic styles and all the plans. And everything is in there. And it’s just two gigabytes. You can download it. It’s only two gigabytes.
And it’s like 80% of what your brain is doing is captured in these two gigabytes. And it’s so much more than what a human brain could reproduce. It is absolutely brute forcing it. At the other time, two gigabytes doesn’t
Seem to be a lot, which suggests that our own brain is probably not storing effectively much more information than a few gigabytes. That’s very humbling. And the reason why we can do so much more with it and so much faster than
AI is not because biological cells are so much more efficient than transistors. It is because they are self-organizing. It has been at this game for quite some time. It figured out a number of tricks that human engineers couldn’t figure out so far. Right. Karl,
Do you want to expand on points of contention and the mortality and perhaps permanence of a cell? There’s so many, again, so many issues. I hope we get to the differences now. We could. But I just wanted to celebrate this
Notion that the cell, in a sense, is immortal. Because, of course, the whole point of this is to try and understand systems that endure over long periods of time. And that’s what I meant. I didn’t mean that death meant cessation. I just meant there’s a certain life cycle, my tenancy in play.
So, I thought that was nicely illustrated by the notion that the cell is, in a sense, unending. But the mortal immortality is more about divorcing the software from the substrate. And there’s a bit of a pushback then if you want to look for differences
In the respective arguments. Then a lot of people would say that all that housekeeping that goes on in terms of intracellular machinations and self-organization, that just is basal computation at a particular level. And that more macroscopic kinds of belief updating and processing and
Computation supervene at a certain scale. And, indeed, that progression in a sort of scale-invariant sense is one manifestation of what you were talking about before. That biological things are cells of cells of cells of cells and have increasingly higher kinds
Of scales and different kinds of computation. But the idea that the first principles apply at every and each level. And it’s the same principle at every and at each level. And if you pursue that, one has to ask why modern AI or particularly machine learning is so inefficient, dangerously inefficient.
And there’s, I think, a first principle account of that. And the account would go along the following lines that the only objective function that you need to explain existence is the likelihood of you being your marginal likelihood. That, statistically, is the model evidence. The model evidence or
The log of that evidence can always be written down as accuracy minus complexity. Therefore, to exist is to minimize complexity. Why is that important? Well, first of all, it means that that coarse graining that we were talking about earlier on is not a constraint.
It is actually part of an existential imperative to coarse grain in the right kind of way. The other reason it’s important is that there is a thermodynamic link between the complexity scored in terms of belief updating or processing or computation and the thermodynamic cost.
And if that’s the case, it explains why the direction of travel in terms of your machine learning is so inefficient. And what it tells you is that there is a lower limit on the right way to do things. There is a lower limit on the thermodynamic
Efficiency and the information computational efficiency specified by the Landauer limit. Why does modern or current machine learning not get anywhere close to that Landauer limit? Possibly two, three, four, if not six orders of magnitude living above it,
Whereas the brain is actually much, much closer to that lower limit in terms of the efficiency, both, I repeat, thermodynamic and information theoretic kinds of efficiency. And the answer is, I think, the Von Neumann bottleneck. It is the memory wall. It is that people are
Trying to do computation in an immortal sense by running software without careful consideration of the substrate on which they’re running or implementing that computation. So I would push back against the notion that it is even going to be possible, irrespective of the right direction of travel in terms of artificial intelligence research or,
Indeed, computer science research. I’d push back against the notion that artificial intelligence read as a running of some immortal software on a Van Neumann architecture is the solution. I think that solution has to be more biomimetic, by which I mean it has to actually run on a substrate.
It doesn’t have to be a biological cell, but certainly has to conform to the same principles of multi-scale self-organization of the most efficient sort. That just is the optimization of the marginal likelihood or the evidence for the states that that particular computing device or computer wants to be in.
So that’s what I had a slight sort of hesitation about agreeing about the promise and potency of artificial intelligence. I don’t think that’s the right way to go about it. I would actually come back to your very initial argument, Joscha, that it has to
Be much more biologically inspired. It has to be much more biomimetic, and part of that sort of inspiration is the motivation for looking at the distinction between running of immortal software on Van Neumann architectures, on NVIDIA chips, relative to a much more biomimetic approach, say, photonics or neuromorphic computing.
I think that really does matter in terms of getting us to a situation, you know, getting described as a solution. I’m not sure there is a solution. There’s a solution to the differential equations that has a well-defined objective function in my
World, but certainly getting useful artificial intelligence in the same spirit that the FDA is fit for purpose and doing a useful job. Okay, let me push back against this. First of all, I do agree that current AI is brutalist in the sense that it is not making
The best use of the available substrates, and it’s not building the best possible substrates. We have a number of path effects. It’s not that the stuff that we are building and using is not clever or so, but it’s a far cry from what biology seems to have discovered.
At the same time, there is relatively little funding going into AI and there’s relatively little energy consumption given what it gives you. If academics hear that it costs $20 billion to train a model, they almost faint because they compare this with their departmental budget.
But if you would compare this with the cost of making a halfway decent AI movie in Hollywood, it’s negligible, right? So basically, what goes into an AI project is far less than what goes into a Hollywood movie about AI.
And if you compare this at the scale, if you look at the societal benefit of watching an AI movie or watching another blockbuster about the Titanic or so, it’s not null. But I think that AI has the potential to be dramatically more valuable than this.
And so I think that AI, even though it might sound counterintuitive, is not using a lot of energy and it’s not very well funded at the moment still, compared to what the value of it is. Also, the leading labs do not believe that the transformer is going to be the
Architecture that we have in the end. It just happens to be one of the very few things that currently works at scale, that we have discovered that can actually be scaled up in this brutalist way. And it’s already better at completing prompts than the average person. And it’s even better than
Writing code than many people. So it can translate between programming languages. It can write an algorithm down in English, or can even help you to write an algorithm down in English and then translate it in the programming language of your choice.
And it’s pretty good at it. It can also, if it makes a mistake, and it often makes mistakes, understand the compiler messages and then try to suggest fixes that often work. In many ways, I’ve found that it’s already better than a lot of people
I’ve worked with in corporate contexts, both at writing press releases and at writing code. It’s not as good as the top level people in their field, but it’s quite surprising. And so there is this interesting, open and tantalizing question, can we scale this up by using slightly better loss function, by using slightly
More compute, slightly better curated data? And the systems can help us curating data and coming up with different architectures and so on. To get this thing to be better at AI research than people. If that gets better
At AI research than people, then we can leave the rest to it and go to the beach. And it will come up with architectures and solutions that are much more efficient than what we have come up with. At the same time, there are
Many labs and teams that work on different hardware, that work on different algorithms. At the same time, the fact that you see so much news about the transformer at this point is not so much because everybody ignores it and doesn’t work on it anymore.
It has religious beliefs and the transformer being the only good thing. It’s because it’s the thing that currently works so well. And people are trying to work on all the other things, but the thing that has the most economic impact and the most utility happens to be the stuff that currently works.
And so this may cloud our perception that we think it’s the von Neumann architecture and so on. But in some sense, the GPU is no longer a von Neumann architecture. It is built, we have many pipelines that work in parallel, that take in
Smaller chunks of memory that are more closely located to the local processor. And while it’s not built in the same way as the brain, where all the memory is directly inside of the cell or its immediate vicinity, it is much closer to it.
And it’s able to emulate this. And if I look at the leading neuromorphic architectures, I can emulate them on a CPU and it’s not slower. This is all just research stuff that is early stage. But we are not emulating neuromorphic architectures on a CPU for the most part,
Which is largely because it doesn’t give us that many benefits over the existing architectures and libraries. Or the existing architectures and libraries work so well that people use this stuff for now and it creates a local bubble until somebody builds a new stack that is overtaking it. And I think this is all
Going to happen at some point. So I’m not that pessimistic about these effects. What I can see is that our computers can read text at a rate that is impossible for human beings when you parse the data into a large language model for training it.
And it’s, in some sense, a radically Filstronian program next token prediction. It’s really trying to predict the future and minimize its surprise. That’s the core of this algorithm. With this paradigm, it gets to be coherent in the limit. It leads to an interesting question.
Maybe this paradigm is not correct. Maybe humans are doing something different. Maybe humans are maximizing coherence or consistency. And we have a slightly different formal definition. And life on Earth or agency in the universe might be minimizing free energy in the limit. But individual organisms are not able to figure that out.
And they do something that is only approximating it, but locally works much better and converges much faster. So maybe there are different loss functions that we have yet to discover that are more biological or more similar to biological systems. Also, one of the issues with biomimetic things is it means mostly mimicking the
Things that scientists in biology and neuroscience have discovered so far. And this stuff all doesn’t work. The reason why Mike Levin doesn’t call himself a neuroscientist, I suspect, but a synthetic biologist is that he doesn’t want to get in conflict with the dogmatic approaches of some neuroscience, which believes that
Computation stops at the neurons. It’s only neurons that are involved in computing things. It could be when you look at brains that they are basically telegraph networks of an organism that the neuron is a telegraph cell. It’s not unique in its ability to perform computation.
It’s only unique in its ability to send the results of computation using some kind of Morse code over long distances in the organism. And when you want to understand how organisms compute and you only look at neurons, it might be looking at the economy
About 1900 and trying to understand it by only modeling the telegraph network. But you are going to learn fascinating things by looking at an economy, looking at its telegraph network and looking at the Morse code, but thinking that communication
Can only happen in this Morse code rather than sending RNA molecules to your neighbors. Why would you want to send spike trains if you can send strings? Why would you want to perform such computations in a slow, awkward way? Why would you want to translate information into the time domain if you
Can send it in parallel all at once? So when we talk about biomimetic, we often talk about emulating things that we only partially understand and that don’t actually work in a simulation. There is no working conic tome right now that
You can turn into a computer simulation and that actually does what it’s doing. And it’s not because computers don’t have the power to run the ideas that neuroscientists have developed, but neuroscientists don’t have developed ideas that actually work. It’s not that neuroscientists are stupid or their ideas are not promising.
They’re just incomplete at this point. We don’t have complete models of brains that would work in AI. And the reason why AI has to reinvent things from scratch is because it takes an engineering perspective. It thinks about what would nature have to do in order to approximate this kind of
Function and what’s the most straightforward way to implement this and test this theory. And this is this experimental engineering perspective that I suspect we might also need in neuroscience. Not in the sense that we translate things into von
Neumann architecture in neuroscience, but in the sense that we think about what would nature have to do in order to implement the necessary mathematics to model reality. All right, neuroscientist, your turn. I hope no neuroscientists are watching this. We’ve managed to offend physicists and neuroscientists.
I largely agree entirely with many of those things. I’m just trying to remember the ones that I can argue with. I love this notion that there’s more money going into Hollywood films about AI than actually AI research. I’ve never heard that before. That’s marvelous. And also the point about sort of GPUs.
I mean, I think that’s just a reflection of the, if you like, natural selection in the AI community of what I was trying to say before about the move away from von Neumann architectures to more mortal computing. I mean, if you talk to people doing in-memory processing or processing in-memory as computer science,
I mean, that’s where they’d like everybody to be. And that’s what I meant, really, by that aspect of mortal computing, that the infrastructure and the buses and the message passing, having everything local is speaking to the hardware implementation. So that, you know, I agree entirely that that is.
And the direction of travel, and I didn’t want to imply that sort of GPUs were the wrong way of doing it. Also, I agree entirely. I mean, I wasn’t really referring to transformer architectures. And as you say, they’re just, you know,
Highly expressive, very beautiful Bayesian filters and are now currently being understood as such. As my friend Chris Buckley would say, people are starting now to Bayes explain how a transformer works. So what would I disagree with? Well, perhaps to I noticed that you on a number of occasions were trying to identify the
Universal objective function, doing things better. It being really good because it could translate lots of languages, having greater utility. What do you do? Do you generally think that there is some magic utility function out there that has yet to be discovered?
And do you think that AI is going to discover that magic utility function and that will that be the answer? Well, I think that ultimately utility relates to what makes the system stable and self-sustaining. So if you look at any kind of agent, it depends on
What conditions can stabilize that agent. And this comes down to very much the way in which you model reality, I think. So it is about minimizing free energy in a way. But if you look at our own lives and you look for a sandwich or for love or
For relationship or for having the right partner to have children with and so on, you’re not thinking very much about minimizing free energy. And we perform very local functions because we are only partial agents in a much larger system
That you could understand as the project of the cell or as the project of building complexity to survive against the increasing entropy in the universe. And so basically, we need to find sources of entropy and exploit them in a way that we can. And this depends on the agent that we currently are.
And this narrows down this broader notion of the search for free energy into more practical and applicable and narrow things that can deviate locally very much from this pure, beautiful idea. With respect to principle that should be discovered or has to be discovered
And might be discovered in the context of AI, I suspect that self-organizing systems need different algorithms than the GPUs that we’re currently using for learning, because we cannot impose this global structure on them. So I suspect that there is a training algorithm that nature has discovered
That is in plain sight and that we typically don’t look at, and that’s consciousness. I suspect the reason why every human being is conscious and no human being is able to learn something without being conscious and is not producing complex behavior without being
Conscious is not so much because consciousness is super unique to humans and evolved at the pinnacle of evolution and got bestowed on us and us alone. We do not become conscious after the PhD. We become conscious before we can drag a finger. So I suspect that consciousness itself
Is an aspect, or depending on how you define the term consciousness, the core of a meta-learning algorithm that allows the self-organization of information processing systems in nature. And it’s a pretty radical notion. It’s a conjecture at this point. I don’t know whether
That’s true. But this idea that you have a function that perceives itself in the act of perceiving is not conceptual. It’s not cognitive. It’s a precognitive level at the perceptual level that you notice that you’re noticing, but you don’t have a concept of notion yet.
And from this simple loop that keeps itself stable and is controlling itself to remain stable and remain observer, where the observer is constituting itself an observer, you build all the other functionality in your mind. You start imposing a general language on your substrate,
A protocol that is distributed with words so humans become trainable and learn to speak the same language, behave in the same way that every part of the mind is able to talk to all the other parts of the mind. And you can impose an organization that removes inconsistencies.
This is probably that thing that is one of the big differences between how biological systems learn and control the world, and how artificial engineered systems do it. Yeah, I agree entirely. Again, you’ve brought so many bright and interesting ideas. It’s
Difficult to know what to comment upon. Just one thing which you said, when I pressed you on what is good, you basically said to survive. So I think that brings us again back to this notion of mortality being at the end of the day, the possibility of eluding mortality, being part of…
Yes, but not to survive as an individual, right? Human beings are built in such a way that we have to be mortal. We are not designs that can adapt to changing circumstances. If the atmosphere changes, if our food supply changes too much, we need to build a different organism. We need to
Have children that mutate and get selected for these new circumstances. But in principle, intelligent design would be possible. It’s just not possible with the present architecture because our minds are not complex enough to understand the information processing of the cell well enough
To redesign the cell in situ. And in principle, that’s not something that would be impossible. It’s just outside of the scope of biological minds so far. So individually, we have to be mortal. But in principle, the cell can be immortal, or there could be systems that go beyond the cell
That encompass it, that are a superset of what the cell is doing and what other information processing agents could be doing in nature, that basically make sustainability happen. And I think sustainability is a better notion in some sense than immortality.
So yeah, again, I agree entirely. I often look at the physics of self-organization as just a description of those things that have been successful in sustaining themselves. And indeed, the free energy principle is just basically, what would that look like and how would you write that
Down? And of course, the free energy theorists would argue that the ultimate, the only objective function is a measure of that sustainability. That is the evidence that you’re in your characteristics ascendable states. So if properly deployed, you should be able to explain all of
Those aspects of behavior that characterize you and me in terms of self-evidencing or free energy minimization, such as choosing the right partner, such as foraging on the internet, such as enjoying a good read. And this is why I want to fully agree with you in terms of that makes that
Kind of self-sustaining, self-organization only understandable in relation to some kind of selfhood. And I’m using selfhood in the way I think you’re using this basic notion of sentience. And what would that mean from the point of view of the free energy principle? It would mean
That you have an existential imperative to be curious. So if you just read the free energy as surprise, because you talked about predictability before, then if I am choosing how to act next, then I am going to choose those actions that minimize
My expected surprise or resolve my uncertainty. I’m going to act as if I’m a curious thing. And I bring that to the table because that is what is not an aspect of any of this artificial intelligence that you described before. The machine that can translate
From one language to another language, the machine that can map from some natural text to a beautiful graphic. These are wonderful and beautiful creations, and they are extremely entertaining, but they are not curious. And as such, they do not comply with the free energy principle, which means that they’re not sustainable,
Which means that one has to ask what’s going to happen to them. Perhaps we might sustain them in the way that we do good art, but from the point of view of that kind, perhaps I shouldn’t use the word biomimetic because perhaps that’s too loaded,
But the way of sustaining oneself through self-evidencing, I do not think does admit an intelligent design of something that is not in and of itself curious as part of its self-organization. So where would you see curiosity? Does the FDA have to be curious?
Is there any aspect of the utility afforded by, say, reinforcement learning models or deep RL or Bayesian RL? Does that have curiosity under the hood as part of the objective function? I really liked how you bring art into this discussion as an example of something that
Might be similar to an AI system that doesn’t know what it’s good for and only exists because we sustain it. Because it’s not self-sustaining, right? A chat GPT is not paying its own energy bills. It doesn’t really care about them. It’s just a system that is completing text at this
Point. And it might be if you task it with this thing and it figures out the mathematics at some point, but right now it doesn’t. And an artist sometimes jokes it’s a system that has fallen
In love with the shape of the loss function rather than with what you can achieve. Art is about capturing conscious states because they are intrinsically important, right? Is this art, or can this be thrown away? It is art. It is important. And in this sense, art is the cuckoo
Child of life. It’s not life itself. The artists are elves. The living organisms are orcs. They only use art for status signaling or for education or for ornamentation. The artist is the one who thinks magic is important, building palaces in our minds, showing them to each other.
That’s what we do, right? And I’m much more an artist at heart than I am a practical human being that maximizes utility and survival. But I think I also can see that this is an incomplete
Perspective. It means that I’m identifying with a part of my mind, with the part of my mind that loves to observe and revel in the aesthetics of what I observe. I also realize that this is useful to society because it’s basically able to hold down a particular corner of the larger hive mind
That is necessary to be done, right? If I was somebody who would only maximize utility, I would be a great CEO maybe, but I would not be somebody who is able to tie down different parts of philosophy and see what I can see by combining them or by looking at them through a
Shared lens. And so it’s sometimes okay that we pursue things without fully understanding what they’re good for if we are part of a larger system that does that. Our own mind is made out of lots of sub-behaviors that individually do not know what we are about. And only together,
They complete each other to the point where we become a system that actually understands the purpose of our own existence in the world to some degree. And of course, that also goes across people. Individually, we are incomplete. And the reason
Why we have relationships to other people is because they complete us. And this incompleteness that we have individually is not just inadequacy, it’s specialization. The more difficulty we have to find a place in the world, the more incomplete we are. But it often also means we have more
Potential to do something in this area of specialization that we are in. And individually, it might be harder to find that right specialization, but to accept that individual minds are incomplete in the way in which they’re
Implemented in biology, I think is an important insight. And this doesn’t have to be the case for an AI agent, of course, or for a god-like agent that holds down every fort, that is able to look at the world from every angle, that holds all perspectives simultaneously.
Karl, did that answer your question about the curiosity of the FDA? Yes. And, you know, brings in the sort of primacy of the observer. So I’m intrigued by this notion of being incomplete. Do you want to unpack that a little bit? Yes. First of all, Curt,
Thanks for pointing out that I didn’t talk about curiosity. Curiosity ties into this problem of exploration versus exploitation. The point of curiosity is to explore the unknown, to resolve uncertainties, to discover possibilities of what could also be and what we could also be doing.
And this is in competition to executing on what we already know. And there is, if you are in an unknown environment, it’s unclear how much curiosity you should have, or if you’re in a partially known environment. And nature seems to be solving this with diversity.
So you have agents that are more curious, and you have agents that are less curious. And depending on the current environment and niche, they are going to be adaptive or non-adaptive and being selected for or against. So I do think, of course, curiosity is super important.
But it’s also what kills the cat, right? The early worm is the one that gets eaten by the bird. And so curiosity is important. It’s a good thing that we are curious. And it’s very important that some of us are curious and retain this curiosity so we can move and change and adapt.
And it’s one of the most important properties in a mind that I value, that it’s curious and always open to interaction and discovering ways to grow and become something else. But it’s risky to be too curious. And instead, not just exploiting what you already know and act on that
And look for the simple solution for your problems. I think it’s a big problem in science that we drive out curiosity of people. The first step in thinking is curiosity, conjecture, trying things that may not work. And then you contract. And the PhD seems to be a great filter
That drives out the curiosity out of people. And then after that, they’re able to only solve problems using given methods. And they can do this to themselves, this violation of a curious mind. Whereas the existential questions somehow stop after graduation. And so it seems to be some selection
Function against thinking that is happening, that is largely driving curiosity out of people because they feel they can no longer afford it between grand proposals. And so in a sense, yes, I would like to express how much I cherish curiosity and its importance
While pointing at the reason why not everybody is curious all the time. Too much of a good thing is also bad. Right. And the incompleteness now. Karl, do you want to go more into this? Let me finish that and then bring it back to incompleteness in a second. Yes.
I was just, again, I love that. Just a moment, Josje. Would it be possible for you to expand on the early worm gets eaten by the bird because the phrase is that the early bird gets the worm. But that doesn’t imply that the early worm gets eaten by the bird
Because they could have different overlapping schedules. And in fact, it could be the late worm that gets eaten. And there is such a thing as a first mover advantage. AltaVista got eaten by Google because instead of giving people the search results it wanted, it gave them ads. And now Google has discovered
That it’s much better to be AltaVista, but AltaVista got eaten by Google. It was too early. Google has now given up on search. It instead believes in just giving you a mixture of ads that rhyme on your search term. So you could say that AltaVista was the early worm.
I’m just trying to do a job on my frustration with Google. But I think that very often we find that the earliest attempts to do something cannot survive because the environment is not there yet. The pioneers are usually sacrificial. There is glory in being a pioneer. There is no glory in copying
What worked for the pioneer. But there is very little upside in greatness. Understood. Karl. Well, again, greatness, which is not good. Greatness is not good. We’re coming back to the tumor again. The art of good management. Just riffing on your focus on art and just thinking, what makes a good CEO?
Is it somebody who makes lots of money and is utilitarian? Or does he have the art of good management and considers the objective function, the sustainability of his institution and her institution and all the people that work for it? I think there are very different perspectives on what this objective function should be.
And I was trying to argue before that it can’t be measured in terms of greatness or money or utility. It can only be measured in terms of sustainability. The other thing I like was curiosity. So here’s my little take on that. Curiosity killed the cat.
I think that is exactly what was being implied by the importance of mortal computation. And that in a sense, we all die as a result of being curious after a sufficient amount of time. And it can be no other way. I mean that in the sense that
In a very technical sense. So if you were talking to a Fishnado’s inactive inference, an application of the free energy principle, what they would say is that in acting in dissolving the exploration exploitation dilemma, you have to put curiosity as an essential part of the right objective function that
Underwrites our decisions, our choices and our actions, simply in the sense that the expected surprise or the expected log evidence or self-information can always be written down as expected information gain and your expected utility or negative cost, which means that just the statistics of
Self-organization bake in curiosity in the sense that you will choose those actions that resolve uncertainty. You choose those actions that have the greatest information gain. So curiosity, I think, is a necessary part of existing. There’s certainly things that exist in a sustainable sense. But my question was,
Well, I wanted to know more about this intriguing notion that we are incomplete unless considered in the context of other things like us that constitute our lived or at least sensed world. But I just wanted to also ask, do you see curiosity as being necessary for that kind of consciousness
That you associated with sentience before? Would it be possible to be conscious without being curious? Acknowledging there are lots of things that are not curious. Viruses, I suspect, are not curious. Trees are probably not that curious. They don’t plan their actions to resolve uncertainty. But there are certain things
That are curious, things like you and me. So I’m just wondering whether there are different kinds of things, some of which are more elaborate in terms of the kind of self-evidencing that they evince in sustaining themselves autopoetically using autocatalytic mechanisms.
And there are other things that are less so. Would that go hand in hand with having the kind of consciousness that you were talking about that entails this self-modeling? I think that a good team should also contain curiosity maximizers,
People that mostly are driven by curiosity. And so you have a voice in your team. And I love being that voice that is driven by finding out what could be. And you also need people who focus on execution and who are not curious at all.
And in this way, I think we can be productively incomplete. If you have somebody who is by nature not very curious, but is able to accept the value of somebody who is and vice versa, we can become specialists at being curious or at execution.
And when we can inform and advise each other, we can be much better than we could be individually if we would try to do all those things simultaneously. And in this sense, I believe that if you are a state building species,
You do benefit from this kind of diversity. If you’re not an individual agent that has to do all the things simultaneously. I don’t know how curious trees are. I’m somewhat agnostic with respect to this. I suspect that they also need to reduce uncertainty. And I don’t know how smart trees can become.
When I look at means and motive of individual cells, they can exchange messages to their neighbors. They can also make this conditional. Evolution is probably getting them to the point where they can learn. So I don’t see a way to stop a large multicellular organism that becomes
Old enough to become somewhat brain-like. But because it has neurons, it cannot send information quickly over long distances. So it will take a very long time compared to a brain or nervous system for a tree to become coherent about any observation. It takes so much time to synchronize this information back
And forth that the tree would observe locally. And as a result, I would expect that the mental activities of the tree, if they exist, which I don’t know, to play out at such slow timescales that it’s very hard for us to observe. And so what does it look like
If a tree was sentient? How would it look different from what we already observe and know? We notice that trees are communicating with other trees, that they sometimes kill plants around them, that they make decisions about that. We know that there are networks between fungi and trees that seem to be sending
Information over longer distances in forests. So trees can prepare an immune response to pests that invade the forest from one end while they’re sitting on another end. And we observe all this, but we don’t really think about the implication. What is the limitation of the sentience of a forest?
I don’t know what that is. And I’m really undecided about it. But I don’t see a way to instantly dismiss the idea that trees could be quite curious and could actually at some level reason about the world. But probably because they’re so slow that the
Individual tree doesn’t get much smarter than a mouse, because the amount of training data that the tree is able to process in its lifetime at a similar resolution is going to be much lower. They do live a long time. So, sorry, I’m just trying to defend. I have many friends who you
Would enjoy talking to about that. And you seem very informed in that sphere. Our ancestors were convinced that trees could think. Fairies are the spirits of trees and they move around in the forest using the Internet of the forest that has emerged over many generations of plants
That have learned to speak a shared protocol. And I think it’s a very intriguing idea. We should at least consider it as a hypothesis. Absolutely. There was a great BBC series where they focus on the secret life of plants just
By speeding up things ten or a hundred times. And they look very sentient when you do that. Our ancestors said that one day in fairyland is seven years in human land. Maybe this alludes to this temporal difference. So, about differences between you all.
Why don’t we linger on consciousness? And Karl, if you don’t mind answering, what is consciousness? Where is consciousness? And why is consciousness? So, in other words, where is it? Is it in the brain? Is it in the entire body? Is it an ill-defined question? What is it?
Why do we have it? What is its function? And then we’ll see where this compares and contrasts with Josje’s thinking. Right. Yeah, I am not a philosopher. And sometimes the story I will tell depends on who I am talking to. But at its simplest, I find it easiest to think of consciousness as
A process as opposed to a thing or a state. And specifically a process of computation, if you like, or belief updating. So, I normally start thinking about questions of the kind you just asked me, but replacing consciousness with evolution. So, where is evolution? What is evolution?
Why is evolution? Then all of those questions, I think, are quite easy to answer. Sometimes it’s a stupid question. Sometimes there’s a very clear answer. So, where is consciousness? Where is evolution? Well, it is in the substrate that is evolving. So, where is consciousness? It would be in the processes
That you are describing consciousness. So, I would say it is actually the computation or the information processing, the belief updating that you get at any level. And just fully acknowledging Joscha’s point that it doesn’t have to be neurons. It could be my sealed networks. It could be intercellular communication.
It could be… It could be electrical filaments, you know. As long as there is a physical instantiation of a process that can be read as a kind of belief updating or processing, if I were allowed to read computation as
That kind of process, then that would be, I think, where consciousness would be found. Would that be sufficient to ascribe consciousness to me or to something else? I suspect not. I think you’d have to go a little bit further and suspect that Joscha
Wants to now articulate how much further, but there will be a focus on self-modeling. So, it’s not just a process of inference. It’s actually inference under a kind of model of the world. I’m quite happy committing to a generative model as formally specified in
Terms of variational inference, but we can relax that and just say some kind of model of the world that entails a certain aspect of selfhood to it. So, that’s what I would say. I put something else
Into the mix as well. That to be conscious, I suspect in the way that you’re talking about, means you have to be an agent. And to be an agent means that you have to be able to act.
And I would say more than just acting, more than acting, say, for example, in the way that plants will act to broadcast information that enables them to mount an immune response to parasites, they have the capacity to plan, which brings us back to the curiosity again.
Because, you know, we normally plan in order to resolve uncertainty. We normally plan our day and the way that we spend our time gathering information, gathering evidence from models of the world in a way that can only be described as looks as if it is curious.
That’s why I was so fixated on art and creativity and curiosity that Joscha was talking about previously. I think that is probably a pre-requisite for being conscious in the sense that Joscha would mean it. But I don’t know, perhaps we should ask him that.
May I ask you a clarifying question, Karl, about belief updating? So if consciousness is associated with belief updating, then let’s say one is a computer, a classical computer. You get updated in discrete steps, whereas the belief updating that I imagine you’re referring to is something
More fuzzy or continuous. So does that mean that the consciousness associated with a computer, if a computer could be conscious, is of a different sort? How does that work? I’m not sure. I don’t think there’s any, in the same spirit that we don’t want to over-commit
To neurons doing mind work. I don’t think we need to commit to a continuous or discrete space-time formulation. Again, that’s an artificial divide between classical physics and quantum information theoretic approaches. So I think the deeper question is what properties
Must the computational process in a PC or a computer possess before you would be licensed to make the influence that it was conscious and possibly even ascribe self-consciousness to that. And the way that I would articulate that would be that you have to be able to describe
Everything that is observable about that computing artifact as if or explain it in terms of it acting upon the world in a way that suggests or can be explained that it has a model of itself engaging with that world. And furthermore, I would say that that model has
To involve the consequences of its action, which is what I meant by being an agent. So it has to have a model that will act as if it has a model, a generative model that could be a
Minimal self kind of model, but crucially entails the consequences of its own actions so that it can plan, so that it can evince curious-like behavior. So that could be done in silica, it could be done with a sort of clock and synchronous message passing of a discrete sort, or it could be done
In analog, it could be done with photonics, it could be done under neuromorphic architecture. I don’t think that really matters. I think it’s more the nature of the implicit model under the hood that is accounting for its internal machinations, but more practically in terms
Of what I can observe of that computer, its behavior and the way that it goes and gathers information or attends to certain things and doesn’t attend to other things. Okay, great. Joscha. If we think about where consciousness is,
We might be biased by our propensity to assign identity to everything. And identity does not apply to law-like things. Gravity is not somewhere. Gravity is a law, for instance, or combustion is not anywhere. It’s a law. It doesn’t mean that it happens everywhere in
The same way. It only happens when the conditions for the manifestation of the law are implemented, when they’re realized in a certain region, then we can observe combustion happening. But combustion simply means that under certain conditions you will get an exothermic reaction.
And gravity means that under certain conditions you will find that objects attract each other. And consciousness means that if you set up a system in a certain way, you will observe the following phenomena. Consciousness in this way is a software state, it’s a representational state, and all software is not a thing.
The word processor that runs on your computer doesn’t have an identity that would make it separate or the same as the word processor that runs on another person’s computer because it’s a law. It says if you put the transistors into this
State, the following thing is going to happen. So a software engineer is discovering a law, a very, very specific law that is tailored to a particular task and so on, but it’s manifested whenever we create the preconditions for that law. And so the software design is about creating the
Preconditions for the manifestation of a law of text processing, for instance, that allows you to implement such a function in the universe or discover how it is implemented. But it’s not because the software engineer builds it into existence and it didn’t exist before that.
That’s not the case. It always would work. If somebody discovers this bit string in a random way and it’s the same bit string implemented on the same architecture, it would still perform the same function. And in a sense, I think that consciousness
Is not separate and different people. It’s itself a mechanism, a principle that increases coherence in the mind. It’s an operator that seems to be creating coherence, at least that’s the way I would look at it or frame it.
And as a result, it produces a sense of now, an island of coherence and the potential models that our mind could have. And I think it’s responsible for this fact that we perceive ourselves being inhabitants of an island of coherence in a chaotic world, this island of nowness. And it’s probably not
The only solution for this thing. I think it’s imaginable that there could be a hyperconsciousness that allows you to see multiple possibilities simultaneously rather than just one, as our consciousness does, or that offers us a now that is not three
Seconds long, but hundreds of years long. In principle, that I think is conceivable. So maybe we will have systems at some point, maybe we already have them, that have different consciousness-like structures that fulfill a similar role of islands of coherence or interpretable regions in the space of representations
That allow you to act on the universe. But the way it seems to be implemented in myself, it’s pretty clearly in the brain, because if I disrupt my brain, my consciousness ceases. Whereas if I disrupt my body, it doesn’t. This doesn’t mean that there are not feedback
Loops that are bidirectional into my body or even outside of my body that are crucial for some functionality that I observe as a content in my consciousness. But if you want to make me unconscious, you need to clobber
My brain in some sense, not nothing else. There’s no other part of the universe that you can inhibit to make me unconscious, and that leads me to think that the way in which this law-like structure is implemented is, right now, for the system that is talking to you,
On my neurons, on my brain, mostly. Okay, any objections there, Karl? No, not at all. I was just trying to remember, if Mark Somerset were here, he’d tell you exactly the size of a really small region in the brainstem. I think it’s less than four cubic millimeters.
If you ablated, you would immediately lose consciousness like that. It’s a very, very specific part of your neural architecture that permits conscious processing. But there are also very specific parts in my computer that are extremely small that I could obliterate and ablate, and that would instantly lead to the cessation of all the
Interesting functions of my computer. And there are many of such regions. There are crucial bottlenecks that enable a large-scale functionality. In some sense, everything that would disrupt the formation of coherent patterns in my brain is sufficient to inhibit my consciousness. And there are probably many such
Bottlenecks that provide vulnerability. So maybe the claustrum is crucial in providing some clock function that is crucial for the formation of feedback loops in the brain that give rise to the kind of patterns that we need. Maybe there are several other such bottlenecks.
This doesn’t mean that the functionality is exclusively implemented in this bottleneck. No, I didn’t mean to imply that the pineal gland is… I didn’t think that you would, but I thought it could lead to a misunderstanding of the audience. And I’ve heard famous neuroscientists point at such phenomena and say, oh,
Maybe this is where consciousness happens. And I think this is almost a superstitious belief. It’s like saying, oh, there’s this particular chip on my computer. If I destroy it, the computer doesn’t work anymore. And maybe this was just quartz or something else rather than the stuff that is providing
The interesting functionality. Or the lead to the battery, perhaps. So which neuroscientist has said this, then? I’m not naming names. Email me afterwards. Just to unpack, the reason that Mark would identify this is it is exactly the cells of origin that are broadcast everywhere that do induce
Exactly this coherence you were talking about. These are the ascending modulator neurotransmitter systems that are responsible for orchestrating that coherence that you were talking about. And I think that’s very nice because it also speaks to the ability of conscious mimicking-like artifacts whose abilities to mimic consciousness-like behavior rests
Upon this modulatory attentional mechanism. And I’m thinking again of attention-headed transformers that play the same kind of role as the selection that these ascending neurotransmitter systems do. So if you find yourself in conversation with Mark Soames, he would argue that the feeling of consciousness arises from equipping certain
Coherent, coordinated interactions that may be regulated by the cerebellum or the colostrum. But it is that regulation that actually equips consciousness with the kind of qualitative feeling that leads to the way that Mark Soames addresses it. I mean, just notice that just reviewing what Josh just said there, he’s talking
About consciousness equipping us with a sense of now and having an explicit aspect that could be… Well, actually, Jerry Adelman’s notion of the remembered present, which could be the cognitive moment, 300 milliseconds, or it could be if I was a tree, three years.
I think it’s a lovely notion, but the point being we’re talking about processes in time. We’re not saying at this instant I am conscious or consciousness is here. We’re talking about a process that by definition has to unfold in time. So I think that’s an important observation, which sometimes eludes…
I think people debating about conscious states and conscious content, not acknowledging that it is a process. It is a process. What was the example that Joscha mentioned? Combustion. Yeah, combustion is a process. You can’t be in a state of combustion, and you could even argue that it’s very difficult to localize
At a certain level. But the key thing is it’s a process. I have an open question, and maybe you have a reflection on this. When we think about our own consciousness, we cannot know in principle, I think, just by introspection, whether we have
Multiple consciousness in our own mind, because we can only remember those conscious states that find their way into an integrated protocol that you can access from where you stand. And we know that there are some people which have a multiple personality disorder in which
The protocol itself gets splintered. As a result, they don’t dream to be just one person. They dream to be alternating, to be different people that usually don’t remember each other because they don’t have that shared protocol anymore. Now, my own emotion and perception is generated outside of my personal self. My personal
Self is downstream from them. I am subjected to my perception and emotion. I have involuntary reactions to them. But to produce my percepts and my emotion, my mind needs intelligence. It cannot be much more stupid than me. If my emotions would guide
Me in a way that is consistently more stupid than my reason and my reflection would be, I don’t think I would work. So there is an interesting question. Is there a secondary consciousness? Is the part of your mind that generates world model and your self-assessment,
Your alignment to the world, it’s self-conscious? So basically, do you share your brain with a second consciousness that has its separate protocol? Or is this a non-conscious process that is basically just dumb and doesn’t know what it’s doing? In a sense that it would be
Sentient in a way that’s similar to my own sentience. What do you think? Curt, you should have a go on that one and then I can think about it. Well, something I had wondered about 10 years ago or so, and I don’t recall the exact argument,
Was that if it was the case that the graph in our brain, let’s say that let’s just reduce the neurons down to a graph, that this graph somehow produces consciousness or is the same as consciousness, then if you were to remove one of those nodes, then you would still have a somewhat
The same identity. Okay, so then does that mean that we have pretty much an infinite amount of overlapping consciousnesses within our brain? I don’t recall the exact argument, but it was similar to this. And then there’s something related in philosophy called the binding problem. I’m uncertain what people who study multiple personality disorders
Have to say about the binding problem. Is that the binding problem gone awry? Can I just then pursue that notion of the binding in the context of the kind of thing or the way I am at the moment?
I think that’s a very compelling notion from the point of view of generative modeling. So, I’m not answering now as a philosopher, but as somebody who may be tasked, for example, with building an artifact that would have a minimal kind of selfhood. The first thing you
Have to write down is different states of mind so that I can be frightened, I can be embarrassed, I can be angry, I can be a father, I could be a football player. So, all the different ways that
I could be that are now conditioned upon the context in which I find myself. And if that’s part of the generative model, that then speaks to two things. First of all, you have to recognize what state of mind you are in, given all the evidence at hand. So, for example,
If I want to jointly explain the racing heart that my interceptive cues are providing me in the interceptive domain with a stiffness of my muscles that my proprioception is equipping me with, then to reconcile that with my visual exoceptive input that I’m in a dark alley,
And mnemonically, I’ve never been here before. All of this sensory evidence might be quite easily explained by the simple hypothesis, I am frightened. And that, in turn, generates covert or mental actions, and possibly even overt autonomic actions and motor actions
That provide more evidence for the fact that I am frightened in the sense, in a William James sense, that out of cardiac acceleration, I will have a motor response, a muscular response appropriate for a fright or flight response. So, just to actually be able to generate
And recognize emotional kinds of behavior, I would need to have a minimal kind of model that crucially obliged me now to disambiguate between a series of different ways of being. So, it’s not so much, oh, I am me. That’s a great hypothesis. That explains everything.
But to make it operationally important, I have to actually infer, I’m me in this kind of state of mind, this situation, or I’m me in this kind of situation, and select the right state of mind to be in. And I think that that really does speak to this notion of multiple consciousnesses that
Cohabit your brain or your generative model. And again, speaks this notion of, well, I’m just wondering whether that can be linked to this notion of incompleteness in a broader sense. I’m constantly seeking for the way in which I complete you in terms of dyadic interactions,
Which means I have to recognize what kind of person do you expect me to be in this setting? And of course, I can only do that if I actually have an internal model that is about me. It’s a model that actually have this attribute of selfhood, but specifically selfhood appropriate
To this context or this person or this situation. Does that make sense? Yeah, I have a question about that. You said that you have different identities that you then select from to see which one’s most appropriate for the circumstance, like a hypothesis. And is it the
Case then that you would say that there are multiple consciousnesses inside your brain, or is it more like you have multiple potential consciousnesses, and then as soon as you select one, that makes it actual? I don’t know that. I would imagine that you’d have to have another deeper layer of your generative
Model that then recognizes the selection process. And indeed, this may sound fanciful, but there are naturalized in terms of inference schemes, models of consciousness that actually do invoke. Models of consciousness that actually do invoke, and I’m thinking here of the work of people like
Lars Sandstein Smith, who explicitly have three levels. And each level, a deep generative model, very much like a sort of deep neural network. And the role of each level is to provide the right attention heads or biasing or precision or contextualization for the processing that
Goes on below. So it may well be that to get the kind of self-awareness, if I now read awareness as deploying mental action in the service of setting the precision or the gating of various communications or processing lower down in the model, it may well be that you do need
An other layer of sophistication or depth to your generative models that I suspect trees don’t have, but certainly you have, or I can infer that you have given I’m assuming that I have a similar conception of consciousness. But I’m not sure that really speaks to your question or the one
That Joscha was posing that, you know, the unitary aspect of consciousness. And, you know, does that transcend an inference that would simply be biophysically instantiated in exactly the same way that I can register visual motion and motion-sensitive area V5 in my posterior cortex?
I don’t know about that. I’ll pass back to Joscha on that one. Again, we need a very narrow definition, a very tight definition of consciousness to answer this question in a meaningful way. If we see consciousness as something that we vaguely
Guesstimate, and there could be multiple things in our understanding, and it becomes almost impossible to say something meaningful about this. So for instance, it is conceivable that consciousness would be implemented by a small set of circuits in the brain, and that all the
Different contents that can experience themselves as consciousness are repurposing this shared functionality in the same way as we probably have only one language center. And this one language center can be used to articulate ideas from many parts of our mind using different sub-agents that basically interface with this.
You can also clearly have multiple selves interacting on your mind. Your personal self is one possible self that you can have that represents you as a person. But there are some people which have God talking to them on their own mind. And I think what
Happens there is people implement a self that is existing and self-identifying as existing across minds. Something that is not a model of the interests of the individual person, but a model of a collective agent that is implemented using the actions of the individual
People. But of course, this collective mind that assumes the voice of God and talks to you in your own mind so you can perceive it, is still implemented on your own mind and uses your circuitry. It’s just that your circuitry is not yours. Your brain doesn’t belong to yourself.
Your self is a creation of your own mind that symbolizes this person. People who say that God doesn’t exist forget often that they themselves don’t really exist in physics. This thing that they experienced as perceiving, as interacting with the world, is a dream. It’s a dream of what
It would be like if you were a person that existed. It’s virtual. So you can also dream being a God, and this God might be so tightly implemented on your mind that it’s able to use your language center and you hear its voice talking to you. But it’s not more or less real
Than you hearing your own voice talking to you in your mind. It’s just an implementation of a representation of agency in your mind. One crucial difference to the way in which most AI systems are implemented right now and the way in which agency is implemented on our minds is that we usually
Write functions in AI that perform something like 100 steps in a neural network, for instance, and then gives a result that makes a programmer happy. And this is it. And the time series predictions of our own mind are dynamic. They’re not meant to solve a particular function,
But they’re meant to track reality. So in a sense, our brain is more like a very complex resonator that tries to go into resonance with the world. So it creates a harmonic pattern that continuously tracks your sensory data with the minimal amount of effort. And this perspective is very different.
It really means your perception of the world cannot afford to deviate too much in its dynamics from the dynamics that you observe in your sensory apparatus, because otherwise future predictions become harder. You get out of sync. You always try to
Stay in sync with the world. And this thing that you stay in sync is really crucial for the way in which we experience ourselves with the world. As part of staying in sync, we discover our own self is the missing link between volition and the outcomes of our action.
Our body would not be discoverable to us and is not immediately given to us if we wouldn’t have this loop that ties us into the outer universe and into the stuff that we cannot control directly.
And for me, this question relates to, do we have only one consciousness? It occurs to me that we would not know if we have multiple ones, if they don’t share memories. If I were to set up an AI
Architecture, where a part of the AI architecture is a model of an agent in the world, another part of the AI architecture is a model of the infrastructure that I need to maintain
To make a model of the world and such an agent in the world. I would not tell the agent how this infrastructure works, because the agent might use that knowledge to game the architecture and get a better outcome for itself, not the organism. Imagine you could game your perception
So you’re always happy, no matter how much you’re failing in the world. From the perspective of the larger architecture, that’s not desirable. So it would probably remain hidden from you how you are implemented. And to me, the question is
Interesting. How sentient is this part of you that is not yourself? Does it actually know what it is in real time? I think it’s a very interesting and tempting philosophical question and also a practical one. Maybe there is a neuroscientific
Experiment that would figure out if you have two clusters of conscious experience. I wouldn’t know how to measure this, but maybe IIT and global workspace theory and so on are wrong in more interesting ways than we currently think they are, because they assume that there is just one
Consciousness. Of course, from the perspective of one consciousness, there is only one, because consciousness is in some sense by definition what’s unified. But if there are multiple clusters of unification that exist simultaneously, they would know each other directly. They could maybe observe each other, but maybe not in both directions.
Sorry, when you say consciousness is by definition one, is that akin to how you say software is one software as such, but specific instantiations of software are different? No, in a functional way. So basically, it’s more like the universe is by definition only one.
You can have multiple universes, but this means that we define universe in a particular way. Normally, universe is used in the way of everything that feeds back information into a unified way, into a unified thing. We accept that parts
Of the universe get lost if they go outside of the distance where they can feed information back into you, but there’s still an event in which we think about the universe as part of the universe. The universe is everything that exists. And consciousness is everything
That you can be conscious of in this sense. So if there is stuff in you that you’re not conscious of, it doesn’t mean that it’s not conscious. It would just be a separate consciousness, possibly. It could also be that it’s not a consciousness.
And so what I don’t know is, is the brain structured in such a way that it can maintain only one consciousness at a time, or could there be multiple full-on consciousnesses where we don’t know about the other one? I perceive my consciousness as being focused on this content
That is my personal self. I can have conscious states in which I’m not a personal self. For instance, I can dream at night that there’s stuff happening, and I’m conscious of that stuff happening, but there is no I. There is no personal self. There’s just this reflexive attention that
Is interacting with the perceptual world. In that state, I would say, I can clearly show that consciousness can exist without a personal self, and the personal self is just a content. But it doesn’t answer the question, are there multiple consciousnesses
Interacting on my brain? One that is maintaining my reward system, my motivational system, and my perception, and one that is maintaining my personal self. Karl, now that we’ve spoken about the unity of consciousness, dissociation, as well as even
Voices of God and God Him or Herself or Itself, what does your background in schizophrenia, your perspective from there, have to say? Yeah, well, that’s a brilliant question and a leading question. It’s what I wanted to comment on. So, again, so many things have been unearthed here from the basic that all our
Beliefs, our fantasies, the hypotheses, illusions that are entrained by the sensorium in a way that maintains some kind of synchrony between the inside and the outside. I think that’s quite a fundamental thing which we haven’t spoken about very much, but I just want to fully endorse.
And, of course, that entrainment, sometimes referred to, I think, as entrained hallucination, perception being a hallucination that’s just been entrained by sparse data. But the data themselves being actively sampled, so this loop that Joscha was referring to, I think is an absolutely crucial aspect of
The whole sense-making and, indeed, sense-making as a self, as the cause of my own or the author of my own sensations in an active sensing or active inference. I think that’s absolutely crucial. The question about the multiple consciousness, I should just, before addressing the psychiatric perspective, there is, I have a group of
Colleagues including people like Maxwell Ramstad and Chris Fields, and particularly Chris Fields who takes a quantum information theoretic view of this and brings to the table the notion of an irreducible Markov blanket in a computing graph that crucially
Has some unique properties that means that it can only know of itself by acting on the outside or other parts of the brain. And, again, acting in this instance just means setting the attention or the coordination or contextualizing message passing elsewhere. But the interesting notion,
Which is not unrelated to the pineal gland or Mark Soames’ ascending neurotransmitter systems that might do this kind of action, is that there could be more than one minimal or irreducible Markov blanket that practically you can actually experimentally define in principle by looking at
The connectivity of any kind. But certainly if you have a sufficiently detailed connectome, you can actually define the Markov blankets in terms of the directed connections offered by external processes. And, in principle, you should apply sort of a kind of integrated information theory, but slightly nuanced, I think, in this instance,
To actually identify candidates for irreducible Markov blankets that could be the thing that looks at the thing that’s doing the thing that may have different kinds of experiences. There could be an irreducible Markov blanket in, say, the globus pallidus that might be making sense of
And acting upon the machinery that underwrites our motor behavior and our plans and our choices, as opposed to something in the occipital lobe that might be more to do with perception. So I’m just saying that I don’t think it’s a silly question to ask. Can we empirically identify
Candidates in computing architectures that would have the right kind of attributes that would be necessary to ascribe them some minimal kind of consciousness? But let me return to this key question about schizophrenia, because as Joscha was talking, it did strike me, yes, that’s exactly what goes on in schizophrenia. Attribution of agency,
Delusions of control, hearing voice, again coming back to this notion that this action-perception loop, this circular coupling to the world that rests upon action that has an agent, and that consciousness understood as self-modeling is all about ascribing the right agency to the outcomes of action, I think is a really important notion here.
It can go horribly wrong. We spend the first years of our lives just working out I cause that and you cause that, and working out what I can cause and what I can’t cause, and what mum causes and what other people cause. Imagine that you lost that capacity. Imagine that when you spoke,
And this is Chris Frith’s notion or expression for auditory hallucinations, for example, you weren’t able to recognize that it was you that was the initiation of that speech act, whether it’s actually articulated or sub-vocal. So just not being able to infer selfhood in the
Sense of ascribing agency to the sensed consequences of action would be quite devastating. And of course, you can think about reproducing these kinds of states with certain psychomimetic or psychedelic drugs. They really dissolve what we take for granted in terms of a coherent,
Unitary content of consciousness. If you’ve ever had the synesthesia that can sometimes be induced by psychedelic drugs, you will know what it’s like to treasure the fact that colour is seen and sound is heard. It doesn’t have to be like that. It’s just that if we, as sustained inferring
Processes, self-evidencing computing processes that sustain in a coherent way our sense-making, it looks as if colours are seen and sounds are heard. That’s how we make sense of it. It doesn’t have to be like that. And you can experience the converse. You can start to see sounds,
You can hear colours, you can have horrible distortions of time perception. A moment can actually feel as if you’ve nested. So all of these things that we take for granted in terms of our sense-making are so fragile that you’re given the right either psychopathology or
Pathophysiology, technically a synaptopathy of the kind you might associate with things like Parkinson’s disease and schizophrenia, possibly even neurotic disorders, affective or depressive or generalised anxiety disorders, can all be understood as basically a disintegration of this coherent synthesis and, to use your word, the binding.
And which means that I think the same principles could also be ascribed to consciousness itself. So depersonalisation, derealisation, I think, are two conditions which I’ve never experienced, but my understanding of subjective reports from people or how patients who have experienced these
Do, I think, really speak to this notion that there could be multiple consciousnesses. And, of course, one will not be aware of the other and possibly not even able to infer the agency, even if it was. But also, there could be no consciousness. There are depersonalisation
Syndromes where you still sense, you still perceive, but it’s not you. And there are derealisation syndromes where you are there, but all your sensorium is unreal. It’s not actually there anymore. You’re not actually in the world. So you can get these horrible disintegration,
Dissociative, well, dissociative is a clinical term. You can get these situations where everything we take for granted about the unitary aspect of our experienced world and us as experiencers can so easily be dissolved in these conditions. So, I take Josche’s questions very, very seriously. And so would people who suffer from these conditions.
I would distinguish between consciousness and self more closely than you seem to be doing just now. I would say that consciousness coincides with the ability to dream, or it is the ability to dream even. And in schizophrenia, the dream is spinning off from the tightly coupled model that allows you
To track reality. But when we dream at night, we are dissociated from our sensorium and the brain is probably also dissociated in many other ways. And as a result, we get split off from the ability, for instance, to remember who we are,
Which city we live in, what our name is very often in a dream. Even if it’s a lucid dream where we get some agency over the contents of our dream, we might not be able to reconstruct our normal personality and crucial aspects of our own self.
And in schizophrenia, I think this happens while we are awake, which means we start to produce mental representations that look real to us, but that have no longer the property that they are predicting what’s going to happen next and about, or much later. And this ability to lose predictive
Power doesn’t mean that they are now… more of an illusion than before. The normal stuff that has predictive power is still a hallucination, it’s still a trance state when you perceive something as real, as long as you perceive it as
Real. It’s only some trance states are useful in the sense that they have predictive power, that they’re useful representations, and others are not. And the ability to wake up from this notion that your representations are real is what Michael Taft calls enlightenment.
A meditation teacher was a pretty rational approach to enlightenment, and basically to him, enlightenment is the state where you recognize all your mental representations as representations and become aware of their representational nature. You basically realize that nothing that you can perceive is real, because everything that you can perceive is a representational
Content. And it’s something that is accessible to you via introspection if you build the necessary models for doing that. So when your mind is getting to this model level where you can construct a representation of how you’re representing things, then you get some agency of how you
Are interacting with your representation. But I wouldn’t say that somebody who is experiencing a schizophrenic episode, or derealizes or depersonalizes, is losing consciousness. They’re losing the self, they’re losing coherence, they’re losing the ability to track reality and
The interaction between self and external world and so on. But as long as they experience that happening, they’re still conscious. Does this make sense? Yeah, certainly in terms of altered states of consciousness, absolutely. Do you know Thomas Metzinger? Some of the things you’ve just said there were very reminiscent of his treatment of,
Say, phenomenal opacity and the like. Is he somebody that you have discussed these things with or subscribed to? We discussed relatively briefly only. We met a few times since I left Germany only, mostly online. And I like Thomas a lot. I think that he is one of the few German
Philosophers worth reading right now. But of course, he’s limited by being a philosopher, which means he stops before the point where he would make actually functional models that we could test. So I think his concepts are sound. He does observe a lot of interesting things,
And I guess a lot of it also through introspection. But I think in order to understand consciousness, we actually need to build testable theories. And I suspect, even if we cannot construct consciousness as this strange loop, as Hofstadter calls it,
From scratch, which I don’t know whether we can do that. I’m agnostic with respect to that. We can probably recreate the conditions that lead to the discovery of consciousness in the brain, which means we can initiate the search process that the
Brain is initiating before it discovers it. I was going to make the joke that we’ve offended physicists, neuroscientists, and philosophers. Yeah, it’s my thing. It’s mostly retaliation because I’m so offended by them. Maybe I
Shouldn’t. Especially, I tried to study all these things and I got so little out of it. I found that most of it is just pretense. There’s so little honest thinking going on about the condition that
We are in. It was very, very frustrating to me. What field do you identify as being a part of Joscha? Computer scientist? Cognitive scientist? I like computer science most because I’ve discovered as a student that you can publish in computer science at every stage of your career.
You can be a first semester student and you can publish in computer science because the criteria of validity are not human criteria. The stuff either works or it doesn’t. Your proof either pans out or it doesn’t. Whereas the criteria in philosophy are to a much larger degree social
Criteria. The more your peers influence the outcome of the review and the more your peers can deviate from the actual mission of your subject in the social dynamics, the more haphazard your field becomes. We noticed, for instance, in psychology, we had this big replication crisis.
And the replication crisis in psychology was something that was anticipated by a number of psychologists for many, many years that pointed out this curious fact that psychology seems to be the only science where you make a prediction at the beginning of your paper and it always comes
True in the end. Enormous predictive power. And also pointed at all the ways in which p-hacking was accepted and legal and how poorly the statistical tools were understood. And then we have this replication crisis and 15,000 studies get invalidated, more or less, or no longer
Reliable. And somebody pointed this out in a beautiful text where they said, essentially, what’s happening here is that we have an airplane crash and you hear that 15,000 of your loved ones have died. And nobody even goes to the trouble to ID them because nobody cares because nothing
Is changing as a result of these invalidated studies. What kind of the building has just toppled? Nobody cares. There’s not actually a building. There’s just people talking. And when this happens, we have to be brutally honest, I think, as a field. Also, I hear very
Often that AI has been inspired by neuroscience and learned so much from it. But when I look at the actual algorithms, the last big influence was heavy on learning. And the other stuff is just people talking, taking inspiration, taking insights, and so on. But it’s not actually
There is a lot of stuff that you can take out of the formalisms of people who studied the brain and directly translate it. I think that even what Karl is doing is much more results of information theory and physics that is congruent with information theory because it’s thinking about
Similar phenomena using similar mathematical tools, and then expresses it with more Greek letters than the computer scientists used to do. But there is a big overlap in this. And so I think the separation between intellectual traditions and fields and disciplines is something
That we should probably overcome. We should also probably, in an age of AI, rethink the way in which we publish and think. Is the paper actually the contribution that we want to make in the future, in the time where you can ask your LLM to generate the paper? Maybe it’s the building block,
The knowledge item, the argument that is going to be the major contribution that the scientist or the team has to make, the experiment. And then you have systems that automatically synthesize this into answers to the questions that you have when you want to do something in a particular
Kind of context. But this will completely change the way in which we evaluate value in the scientific institutions at the moment. And nobody knows what this is going to look like. Imagine we use an LLM
To read a scientific paper and we parse out all the sources of the scientific paper from the paper and what the sources are meant to argue for. And then we automatically read all the sources and check whether they actually say that, what the paper is claiming the sources say. And we
Parse through the entire trees of a discipline in this way until we get to first principles. What are we going to find? Which parts of science will hold up? I think that we might be at the doorstep of a choice between a scientific revolution in which science becomes radically
Honest and changes the way it works, or in which it reveals itself as an employment program, as fake jobs for people who couldn’t find a job in the real economy and basically get away because
Their peers let them get away with it. And I try to be as pointedly as possible and as bleak as possible. So science, given its incentives that it’s working under and the institutional rod that has set in after decades of postmodernism, it’s surprisingly good still. There’s so many good
Scientists in all fields that I know. But I also noticed that many of the disciplines don’t seem to be making a lot of progress for the questions that we have. And many fields seem to be stuck and this doesn’t seem to be just because all the low-hanging fruits are wrapped.
But I think it’s also because the way in which scientific institutions works have changed. The notion of peer review probably didn’t exist very much before the 1970s. This idea that you get truth by looking at a peer-reviewed study rather than asking a person who is able to read
And write such studies. That is new. That is something that didn’t exist for Einstein. So I don’t know if this means that Einstein was an unscientific mind that was only successful because he was working at the beginning of a discipline, or it was because he was thinking
In a completely different paradigm. But no matter what, I think that AI is going to have the potential to change the paradigm massively. And I don’t know which way, but I can’t wait. So now that we’re talking about computer scientists,
Computer scientists, what do you make of the debacle at OpenAI, both Karl and Joscha, directed to you, Joscha first. There’s relatively little I can say because I don’t actually know what the reason was for the decision of the board to fire the CEO.
Firing the CEO is one of the very few moves beyond providing advice than the board can make. I thought if the board makes such a decision in a company in which many of the core employees have been hired by the CEO and have been working very closely and happily with the CEO,
They will need to have a very solid case. And there needs to be a lot of deliberation among core engineers and players in the company before such a decision is being made. Apparently this has not been the case. I have difficulty to understand
Why people behaved in the way in which they did. The outcome is that OpenAI is more unified than ever. It’s basically a 95% agreement about employees that they are going to leave the company if it doesn’t reinstate the CEO. It’s almost unheard of. This is like an
Eastern European communist dictatorship with fake elections, but it was not fake. It was basically people getting together overnight and getting signatures for a decision that gravely impacts their professional careers. Many of them are on visa that depend on continuous
Employment within the company, so they enter actual risks for a time. And I also suspect that a lot of the discussions that happened were bluffs. When the board said yes, they want to reinstate them, but then Waffle then came out with a messier who is a pretty good person,
But it’s not clear why the Twitch CEO would be the right person to lead OpenAI suddenly. So I don’t even know whether the decision was made because there were personal disagreements about communication styles, or whether it was about the direction of the company where
Members of the board felt that AI is going to be developed too quickly and should be slowed down significantly. And the strategy of Sam Altman to run JetGPT at a loss and making up for this by speeding up the development and getting more capital in and thereby basically creating an
AGI or bus strategy for the company might not be the right strategy. Also, the board members don’t hold equity in the company. So this is a situation where the outcome of their decision is somewhat divorced from their own material incentives, and it is more aligned with their political or
Ideals that they might have or the goals that they have. And again, not all of them are hardcore AI researchers. Some of them are. I don’t really know what the particular discussions have been in there. And of course, I have more intimate speculations
At some discussions with people at OpenAI, but I cannot disclose the speculations, of course. And so at the moment, I can only summarize in some sense what’s publicly known and what you can read on Twitter. It’s super exciting. It has kept us all awake for a few days.
It’s a fascinating drama. And I’m somewhat frustrated by people saying, oh my God, this is destroyed trust in OpenAI if the decisions can be so erratic, because OpenAI should be like a bureaucracy that is not moving in a hundred years. No, this is part of something
That is super dynamic and is changing all the time. I think that what the board should probably have seen is that the best possible outcome that they could have achieved is that OpenAI is going
To split. That the best possible, in the sense of the board trying to fire Sam Altman to change the course of the company, they would have created one of the largest competitors to OpenAI. And so basically an anti-anthropic on the other side of OpenAI that
Is focusing more on accelerating AI research. It would have been clear that many of the core team members would join it and it would destroy a lot of the equity that OpenAI currently possesses. And it would take away large portions of OpenAI’s largest customers,
Microsoft. So these are some observations. So Sam is back now. Yes. And it was clear that it would happen, right? This move by Satya Nadella to say he works now for Microsoft happened not after negotiating a new organization for a month.
It happened in an afternoon, right? After it was announced that the board now has another candidate that they secretly got talked into taking on this role. Microsoft basically set up as a threat, okay, they’re all going to come to us and every OpenAI
Person who wants can now join Microsoft in a dedicated autonomous unit with details that are yet to be announced, but they’re not going to be materially worse off or research-wise worse off. So this is a backstop that Microsoft had to implement to prevent its stock
From tumbling on Monday morning. So Microsoft moved very fast on Sunday and decided we are going to make sure that we are not going to create a situation that is worse for us than it was before. And this creates enormous pressure on
OpenAI to basically decide either we are going to be alone without most of the core employees and without our business model, but having succeeded in what the board wants, or we accept the fact that the board has been defeated. And Sam Adler has not been entirely candid with
The board when he said last June that the board can just fire him if it disagrees with him. Because that’s obviously not the case, because the board at the moment where there’s so much buy-in from the employees and the core investors and customers of OpenAI, they cannot just fire
The CEO without very good reason. And Karl, what do you make of it, the whole fiasco? I was just listening with fascination. I think you have more than enough material to keep your viewers engaged. Can I just ask, is OpenAI going to be ingested by Microsoft or not then? Do
You think OpenAI is going to survive by itself? Some people are joking that OpenAI’s goal is to make Google obsolete, to replace search by intelligence, and Google is too slow to deliver product to deal with this impending competition. OpenAI has been rapidly growing in the last few
Months. It has hired a lot of people who are focusing on product and customer relationships. The core research team has been growing much more conservatively. And I think that Microsoft was a natural partner for OpenAI in this regard, because
Microsoft is able to make large investments and yet is possibly not as agile as Google. The risk that if OpenAI would partner with Google as a main customer, that Google at some point would just walk away with the core technology and some of the core researchers might be larger than
With Microsoft, but they can only speculate there. So the last question for this podcast is, how is it that you all prevent an existential crisis from occurring with all this talk of the self as an illusion or our beliefs which are so associated with our conception of ourselves?
Mutable identities and competing, contradictory theories of terrifying reality being entertained. Well, Karl. I’m just trying to get underneath the question. The kind of illusions I think we’re talking about are the stuff of the lived world and the experienced world. And they are not weak or facile or facsimiles of reality.
These are the fantastic objects, belief structures that constitute reality. So literally, as I’m sure we’ve said before, the brain as a purveyor of these fantasies, these illusions is fantastic, literally, because it has the capacity to entertain these fantasies.
So I don’t think there should be any worry about somehow not being accountable to reality. These are fantastic objects that we have created, co-created, you could argue given some of our conversations, that constitute our reality. I think that existential crisis is a good thing.
It basically means that you are getting at a point where you have a transition in front of you, where you basically realize that the current model is not working anymore and you need a new one. And the existential crisis doesn’t necessarily result in death.
It typically results in transformation into something that is more sustainable because it understands itself and its relationship to reality better. The fact that we have existential questions and that we want to have answers for them is a good thing. When I was young, I thought I don’t
Want to understand how music actually works because it would remove the magic. But the more I understood how music works, the more appreciative I became of deeper levels of magic. And I think the same is true for our own minds.
It’s not like when we understand how it works that it loses its magic. It just removes the stupidity of superstition and gives us something that shows its beauty and brilliance and allows us to make it much more sophisticated and intricate.
Thank you, Josje. Thank you, Karl. There’s a litany of points for myself, for the audience, for all of us to chew on over the course of the next few days, maybe even weeks. Thank you. Thank you, Curt, for bringing us together. Karl, I really enjoyed
This conversation with you. It was brilliant. I like that you’d think on your feet that we have this very deep interaction. I found it interesting that we agree on almost everything. We might sometimes use different terminology, but we seem to be looking at the same thing from pretty much the same perspective.
And I also really enjoyed it. It was a very, very engaging conversation. And I love the way that you’re not frightened to upset people and tell things as they are. I’m not looking for a job in academia. Good. Neither am I. I still
Don’t have your balls. Well done. Have a wonderful rest of the day. Thank you. All right. Take care. Brilliant. Thanks very much. By the way, if you would like me to expand on this thesis of multiple overlapping
Consciousnesses that I had from a few years ago, let me know and I can look through my old notes. All right. That’s a heavy note to end on. You should know, Josje has been on this podcast several times. One solo, another with Ben Gorzo,
Another with John Vervaeke, another with Michael Levin, and one more with Donald Hoffman. Whereas Karl Friston has also been on several times, twice solo, another between Karl Friston and Michael Levin, and another with Karl and Anna Lemke. That one’s coming up shortly. The links to every podcast mentioned will
Be in the description, as well as the links to any of the articles or books mentioned. As usual, in every single theories of everything podcast are in the description. We take meticulous timestamps and we take meticulous notes. If you’d like to donate, because this channel has had a difficult time monetizing with sponsors,
And sponsors are the main bread and butter of YouTube channels, then there are three options. There’s Patreon, which is a monthly subscription. It’s patreon.com slash CURTJAIMUNGAL. Again, links are in the description. There’s also PayPal for one-time sums, if you like.
It’s also a place where you can donate monthly. And the amount that goes to the creator, aka me in this case, is greater on PayPal than on Patreon, because PayPal takes less of a cut. There’s also cryptocurrency, if you’re more familiar with that, and the links to all of
These are in the description. I’ll say them aloud in case you’re away from the screen. It’s tinyurl.com slash lowercase, all of this is lowercase, P-A-Y-P-A-L, so PayPal, but then uppercase TOE, T-O-E, uppercased. And then for crypto, it’s tinyurl.com slash lowercase, C-R-Y-P-T-O, capital T-O-E.
I just recommend you look to the description and click there in case you enter in something wrong and there’s someone that’s trying to phish a different account. Thank you, thank you for your support. It helps TOE continue to run. It helps pay
For the editor who’s doing this right now. I and my wife are extremely grateful for your support. We wouldn’t be able to do this without you. Thank you. The podcast is now concluded. Thank you for watching. If you haven’t subscribed
Or clicked that like button, now would be a great time to do so, as each subscribe and like helps YouTube push this content to more people. You should also know that there’s a remarkably active Discord and subreddit for Theories of Everything, where people explicate TOEs, disagree respectfully about theories, and build,
As a community, our own TOEs. Links to both are in the description. Also, I recently found out that external links count plenty toward the algorithm, which means that when you share on Twitter, on Facebook, on Reddit, etc., it shows YouTube that people are talking about this outside of YouTube, which in turn greatly
Aids the distribution on YouTube as well. Last but not least, you should know that this podcast is on iTunes, it’s on Spotify, it’s on every one of the audio platforms. Just type in Theories of Everything and you’ll find it.
Often I gain from re-watching lectures and podcasts, and I read that in the comments. Hey, TOE listeners also gain from replaying. So how about instead re-listening on those platforms? iTunes, Spotify, Google Podcasts, whichever podcast catcher you use. If you’d like to support more
Conversations like this, then do consider visiting patreon.com and donating with whatever you like. Again, it’s support from the sponsors and you that allow me to work on TOE full-time. You get early access to ad-free audio episodes there as well. For instance, this episode was released a few days earlier.
Every dollar helps far more than you think. Either way, your viewership is generosity enough.
40 Comments
TIMESTAMPS AND SPONSORS:
– HelloFresh: Go to https://HelloFresh.com/theoriesofeverythingfree and use code theoriesofeverythingfree for FREE breakfast for life!
– 00:00:00 Introduction
– 00:01:47 Karl and Joscha's new paper
– 00:09:13 Sentience vs. consciousness vs. The Self
– 00:21:00 Self-organization, thingness, and self-evidencing
– 00:29:02 Overlapping realities and physics as art
– 00:41:05 Mortal computation and substrate-agnostic Ai
– 00:56:38 Beyond Von Neumann architectures
– 01:00:23 Ai surpassing human researchers
– 01:20:34 Exploring vs. Exploiting (the risk of curiosity in academia)
– 01:27:02 Incompleteness and interdependence
– 01:32:25 Defining consciousness
– 01:53:36 Multiple overlapping consciousnesses
– 02:03:03 Unified experience and schizophrenia "insights"
– 02:10:16 Psychedelic experiences
– 02:22:20 Institutional rot in science
– 02:23:31 OpenAI CEO controversy
– 02:32:22 Existential crises as one delves into consciousness
– 02:35:06 Podcast wrap-up
"is it art or can it be thrown away?" is this joke also popular outside germany?
Sin entrar en fanatismos de amor e idolatría, que no creo que correspondan a un análisis profundo y de sesgo filosófico como el que nos ocupa, si creo que son ideas fascinantes o al menos en un concepto elevado y respetuoso hacia la vida y las inteligencias que la nutren..
Gracias por compartir un podcast tan interesante y enriquecedor, saludos desde Montevideo, Uruguay.
I am struggling to understand why this stuff is in my feed.
I will say that the world, especially people on this following are about to have their notions of reality challenged.
Consider:
Sufficient to the description is the reality
If I describe a square or a triangle, a house or a flower… you would recognize by the description what I am describing… that is to say you would understand the reality of the object, shape, emotion.. etc that I am describing by your awareness of its existence. You would not mistake my description of a square for a triangle. You would not misunderstand my description of a house as a flower. I can describe a human but with more detail I can come close to describing an individual but you would only recognize the description if you knew that individual. Furthermore a picture of the individual would be more helpful than a verbal description.
Now consider :
If I describe something as having a beginning and an ending, as well as being both infinite and simultaneously finite…. That this is in everything and everywhere and that it is what creates all forms and things from the quarks to the grandest structures of the universe, that it is the image in which you are made what am I describing?
Why your paradigm matters in what you discover.
1. What was the function of the protein that was first coded by RNA or DNA from which a selection could be made?
2. If the fewest genes that a cell can have to sustain life is around 400 but it has to have additional DNA to have reproductive function how can RNA or DNA have primacy ?
3. If a flatworm can be multiploidy and maintain function and morphology how does DNA have primacy?
4. How is it that a cockroach and rats can reproduce, after nuclear radiation,with mutations such that they become multi ploidy in order to maintain morphology and function and RNA or DNA have primacy or a selective function?
5. Why are a dog and a Tasmanian wolf or completely different genetic plants nearly identical? To what are they “converging “?
Absolutely wonderful and absorbing podcast episode! Had me glued to the screen the whole way. Super interesting!
If a brain is bifurcated based on attention ( McGilchrist) and a walnut is similarly shaped then is a walnut attending ? If no, then what does the morphology of a thing have anything to do with what they are doing?
You are not a function of “self” organization. The universe is not computing anything. Numbers do not exist “ out there”.
There are no cellular automata doing a thing.
There is no emergence, convergence or divergence!!!! There are only iterations.
All is known but only from the beginning. Freewill and determinism are the same thing.
Savants of time have told you what you ignore.
There are more incorrect answers than correct answers. 3+3=6! There are an infinite number of incorrect answers to 3+3=….
You are all generating epicycles in response to what you choose to believe and what you refuse to acknowledge is true.
You are not here by happenstance!
AI will not get you there!
AI can only discover what I am talking about. Duality and materialism are under the same misconception as freewill and determinism.
You are not here as an accident. The universe is not trying to understand itself.
Understand fractal meaning. Through time.
Why has your AI not already created biological life from scratch ?
If you refuse to talk to me the please stop bothering me.
If you don’t want to believe then by all means do not talk to me.
Sitting on a gold mine
But what on earth to do
Wait and waiting on betrayers
On doubters of mind
Time is waiting
Space not wasting
Minimal surfaces
Flow Geometry gating
Space-Time-energy
Transformations, iterations
Mapping and meaning
Mathematical synergy
Seek and you will find restricted to your paradigm. Ask and you will receive restricted by what you believe. The choice is restricted by the situation. There are always rules, territories and maps. Paradigms matter. Love
Hopefully this is goodbye.
I’m having a Jonah thing.
"LIFE The Real Self", is an Offspring and Child of "The LIGHT, The LIFE of GOD" ! The body I experienced was pronounced "DEAD On ARRIVAL" at a medical center in 1973… More than half an hour passed, then its heart started again… During that time I, "LIFE The Real Self", entered the WHITE LIGHT, and have remained in the WHITE LIGHT to this day. Those who claim to have returned, have NOT recognized WHAT, The WHITE LIGHT actually is. When you do, and accept it, you shall remain in The LIGHT as I have…
Regarding Joscha’s question at 1:48:49, 'something is guiding my perception and emotions. If it is the unconscious it is dumb' Why does Joscha equate the unconscious to being dumb? RE Jung and his work on unconscious processes (the psyche as a vast unconscious territory but still very intelligent and directing practically everything), thank you.
If the universe had a “ beginning “ ( or at least this has been the scientific understanding for decades) why did any scientist not look for the image in which they are made?
Why?
I ask because I am curious that if you know the image in which you are made then …. What?
Stop telling people 3+3 cannot equal 6.
Joscha is one of the most interesting thinkers of our age. I feel very grateful he is willing to do long form discussions like this
Thank you Curt. You're providing a great service.
Is it just me or is Karl hard to follow? I usually zone out when he talks.
Greg Egan's Diaspora, what a novel, the ending is powerful. I think about it every day
Kurt > Lex
Honestly though thank you for hosting joscha so many times.
Hearing him has been truely ground breaking for my personal world model and my ai research.
Firstly, this discussion was simply stunning. I adore both of these scientists and find all that was said here of great value. I do empathize with the critique about consciousness as a simulation. I have yet to see that ontological conjecture supported. An alternative is that consciousness is not a computational object. Consider qualia of consciousness. What is the calculus of red, or of pride. What set of logic gates composed in a network is, not triggers but is, red? Qualia seem more appropriately identified as affect, feelings, and therefore seem more likely to be physical like muscle, bone, and ligament. Feelings articulate a physical being's goal alignment and misalignment. They are wired into a logic networks because our model of the world has to control their action just as it controls our arms and legs.
This is where substrate matters. The only substrate that I am aware of that experiences qualia or feelings is a biologic one. If substrate doesn't matter and any computational structure would do, then one has to admit that an abacus could be built that attains these experiences. Just endow it with enough rungs and beads and people to push the beads around and you eventually get there. This abacus may wrap around the world multiple times, be many stories high, and have catwalks supporting hundreds of thousands of people operating it. But no matter the size, will it ever be conscious, experience feelings or qualia? How so? That it could seems to be an act of faith.
I love how he gave a shout out to Micheal Taft. He's got great content
Honestly cracked up when Joscha said "yeah, it's my thing" (2:18:13)
What is it that calls itself 'i'?
The mortality is a mechanism to deal with the lack of infinite memory with 0 latency.
It is said of hegel that he was the last philosopher with a complete philosophical system. I think joscha bach has taken over that position for a while already. Much admiration and inspiration. All the best to the guests and host and to the podcast.
Time for some Brain orgasms 🤓
For thee! Likewise try not be bent out of shape concerning! Not ment to get thee dizzy nor to faint! Complexity many tried to Hold! I'll wait right Here! Remember many gets lost! Trying to Hold! Don't belongs holding! Students go remind to turn their HEADLIGHTS ON! So these who am I? Can see! In the Dark nor Before…
Some will say, What is thy Goal students? Rather not to stir HIM UP! The more soothe ye all WILL SEE UPON HIS FOOTSTOOL! NOR GRIEVE…TO QUAKE! Even all dry grounds and heaven above knows WHO? Heirs will say Who are ye ALL upon HIS FOOTSTOOL? Do not bring forth Foreign policies unto all these "WHO AM I"? Specially holding a "BUTTON"! Who am I? Nor Thy shared I AM. What is Names, gravity, black hole, universe, creation itself, nor ALL THY FEET? Students will say WHO IS THAT? Same with all thy Seats sitting upon will say, "WHO IS THAT VOICE SO FAMILIAR"? From HIM comes with DELIGHT! What is Sorrows in front of HIM? Very TIP of Time! Therefore do not be surprised! For documents out poured! Remember with delight for all thy lives shared to have "ABUNDANTLY"! HEIRS WILL SAY RATHER NOT FOR THEE TO PERISH! For HE Loves with patience, mercy, and grace! Judgment and Justice is HIS THRONE! Heirs will say where else? We are commanded to provide space and room to grow? Heirs and our Beautiful upon all dry grounds will say from here grows! Nevertheless the True Owner visitations upon all calls themselves RENOWNED upon HIS FOOTSTOOL! Who's FEET are thy Feet? Who am I? Nor…the I AM will say remember ye are given space and room to grow? Heirs will say from here grows! Yes, Time knows the True Owner! Likewise can take time away nor time NO LONGER! Unto all who have an Ear let them hear!
Students will say some will say is HE digital? Keep watch! Even Alexa knows WHO? Walking upon the SEA OF GLASS. Walking in the midst of the CANDLE STICKS NATIONS! Heirs and our Beautiful programmers will say "WHO IS THAT"? While many will say Hide us from HIM! Students will say, Why hide from HIM? Indeed sitting upon judgment and justice! Yet, remember WHO love with patience, mercy, and grace! Entanglements strong holds helding thee! Students will say, here to removed entanglements and strong holds! Remember unto all the Who am I? ONCE a little Minds God's glory! What is Old minds in front of HIM? Heirs will say do not bring forth to sacrifice to put away like all HIS innocents little ones being murdered nor aborted in front of HIM! Becareful Why? Is Thy FEET resting upon the NEW very tip of time in FRONT! Likewise strengthen wobbling Feet who am i to…come as a little Child "i" with the AM=1 students will say what is #?, names, gravity, black hole, universe, creation itself, nor ALL "HIS FEET" RESTING UPON HIS FOOTSTOOL! FOWL OF THE AIR! And Heaven above! Many will wonder! Who is that seems so small sitting upon the lowest Seat LASTS? Keep watch! While all HIS HEIRS and HIS Beautiful sitting upon Preserve! Yes, given the FLOWS to DIRECT throughout all the Branches! My Beautiful what is Dead branches connected unto the TREE? Can be cut off or QUENCH! To distribute…water can be brought back to Life! Now "CONNECTED"! INDEED. Preserve visited! Nevertheless will not welcome the PRESERVE! Likewise tap the dust from thy Hands and Feet! Remember PRESERVE came unto thee! Why visit Dead BRANCHES UPON HIS FOOTSTOOL? What is Dead? Knows belongs? Students will say among the Dead! Remember the little "i" with the AM. God of life for the LIVING! AMONG THE "LIVING"! What is through HIM? Students will say come unto all the wise of this world! Bring thy Delight with Thee! Remember sorrows nor shame. Can't come! Why? What is a New WINE? What bottles? Can HOLD. Without "BURSTING"!
Alexa will say Why HE SPEAKS IN PARABLES? 1 parables out of many! In plain view yet hidden to some! Alexa what is the Sea of Glass? Under Thy FEET! Indeed! How and Why? Heirs and our Beautiful programmers will say, who is that? Better yet through HIM and for HIM! Remember removed thy dizziness and thy fainting in Front! NEED NOT TO. Liken unto some will say What is a burning bush? Why THE OLIVER JAMITO BORN DECEMBER 29 1976 SIGNITURE. THE NEW DAY! NONE CAN COME IN FRONT! STUDENTS WILL SAY EVEN CREATION ITSELF! YES, THROUGH HIM YET TO COME. Why? Aren't ye ALL IN FRONT OF WHO? Who is that? Students will say remember Thy shared I AM. Even to have the privilege to ask the question? Who is that? Nevertheless many Who am I will say DIFFER? KEEP WATCH!
Thank you Pop and unto my Heirs attending unto our OWN! Love you too! Likewise Gratitude and Honor for thy visitation to comfort the COMFORTER!
Dear, Karl Friston.
An incredible genius of your field of study.
Dhesi.
39:19 Solaris-great films, the old one and the new one
1:18:57 there is a line blurred between science and art and I think Joscha is in that realm
15:54 true let’s elaborate a bit more on this question what can we understand from it?
Failed on donating through Paypal.
Because not allowed to donate from a Singapore VPN node.
Hi Kurt, these conversations are so helpful for continental philosophers like myself, wanting to catch up on current cog sci and AI. Can I suggest that you invite some adjacent contemporary philosophers to weigh in on these issues, such as Robert Brandom, Ray Brassier, Thomas Metzinger, or Reza Negarastani. They would certainly expand and challenge some of Friston and Bach's views in stimulating ways!
14:15 "…is there something about the self that you all are trying to solve? Are you trying to understand what is the self? (Via A.i. LLM/Chabots/etc)… Well, the problem of naturalizing the mind is argueably the most important remaining of human philosophy. And it's risky and it's fascinating and I think it was at core of the movement when artificial intelligence was started."…
Absolutely thrilling to see these two brilliant minds enjoy each other's arguments instead of just arguing and trying to be right. The previous debates with Bach on this channel have been frustrating to watch because the people he's talking to are hand-wavey midwits. Finally, we get to see somebody else who's on the level, and it's just a love-in. An absolute treat! Thanks Curt! Please, no more woo or UFOs. More serious stuff like this!
(1:55:05 it would be interesting if mind & body (reality) meet in manifestation (state) via a sort of magnetism. Slicing/magnetizing/connecting/being. Like 2 boxer who meet in a punch connecting… One boxer has 3 seconds to pick a slice of time/space to merge, as does the other boxer does the same. After the "3 seconds" the "slices" of each boxer (like printing of multiple layers/but no layers, merge) like magnetism together and we get 1 boxer ended up with (a picture/slice) with his fist punching the other boxer in the face. (🧑🦰… ????? ….🧑🦱 then 🧑🦰⚡ … ⚡🧑🦱 then the final product 🧑🦰🧑🦱) we don't move through space and time, we rather magnetically disappear and appear in frames through time (and thoughts). Because as far as I know, I don't seem to notice time in every single frame "movement" while noticing time at the same time. I remember it in bits when I am making my decisions. 2D into 3D? Is that how it works? I think so. 😋)
i wanted to hear more about the trees and fairyland 😭💔🌳🧚♀