Robin Hanson is someone who has thought about AI futures the most and in the greatest detail. He’s skeptical of most intelligence explosion stories out there, but he does expect that soon enough, biological humans will be surpassed in all economically valuable capacities by non-biological agents and that we will no longer be the “main characters” of history.
Robin’s view is that these future non-biological agents are going to be a lot like us, at least when they first come on the scene. He thinks that the most plausible versions of them, whether they be something like scaled up LLMs or (his most likely expected outcome) human brain emulations, are going to probably have a lot of our values and psychological features as deeply ingrained design features. He thinks that these values and psychological features will shift over time through the same processes of cultural evolution that humans have undergone. This is worrying to him, not because he thinks these agents will be out of our control, but because he thinks humans have in fact got themselves into a maladaptive state of cultural drift. He worries that non-biological agents could just as easily end up in a state of maladaptive drift in a future civilization.
As far as human control over future non-biological agents does, Robin doesn’t think that we should expect or hope to have much. He thinks that these agents will meaningfully be our descendants, and that we should expect and want to control them only in the same ways and to the same extent that we expect and want to control our biological descendants. To him, this basically means that we probably should not expect or hope to control them very much at all.
In this episode, I try to get a better understanding of all of these views. We start by talking about the unity of intelligence, move into what we should expect culturally or psychologically from AIs, and end up talking about how we should think about values in the very long run.