Sy Borg wrote:
There is a blurred line between agency without sentience and agency with sentience, so you are right that the ideas are debatable. Consul and I debated this for years Still, the fact remains that the biosphere was once not sentient and now it is. Sentience has evolved from the non-sentient before. In fact, sentience emerging is the broad story of each of our lives.
My guess is there won't be a "lights on moment" in The Awakening of the Machines, but a gradual dawning, as happened with biological life.
I already extend life to beyond just biology. I see biological life as a phase of a larger dynamic. If life/sentience/existence is to be anything but a rare blip in a barren universe, then it needs to be at least interplanetary. Yet biology and space don't mix. Even a brief stay in the ISS is hard on the body. The story can only continue with non-biological sentience.
Even if humans die out and non-sentient AI continues, it may be a period of "sleep" before sentience re-emerges. In the end, either AI can eventually become sentient or the future of the universe is just plasma, rock, and mindless machines doing pointless tasks. Then again, in a sense, it already is.
The facts that we are aware of are that biological life emerged from inanimate matter and then eventually sentience emerged from that biological life. There’s also the fact that we have seen no signs that in the vast universe outside or Earth, the two processes I just mentioned have taken place, so Earth is our only reference. What has happened on Earth, though, is entirely contingent, so anyway, even under similar initial conditions, which cannot be guaranteed, there’s no good reason to expect that things will come out the same.
Now, if that is not enough, there’s a bigger problem for the analogy between the emergence of life or sentience and the expected emergence of new sentience from lifeless forms. If the contingent processes of inanimate matter (A) produced the contingent process of animate matter (B), which then produced the processes that gave rise to sentience (C), it cannot be logically inferred that A has the potential to produce C directly, without B. Even worst, absent B or C, that is, if everything is reset to the conditions where only A exists, if one could argue that there’s a chance that B or C will reappear, none of that allows us to infer, being B or C present, that a new process D (artificial intelligence) will emerge, no matter that you name it as an existing process and put the adjective “artificial” before it. We already have life and intelligence, so what then is supposed to emerge new and from what? Of course, it is more likely that once the playing with words is avoided, we are left with the proposition that what is to be produced is a property that already exists in certain sentient entities (created out of the contingencies of inanimate matter), but this time would exist in non-sentient ones. Again, we lack any precedent for that possibility. The only precedent we have is that property arising from living organisms.
These problems cannot be solved by inverting the concepts and calling life a larger dynamic that encompasses non-life or intelligence as a larger dynamic that encompasses non-sentience, given the minimal evidence of those things outside of our ridiculously unimportant planet. The workaround, which involves reconceptualizing life and intelligence, is deeply problematic: whatever comes out of it can’t be “life as we know it” or “intelligence as we know it”. That’s a major issue for current discussions about AI, which focus on assigning properties, inferring relations and making predictions based on life and intelligence as we know it. They are always there and are never bypassed, so reconceptualizing simply will not work.