"The real question is not whether machines think but whether men do."
-- B F Skinner
There is a vitalism[1] streak in some AI commentaries. People cannot quite define or measure human consciousness and yet readily conclude that machines have not achieved/will not achieve any meaningful form of intelligence.
The people who think AI is conscious without any definition or measure of consciousness. Conflate consciousness with computation.
The only reason we do this is because our computation organ seems to also be our consciousness organ.
We don’t understand consciousness well but we do understand computation.
Given infinite time and space all the computation in any AI system can be done manually using sticks and stones. If it likely that system of sticks and stones become conscious?
It is ethically very dangerous to extend the same care to a convincing computation pulls at our evolutionary heartstrings as one would to a sentient creature that is capable of pain and suffering.
Largeness isn't sufficient, but it is certainly necessary. By largeness I mostly mean a state space which is effectively inexhaustable. Humans are finite, but human state has ever recurred.
I don't think the sticks and stone argument works. It sounds like you are saying something like "well we can imagine in principle performing a computation just like brains do with sticks and stones but you dont think the sticks and stones are conscious do you?"
But yes of course it would have to be conscious, that's what equivalent means. Otherwise you are saying you left something out, or that brains are special computers.
Why do I read these hoping to find novel insights or a different way to see things:
> [...] There are two ways to look at this issue. Some researchers think that the kind of intelligence found in biological agents is cut from a fundamentally different cloth than the kind of statistical pattern-matching large models excel at. For these skeptics, scaling up existing approaches is but a fool’s errand [...]
> Others would argue that large pre-trained models are already making strides toward acquiring proto-intelligent abilities. [...] One could see these capacities as very preliminary ingredients of intelligence [...] and there are compelling reasons to doubt that simply training larger models on more data, without further innovation, will ever be enough to replicate human-like intelligence.
> Regardless of how we answer these questions, we need to tread carefully when deploying large pre-trained models in the real world; not because they threaten to become sentient or superintelligent overnight, but because they emulate us, warts and all.
> In a particularly influential article, Emily Bender, Timnit Gebru, and colleagues ...
This quote was included to virtue signal. It assured the article would be garbage.
The thesis (buried a dozen paragraphs in) is
> I suggest much of what large pre-trained models do is a form of artificial mimicry.
> Here’s the thing about mimicry: It need not involve intelligence, or even agency.
> a kind of biological mimicry that can be seen as solving a matching problem
> Artificial mimicry in large pre-trained models also solves a matching problem
> We also need a better understanding of the mechanisms that underlie the performance of large pre-trained models to show what may lie beyond artificial mimicry.
Wait what? The whole thesis is that AIs are parrots. But parrots are living intelligence creatures.....
But the entire argument is smuggled in on the assumption
> Parrots repeat phrases without understanding what they mean
I see people assert this all the time. Intelligence is the ability to achieve goals. Consciousness is contents of what you are aware of.
The definitions are irrelevant. People want a definition to do demarcation. But demarcation is boring. I don't care whether some threshold entity is on this or that side of the line. The existence of some distinct territories is enough for me.
There is no reason to imagine AIs are prohibited from occupying territory on both sides of the line.
AIs aren't climbing up a ladder, non local exploration is possible
A more precise definition by François Chollet: The intelligence of a system is a measure of its skill-acquisition efficiency over a scope of tasks, with respect to priors, experience, and generalization difficulty
So a system is more intelligent if it can solve harder tasks with fewer trials and prior knowledge. Intelligence is always defined over a scope of tasks, for example human intelligence only applies to the space of tasks and domains that fit within the human experience. Our intelligence does not have extreme generalization even though we have broad generalization (defined as adaptation to unknown unknowns across a broad category of related tasks).
I read it more as asking about deep-NN models specifically.
If it really was trying to suggest that computation isn't universal or that our intelligence is non-physical or something, that would be a whole different problem.
> AGI is inevitable because computation is universal and intelligence is substrate independent.
People who say this slide invisibly from "computers exist" to "you could simulate a human in a computer" to "AGIs exist and have all the properties granted to them in science fiction stories about AGIs". Mostly they do this by forgetting if they have a specific or vague definition of the word "intelligence".
Can it imagine how a human would cook an egg if you only fed it natural language text about doing that? (No, language doesn't describe muscle memory.) Could it cook an egg if it was controlling a robot? (Not without practice.) Would it actually do that if you told it to? (Maybe. Who says it has motivation? All we know is it has intelligence.)
So, the theories that I've been reading present the picture that what AGIs have already has a sort of consciousness that isn't anything like human consciousness, because these agents really only have a subset of the processing that humans have, which in most cases these days, is language.
Human brains develop by performing movement, in fact, much of our brain is specialized for performing movement, judging angles and speeds, testing if we can jump across a puddle safely. The AGI would need to be trained on these things and the language associated with them. Your example with the egg, I think, is illustrative of the amount of training the AGI would need to go in order to learn, and that training would become part of its "personality".
There is also a set of desires that humans have, such as survival, companionship, hunger, etc. These would also have to be trained. These desires would have to be trained into the AGI as well.
The theory is that if we would have to train an AGI the same way we do humans in order to get it to be close to human consciousness. That doesn't mean it wouldn't have a type of consciousness along the way, maybe closer to what we experienced as we grew from children to adulthood.
Anyway, just conclusions from what I've been reading.
> AGI is inevitable because computation is universal and intelligence is substrate independent.
Let's say Human Intelligence and AGI are subsets of General Intelligence, we don't know whether GI will actually subsist without the biological drive, AGI's might decide that hoarding resources/knowledge is futile in a finite universe and shut themselves down then reaching the halt state of this paradox. It might never be possible to achieve an AGI too different from the stealing/warmaking/miraculous HI.
I dont see how this is anything except lack of imagination. To imagine that life can only be made of meat is pretty strange. To imagine the only way of "being" in this universe, is to be nearly identical in construction and behavior to humans is absurd.
This doesn't sit well with me. Parrots can repeat sounds. AI models can compose phrases into coherent output. Very different. One is just funny and the other can be really useful.
I think the Stochastic Parrots name is an attempt to disparage and limit LMs by a group of people who only see them as discrimination amplifiers.
> AGI is inevitable because computation is universal and intelligence is substrate independent.
That says nothing about timescale. There's no guarantee that the current favored approach will work. Or even that we'll figure it out before we go extinct for whatever far-future reason.
The path to AGI is shrouded in mist and we don't know which way to take. If you directly optimise towards an objective you might not reach it unless you're already very close to see the last few steps.
The best solution is to try everything, build the necessary stepping stones. Some of them will be useful in hindsight, we just don't know which. Biological evolution does the same. Culture evolves in a similar way.
For example, did Newton have any idea about the utility gradients will have in learning models of language? Probably not. But his contribution paved the way to GPT-3. From his position it was just impossible to foresee, from our position it seems trivial. You can learn the basics of transformer neural nets in a course of 8h on YT.
So we don't need guarantees if we are ready to try out everything. And that's exactly what the deluge of AI papers does.
> I suggest much of what large pre-trained models do is a form of artificial mimicry. Rather than stochastic parrots, we might call them stochastic chameleons. Parrots repeat canned phrases; chameleons seamlessly blend in new environments. The difference might seem, ironically, a matter of semantics. However, it is significant when it comes to highlighting the capacities, limitations, and potential risks of large pre-trained models. Their ability to adapt to the content, tone, and style of virtually any prompt is what makes them so impressive—and potentially harmful.
> Here’s the thing about mimicry: It need not involve intelligence, or even agency. The specialized pigment-containing cells through which chameleons and cephalopods blend in their environments may seem clever, but they don’t require them to intentionally imitate features of their surroundings through careful analysis.
The author thinks reducing models to a parrot is too lowly, because in his conception parrots are just dumbasses who repeat phrases at random apparently. So he invents the term "stochastic chameleons", which is a downgrade - parrots are much smarter, and language is far more complicated than the adaptations chameleons and cephalopods do. But the heavy-lifting comes down to the term "intentionality", which pervades the entire article. The author equates intelligence to intentionality, a big mistake.
Another issue is this constant bemoaning of models as susceptible to human biases. Language models can be trained to be racist, sexist or hateful - so what? What does this have to do with mimicry and intelligence?
We know how ML models work: A machine-learning model finds patterns in the input data, looks at its vast statistical knowledge (lossy-encoded as multi-dimensional latent space), and then generates patterns which are statistically close to the input data.
If this instance of 'statistically close' corresponds very closely to what humans perceive as 'semantically close', we perceive this as parroting.
If this instance is somewhat distant but still connected to what humans perceive as 'semantically close', we perceive this as creativity.
If this instance of 'statistically close' is very different from what humans perceive as 'semantically close', we perceive this as an error, a glitch, a fluke, or the model not working well.
This is complicated by two things:
(a) we have a very limited understanding of what 'semantically close' means.
(b) we have a very limited understanding of how human brains work.
> If this instance of 'statistically close' corresponds very closely to what humans perceive as 'semantically close', we perceive this as parroting.
Could it be that with scaling the statistical model can become good enough for semantics? Quantity has a quality all its own / The map is the territory.
I believe we're seeing emergence from low level statistical patterns to semantics, especially in GPT-3, DALL-E and AlphaGo.
Perhaps the most important notion of semantics is that of referent. Pretty sure you can't develop that without an agent that can interact with the environment.
This is a good article among a sea of bad articles on this topic, taking a balanced view. It doesn't buy the "stochastic parrots" line but also doubts "scale is all you need".
There is a vitalism[1] streak in some AI commentaries. People cannot quite define or measure human consciousness and yet readily conclude that machines have not achieved/will not achieve any meaningful form of intelligence.
[1] https://en.wikipedia.org/wiki/Vitalism