> It's not fair to judge AI by asking it for objective answers in questions requiring value judgement... especially when it's been "aligned" to within an inch of its simulated life to avoid bias.

They aren’t aligned to avoid bias (which is an incoherent concept, avoiding bias is like not having priors), they are aligned to incorporate the preferred bias of the entity doing it the alignment work.

(That preferred bias may be for a studious neutrality on controversial viewpoints in the surrounding society as perceived by the aligner, but that’s still a bias, not the absence of bias.)

> Again, that's all we do. We train extensively until we "get it." Monkey-see, monkey-do turns out not only to be all you need, so to speak... it's all there is.

Which is fine for us humans, but would only be fine for LLMs if they also had continual learning and whatever else was necessary for them to be able to learn on the job and be able to pick up new reasoning skills by themselves, post-deployment.

Obviously right now this isn't the case, so therefore we're stuck with the LLM companies trying to deliver models "out of the box" that have some generally useful reasoning capability that goes beyond whatever happened to be in their pre-training data, and the way they are trying to do that is with RL ...

Agreed, memory consolidation and object permanence are necessary milestones that haven't been met yet. Those are the big showstoppers that keep current-generation LLMs from serving as a foundation for something that might be called AGI.

It'll obviously happen at some point. No reason why it won't.

Just as obviously, current LLMs are capable of legitimate intelligent reasoning now, subject to the above constraints. The burden of proof lies on those who still claim otherwise against all apparent evidence. Better definitions of 'intelligence' and 'reasoning' would be a necessary first step, because our current ones have decisively been met.

Someone who has lost the ability to form memories is still human and can still reason, after all.

I think continual learning is a lot different than memory consolidation. Learning isn't the same as just stacking memories, and anyways LLMs aren't learning the right thing - to create human/animal-like intelligence requires predicting the outcomes of actions, not just auto-regressive continuations.

Continual learning, resulting in my AI being different from yours, because we've both got them doing different things, is also likely to turn the current training and deployment paradigm on it's head.

I agree we'll get there one day, but I expect we'll spend the next decade exploiting LLMs before there is any serious effort more on to new architectures.

In the meantime, DeepMind for one have indicated they will try to build their version of "AGI" with an LLM as a component of it, but it remains to be seen exactly what they end up building and how much new capability that buys. In the long term building in language as a component, rather than building in the ability to learn language, and everything else that humans are capable of learning, is going to prove a limitation, and personally I wouldn't call it AGI until we do get to that level of being able to learn everything that a human can.