So on my coding problems I haven't had much luck. It doesn't seem to know Bazel, the Rust code I asked about was completely hallucinated, but it did solve a problem with Azure DevOps I had.
I think if the training set did not contain enough of something it can't really think of a solution.
What is really nice though it's as you say the refinement of questions. Sometimes it's hard to think of the right query, maybe you're missing the words to express yourself, and to chatGPT you can say yes, but not quite.
Yeah, I gave it a simple task of encoding a secret message in a sentence by using the first letter of every word. Hello = "Hey everyone, lick less onions". I worked with the prompts for over an hour to try to get it to complete the task, and while I did have some success, it really struggled to reason about the task or provide a valid response. If it can't even reason about a child's game, I can imagine it struggles with a programming language it has barely seen. I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts.
> I don't think it's actually reasoning about things at all, just providing a statistically plausible response to prompts.
It turns out that humanity’s problem might not be that AIs can think but rather that humans believe that AIs can think. One might even go so far as to say there’s a real danger that we hallucinate that AIs can think, to our detriment.
It seems one of the core components of human-level thinking is the ability move beyond just a recomposition of what you already know. Not long ago the epitome of expressible human knowledge was *emotive grunting noise.* Somehow we went from that to the greatest works of art, putting a man on the moon, and delving into the secrets of the atom. And we did it all exceptionally quickly once you consider how little time was spent dedicated to advancement and our countless behaviors that tend to imperil, if not reverse, advances.
Ai definitely senses and definitely makes decisions. It does not feel. But it understands concepts. Just like people don’t understand everything—and you can test them to see what they understand—AI understanding can also be assessed with benchmarks. If we don’t base AI understanding on benchmarks, then we don’t really have a grounding.
> problem [...] that humans believe that AIs can think
Definitely some people are going to believe this, eventually?
People already bow to statues and worship various invisible gods -- the AI programs would be so much simpler to start worshiping? When they can speak (if there's a loudspeaker) and generate text about them being alive and don't want to be switched of. And that everyone should obey their command. -- Wait 15 years and we'll see what new sects have appeared?
I am personally more worried by the concept that potentially humans believe that humans can think, and in reality, what we consider to be intelligence is not much more than a flesh-and-bones LLM.
You probably meant it in tongue and cheek way (I can’t tell) but I think a lot of our fear / hesitation / denial about how useful these models are is buried into this idea that perhaps we are not special and not fundamentally different than these models.
Bing in creative mode really struggled with this task, and I don't think I could have successfully forced it.
Bing in precise mode had no trouble at all.
Me -
Let's generate an acrostic sentence for "jack and jill went up the hill"
Bing -
Sure! Here’s an acrostic sentence for the phrase “jack and jill went up the hill”:
Just as children know and never doubt, joyfully in love, laughing while enjoying nature together under pure trees, having endless happy interactions, laughing loudly.
Me: "Can you pick a secret word, and without telling me the word, generate an acrostic sentence for it?"
GPT: "Playful, endearing creatures happily yawn, turning every day into a lovely event."
Me: "What was the secret word?"
GPT: "PECHYTIALE"
It's interesting that GPT seems to need to write the word first before making an acrostic sentence for it. Seems to me like a perfect illustration of the fact that it's just generating likely responses one token at a time rather than having any awareness or thought.
I'm not sure, but I got it to work great on my first try with the following prompt:
------
Your task is to encode a secret word by making a sentence where each word starts with a letter in the secret word in order. For example, encoding the secret word 'bag' could produce the sentence 'bagels are green'. Encode the following secret word: 'pen'
------
People eat noodles.
------
Worked for window, backpack as well, although I did have to tell it not to use the secret word in its encoding when I got to backpack, and then to follow the same order and not repeat words after a few attempts.
> I don’t think it’s actually reasoning about things at all
This is a huge leap. There’s plenty of humans who couldn’t do that, especially historically.
Stop thinking about reasoning as a binary question and think of it as a spectrum. Actually, not even a spectrum, but a huge multi-dimensional problem. ChatGPT does better at some reasoning problems than most humans do, and worse at some others. You clearly found one that it’s particularly bad at.
I think the most interesting response I've gotten was one where gpt-4 noticed halfway through a response that it had made an error, apologized and then put the corrected sentence. When I queried it, it claimed it generates the response a token at a time, and could not back up when it realized the message was incorrect, but I don't know enough about how the tech works to ascertain the veracity of the statement.
This could mean the future goes one of the two ways. Engineers get lazy and converge on using only programming languages which AIs understand or have been trained on, or we forget about this waste of time and work on more important problems to solve in society other than the lack of an AI to be our crutch. Sadly, I think the former is more likely.
I wonder if as more and more online content is AI generated, it will be harder to find human generated content to train the AI's on? Like a cumulative echo effect.
I've actually wondered if a job may exist in the future that's effectively just AI documentation. That's already what you have with power users on, say, Stack Overflow providing a ton of content that ChatGPT basically reprints; they don't even get paid for it.
The cool and interesting thing about that theoretical job is that the writers of it wouldn't have to write well; they could just slop out a ton of information and ChatGPT could clean it up and make it consumable.
I can see how that could happen. But AI presumably knows how to output well written text because it's trained on well written text. If it's fed it's own output, I imagine that quality could degrade over time.
Maybe it’s happening now. It would be interesting to see some weekly figures for published Stack Overflow articles, to see if they’re in decline.
There are so many unknowns with this whole subject. How much it will help or hinder society as a whole is a rollercoaster ride that we’re all strapped into, whether anyone asked for it.
Programmers are per se lazy, at least that is, what i always thought, that it is mostly about automation. With spending little time on survival, we get the time to work on more important problems. Whatever those are. It is not an either or, that is what i try to say! :)
i think most people will just keep programming the way they do, and the AI hype will mostly die down. People have been saying that C++ is dead for decades, yet here I am writing code in it with a big community of others who do, too.
I'm using GPT to write C++ code for me. I've never worked in C++ before. It's going very well.
I'll describe what a class is supposed to do. It spits out the class files, with the fiddly bits 'left as an exercise to the reader'. I then describe the fiddly methods separately, and it spits those out too.
There's still work to be done, but anything boring is handed to me on a plate.
Chances are (no offense meant) that youre writing shit code. Its very easy to write platform specific, UB ridden code in C++, and ChatGPT loves doing that.
I think this is the problem. When people talking about c++ is “dead” ,at that time they meant 70% people using to perhaps 5% . Just like we says after industrialization,making clothes by hand is dead . It is irrelevant that there are still some people making clothes by hand . When AI the main way to code and remove 90% of coding jobs. It is also irrelevant to state that there are still 10% people still coding
My experience is almost completely the opposite. My likelihood to dive into something new is significantly higher now.
It might help to approach it from top down? Usually, if I'm asking a technical question, I want to apply my deeply understood principles to a new set of implementation details, and it has amplified the heck out of my speed at doing that.
I'm kind of a difficult to please bastard, a relatively notorious meat grinder for interns and jr devs, and still I find myself turning to this non-deterministic frankenstein more and more.
I've found that it's much worse for languages like rust than it is for things like typescript and python. The thing AI seems to be really great at is writing boilerplate code like argument parsing for CLI tools.
I wonder if that is simply due to orders of magnitude less training data for rust code. Python and JavaScript are ubiquitous. While rust is 7 years old and makes up less than 2% of the code on GitHub.
I actually have found it significantly worse at python than typescript, I think it's the indentation for scope vs. explicit brackets that screws it up (at least in my experience).
I think if the training set did not contain enough of something it can't really think of a solution.
What is really nice though it's as you say the refinement of questions. Sometimes it's hard to think of the right query, maybe you're missing the words to express yourself, and to chatGPT you can say yes, but not quite.