The CUDA moat is real for general purpose computing and for researchers that want a swiss army knife, but when it comes to well known deployments, for either training or inference, the amount of stuff that you need from a chip is quite limited.
You do not need most of CUDA, or most of the GPU functionality, so dedicated chips make sense. It was great to see this theory put to the test in the original llama.cpp stack which showed just what you needed, the tiny llama.c that really shows how little was actually needed and more recently how a small team of engineers at Apple put together MLX.
Absolutely agreed on the need for just specific parts of the chip and tailoring to that. My point is bigger than that. Even if you build a specific chip, you still need engineers who understand the full picture.
They do have such a dedicated chip, the MAIA 100 chip which is an in-house chip, and it is a chip that was designed in the era of transformers, and this is what is being discussed in the interview.
I missed that, it’s been a few years since I’ve paid attention to MS hardware and it is very possible that my thoughts are out of date. I left MS with a rather bad taste in my mouth. I’m checking out the info on that chip and what I am seeing is a little light on details. Just TPUs and fast interconnects.
What I’ve found; MIAI 200 the next version is having issues due to brain drain, and MIAI 300 is to be an entirely new architecture so the status for that is rather uncertain.
I think a big reason MS invested so heavily into OpenAI was to have a marquee customer push cultural change through the org, which was a necessary decision. If that eventually yields in a useful chip I will be impressed, I hope it does.
“(talking about when he tells his wife he’s going out to buy an envelope) Oh, she says well, you’re not a poor man. You know, why don’t you go online and buy a hundred envelopes and put them in the closet? And so I pretend not to hear her. And go out to get an envelope because I’m going to have a hell of a good time in the process of buying one envelope. I meet a lot of people. And, see some great looking babes. And a fire engine goes by. And I give them the thumbs up. And, and ask a woman what kind of dog that is. And, and I don’t know. The moral of the story is, is we’re here on Earth to fart around. And, of course, the computers will do us out of that. And, what the computer people don’t realize, or they don’t care, is we’re dancing animals.”
He ignores his wife's suggestion because, among other things, he wants to see some great looking babes. Maybe this isn't a guy whose philosophy I want to follow.
I understand the point of the quote. My point was that if someone advocates for something I consider immoral, I prefer to disregard the overall philosophy that lead the person to advocate for that.
If the quote threw in some casual racism, or advocated for stealing the envelope instead of paying for it, I would similarly disregard the overall philosophy.
The terms have changed gradually over the years and now we are boiling IDEs on the iPad.
My plan is to ship something that is both a great iPadOS app and operates within the confines of the AppStore restrictions.
I find restrictions as a powerful motivator to think about a problem differently. Lots of great art (and software) is great when it explores and brings to light what’s possible with the limitations of a medium.
I love the spirit here, but the limitations on iOS are not the limitations of the medium. Mobile computing has lots of interesting and inspiring limitations, we don't need apple to draw artificial squircles we can't cross in an api.
20 years is a generation, however for many of us, Apple's walled garden was a refreshing concept versus the mobile operators gardens.
First of all, getting SDKs was akin to console devkits, back in 2004 getting a Symbian SDK was still a commercial only product for example, same for Windows CE/Pocket PC,...
Followed by about 80% tax, only to be listed on mobile phones magazines, with the SMS code to trigger the application download.
Hence why everyone rushed for the garden, it was indeed easier to be creative in Apple land.
