I don't see why everyone is so surprised by this. Google already provides the default search engine for iOS, and it's clear that Apple's on-device stuff is going to be years away. They don't have the infrastructure to do this themselves either, so they're going to need a partner. Steve Jobs always saw Apple and Google as natural partners, he just couldn't get over that Google launched Android.
I think Apple by now realizes that Android was a necessary defensive play by Google aimed not at them, but at Microsoft, so maybe this partnership could be revived. Probably going to be more difficult now in the current regulatory environment, however.
While not immediately, ultimately Android was very explicitly a hedge against Apple, not Microsoft. Seems this became clear at Google near instantly after the iPhone announcement. Google actively shifted it's Android development toward touch screens directly as a result of the iPhone, for instance. By the 2010 Google IO keynote they made this explicit with Vic Gundottra's speech describing the mission of Android according to Andy Rubin (it's original creator): "If Google did not act, we faced a draconian future. A future where one man, one company, one device, one carrier would be our only choice. That's a future we don't want."
This was a reference to Steve Jobs, Apple, iPhone and AT&T (originally Cingular).
The reason for Apple to do this will be exactly why Microsoft is happy for OpenAI to provide the Models. If anything goes wrong it's not Apple or Microsoft's fault. It is the SaaS provider who is to blame.
For apple this will mean Google's stock takes the hit while theirs remain unaffected. Unfortunately for OpenAI/MS, OpenAI is not public and so everyone is using MS as a proxy for trading on OAI. So if something goes wrong in the Bing/CoPilot world, MS stock will take the hit.
Given all the regulatory scrutiny on the Google-Apple search deal, I believe we'll get an option to choose the 'AI Backend' similar to how EU users get to choose Search Engine.
In effect a lot more people are aware of Google logo than OpenAI so that's what most common people will choose, while maybe most tech-savy folks will end up choosing OpenAI.
This will be Apple's way to hedge against any further regulation option and they get to market iPhone as a AI aggregator similar to the AppStore.
It either indicates Apple isn't sure it can catch up or that they simultaneously realize they need to provide these AI features but don't think AI as a marketing component will have the staying power to make the investment worth it.
I can't say I disagree very much, as the AI hype is quite overblown.
That would also explain why they didn't pick something like Mistral or Stability. Newcomer with no strong legal dept can easily be tanked if people catch on that training infringes and legal winds change. Google is less likely to fail catastrophically when it comes to that.
Is Apple making profit out of generating and selling derivative works from music and movies without compensating the original authors? If not, I'm not sure how it relates to the current breed of generative AI.
Mine being that large language and diffusion models are trained on "whatever the heck is there, available for scraping on the broad internet, indiscriminately".
I don't think Apple's money or it being in bed with the Majors solves either side of "acknowledging authorship when none is provided/known", "obtaining formal consent for commercial use" or "fairly compensating a creators base" when it is larger than all humans on earth.
I agree that this is very different from traditional content agreements. Apple is a very conservative company. They may not want to be trailblazers when it comes to working out how compensation of creators is going to work in an AI context.
What appears to be happening right now is that Google, OpenAI, et.al. are negotiating individual agreements with large publishers if and when threatened with legal action.
I'm not sure this will be sufficient. There will be calls to introduce some sort of AI tax or other forms of collective compensation schemes to funnel money to independent creators, small businesses, museums, etc.
This is all very much up in the air and it will take many years if not decades to settle.
Apple is doing it, but doubt they'll use something in-house that's still very much a work in progress when they can actually get paid by Google to use Gemini this year.
Everyone wants/wanted Google as a search engine because it was the best. No users thinks Gemini is the best. In fact it is being mocked by media right now.
>It falls a bit on programming tasks, but I don't use it for that.
I suspect a lot of us mediocre coders probably are using these chat AIs primarily for this. ChatGPT's been a pretty solid replacement for what used to be Googling for "How do I do X in Y?" kind of queries. Obviously, that's not really what Siri needs to be.
I've been using Gemini in lieu of Google search where I'm asking a question I want an answer to rather than searching for a website. Microsoft is doing this with Bing, it tries to understand when you're searching for web sites and when you're searching for answers. I imagine Google will eventually integrate these offerings.
Where Microsoft's integrated approach presents challenges is Gemini provides for a conversational approach (context). If I ask a question and subsequently need to provide further clarification, I can just provide the clarification. I don't need to re-ask the entire edited question. It's amazing how fast I've become accustomed to this conversational approach.
WRT Apple, Siri has become irksome. She can't reliably handle basic tasks on your phone, much less general-purpose questions. At this point I think it's going to become the case that AI is a commodity, and I don't see what Apple gains by building their own commodity service.
What's surprising to me is not the partnership but that they are outsourcing AI to another company. This is a big minus for the future growth of Apple. Google will get better at its AI offerings while Apple will get weaker. I'm a bit shocked that Apple is in such a spot. I was sure they had a big AI initiative that they would shortly unveil. It seems now that it was wishful thinking on my part. In the past, Apple has been able to move forward by buying products that they can then improve by giving them the Apple treatment and selling them to their fans. AI takes so many resources I suspect they won't be able to buy their way into a killer new product. Apple has been stellar at hardware. I hope that they can keep up their hardware lead since they are blowing their AI offerings.
> I think Apple by now realizes that Android was a necessary defensive play by Google aimed not at them, but at Microsoft
Huh? Could you elaborate on this? Maybe I am missing some historical context, but it feels like Android is very much eating into Apple's market share, not into Microsoft's. How does this reasoning work out?
They did not have to do much though, Microsoft killed Windows Phone all on their own. Windows Phone 7 was pretty great, but Microsoft threw all early-adopters under the bus by then releasing Windows Phone 8. None of the Windows Phone 7 devices got an upgrade to WP8 and WP8 apps would not run on WP7.
I knew a bunch of folks who were very sour after how Microsoft handled WP7 -> WP8 and moved to other platforms. And these were the people that were championing Windows Phone.
That was actually the second time they through earlier users under the bus. I had a WP6 and was promised (at the time I bought it) that it would be upgraded to WP7 when that came out. Then Microsoft decided not to. Then they shit all over us by decided to shut down the app market. So my not even year-old phone became essentially worthless very quickly. I switched to Android and that was the last time I ever paid for a Microsoft product. It also turned me into an anti-advocate for Windows phones. I still cringe when I think of that perfectly good flagship (at the time) level hardware sitting worthless in a drawer
I am surprised that people are surprised about this.
iCloud runs mostly on GCP. All of the storage in iCloud relies on Google Cloud Storage.
PS. This is the reason why many people call Vercel, the Apple of the dev world. Vercel offers services that have great user experience, but under the hood are mostly wrapped AWS services.
Apple sending my data to Google would be one of the few actions they could take to make me consider going through the pain necessary to leave their ecosystem.
They take billions a year to do that with your search, and have been at it for decades. They collect the same amount, if not more, of personal data. I really don't see why people elevate Apple to higher standards there. Because they also sell high margin gadgets? They are a for profit, sit on a goldmine, and implement as many lock-in patterns to make sure you can't self host/escape this collection.
Part of the appeal of an iGizmo is not sending so much data to Google. I really doubt this is happening, and if it is, Apple will be hosting the models.
It shows encrypted streams that neither you or I can decipher. By default, either will spend its day scraping your surroundings (wifi and bluetooth) and report it back home, unless you opt-out (which both allow). Both are equally evil in my book.
Can’t you root both of those to setup a proxy to decrypt and see for yourself? If it’s encrypted, how can you tell that’s what it's doing to say with such certainty there?
On Android the certificate pinning makes it very hard even with root. On iPhone where the owner of the phone (Apple) actively fights against your ability to gain root, I can't imagine it's easier, but if it is I'd appreciate being corrected.
mitproxy lets you one tap install a config profile that does it. You know like you sometimes need to do in Korea or Kazakhstan... It's routine.
But I don't get you. You complained that droid makes it hard and apple makes it impossible. But it would be better for average user security if they could not do it (aka "did not own the device" in anti-apple propaganda), right?
The parent is right, though. Both Google and Apple send encrypted telemetry that you cannot MITM or decrypt a-la HTTPS or TLS. The average iPhone and average Android phone lights up like a Christmas tree in Wireshark - some of it can be reverse-engineered with TLS or DNS abuse, some of it is RSA encrypted against the hardware root-of-trust.
Apple's mea-culpa is that unlike Android they do not ship an Open Source OS ROM for developers to modify. Google's telemetry can be entirely neutralized by removing Google Play services and using Android without Google software. iPhones don't have that escape hatch, leading to a pretty literal limitation of how you "own" your phone and the software on it. On top of that, iOS has a permissions architecture Apple designed to give the user second-class control over the network. You cannot MITM Apple services - they will go around whatever user-land profile you think you've set. On top of that, there are modem emissions that you're never going to catch with a MDM profile hack and certificate pinning. You have fully drank the kool-aid if you think an empty aircrack-ng screen means "you won" against the multitrillion dollar company and coalition of government regulatory bodies.
> But I don't get you. You complained that droid makes it hard and apple makes it impossible.
I didn't complain about anything, I just stated the facts, with a possible exception regarding the snark about how Apple "owns" the device, although I do think that's a defensible position since they have higher access to it than it's "owner". I do think it's shitty though that they don't provide a way (even with some hoops) for the "owner" of the device to get the highest level of access to it, but that wasn't in the comment.
> But it would be better for average user security if they could not do it (aka "did not own the device" in anti-apple propaganda), right?
Why would that be better? I highly doubt it would make any difference at all to the average user. I doubt it even impacts the majority of power users.
The people who are impacted by these restrictions are the technical users who want to capture and inspect their own device's traffic, usually on their own network. Conveniently, these are also the researchers who might publish blog posts and articles about what kind of data and surveillance the device is sending home about the user, without their knowledge.
It would be better because if not then someone can turn it off. Automatically or by misinforming the user or by requiring it etc. Like now on ios you apparently just need to install a profile, maybe it's too easy.
Well an MDM profile isn't going to decrypt iCloud data or Apple telemetry. It's basically the same dangerous power your ISP and DNS provider wields, but nobody is about to suggest banning those for user safety too.
That's the point, indeed. Your ISP and DNS can technically intercept your traffic, but it's pointless since TLS exists. Similarly, you can Wireshark an iPhone using MDM profiles but Apple doesn't respect your profile in the first place. Third-parties have no obligation to show you their traffic either, and many don't.
They don't need to. I'm not sure if you're aware, but it's actually possible to encrypt traffic using things other than TLS. A regular app on non-jailbroken iOS can completely circumvent TLS decryption. First-party Apple Apps will bypass your profile and custom CA.
Ah. You are saying they would encrypt on top. Sounds inefficient but I guess reasonable if you think about people like Kazakh government or Korean institutions requiring everyone to add a CA just to live a life. So without extra encryption they could snoop on that too. We can't have nice things...
(It's still possible to compare how much a blank device phones home but perhaps we wouldn't know all the details of what it talks about)
> So without extra encryption they could snoop on that too.
This is basically the crux of your argument. I mostly agree with you - neither Apple nor Google do enough to protect user traffic in the big-picture. You can Wireshark a lot of data off both OSes, the throughput is even scarier when you track radio emissions.
That being said, a lot of people have taken notes from Apple's "protect user privacy" shtick. Many logging libraries contain the app-equivalent of screen-recording baked in to the app framework, enabling a pipeline where PII gets ingested as a part of the logging process. Startups that incorporate these processes then brag about their self-imposed security compliance as a result of their own ass-backwards philosophy. And these aren't even the bad guys!
Companies like TikTok and Facebook collect lord knows how much information, and use the same "security" tautology as their scrappy startup peers. They consciously stretch the limits of their API capabilities, and then turn around and make puppy-eyes whenever regulators act concerned. Meanwhile, the actual users of these smartphones aren't empowered to regulate their own device's security. They can't turn off their phone because the modem is still on. They can't firewall Facebook analytics when the app is closed. They can't even stop their notifications from being snooped on without disabling the feature altogether. Where's Apple or Google when that's under scrutiny?
It's a bit tangential, but this is why I think Apple made an enormous mistake attempting to commodity privacy. Privacy is idealistic - there will always be perennial exploits on the iPhone to prove them wrong. Because Apple commits to imperfect, conditional privacy, scummier-and-scummier companies can follow their imperfect lead and make the same claims. And because none of them are as big as Apple, they rarely take flak when their systems fail. Apple's attempts to market security is like watching a leading F1 driver start turning into a tailspin, and taking the rest of the racers with them in a firey crash.
I'm not sure marketing privacy is the same as commodifying it, if anything commodifying something makes it less marketable...
Most marketing hinges on non commodities, take coca-cola for example, it's sugar water with a bit of caffeine, if they marketed that they'd be nothing, all of their marketing is about other stuff, intangibles
Kind of like Netflix's proverbial "chill"
Comparatively I'd say privacy is among better things to market
I can't even tell whether it's sarcasm… All those services are closed-source, exchanging over binary protocols, of which there is no public description/documentation, and no stability guarantee.
You overdramatize, they mostly just push json around. mitmproxy is your friend. And since you only need to see for yourself once who cares about stability.
I share your attitude towards inspecting your devices’ traffic being an inaliable right, but AFAICT this hasn’t been the case for a while now.
I believe on Android MITMing even most third party applications (that make zero-to-no effort to prevent this) requires a rooted phone or an emulator running and older Android (8) without Google Play Services and doing a little bit of RE (for instance using some Frida user scripts to patch the apk to circumvent the certificate pinning). I reckon MITMing the actual traffic Google itself can collect would require a lot more RE and network wizardry than I’m even aware of (feel free to link some reading though). Here’s a recent walkthrough I saw in the wild: https://youtu.be/c4wS9n7yilA?si=xAfwCyWIzdrvOiHc
For Apple devices afaict since rooting was…ahem rooted out, no viable amateur-DIY methods for monitoring your devices traffic exist.
I know everything is open source if you’re good enough at assembly but at some point it’s gone from something a tinkerer can do to something you need significant talent and in-depth knowledge to do.
I’d love to read any write-ups or guides to the contrary though.
This is such a dramatic turnaround from the way Apple has positioned itself in the last few years you have to really wonder about their confidence level for in-house AI efforts.
An alternative read is that AI in its current form can produce such unpredictable results they're content for now to use Google as a scapegoat for any oddities.
If Apple were to partner with anyone, I would suggest Perplexity, unless there is something about the way Perplexity operates, its business model, etc. that may make it an unstable partner down the road.
Perplexity does have good competition in the ‘better web’ product space, but for right now Perplexity just makes searching and exploring more fun. For a few years, I think Apple would do well to have Siri partially wrap Perplexity services as appropriate.
Long term, I hope Apple does everything on device with their own models with firm privacy guarantees.
If AI features on mobiles get that important, like websearch was for the internet. Why should i buy an apple product if it uses the same assistant (or worse, as it runs native in the android world?)
Better integration? nicer or more powerful device? som exclusive apps or servcies? Same as today... Android phones do most of the same things an iPhone does but people who choose iPhone have their reasons to do so.
But currently only a small subset of Android devices come with all of Google’s current phone AI features out of the box, Pixel 8 Pro and the Samsung S24 lineup.
You can install the Gemini app on other phones, but last time I checked the install counts, it’s only a small percentage of the Android user base.
(I’d also be surprised if a large population would be willing to pay the $20 monthly Google One tier to use Gemini for private use.)
No, afaik, it's still a region locked feature. US only. Pretty convenient that all popular product reviewers are US based and advertise the Pixel 8 as an AI powered device.
NVIDIA chips are better for training due to the sheer prevalence of CUDA and NVIDIA being ahead of the curve in providing data centre hardware to scale up. They’re also not keeping stuff for just themselves.
When it comes to inference, most of that isn’t really relevant. TPUs or any other specialized hardware will have a higher ROI. But Google aren’t giving them to others either
I know you’re asking for a link, but it’s not really something other than Google can prove. However it is a reasonable extrapolation for anyone familiar with the product space to make.
I think there are different size Gemini models and some of them can fit on a phone with TPUs or similar and since each user will use their own phone, it offloads the computation.
That's primarily just because Google is doing a poor job offering them to people. Bad documentation, and few trust Google enough to want to tie themselves to Google's trusted hardware.
Also, Google is using all these TPUs for their own training and inference at the moment.
It takes a lot of effort to make the current version of a product much worse than the initial version. They did the same with the podcast app which used to be amazing, the new one I’m constantly fighting against because it keeps thinking I want to relisten to a podcast I just listened too, also the new version is slower and buggy.
>It takes a lot of effort to make the current version of a product much worse than the initial version.
My favorite example of this is how Siri used to give an audio confirmation that it is hearing me. This went away several iOS versions ago, and now I have to wait a few seconds then hope that I hear Siri's response; if not, I only then know that Siri did not hear my "Siri, ____" and have to start again. Even worse, maybe one out of ten times Siri does provide an audio confirmation at the start of listening! If there is some hidden iOS setting I can set to fix this, please let me know because I sure can't find it.
Siri only does that when it hears "Siri" and nothing else, so a wait is still required. When I began using Siri it beeped when it began listening for more after "Hey, Siri", so I knew immediately that I could continue speaking.
Apple put themselves in the corner with their privacy-first approach and local computations. To evolve AI you need to get all the data you can. It’s unsurprising that Apple progress was slow. And now they will pay to the Google who doesn’t hesitate to extract data from the users.
That's not the issue. LLMs are not trained on whatever crap you can scrape from users text messages or emails. That is relatively useless for training.
Wouldn't the issue be Gemini would need access to the users personal information in order to act on it? Yes there are applications where this wouldn't be needed, but I can't help but think that the number one application of ai would be adding to the capabilities of the personal assistant. To me a personal assistant that couldn't draw on the context of my location, calendar, or communications wouldn't be much improvement.
If you believe the memes, Siri is already quite capable of searching Google for you.
> If you believe the memes, Siri is already quite capable of searching Google for you.
How is that a meme? It casually searches Google and summarizes the result when required. Siri runs locally on iOS and needs to use a search engine to lookup information.
Is Apple really failing? $30B net profit per quarter. Who knows what they are researching behind the scenes. They can afford to hire any top AI researchers. Siri is optimized for cost and performance on small devices. Works very well considering that it runs locally on the iPhone hardware.
No one seems to have mentioned what is most obvious to me about this: if this goes through, Google will corner the market on mobile (on-device) AI. They already have Android users, this would give them iOS users too. Not a large market right now, but the way things are heading it could prove significant in the future.
This seems asymmetrically good for Google, less so for Apple. If Gemini proves to be an indispensable feature embedded in future iPhones it assists Google in branding and fine tuning. Meanwhile Google can do the same Android and lock down certain features for iPhone users
I feel like it will certainly need to be a hybrid approach for the moment. They’ll get a long way running small highly specific and tuned models locally on the neural engine for a lot of stuff. But, at some point, people are going to expect to ask questions and have something in the league of a Mistral/Claude/ChatGPT talk back and that’s just not possible with today’s hardware. Over time I expect more and more of that will get moved locally, and less will hit the “escape hatch” of throwing things to a giant LLM.
If they end up renting rather than building that capability, it says to me that they’re betting on being able to move most of that locally to the phone on a relatively short time horizon, which is interesting.
>Over time I expect more and more of that will get moved locally
I'm not so sure whether this is the direction of travel. The economics are pretty harsh for local, general purpose AI. And battery will always be a limiting factor.
If you're sending a few hundered requests per day to a very powerful AI then sharing the cost of the inference machinery with others has overwhelming economic advantages. And it will need access to current data anyway, so it can't be completely local.
There will be tasks such as keyboard autocomplete and a lot of other specialised tasks where latency or privacy is more important than quality. So yes I do believe in hybrid. But I think the cloud will always do the heavy lifting when it comes to more general tasks.
I can imagine an alternative scenario where a lot of AI processing happens on Mac and PC and mobile devices benefit from that. But many people don't have powerful desktop devices. So I'm not sure Apple can rely on this.
The article is pretty clear that it's going to be a service offered as a secondary option instead of core functionality.
>Apple is preparing new capabilities as part of iOS 18 — the next version of the iPhone operating system — based on its own AI models. But those enhancements will be focused on features that operate on its devices, rather than ones delivered via the cloud. So Apple is seeking a partner to do the heavy lifting of generative AI, including functions for creating images and writing essays based on simple prompts.
I'm still expecting this is the end goal. But I've started to question what inference on device will do to battery life. I suspect it will drain devices pretty quickly but if anyone has any studies to suggest otherwise please let me know!
I don't think this excludes doing their own thing. It would be in the Apple playbook to have more than one team working on different approaches internally and still negotiate with Google about a possible partnership.
I can't see how they can delivery 'thoughtful integration' with the Apple services when work is being done on Googles servers, but perhaps the "efficient inference on device" (which could be the biggest product leap for Apple since the iPhone) is taking too long and they need a stop gap to stop the downward pressure on their shareprice.
> Gemini has captured the imagination of consumers and businesses
Citation needed. I’m fairly certain this is all about scale. OpenAI and Anthropic have obvious capacity issues and Apple obviously doesn’t want to get in bed with Microsoft. It’s a smart bet short term if they get data on what people are actually doing with Gemini.
After Gemini's embarrassing "woke images" debacle last month, probably a lot of people wouldn't want it on their phones, nor on their desktops/laptops. Until Gemini is cleaned up and all the political biases removed, it's a tainted product, at least among a lot of vocal influencers, if not the masses.
I don't know anybody not in the tech world who even knows about that, let alone would care enough to change their platform. My guess is pretty much all average consumers, and 80% of technical consumers, don't care at best and actively applaud Gemini at worst. I doubt this is a problem at all
Imagine a world where every phone gets turned into a useful digital assistant that can order things for us online via APIs. I wonder if Amazon is ready for this.
I think the more interesting world is where AI assistants overtake the GUI as the primary interface for many operating system functions.
It's my belief that the lasting innovation of LLMs will be finally making voice a true first order interface for our computers. "We'll just be talking to them" is something I've heard since the 90s and it seems like we can finally see that reality on the horizon.
That sounds endlessly frustrating. Imagine trying to use your computer and the primary way to accomplish anything is to convince/prompt-engineer some two-bit LLM into doing it. First you have to convince it that what you're doing is ethical and doesn't violate its safety guidelines, then you have to get it to actually translate your natural language into the actions you want to perform, which will be more difficult the less common those actions are, and finally you need to check its action plan for any subtle errors or misinterpretation. It would be like operating a computer by way of a phone call with an annoying relative who doesn't quite know how to use one.
This will be the case until we have AI agents that are as intelligent and competent as a top-percentile human personal assistant. At the point where that's possible though, I doubt the future of phone interfaces will be the most interesting change.
When it comes to GUI, I can click faster than I talk. Sometimes typing would be a bit faster as well, in case you need to correct an error. Also, imagine a bunch of people talking all day long in an open space instead of mouse clicking/typing.
Brain to machine signaling interface is the logical step forward if we really want to be faster at input.
This is only true for the affluent people in the world.
Every person I personally know off will reasearch the shit out of the products including coupons available before making any kind of medium-high value purchase.
The digital assistant will need to be impartial and help in comparison with other similar product. Then again different people have different comparison criteria. Based on context I might be okay with a chinese knockoff while most of the time I might prefer the item that will last the longest.
> Imagine a world where every phone gets turned into a useful digital assistant that can order things for us online via APIs. I wonder if Amazon is ready for this.
Due to its size and infrastructure Amazon has advantages of choice, price, delivery and service. No digital (or human) assistant is going to be able to find a better deals on most items and then negotiate for it to be delivered tomorrow, free or cheaply, with a fairly good, no quibble, return policy.
Not at all. I'm lazy. Why do I have to open the Amazon app to search for what I want and click one button. Why can't Siri do it for me and take a percentage of the sale? Also I don't want to be limited to Amazon only so why not allow approved merchants to plug their backends into the search that Siri does?
I think Apple by now realizes that Android was a necessary defensive play by Google aimed not at them, but at Microsoft, so maybe this partnership could be revived. Probably going to be more difficult now in the current regulatory environment, however.