The even bigger hurdle is selling token based pricing to normal (non-dev) users.
"You pay an indeterminant amount of money to ask a question and you might not even get the response you want without spending even more money" doesn't appeal to most people who aren't gamblers and explaining how "thank you" at the end of a long exchange can be expensive due to context is an even harder thing for an average person to swallow.
Token cost going up/down like a yo-yo also doesn't help. Normal users NEED fixed costs and don't want to expend energy constantly keeping up with the AI meta. "My subscription lasted much longer last month" isn't a winning problem either.
I think Apple is correct that Local LLM for most things is the future.
It reminds me of the Chinese bike wars where everyone was slashing prices trying to keep marketshare until the bubble burst and everyone lost billions.
In the coding realm, I think we'll be seeing 35, 70, and 150B models sold where you pay a few hundred to a few thousand dollars up front and get a year of monthly/bi-monthly updates where they've trained it on new coding documentation and repos.
Humans have good spatial memory and having a handful of statically-positioned desktops in a 2D plane makes navigation intuitive and consistent.
The real issue is how the ORDER of the desktops changes all the time which messes with that spatial memory and kills a lot of the productivity improvements. A consistent straight line would still be worse than a grid, but still MUCH better than the current situation.
If AI could outperform humans, Anthropic would NEVER release that model. Instead, they'd use it to create a new google, photoshop, office, windows, etc for cheap then undercut all those companies and taking over the entire software industry.
It can outperform humans, just unevenly. You’ll see a lot of the same dynamics as you see with Mythos which, tbh is kind of refreshing. I get the sense that Dario while of course forced to ruthlessly run a company is genuinely interested in figuring out how to roll this out as ethically as he can.
I remember when React's vdom renderer was faster than anyone and it wasn't even close. InfernoJS uses a vdom and is still one of the fastest JS frameworks out there, so I don't know that diffing is always inherently slower.
They no longer accept world records for not sleeping because the record breakers have universally suffered lifelong cognitive damage.
We know more generally that people who get decreased amount of sleep suffer increased rates of physical and mental health issues.
It is not a very big leap from "causes permanent damage" to "enough permanent damage can cause death" and of course, keeping someone awake until they are hurt or killed is deeply unethical, so even if it could be proven in other species, you'd still be here arguing that 'they aren't humans".
We'll set aside performance because that could be fixed with a renderer rewrite.
The biggest issue when I worked with it was weaving a spiderweb of bindings that eventually trap you. At some point, you wind up spending most of your time fighting weird bugs that show up in places very far removed from the bindings that actually caused the bug.
"You pay an indeterminant amount of money to ask a question and you might not even get the response you want without spending even more money" doesn't appeal to most people who aren't gamblers and explaining how "thank you" at the end of a long exchange can be expensive due to context is an even harder thing for an average person to swallow.
Token cost going up/down like a yo-yo also doesn't help. Normal users NEED fixed costs and don't want to expend energy constantly keeping up with the AI meta. "My subscription lasted much longer last month" isn't a winning problem either.
I think Apple is correct that Local LLM for most things is the future.
reply