The fats in Chinese Water Snakes are rich in omega 3s and do have genuine benefits to consumption. The problem with snake oil wasn't that it was useless. The problem was with hucksters selling it as a cure-all for everything from cancer to syphilis. The metaphor is pretty apt IMO.

Exactly what I was thinking. It's not that snake oil sales people sold totally useless stuff, its just that the stuff they sold did not deliver the value that was promised. Another example that is still going on today. There is a community of people that swear the ingesting silver prevents all kinds of things, even so far as a cure for cancer. It's snake oil, but it doesn't mean it doesn't have any medicinal purposes. Silver does have anti-microbial properties and can be used topically to manage infections.

Yes, I got that Google summary too. But "snake oil" patent medicine didn't contain snake oil.

"Snake oil" refers to something sold as a medicament that has no beneficial effect.

Not at all, they can just use the petabytes of data sold on the cheap by American and Chinese companies instead of their own.

Not nearly as much as you might think. 1.2kw where I live translates to about $0.12/hr, and that's when running full clip. If you have a decent solar hookup, it's small fraction on a sunny day.

The expensive part is the upfront hardware cost and the electrical system upgrade you'll need to give your house.

> There are hundreds of ligands that interact with the same receptors.

Except the ligands matter, binding site is massively important to drug design. As is the behavior of the molecule beyond that. A 5HT2A agonist that's also an irreversible and potent dopamine agonist is obviously a non-starter. Minor modifications of a molecule produce wildly different and very unpredictable effects. Pharmacodynamics and pharmacokinetics are the bottleneck of drug research, and they take quite a lot of work to understand.

> We can now use powerful computers to come up with countless variations of drugs that activate the receptors involve and study them rapidly.

"Research chemical" is common parlance, and it's synonymous with "dangerous gutter drug" because you end up with nasty little molecules like what's found in the 25-NB or FLY families, or something like MDMB-CHMNACA. If it ain't broke, don't fix it. Our algorithmic and predictive power in pharmacology is one of the absolute worst out of all the sciences. The absolute state of this naive futurist mindset that we can move fast in drug research is absolutely horrifying to even suggest. That's not where the state of the art is, and I'd put big money on us not getting there for another 100k years or so.

Limitation is the wrong way to think about things when computational equivalence is in play. It's about mental foundation. Lisp at its core is like driving a Turing machine, Clojure is not.

AllegroLisp is very far behind SBCL.

AI hardware is for inference, not training. Training uses normal HPC crap. Superpods aren't really power efficient, it's kind of a meme, and it stems from limiting the power draw of other components by having less of them. It's more of a rounding error.

> you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.

Costs spread over a large population, it really doesn't matter. You're not getting hundreds of thousands of people to pitch half their monthly electric bill to pay for someone else's datacenter. They will pay the electricity themselves quite happily though, if all they need to do is give you compute. This isn't new.

Interconnect is the bottleneck for distributed training, nothing else really.

You got it wrong. Inference can use crap GPU's. Training needs the 100x more expensive big guns. Our training machine is 100x more expensive than our inference machine.

What I'm saying is those 100x more expensive big guns are just normal GP HPC howitzers. Systems that are exclusively designed for AI and nothing else are more or less all just edge inference TPUs.

How is the result of training stored? How big is that? It seems reasonable to assume we’ll eventually plateau and all we’ll need is relatively infrequent training.

I agree, leave the training to open source federations that roll out like operating systems. Minimal training over time.

Then have inference go down to the next layer to use those models as a P2P decentralized network.

Maybe like open router could tap federation networks.

Not so often. The GPU's are running 100% for 3 weeks for a training run. We do images only, but it's the same process. And then we can use the costly GPU's for inference, local model coding agents. Training is about 4x a year. But it depends what ideas the PM or the costumers have. If they has more, more training tasks. Eg. more viruses to detect.

> AI hardware is for inference, not training

Not sure what you are referring to, unless you don't think h100/h200/b200 are "AI hardware"

> Superpods aren't really power efficient

Maybe not compared to a specialized rig with multiple 4090s, but that is the best case for consumer hardware - the vast majority will be dramatically less efficient than that

Anyway, I agree the interconnect is by far the biggest obstacle and seems insurmountable, I should probably have led with that.

I wouldn't consider those AI hardware, no. They're normal GPGPUs.

That makes no sense. It’s basically the same calculations for training as well.

Bit of a doozie though, that one.

I recall getting really excited over hinton's FF foray, right before he bailed on AI as a societal direction (which, if anyone ever had the right, I suppose he does). If one squints, one can see a backprop-free base being much easier to train on geographically distributed and heterogenous hardware.

Are you sure most of frontier cost isn't inference in RL environments?

Deepseek's base models aren't censored.

Which is going out of control. Something not under control is out of control. If I jump out of a moving car, I deliberately relinquished control of it. It's still out of control. What a silly semantic game.

The last few iterations show a logarithmic curve at best tbh. If we are to see a major advance, it'll be something like the implementation and infrastructure for byte-level transformers.