I didn't down vote you, but you aren't really adding anything to the conversation. This type of pithy comment might be fun on Reddit, but at HN, we try to provide more constructive, and information rich, comments.
I have a similar rig, but a 10th gen i7, 128GB of DDR4, 2TB NVMe, and dual RTX 3090s. Built it in 2021 for crypto mining, it ended up paying itself off just before Eth went PoW. I kept it mining even after profitability, because it ran warm enough to heat my apartment leaving my HVAC on fan-only to circulate the heat around.
After winter, I started playing with various other GPU loads until LLMs and SD became easy enough to use. Now it's my experimentation machine.
It's already paid for itself, so anything I sell it for would be profit, but it is still super nice for running local LLMs that power various projects "for free".
I think a lot of the HIPAA compliance can be signed away when you authorize them to send your medical information over email/voicemail/sms, but I'm not a lawyer, and my doctor doesn't email me anything but a link to log in to their EPIC portal.
I'm not a very smart person, so take what I say with a grain of salt.
I think the path forward will have agents that use models that are individually specialized tasks (some might use a bigger model, some might use smaller models), then orchestrators that are good at knowing when to use which agent type.
I've played around with this in my own tiny coding agents, for TTRPG NPCs, and even a small experiment where LLMs controlled a MUD client as an NPC that played the game with you (only 5 rooms in the experiment).
Basically, break the tasks down into chunks so you don't have to use generalist models for everything, and can chose the right model for the job.
I'm also running all of this locally, where a generalist foundation model doesn't work, and heavily quantized models don't perform well for all tasks, so for unlimited token budgets, my solution is probably overkill.
"Orchestrator" pattern, "only use a big model to do big thinking, use smaller models to do grunt work" is probably what the field would converge to, eventually. Perhaps in form of "dynamic sparsity" - i.e. a family of closely related models allowing inference to transition from 1B class to 100T class on a dime, complete with something like joint KV cache.
But it's a hard pattern to pull off, so I'm not sure how soon we'll see it in action.
I agree that HN is "social media", but I'm starting to wonder if Facebook/Twitter/TikTok/Reddit/YouTube aren't "social media", but instead a new category of media tangentially related as OP posted to cable news. Something like "attention media" where your attention is the point of it.
HN, on the other hand, your attention matters less. They aren't paying for this platform using our "attention" necessarily. I'm sure it is a way to curate an audience of tech-enthusiasts where they can exploit our knowledge and push their investments in front of our eyes.
I like HN for that reason, I don't feel like I'm the product as much as with other attention-seeking platforms.
I don't have the data to quantify it, but M-F, 6a-4p, maybe once an hour or so just to check the headlines. If I have comments, I might check my threads to see if I need to respond. On the Weekends, though, I might check it in the morning and again before bed just to see if anything interesting happened.
But it isn't like YT (which is running in the background nearly 24/7) or Reddit, that I habbitually check. Those feel way more addictive. Same with Instagram, but I don't really care for short form content, so it doesn't capture me the same way as news and long form videos.
Well just to offer another data point, I check HN far more often than any of the others. Many times a day. I consider it far more addictive - there's usually something interesting, and scanning is low investment. Youtube requires headphones and willingness to block out the world for 15+ minutes at a time. Facebook just doesn't have much interesting in it anymore since friends stopped posting.
I feel like all of this is fine? HN is winning the attention game for a niche audience of people vaguely like me. TikTok is winning the attention game for other kinds of people. I don't understand why we have to agonize over this. What would you rather people spend their attention on? What would you rather spend your own attention on? Why don't you?
I agree that HN is addicting. When I have a blog post that I find on the front page, I get drawn in for multiple days straight, but I feel like it is a better/safer addiction. Like vaping vs cigarettes. Weed vs alcohol. Opium vs fentanyl.
TikTok is low-fat, high-sugar ultra processed "diet" food. HN is a fatty cut of steak with mushrooms and onions. Both are terrible for you everyday, multiple times a day, but one of them is arguably worse. But either one of them in moderation, is actually good for your soul lol.
I'm in a similar boat, and it annoys the shit out of my wife, and any family member that thinks I'm "just cheap" or a "budget shopper".
I'm a value shopper. I was happy with the McDouble when it was under $1, but not now that it is nearly $2. It isn't worth that, but the BK Double Cheeseburger IS worth it even for $1 more than the McDouble (at least in my location).
Same is true for almost everything I buy. A certain fabric I want in my button ups, or a stich type I want in my pants. I hyper inspect almost every item of clothing I'm even contemplating buying. Same with furniture, with tools, with appliances.
It is almost a compulsion. I cannot pay more for something than I think it is worth... unless I REAALLLLLY want it.
reply