It should improve agents' action selection by allowing them to evaluate actions' effects before performing them.
An agent using only a regular LLM has no real way to predict the results of its actions. It has to just take an action based on its training data and hope it's the right one. With a world model like this, it could do a second pass before each action to catch mistakes.
I don't know if this actually delivers yet, but if it does it might help make agents more usable.
One can only imagine how much money Intel would have made from Optane during the ongoing RAM shortages. It would be absolutely perfect for warm KV cache, and potentially good for MoE expert offloading.
The silenced errors aren't guaranteed to be memory leaks or use after frees. There are some situations where memory is being handled properly, but the borrow checker isn't able to prove it.
One example might be a tree-like struct where a parent and child have references to each other. Even if everything is cleaned up properly, the borrow checker has no way to know that when the struct is created. Solving it requires unsafe at some point, usually through something like RefCell.
If speed is a concern, why did you all stick with Synapse (essentially single-threaded due to the GIL) over moving to Dendrite? As far as I can tell, Dendrite is, for all intents and purposes, abandoned.
Because we didn't have enough people or cash to do a good job of simultaneously writing two servers, and as Synapse had gone into production across *.gouv.fr and other critical deployments, we instead frantically backported Dendrite's main novelties to Synapse - adding instead worker processes to Synapse so it could easily scale beyond the GIL: https://matrix.org/blog/2020/11/03/how-we-fixed-synapse-s-sc...
The hope was always that we would then get back to Dendrite and be able to invest in it and migrate over, but the cash situation got worse in 2022 due to Matrix being more and more successful (causing the available $ in the industry to be flow to integrators rather than the upstream project), and instead we had to essentially park Dendrite dev in 2023 other than for critical fixes.
Meanwhile, to try to fix the $ situation, we added Rust workers to Synapse as "Synapse Pro" to give customers a reason to actually route money to us as the upstream project, and nowadays Element is actually on a more economically sustainable path. However, at this point the likelihood is that rather than progressing Dendrite we'll instead look to add more Rust to Synapse and fix its resource usage. That said, others are of course very welcome to continue progressing Dendrite forwards, and I personally find it super depressing that we failed to progress both servers at the same time.
Matrix team is doing a solid job of running - Keep it up and keep eating the Slack/Teams marketshare up with competitive features and pricing. Additional business considerations like HQ location costs, tax liabilities, and talent pool availability on paper also affect what you have to work with. London tax, talent, and labor pay versus Austin for example.
Also I got your name wrong last time - I apologize for that.
What’s the best way to fund development of this stuff? I’m aware of donating to the matrix.org foundation, but as far as I can tell none of that goes towards funding server and client implementations since those are Element instead of the foundation.
> the likelihood is that rather than progressing Dendrite we'll instead look to add more Rust to Synapse
I thought the goal of Dendrite was decentralization done right? Namely, the ability to run a homeserver from the very phone one is using the client on?
Dendrite did subsequently switch to powering the P2P Matrix work… which also got paused in 2023. We’re currently resurrecting it, but it’s not clear whether Dendrite will be the clientside server impl.
As someone who as attempted to use React Native for Windows, I can tell you that the "native" XAML doesn't make things any better. If it was using web technologies I wouldn't need to manually modify RNSVG to fix segfaults when an SVG goes offscreen.
I'd use caution with the Mi50s. I bought a 16GB one on eBay a while back and it's been completely unusable.
It seems to be a Radeon VII on an Mi50 board, which should technically work. It immediately hangs the first time an OpenCL kernel is run, and doesn't come back up until I reboot. It's possible my issues are due to Mesa or driver config, but I'd strongly recommend buying one to test before going all in.
There are a lot of cheap SXM2 V100s and adapter boards out now, which should perform very well. The adapters unfortunately weren't available when I bought my hardware, or I would have scooped up several.
I've seen the sxm2 (x2) with pci extension cards out on ebay for like $350.
The 32gb v100s with heatsink are like $600 each, so that would be $1500 or so for a one-off 64gb gpu that is less overall performant than a single 3090.
An agent using only a regular LLM has no real way to predict the results of its actions. It has to just take an action based on its training data and hope it's the right one. With a world model like this, it could do a second pass before each action to catch mistakes.
I don't know if this actually delivers yet, but if it does it might help make agents more usable.
reply