Super Testnet's avatar
Super Testnet 2 months ago
Does anyone know of research along the lines of the following assumptions? On an average day in bitcoin, I assume most of the utxos consumed are relatively recent, and most of the utxos created are soon consumed. How much memory do you need to store 100% of them? 90%? 80%? 70%?

Replies (5)

Super Testnet's avatar
Super Testnet 2 months ago
Yes. This is the first project where vibe coding has proven helpful to me: I used an LLM to implement my protocol for checking if a pruned utxo is unspent, and it seems to work. So now I'm thinking of making a node implementation that lets users prune the utxo set, and when that setting is used, I'm wondering what amount of memory I should allocate to store the most recently created utxos. The more you keep, the less network queries you have to make. Perhaps I should just do some tests with different memory allocations and make a chart to visualize the results. This protocol would massively slow down IBD if it prunes too aggressively, but I bet it would be a decent tradeoff if the numbers come out to something like, "you need 500 megabytes of memory to store 99% of the utxos that get created and consumed in a given day, and 1% of the time you'll have to do a network query because a pruned utxo got spent."
Eric Voskuil is probably someone who knows this stuff since he's working on a alternative node implementation.
anything that a mini pc or a $5 vps could run would be great. so that's the limiting factor then I'd shoot for 1GB memory or less. If so then the 500MB should be sufficient, if not then tap it up to 750MB. Bitcoin is super efficient compared to other modern forms of computing but i won't start ranting here :D and yes i agree the. more you keep the less queries, caching like a sweet spot