jb55's avatar
jb55 _@jb55.com 2 months ago
did you know you can get the bitcoin whitepaper from your pruned node? Its stored in the utxoset permanently. Heres a one liner i created to extract it seq 0 947 | (while read -r n; do bitcoin-cli gettxout 54e48e5f5c656b26c3bca14a8c95aa583d07ebe84dde3b7dd4a78f4e4186e713 $n | jq -r '.scriptPubKey.asm' | awk '{ print $2 $3 $4 }'; done) | tr -d '\n' | cut -c 17-368600 | xxd -r -p > bitcoin.pdf This is why we prefer people use large OP_RETURNs instead of 947 outputs that can’t be pruned. OP_RETURNs are probably unspendable so they will never end up in your pruned nodes utxoset. They also produce smaller blocks than inscriptions. Unfortunately they are 4x more expensive… but lifting the filter restriction is one small thing we can do to encourage it over other options that are much worse for bitcoin spam-wise. You definitely don’t want illegal, unprunable stuff in your utxoset permanently, which is why i am running core v30 to do what i can to disincentivize this spam.

Replies (124)

jb55's avatar
jb55 _@jb55.com 2 months ago
even with full archival nodes, if you don’t have txindex on you won’t be able to easily extract the op_return from a txid.
Jeff Swann's avatar
Jeff Swann 2 months ago
Having to download & verify the data to begin with is the real problem. Encourgaing one type of spam to prevent another just seems retarded. We should just find ways to discourage all spam & work to decentralize mining so that bitcoin is used as it was intended.
You’re right. Just all mine need to be full nodes. lol It might raise the hardware barrier, but you still have to use IBD and prune after the download finish’s anyway, it’s the same requirement up front. Less IBD nodes increases centralization and relies on more trust. Saying “we’re making this change, you can just prune” is the worst for decentralization of the nodes. It’s not theoretical. We’ve already seen it! Wallets default to public Electrum servers. People syncing through Blockstream.info. Light clients like wallet-as-a-service models. Some of these users don’t even know that they’re using a trusted service. These are fine options but become points of failure when only option for a node is “just prune it dude”
jb55's avatar
jb55 _@jb55.com 2 months ago
I am constrained by the properties of the system today and am encouraging the least bad way to move forward. of course i agree if there is a new proposal that fixes this i will be interested, but as of today any restrictions on op_return will only push people to worse spam variants.
jb55's avatar
jb55 _@jb55.com 2 months ago
of course it is, not everyone can run a full archival node. Making it easier for more people to run nodes -> more decentralized. Pruned nodes (aka Full nodes) are still fully validating, they just don’t store all the blocks.
One of the things keeping node runners interested and leveling up their skills is running services adjacent to the node, like block explorers and lightning. Without indexing, pretty much anything interesting you want to do with a node is off the table.
jb55's avatar
jb55 _@jb55.com 2 months ago
of course it does, it reduces the storage requirement, the most important requirement. Not everyone has terabytes of storage to spare on their device.
it doesnt though, you still have to download the whole thing before you can prune. You still need terabytes of storage to start a bitcoin node. Once its download the requirements change. But to join the network, its the same hardware requiremnent for IBD w/ Prune or IBD w/o prune
jb55's avatar
jb55 _@jb55.com 2 months ago
this is false. you do not need terabytes of storage on a pruned node. you need the same amount of network *bandwidth*, sure, but pruned nodes prune as they are downloading. They only keep as much as they need to rollback incase of a reorg.
SatsAndSports's avatar
SatsAndSports 2 months ago
If you really care about miner decentralisation, you would: Encourage the use of the open-source StratumV2, not Ocean's closed-source Datum system and you would help to ensure that mempools can accurately predict the next block, making it easier for small miners to stay at the tip and propagate their new blocks
Man.. you've lost the plot, honestly. Alright, Will, I'll definitely respect your argumentation. Keep going at it for sure. Its the way things should be. My only wish/advice is, you also take an honest look at the reaction from node runners and take that into account as you support a change on core. If you ask me, looking at the reaction from an HONEST point of view, it tells me a substantial amount noderunners disagree with whatever logic you and core devs are presenting. And despite whatever you might think, "a substantial amount of noderunners" disagreeing on a changr should raise serious concerns about that change.
jb55's avatar
jb55 _@jb55.com 2 months ago
how have i lost the plot exactly. Most people disagreeing with me don’t seem to understand the technical nuance of the spam issues at play. I believe utxospam is much worse than op_return spam, so i am acting accordingly. If “pleb” noderunners disagree then so be it. They can continue living in an alternate reality that doesn’t affect me.
That doesn’t make sense. You have no control over the child porn that will be shared in OP_RETURN over your node when running core. The same thing happens when you run a TOR relay.
Serving Bitcoin's avatar
Serving Bitcoin 2 months ago
@jb55 is correct. Pruned nodes can work with all kinds of sub-1TB disk sizes. We even sell refurbished pruned nodes that have only a 256 GB SSD that work quite well. image
Judge Hardcase's avatar
Judge Hardcase 2 months ago
"I'm doing my part to disincentivize the KKK from having public parades by encouraging them to rent a private hall at 4x the expense - so that the rest of us don't have to see." Solid plan. 👍
The "lost the plot" part was directed at the note I replied to. Obviously all nodes being full nodes is not bad for decentralization. I think/hope your rationale is -- if all nodes HAVE to be full nodes --> there would be fewer full nodes --> which would be bad for decentralisation. But your phrasing is easily misinterpreted to something that sounds like people should prefer to run pruned nodes. Thats the way I read it the first time. My point with respecting the signal from a substantial amount of noderunners is, since the problem the proposed change seeks to fix and the response is as controversial as it is, no change should be implemented. A better fix should be given time to be thought of.
jb55's avatar
jb55 _@jb55.com 2 months ago
is this the contiguous data argument? because my one liner above pulls and combines data non contiguously. It’s a pretty weak argument.
You are getting closer to a valid argument, but it remains incomplete. Filters still provide resistance to spam, making it more expensive the more nodes filter it. Your argument is essentially: "Since Core doesn’t filter spam in OP_RETURN or the UTXO set, and UTXOs are less efficient for the network, then spammers should be allowed to use OP_RETURN." The real question is whether spam should be filtered. The only counterargument presented is "filters don’t work," which is a red herring. The argument against filters that do work is that they are centralizing, but this claim is made without evidence. Yes, IP RBLs are centralized, but they are an ancient and ineffective solution, especially in the era of Tor. Bayesian filters are far more effective, though they can be poisoned; again, an old problem. The fact that we don’t have perfect filters is not an argument against using filters at all. The assertion that every mempool on every node must be identical is presented without justification, and that requirement itself is a centralizing force. This leaves us with: "Core devs are dedicated and underpaid and shouldn’t have to argue with non-experts; they should just do what they’re good at, which is writing software." This is the most pathetic argument of all. It relies on argumentum ad authoritatum and argumentum ad passiones, and implies that we should entrust our future and fortune to people who are wholly socially inept. Not only is this a highly questionable request, it also leaves open the possibility that these same developers might be unable to defend their "technical" positions against more cunning and malevolent influences; whose presence we are fully aware of.
Judge Hardcase's avatar
Judge Hardcase 2 months ago
Not at all. It's the offering a path for bad actors to sacrifice their own self interest for the sake of community health is pure hopium argument.
jb55's avatar
jb55 _@jb55.com 2 months ago
if bad actors wanted to do harm at their own economic expense they would just put them in the format above, which filters wouldn’t stop. If they wanted to be economical they would just use inscriptions. both cases have nothing to do with op_return
Judge Hardcase's avatar
Judge Hardcase 2 months ago
So, it sounds like we're in agreement: since neither have anything to do with op_return, lifting op_return filters has nothing to do with disincentivizing non-standard use of inscriptions (i.e. bad actors).
jb55's avatar
jb55 _@jb55.com 2 months ago
yes i agree, which is why i also believe lifting op_return is a very small incentivization tweak for lazy, non bad actors to use op_return for their app protocols instead of something dumber like multisig pubkey outputs
Judge Hardcase's avatar
Judge Hardcase 2 months ago
I see: being too "lazy" to design your app protocol not to damage the health of the network qualifies as a "non bad actor".
Judge Hardcase's avatar
Judge Hardcase 2 months ago
At best, it's willful neglect - which is a form of malice; not adequately explained by stupidity alone.
jb55's avatar
jb55 _@jb55.com 2 months ago
this is the conundrum. to stop this multisig transaction would require you to censor a valid bitcoin transaction.
That doesn’t really respond to the preference that Bitcoin be a monetary transaction protocol for the peer-to-peer electronic cash transfers. If someone wants to play with blockchain file storage, Solana looks like a great option.
BTC_P2P's avatar
BTC_P2P 2 months ago
Thank you for continuing to educate. It’s helped me as a non-technical understand this entire situation much more thoroughly.
jb55's avatar
jb55 _@jb55.com 2 months ago
not really, unless you want something to be super censorship resistant and have it replicated across many computers across the world. you just have to pay a decent amount of coin to do this.
IMHO that means monetary transactions only. No image or other non-monetary data warrants the bloat created down the road on the main chain. Even the Bitcoin white paper doesn’t need to be imbedded in the Bitcoin time chain. Just because something can be done, doesn’t mean it should. It helps to think long-term and all users. Hardware requirements for node runners has already become much more expensive and 3rd world plebs deserve their financial sovereignty just as much as we do.
Your technical arguments appear to me to be sound and worthy of very serious consideration and debate. However, it’s important to acknowledge that many in the community feel Core initially dismissed their concerns and then doubled down, which fostered division and mistrust. While the technical rationale holds and is reasonable, the perception of arrogance has fueled ongoing controversy. Many believe Core leaders could have done more to build consensus and dialogue before pushing a fracturing change. Core devs feel insulted, attacked and unappreciated, personally threatened. This is more a community, political, leadership and governance issue than it is a technical issue. IMHO.
The question is if there will be a significant demand for spam if you need to find workarounds like this because there is no standardized easy way to do so. Maybe it’s not a good idea to make spam more easy.
WildBill's avatar
WildBill 2 months ago
Haha still can’t be objective, can ya? It’s not hard. Just remove emotion and compare objective sets.
WildBill's avatar
WildBill 2 months ago
Kaspa solves this and he knows it. He is too prideful to admit it.
WildBill's avatar
WildBill 2 months ago
Kaspa solves this and he knows it. He is just too prideful to admit it.
jb55's avatar
jb55 _@jb55.com 2 months ago
Let's recap: - they were technically right - misinformation campaign to discredit and shame them publicly was initiated - this noise completely overwhelmed Bitcoin dev process and distracted people for months if anything all this did was to further separate dev from angry mobs. not sure it accomplished it's intended goal you described. best way to make change in the dev process is to become involved, gain credibility amongst current devs, and then put in your well informed technical review on these changes. this is what everyone else involved in the dev process has done and will continue to do so, even in the presence of misinformed angry mobs.
Szabo has a degree in computer science and a degree in law. He says: "illegal content in a contiguous standard format, thus readily viewable by standard software, is more likely to impress lawyers, judges, and jurors, and thus is legally more risky, than data that has been broken up or hidden and thus requires specialized software to reconstruct. Such demos would be part of convincing these legal decision-makers that a defendant node operator had knowledge of the content." And remember, this issue is not 100% technical and cannot be fully understood using technical arguments only.
jb55's avatar
jb55 _@jb55.com 2 months ago
he is wrong, my post above shows that its just as easy to extract data from non-contiguous sequences. the "readily viewable by standard software" is my one liner above. it doesn't require specialized software.
jb55's avatar
jb55 _@jb55.com 2 months ago
i did not have custom or complex software in this line. everything here is readily available on any linux machine running bitcoin. this is such a weak argument.
Your line is gibberish to a lawyer. Would an Op_Return extraction be simpler? Yes. Would it require fewer Linux utilities to extract? Yes. Is it easier for less technical person to accomplish? Yes. Case closed.
jb55's avatar
jb55 _@jb55.com 2 months ago
here is the equivalent op_return one liner bitcoin-cli getrawtransaction 6ae4de7542bebb0884b08db8265d7567b27673034da179980c632b8d592ffe9b true \ | jq -r '.vout[] | select(.scriptPubKey.type=="nulldata") | .scriptPubKey.asm' \ | sed 's/^OP_RETURN //' \ | xxd -r -p I don't think its much different.
jb55's avatar
jb55 _@jb55.com 2 months ago
and even this one requires txindex, which most nodes don't enable. so you could argue the utxo one is simpler and more universal among nodes. this argument just doesn't work man.
several related applications (etectrum, lnd, etc) require non-pruned node. If your node is pruned you need to trust on a third-party non-pruned node or service ! do you think this is good for decentralization ?
But it doesn't make it any more expensive because more nodes filter it... Nearly every node filters dust. I made a dust tx that paid normal fee to prove this and it went through just fine.
Thank you for providing this. Sorry for a late reply, but after looking it seems this directly targets inscriptions and ordinals. ie. this is a cat and mouse game. We can do that and break ordinals and inscriptions... until they change their protocol to just do it a different way. Then devs would have to explicitly update DatacarrierBytes() with another case to catch it. like ad blockers... They work until the ads use a different scheme and the blocker no longer works until the devs implement a new blocker. It's not a real solution to the problem. This is just whack-a-mole imo. What’s the real “fix” A consensus rule that says: “No witness stack element may exceed X bytes, no witness script may exceed Y bytes, no input may have more than Z stack elements.” But this requires a hard fork.
jb55's avatar
jb55 _@jb55.com 2 months ago
i think more people running nodes is a good thing. once they are comfortable running a pruned node they can choose to upgrade to a more fancy setup like archival nodes and lightning nodes + block explorers.
of course more nodes are always better, but with terabyte ssd getting cheaper and cheaper, running a pruned node isn’t worth it given the current size of bitcoin’s full blockchain. Don't you think so ? Another thing is if the blockchain size explodes due to some kind of "vitalikization".
WildBill's avatar
WildBill 2 months ago
Well that’s the disagreement, isn’t it?
Brunswick's avatar Brunswick
@calle Calle, I appreciate the clarity and depth of your post. I agree with you on the importance of assuming good faith. Everyone here is trying to strengthen Bitcoin, not weaken it. But the changes currently being made around OP_RETURN size and relay fee policy are not being carefully examined. They are introducing new problems that are more immediate and concrete than the hypothetical problems they are claimed to address. You frame the issue primarily as a question of how to prevent blockchain spam, with fees as the only decentralized answer. That misses two key layers. First, the blockchain itself already has structural spam filters built into the protocol: the block size limit and the 10-minute block interval determined by difficulty adjustment. Those are what prevent infinite transactions, not fees. Fees determine priority within the fixed block space, but they are not the mechanism that sets the boundary. The size and timing constraints were deliberately chosen so that individuals could afford to keep a full copy of the chain on inexpensive hardware. This was the original economic model, and it remains central to Bitcoin’s decentralization. Second, you are not addressing the relay mesh. The change to allow OP_RETURN data up to 100 kilobytes per transaction, combined with the sub-sat per vbyte relay policy introduced in v29, creates a completely new class of abuse. It is one thing for someone to pay a miner directly to insert 100 kilobytes of arbitrary data into a block. That has always been possible by paying a mining pool directly. It is another thing to let anyone use the peer-to-peer relay mesh as a free content distribution network by pushing 100 kilobyte transactions through the network at negligible potential cost, knowing they will never be mined. Before v29, the minimum relay fee was around 1 sat per vbyte. This meant that using the relay network carried a real economic cost. Lowering that threshold to 0.01 sat per vbyte or less removes the cost. A spammer can now broadcast large transactions for essentially nothing. These transactions will sit in mempools, consume RAM, eat bandwidth, and crowd out legitimate activity, without ever reaching a block. There is no fee market mechanism that solves this, because the spammer is not actually paying for block space. They are abusing the transport layer itself. This is not just a blockchain problem, and it is not just a future problem. It’s a relay layer problem today. If this policy remains in place, the peer-to-peer network becomes an unpriced commons, and like every unpriced commons, it will be abused. The bandwidth and memory requirements to run a node will increase dramatically. Ordinary node operators will be forced out, and centralization pressure will increase. That is the opposite of what Bitcoin’s design intends. Your analogies to CAPTCHAs, logins, and ad blockers are also misplaced. Those are rate-limiting mechanisms for identity and access control in centralized systems. OP_RETURN filtering and relay policy are content filters. A better analogy is email spam filtering. Each mail server chooses its own rules. There is no central authority dictating identical behavior. Nodes deciding what they are willing to relay is not central planning. It is local policy and a critical component of node sovereignty. There is also a larger pattern at work here. Developers, by nature, tend to look for technical solutions to potential future problems. But the market is not calling for these changes. The problem being described is not present today. The solutions being introduced do not solve any actual problems, but they do create new ones and worsen existing issues. We are not facing a flood of legitimate transactions that can’t get through because relay fees are too high. What we are doing is opening the door to large-scale abuse of the relay mesh and the blockchain without any clear benefit. I also want to call attention to something in your post that appears indirectly, but is important. Your closing argument is structurally the same as Peter Todd’s tail emission argument: that someday in the future miners may not have enough incentive, so we must change the protocol now before it’s too late. This is a speculative future scenario. We don’t know if it will happen, when it will happen, or what the economic environment will be if it does. Changing fundamental network policies today based on hypothetical future conditions is not sound systems engineering. If such a problem arises, the tools and circumstances available in that future will almost certainly differ from those we have now. Solving a future potential problem with present tools is not only uninformed by definition, it is most certainly wrong. Developers are important, but they are not the sole authority on how the network should function. The market is. And the market has already voted, through real use and sustained preference, for the current structure of the Bitcoin blockchain and relay policies over thousands of alternatives. The strength of Bitcoin lies precisely in its ability to let market forces determine what works, not in preemptively changing fundamental parameters based on speculative scenarios. We should not make these kinds of changes lightly, and although you may have been involved in the discussion for two years, the discussion is not over. The decision is not final, nor will it ever be. We already have open-source forks, more than one viable alternative with more to come. Allowing 100 kilobyte payloads combined with sub-sat relay fees creates immediate, predictable problems, while claiming to address problems that may never materialize. The proper course is to let the market surface real issues, then address them when they exist, not to introduce new vulnerabilities based on hypothetical futures. Respectfully, the changes being proposed do not solve real problems. They create them. View quoted note →
View quoted note →
jb55's avatar
jb55 _@jb55.com 2 months ago
and it still doesn't work, since it would be easy to get around with librerelay. lots of wasted dev time for nothing.
So you answered the question right? Spam was under control until the taproot change, and inscriptions in 2023. IMO- It was another spam event, which was handled before in Bitcoin history: satoshi dice, colored coins, and counterparty. All blocked by policy filters and SOCIAL DISINCENTIVES The difference today is the core dev team redefined the documentation and pretended a bug was a feature. Plebs are now saying enough is enough.
Let me be clear, I have great respect for you and everything you’ve done for NOSTR and Bitcoin, as well as the other core devs and leaders in the bitcoin community. You can be technically right and still struggle to manage perceptions. When perception and reality don’t equate, perception becomes reality. I get that this Is far more personal and raw for you than it is for most and I understand why you feel attacked and betrayed by a large chunk of the community. My only motivation is to try and encourage people to consider others perspectives and try to reconcile where possible. I understand that may not be possible in many cases and that people will do so on their own time. I appreciate your willingness to engage with conversation. I hope that people will step up to help manage perceptions and that our community can come to a consensus soon.
Apiarium's avatar
Apiarium 2 months ago
If everyone is using only pruned nodes, new nodes won't be able to download the blockchain, bitcoin will die
You get to decide your own semantics, but for yourself only. I'd argue its silly to think pruned node = full node and full node != full archival node. Why have a mode called pruned then! Obviously full node = full archival node and PRUNED NODE IS PRUNED and cannot serve IBD.
jb55's avatar
jb55 _@jb55.com 2 months ago
full means fully validating. Its not “me deciding my own semantics “. Its literally what it means
Jeff Swann's avatar
Jeff Swann 2 months ago
Storage is not the most important requirement, bandwidth has always been the primary concern. Real decentralization depends on the number of full nodes. If we are not lowering the barrier to full node operation then we are not improving decentralization.
Cody's avatar
Cody 2 months ago
Hmm I'm not sure I agree here. Yes in places with shitty Internet bandwidth is more of a concern, but I the barrier to entry(device cost) doesn't increase with bandwidth, but it does with storage. I also don't think most people are turned away by either of these constraints. I think most people don't care or transact enough to bother running a node.
jb55's avatar
jb55 _@jb55.com 2 months ago
storage has always been the main concern, jeff is just completely wrong here. you have to download the whole chain regardless of pruned or unpruned.
Jeff Swann's avatar
Jeff Swann 2 months ago
Utxo spam which is cheaper to produce will not go away because you invited op return spam, you will get more of both.