What's happening with Gossip?:
Implementing NIP-46 support in gossip has been far more difficult than I expected, at every step of the way, to the point that I've become dejected. I'm primarily motivated by progress and currently I am demotivated by my lack of completing anything useful for the end users, and have become too easily distracted to the point that I think it would be fair to send some bitcoin back to OpenSats as a refund.
For a long time there was a rust compiler bug around lifetime erasure ('_ vs '0) of data sent across async points. But I noticed that wasn't happening about a month ago.
Then all the signing/encrypting/decrypting functions needed to be made async, which bubbled up to more and more code needing to be async.
Then I went ahead and defined a Signer trait implemented by anything that could sign things... but that had too many functions (NIP-46 only does some) so it had to be broken up into 3 or 4 traits over time.
Then I built an Identity which could be any of (None, PublicKey, PrivateKey, RemoteSigner), and also a gossip client identity that wrapped this. And it had to be Serialize/Deserialize to save it.
Then I needed a network-level Client that a nip46 signing client could utilize to communicate with it's remote signer, without being tangled up in the "minions" I currently have.
Then I needed the nip46 signer itself that uses this Client and implements Signer, plugged into Identity and Serialize/Deserialize.
Then I needed to migrate how your identity loads and saves to this new serialized Identity (instead of just keys as it used to be).
Then there were lots of bugs around where detection of whether you could sign things or whether things were unlocked was being done wrong.
It starts to work... you can connect to the signer and you can learn your public key, and you can then get a feed. But it quickly gets jammed up and gossip freezes.
So now I think I should use tokio_console to debug it, which requires me to instrument lots of code.
And I'm just so tired of this.
Replies (13)
this is my exact fear about switching to async signing in notedeck, it looks like you are confirming all my fears.
Just hang tight. I just had a 2-3 weeks debug session behind myself to find a recursive rendering issue that was eating all CPU. We've all been there. Sometimes our job is hard, but that is part of it. Don't give up, just keep grinding and whenever you feel, reach out to your peers for help.
Mike Dilger ☑️
What's happening with Gossip?:
Implementing NIP-46 support in gossip has been far more difficult than I expected, at every step of the way, to the point that I've become dejected. I'm primarily motivated by progress and currently I am demotivated by my lack of completing anything useful for the end users, and have become too easily distracted to the point that I think it would be fair to send some bitcoin back to OpenSats as a refund.
For a long time there was a rust compiler bug around lifetime erasure ('_ vs '0) of data sent across async points. But I noticed that wasn't happening about a month ago.
Then all the signing/encrypting/decrypting functions needed to be made async, which bubbled up to more and more code needing to be async.
Then I went ahead and defined a Signer trait implemented by anything that could sign things... but that had too many functions (NIP-46 only does some) so it had to be broken up into 3 or 4 traits over time.
Then I built an Identity which could be any of (None, PublicKey, PrivateKey, RemoteSigner), and also a gossip client identity that wrapped this. And it had to be Serialize/Deserialize to save it.
Then I needed a network-level Client that a nip46 signing client could utilize to communicate with it's remote signer, without being tangled up in the "minions" I currently have.
Then I needed the nip46 signer itself that uses this Client and implements Signer, plugged into Identity and Serialize/Deserialize.
Then I needed to migrate how your identity loads and saves to this new serialized Identity (instead of just keys as it used to be).
Then there were lots of bugs around where detection of whether you could sign things or whether things were unlocked was being done wrong.
It starts to work... you can connect to the signer and you can learn your public key, and you can then get a feed. But it quickly gets jammed up and gossip freezes.
So now I think I should use tokio_console to debug it, which requires me to instrument lots of code.
And I'm just so tired of this.
View quoted note →
@ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ does #realy and #bunk fix this? 🤔
well are you burned out or just feel like it's too much work ?
if burned out - take a break ( work on something else ).
if too much work - teach somebody how to do the work for you.
NOT ME ! ! !
you guys need to get better at collaborating.
what if you get a stroke tomorrow - will Gossip die ?
you need to learn to delegate to where you only provide the vision and others do the work.
Everything you mentioned is exactly what the last 5 months of work on noStrudel was for me. I already had everything async (because nip-07) but I still had to massively refactor how accounts worked.
Although I think I came out ahead since I ended up building a reusable library
Well I didn't intend to confirm fears, I was trying to make a confession. But the chips fall wherever they fall. If the UX isn't terrible as it stands, you might be better steering clear of this. Rust as a lot of "safety" but the compiler doesn't help you with deadlocks and other lock contention issues.
It makes sense that if you started with nip07 and async, then nip46 isn't a big leap. Also javascript isn't persnickitty about "ownership" and "lifetimes".
Curious, what are the main UX hits to not implementing async? And are they mainly in play if there's a long wait for the user to approve on their end? (Not so much in play if just an extra second or so of latency?)
I use #gossip daily, and really appreciate it. Sadly, I don't know rust (or whatever else is used) so I can't really help--though I tried, specifically to help with NixOS, but it is beyond my capacity and bandwidth. Hope you stick with it because it's a good client. Thank you. 🙏
Still using gossip, it's a great client. Reading the NIP-46 spec, i can see why it could be tricky to implement in an egui client.
I think I've found the problem. It wasn't that bad. But still the whole process was kinda long.
The UI is on it's own thread and shouldn't be getting blocked, but there are probably some statements in there where it thinks it's making a quick function call but due to a bug that function is not returning.
If I didn't implement async in the main code, then everything would be on the same thread, using a single CPU core, and all activity would be linearized... I'd have to wait for one relay to respond before I could ask the next relay. So async is vital in general. If just the signer wasn't async for its NIP-46 part, all operations like sign, encrypt and decrypt would block until the nip-46 relay replied, and that would slow things way down, but the UI wouldn't freeze.
