jb55's avatar
jb55 _@jb55.com 2 years ago
I’ve been playing with the new nostrdb query engine and this gave me an idea:💡 When you get text search results, it would be neat if your timeline jumped to that point in time. Kind of like when you search for old things on telegram. In other news, i finished tag queries in nostrdb today, finalizing the last thing I needed for the switchover to nostrdb in iOS. Going to start on this tomorrow 👀. I started building nostrdb 6 months ago, turns out building a high performance database from scratch in C takes some time, what a wild ride it’s been. Now the cool stuff begins 🤠 Things that are possible now: - contact list restoring - fast account switching - full local search history - fast seeking to any point in time, infinite local scrolling - efficient syncing between nostrdb and strfry nodes (soon) - full validation and rogue relay protection, gossip model is viable now. - note history cloud backups - properly saved bookmark storage - full scrollable dm history - dm and note read states - proper follow counts - proper note stat counting - follow / unfollow tracking - fast nostrscripts for custom lists and filtering Guess i better get to work! #nostrstandup image

Replies (30)

iefan 🕊️'s avatar
iefan 🕊️ iefan@primal.net 2 years ago
This looks promising; we just have to get it right once. It can solve many problems, from speed to reliability, and most importantly, it can do it in the right way.
HoloKat's avatar
HoloKat 2 years ago
Aww man, I kinda liked spotty follower counts, good reason to ignore it.
jb55's avatar
jb55 _@jb55.com 2 years ago
I also wanted a database I could use in damus android+notedeck, if I built on coredata it would be a bunch of duplicated and wasted time. It’s nice I can use this in my multi-platform client as well. I want to use this as a way to quickly build new native nostr clients without having to do all the hard stuff each time.
1 replies ↓
jb55's avatar
jb55 _@jb55.com 2 years ago
Yes damus use to use coredata for our profile database. It was extremely slow and had many race conditions and contention issues. Switching to nostrdb made profile search so much more stable and faster. It’s actually live on the appstore version. The remaining work is replacing in-memory and from-network notes with nostrdb. There is a lot of duplicate work that is happening atm because notes are processed twice. So the perf boost should be substantial. I am also going to rework timeline ui rendering so that it’s not as slow, as that is the slowest part of damus atm.
1 replies ↓
jb55's avatar
jb55 _@jb55.com 2 years ago
Cool thing about nostrdb is that it is safe to query it concurrently across threads. reads don’t require locks because of lmdb’s mmap + copy on write mechanism. Most queries are fast enough to run on the main thread (0.01ms for pulling 10 notes), but it’s nice that you can safely offload queries to threads if you need to. Fulltext queries can sometimes take 70ms because of the compressed key index I’m using, so I leverage threads for those situations.
1 replies ↓
Dimi's avatar
Dimi 2 years ago
I guess simple storage/recall is a prerequisite to this, right? I’ll put up 100k sats
1 replies ↓
ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ's avatar
ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 2 years ago
almost all the clients on the web do this, some worse than others coracle's compose field remembers if you accidentally unselect it in a reply but once you change context it forgets also the main compose button , now post+ up the top right, idk if it remembers prior inputs, probably not primals is really easy to drop out of the input field and then poof it's gone ah yeah, i don't really use any others, they are all shit xD
TKay's avatar
TKay tkay@bitcoindistrict.org 2 years ago
That’s impressive. Can I ask a few questions? Is the db for one’s notes and followers profiles? Or is it everything close to a relay copy? What’s the expected size of the db?
Dimi's avatar
Dimi 10 months ago
Nice work! Thanks 🙏
↑