Thread - Nostr Hypermedia

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 1 month ago

#orly🦉 #devstr #progressreport today finally i got the benchmarks to all work, and includes the new dgraph and neo4j graph database drivers. they are quite realistic and exhaustive benchmarks, based on an actual collection of real events that i also use in encoder tests to validate my json codec. just brief comments but you should be able to see that next-orly-badger is the fastest relay in the set, by about 10% at some things and a little more at some others. there is test conditions i could create that demonstrate the benefits of the filter query results cache, and the key-table inlined events under 1kb in size (these are iterated first and are the most numerically common size in the data set. such as fetching threads, reactions, and simulating a large userbase of user interested in some new events, they only get decoded once and sent to all matching filters until 5 minutes passes and nobody asks for that filter again. the cache is ZSTD compressed, 512mb in size, providing 1.6gb of cached query results instantly if they are hot. also, you can clearly see, that the baseline latency of most network and sqlite event stores is slower, 600 for sqlite and dgraph and 700 for the neo4j. the graph query schema might not be ideal for the task but implements NIP-01 filter queries https://git.nostrdev.com/mleku/next.orly.dev/raw/branch/main/cmd/benchmark/reports/run_20251119_114143/aggregate_report.txt ================================================================ NOSTR RELAY BENCHMARK AGGREGATE REPORT ================================================================ Generated: 2025-11-19T12:08:43+00:00 Benchmark Configuration: Events per test: 50000 Concurrent workers: 24 Test duration: 60s Relays tested: 8 ================================================================ SUMMARY BY RELAY ================================================================ Relay: next-orly-badger ---------------------------------------- Status: COMPLETED Events/sec: 17949.86 Events/sec: 6293.77 Events/sec: 17949.86 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.089014ms Bottom 10% Avg Latency: 552.633µs Avg Latency: 749.292µs P95 Latency: 1.801326ms P95 Latency: 1.544064ms P95 Latency: 797.32µs Relay: next-orly-dgraph ---------------------------------------- Status: COMPLETED Events/sec: 17627.19 Events/sec: 6241.01 Events/sec: 17627.19 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.103766ms Bottom 10% Avg Latency: 537.227µs Avg Latency: 973.956µs P95 Latency: 1.895983ms P95 Latency: 1.938364ms P95 Latency: 839.77µs Relay: next-orly-neo4j ---------------------------------------- Status: COMPLETED Events/sec: 15536.46 Events/sec: 6269.18 Events/sec: 15536.46 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.414281ms Bottom 10% Avg Latency: 704.384µs Avg Latency: 919.794µs P95 Latency: 2.486204ms P95 Latency: 1.842478ms P95 Latency: 828.598µs Relay: khatru-sqlite ---------------------------------------- Status: COMPLETED Events/sec: 17237.90 Events/sec: 6137.41 Events/sec: 17237.90 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.195398ms Bottom 10% Avg Latency: 614.1µs Avg Latency: 967.476µs P95 Latency: 2.00684ms P95 Latency: 2.046996ms P95 Latency: 843.455µs Relay: khatru-badger ---------------------------------------- Status: COMPLETED Events/sec: 16911.23 Events/sec: 6231.83 Events/sec: 16911.23 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.187112ms Bottom 10% Avg Latency: 540.572µs Avg Latency: 957.9µs P95 Latency: 2.183304ms P95 Latency: 1.888493ms P95 Latency: 824.399µs Relay: relayer-basic ---------------------------------------- Status: COMPLETED Events/sec: 17836.39 Events/sec: 6270.82 Events/sec: 17836.39 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.081434ms Bottom 10% Avg Latency: 525.619µs Avg Latency: 951.65µs P95 Latency: 1.853627ms P95 Latency: 1.779976ms P95 Latency: 831.883µs Relay: strfry ---------------------------------------- Status: COMPLETED Events/sec: 16470.06 Events/sec: 6004.96 Events/sec: 16470.06 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.261656ms Bottom 10% Avg Latency: 566.551µs Avg Latency: 1.02418ms P95 Latency: 2.241835ms P95 Latency: 2.314062ms P95 Latency: 821.493µs Relay: nostr-rs-relay ---------------------------------------- Status: COMPLETED Events/sec: 16764.35 Events/sec: 6300.71 Events/sec: 16764.35 Success Rate: 100.0% Success Rate: 100.0% Success Rate: 100.0% Avg Latency: 1.245012ms Bottom 10% Avg Latency: 614.335µs Avg Latency: 869.47µs P95 Latency: 2.151312ms P95 Latency: 1.707251ms P95 Latency: 816.334µs ================================================================ DETAILED RESULTS ================================================================ Individual relay reports are available in: - /reports/run_20251119_114143/khatru-badger_results.txt - /reports/run_20251119_114143/khatru-sqlite_results.txt - /reports/run_20251119_114143/next-orly-badger_results.txt - /reports/run_20251119_114143/next-orly-dgraph_results.txt - /reports/run_20251119_114143/next-orly-neo4j_results.txt - /reports/run_20251119_114143/nostr-rs-relay_results.txt - /reports/run_20251119_114143/relayer-basic_results.txt - /reports/run_20251119_114143/strfry_results.txt ================================================================ BENCHMARK COMPARISON TABLE ================================================================ Relay Status Peak Tput/s Avg Latency Success Rate ---- ------ ----------- ----------- ------------ next-orly-badger OK 17949.86 1.089014ms 100.0% next-orly-dgraph OK 17627.19 1.103766ms 100.0% next-orly-neo4j OK 15536.46 1.414281ms 100.0% khatru-sqlite OK 17237.90 1.195398ms 100.0% khatru-badger OK 16911.23 1.187112ms 100.0% relayer-basic OK 17836.39 1.081434ms 100.0% strfry OK 16470.06 1.261656ms 100.0% nostr-rs-relay OK 16764.35 1.245012ms 100.0% ================================================================ End of Report ================================================================

Replies (16)

it's me, real fake 1 month ago

whatever you do, as long as you name the plugin to Neo "nostr4j" then you're winning

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 1 month ago

it's not a plugin, it's just a driver. all of them are available in the binary. the graph schema already implements nip-01, to make it really useful, it just needs to make more vertices to cover follow, mute and report vectors and another independent server can share the neo4j (or dgraph) instance with the relay and always know the graph is current. when it replaces and deletes events, it deletes all of the edges associated with it.

1 replies ↓

Pip the WoT guy pip@vertexlab.io 1 month ago

ups, it's on my own account.

GitHub

GitHub - pippellia-btc/rely: A framework for building custom Nostr relays you can rely on. Designed for the best developer experience.

A framework for building custom Nostr relays you can rely on. Designed for the best developer experience. - pippellia-btc/rely

Pip the WoT guy pip@vertexlab.io 1 month ago

It should be definitely faster than khatru given that it uses inverted indexes for subscriptions, IF the benchmark has REQs as well as EVENTs.

Pip the WoT guy pip@vertexlab.io 1 month ago

these inverted indexes are map --> set of subscription ids. It's very similar to how strfry works.

GitHub

rely/dispatcher.go at main · pippellia-btc/rely

A framework for building custom Nostr relays you can rely on. Designed for the best developer experience. - pippellia-btc/rely

I rolled my own implementation of small sets, which are exactly that, data structures that work when you want a set with less than 1000 elements. It's very useful because it avoid using maps and performing more allocations when you just want a small set.

GitHub

GitHub - pippellia-btc/smallset: A memory-efficient, slice-backed implementation for small sorted sets in Go.

A memory-efficient, slice-backed implementation for small sorted sets in Go. - pippellia-btc/smallset

Rely originally was like khatru, so to broadcast an event it would iterate over all subscriptions, see if they match the event, and then write. Then I was able to double the efficiency by introducing these inverted indexes, AND by marshalling the event only once, and then write the bytes to each, instead of marshalling once for every client.

it's me, real fake 1 month ago

do you have a repo? I've made a number of neo4j-backed applications.

1 replies ↓

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 1 month ago

next.orly.dev will redirect you there I built a driver for neo4j because it made it simpler - they can concurrently use it directly while the relay writes to it, no complicated sync. Just needs a few more vertexes to cover the other.. hmmmm

Pip the WoT guy pip@vertexlab.io 1 month ago

Hey @ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ, I made this.

GitHub

GitHub - pippellia-btc/rely-sqlite: A nostr relay using the rely framework and sqlite as the database

A nostr relay using the rely framework and sqlite as the database - pippellia-btc/rely-sqlite

As the name suggest is just, rely + nostr-sqlite.

Pip the WoT guy pip@vertexlab.io 1 month ago

thank you man, and yeah I understand your frustration with javascript, I fucking hate it as well

Pip the WoT guy pip@vertexlab.io 1 month ago

the relay logs to stdout when it's running. logs like INFO relay is starting up INFO relay is running address=<your address> But, you can change the logger by imusing the WithLogger functional option in the rely.NewRelay

Pip the WoT guy pip@vertexlab.io 1 month ago

yeah It does it. Here is the relay ServeHTTP method.

GitHub

rely/relay.go at main · pippellia-btc/rely

A framework for building custom Nostr relays you can rely on. Designed for the best developer experience. - pippellia-btc/rely

As you can see it supports nip-11, websocket, otherwise it return bad request

semisol semisol@nostr.land 1 month ago

I'd say a good Rust async runtime would be competitive, but you get into the high-cost low-benefit part of the "cost(time)/benefit" curve pretty fast

1 replies ↓

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 1 month ago

yeah, that lines up with my observations too. it did take a lot of work to iron out the bottlenecks though. khatru badger in particular gets a big score on the ranking because of the way its badger event store just flings events but i bet it fails to always get them in order on a relay that has had old events loaded up after a lot of newer ones. it depends on sequence of events being stored to order the results. orly has a special index to make sure they always are delivered in the explicit order of timestamp, and if you read nip-01, you will see. fiatjaf cheated on that with khatru. and relayer.

1 replies ↓

semisol semisol@nostr.land 1 month ago

index optimization is crucial… on NFDB 2.1 I have worked on some stuff: - some index size optimizations - move the event header (content stripped + only indexables) into the event store as before it was duplicated across multiple dbs sharing the same eventstore - a faster “exists in any one of these relays” lookup with event stores, that also can return event lifecycle info - better event lifecycle state (NFDB supports a soft-deleted stage, and for large events like follow lists it is inserted in multiple transactions due to FDB size limit) - unified deletion index that can be shared across all relays - unified event state data (encoded with capnp, contains lifecycle + expiration + index version + size + …) - “size” accounting based on any metric - global event cache

1 replies ↓

semisol semisol@nostr.land 1 month ago

also who does not order by created_at, and I forgot to mention the “updates log”: this basically allows anything to subscribe to deletions/insertions on the relay as a log and act on it

1 replies ↓

ᴛʜᴇ ᴅᴇᴀᴛʜ ᴏꜰ ᴍʟᴇᴋᴜ me@mleku.dev 1 month ago

fiatjaf's eventstore is designed in mind to fling events out of the database as soon as they are found. idk about the other implementations aside from badger but he depends on event store sequence numbers for ordering. orly has an index for taking event serials and sorting them by created_at, filtering out authors, and cutting a since/until out of the result set. once it does that it fetches the binary, decodes, encodes to json and puts it on the wire. probably the networked database engine drivers he wrote return sorted results but that's all on the RDB engines it uses. hm i better check this part of these two new graph database drivers i wrote. graph dbs require a different scheme for searches using graph traversal, so that pubkey/kind/created at index i made for sorting events before fetching them to decode probably isn't optimal.