index optimization is crucial… on NFDB 2.1 I have worked on some stuff:
- some index size optimizations
- move the event header (content stripped + only indexables) into the event store as before it was duplicated across multiple dbs sharing the same eventstore
- a faster “exists in any one of these relays” lookup with event stores, that also can return event lifecycle info
- better event lifecycle state (NFDB supports a soft-deleted stage, and for large events like follow lists it is inserted in multiple transactions due to FDB size limit)
- unified deletion index that can be shared across all relays
- unified event state data (encoded with capnp, contains lifecycle + expiration + index version + size + …)
- “size” accounting based on any metric
- global event cache
Login to reply
Replies (1)
also who does not order by created_at, and I forgot to mention the “updates log”: this basically allows anything to subscribe to deletions/insertions on the relay as a log and act on it