NFDB: Scaling Relays

I have been recently building NFDB, a new relay DB. This post is meant as a short overview.

Regular relays have challenges

Current relay software have significant challenges, which I have experienced when hosting Nostr.land:

• Scalability is only supported by adding full replicas, which does not scale to large relays.
• Most relays use slow databases and are not optimized for large scale usage.
• Search is near-impossible to implement on standard relays.
• Privacy features such as NIP-42 are lacking.
• Regular DB maintenance tasks on normal relays require extended downtime.
• Fault-tolerance is implemented, if any, using a load balancer, which is limited.
• Personalization and advanced filtering is not possible.
• Local caching is not supported.

NFDB: A scalable database for large relays

NFDB is a new database meant for medium-large scale relays, built on FoundationDB that provides:

• Near-unlimited scalability
• Extended fault tolerance
• Instant loading
• Better search
• Better personalization
• and more.

Search

NFDB has extended search capabilities including:

• Semantic search: Search for meaning, not words.
• Interest-based search: Highlight content you care about.
• Multi-faceted queries: Easily filter by topic, author group, keywords, and more at the same time.
• Wide support for event kinds, including users, articles, etc.

Personalization

NFDB allows significant personalization:

• Customized algorithms: Be your own algorithm.
• Spam filtering: Filter content to your WoT, and use advanced spam filters.
• Topic mutes: Mute topics, not keywords.
• Media filtering: With Nostr.build, you will be able to filter NSFW and other content
• Low data mode: Block notes that use high amounts of cellular data.
• and more

Other

NFDB has support for many other features such as:

• NIP-42: Protect your privacy with private drafts and DMs
• Microrelays: Easily deploy your own personal microrelay
• Containers: Dedicated, fast storage for discoverability events such as relay lists

Calcite: A local microrelay database

Calcite is a lightweight, local version of NFDB that is meant for microrelays and caching, meant for thousands of personal microrelays.

Calcite HA is an additional layer that allows live migration and relay failover in under 30 seconds, providing higher availability compared to current relays with greater simplicity. Calcite HA is enabled in all Calcite deployments.

For zero-downtime, NFDB is recommended.

Noswhere SmartCache

Relays are fixed in one location, but users can be anywhere.

Noswhere SmartCache is a CDN for relays that dynamically caches data on edge servers closest to you, allowing:

• Multiple regions around the world
• Improved throughput and performance
• Faster loading times

routerd

routerd is a custom load-balancer optimized for Nostr relays, integrated with SmartCache.

routerd is specifically integrated with NFDB and Calcite HA to provide fast failover and high performance.

Ending notes

NFDB is planned to be deployed to Nostr.land in the coming weeks.

A lot more is to come. 👀️️️️️️