4.4.0

HarperDB 4.4.0

9/27/2024

Native Replication

HarperDB has a completely new native replication system which is faster, more efficient, secure, and reliable than the previous replication system. The new system (codenamed "Plexus") uses direct WebSocket connections between servers with highly optimized encoding and is driven by direct tracking audit/transaction log for efficient and flexible data transfer. This replication has improved resilience with the ability to reach consensus consistency when one node goes down through cross-node catch-up. Network connections can be performed over the existing operations API port or a separate port, for improved configurability.

The native replication system is much easier to configure, with multiple options for authentication and security, including PKI/mTLS security that is highly robust and easy to use in conjunction with existing PKI certificates. Replication can be configured through explicit subscriptions or for automated replication of all data in a database. With automated replication, gossiping is used to automatically discover and connect to other nodes in the cluster.

Sharding

The new replication system also includes provisional support for sharding. This sharding mechanism paves the way for greater scalability and performance, by allow data to be distributed across multiple nodes.

Replicated Operations

Certain operations can now be replicated across the cluster, including the deployment and management of components. This allows for a more seamless experience when managing a cluster of HarperDB instances. Restarts can also be "replicated", and if used, will perform a rolling restart of all the nodes in a cluster.

Computed Properties

Computed properties allow applications to define properties that are computed from other properties, allowing for composite properties that are calculated from other data stored in records without requiring actual storage of the computed value. For example, you could have a computed property for a full name based on first and last, or age/duration based on a date. Computed properties are also foundational for custom indexes. See the schema documentation and Resource API for more information.

Custom Indexing

Custom indexes can now be defined using computed properties to allow for unlimited possibilities of indexing, including composite, full-text indexing, vector indexing. Again, see the schema documentation for more information.

Dynamic Certificate Management

Certificates are now stored in system tables and can be dynamically managed. Certificates can be added, replaced, and deleted without restarting HarperDB. This includes both standard certificates and certificate authorities, as well as private keys (private keys are not stored in table, they securely stored in a file).

Status Report on Startup

On startup, HarperDB will now print out an informative status of all running services and ports they are listening on.

Support for Response object

Resource methods can now return a Response object (or an object with headers and status) to allow for more control over the response.

Developer/Production Mode for Configuration

When using interactive installation (when configuration is not provided through arguments or env vars), HarperDB now provides an option for developer or production mode with a set of default configuration for each mode better suited for developer or production environments.

Export by Protocol

Exported resources can be configured to be specifically exported by protocol (REST, MQTT, etc.) for more granular control over what is exported where.