Load Balancing and Node Availability
A Harper Fabric cluster serves your applications from one hostname, the cluster's Fabric hostname (for example my-cluster.my-org.harperfabric.com) or a custom domain pointed at it. Fabric's load balancer picks a node of the cluster for each new connection to that hostname. This page covers how it decides which nodes may receive traffic, and how to read or change that decision for one node.
How Fabric decides where traffic goes
The load balancer checks each node with a GET /status request to that node. A node that answers 200 may receive new connections. A node that answers anything else, or does not answer, receives none until it answers 200 again.
/status is served by a component that Fabric deploys to every node of the cluster, named status-check. Its source is public: HarperFast/status-check-fabric. It answers from a single record on the node, the availability status, which Harper's set_status and get_status operations write and read:
availability record | GET /status answers |
|---|---|
Available | 200 |
Unavailable | 404 |
| No record | 404 |
Fabric writes this record itself. For example, it takes a node out while it upgrades that node, and puts it back afterwards.
Each node has its own record
Harper keeps status records in the system.hdb_status table, which it creates with replication turned off. Every node therefore has its own availability record: writing it on one node changes only that node. The record is stored on disk, so it keeps its value when Harper restarts.
Because each node answers for itself, a request meant for one node has to reach that node. Through the cluster hostname, the load balancer decides which node receives it. Use the node's own hostname, which only that node answers.
The status-check component
GET /status
Unauthenticated.
200with the body"Status endpoint is reporting for duty."when the node'savailabilityrecord isAvailable.404with an error titledStatus endpoint is reporting downtime.otherwise, including when the node has noavailabilityrecord.
curl -i https://<node hostname>/status
POST /status
Sets the node's availability record to Available. Requires a super_user: the component answers 204 on success, 401 to a request without credentials, and 403 to any other user.
DELETE /status
Sets the node's availability record to Unavailable. Authorized the same way as POST.
To change the record yourself, use the Harper CLI rather than these routes, as Taking a node out of rotation shows. It needs no password on the command line, and from v5.3.0 the user it runs as does not have to be a super_user.
Fabric routes traffic by this component. Do not remove it, and do not deploy another resource that serves /status.
This is not the @harperdb/status-check npm package, which keeps its own hdb_status table in the data database. Fabric does not read that table, and the package also serves /status, so do not deploy it on a Fabric cluster. On a self-managed cluster, Health Checks and Traffic Admission covers it.
Taking a node out of rotation
To stop new traffic reaching a node, set its availability record to Unavailable with the set_status operation. To let traffic back, set it to Available. Send each command to the node's own hostname. From your own machine, log in once and the CLI keeps the token:
harper login https://<node hostname>
harper set_status id=availability status=Unavailable target=https://<node hostname>
harper get_status id=availability target=https://<node hostname>
harper set_status id=availability status=Available target=https://<node hostname>
get_status returns the record, including __updatedtime__, the time of its last write.
From a GitHub Actions workflow, run the same commands with workload identity (OIDC), so the workflow holds no Harper credential at all. Deploying from a CI/CD Pipeline shows how to set up the trust policy and the user it maps to.
That user does not need to be a super_user. From v5.3.0, a role whose operations list names only set_status and get_status can run those two operations and nothing else.
Connections already open to the node stay open: the load balancer only decides where new connections go. Fabric writes the same record, so a value you set can be replaced by Fabric's own operations on the node.
An application can also feed its own health into this record, so that a node taking errors leaves rotation while its peers keep serving. See Taking an Unhealthy Node Out of Rotation.