Clustering

Cluster Set Routes

Adds a route/routes to either the hub or leaf server cluster configuration. This operation behaves as a PATCH/upsert, meaning it will add new routes to the configuration while leaving existing routes untouched.

Operation is restricted to super_user roles only

operation (required) - must always be cluster_set_routes
server (required) - must always be hub or leaf, in most cases you should use hub here
routes (required) - must always be an objects array with a host and port:
- host - the host of the remote instance you are clustering to
- port - the clustering port of the remote instance you are clustering to, in most cases this is the value in clustering.hubServer.cluster.network.port on the remote instance harperdb-config.yaml

Body

{
    "operation": "cluster_set_routes",
    "server": "hub",
    "routes": [
        {
            "host": "3.22.181.22",
            "port": 12345
        },
        {
            "host": "3.137.184.8",
            "port": 12345
        },
        {
            "host": "18.223.239.195",
            "port": 12345
        },
        {
            "host": "18.116.24.71",
            "port": 12345
        }
    ]
}

Response: 200

{
    "message": "cluster routes successfully set",
    "set": [
        {
            "host": "3.22.181.22",
            "port": 12345
        },
        {
            "host": "3.137.184.8",
            "port": 12345
        },
        {
            "host": "18.223.239.195",
            "port": 12345
        },
        {
            "host": "18.116.24.71",
            "port": 12345
        }
    ],
    "skipped": []
}

Cluster Get Routes

Gets all the hub and leaf server routes from the config file.

Operation is restricted to super_user roles only

operation (required) - must always be cluster_get_routes

Body

{
    "operation": "cluster_get_routes"
}

Response: 200

{
    "hub": [
        {
            "host": "3.22.181.22",
            "port": 12345
        },
        {
            "host": "3.137.184.8",
            "port": 12345
        },
        {
            "host": "18.223.239.195",
            "port": 12345
        },
        {
            "host": "18.116.24.71",
            "port": 12345
        }
    ],
    "leaf": []
}

Cluster Delete Routes

Removes route(s) from hub and/or leaf server routes array in config file. Returns a deletion success message and arrays of deleted and skipped records.

Operation is restricted to super_user roles only

operation (required) - must always be cluster_delete_routes
routes required - Must be an array of route object(s)

Body

{
    "operation": "cluster_delete_routes",
    "routes": [
        {
            "host": "18.116.24.71",
            "port": 12345
        }
    ]
}

Response: 200

{
    "message": "cluster routes successfully deleted",
    "deleted": [
        {
            "host": "18.116.24.71",
            "port": 12345
        }
    ],
    "skipped": []
}

Add Node

Registers an additional HarperDB instance with associated subscriptions. Learn more about HarperDB clustering here.

Operation is restricted to super_user roles only

operation (required) - must always be add_node
node_name (required) - the node name of the remote node
subscriptions (required) - The relationship created between nodes. Must be an object array and include schema, table, subscribe and publish:
- schema - the schema to replicate from
- table - the table to replicate from
- subscribe - a boolean which determines if transactions on the remote table should be replicated on the local table
- publish - a boolean which determines if transactions on the local table should be replicated on the remote table
- start_time (optional) - How far back to go to get transactions from node being added. Must be in UTC YYYY-MM-DDTHH:mm:ss.sssZ format

Body

{
    "operation": "add_node",
    "node_name": "ec2-3-22-181-22",
    "subscriptions": [
        {
            "schema": "dev",
            "table": "dog",
            "subscribe": false,
            "publish": true,
            "start_time": "2022-09-02T20:06:35.993Z"
        }
    ]
}

Response: 200

{
    "message": "Successfully added 'ec2-3-22-181-22' to manifest"
}

Update Node

Modifies an existing HarperDB instance registration and associated subscriptions. This operation behaves as a PATCH/upsert, meaning it will insert or update the specified replication configurations while leaving other table replication configuration untouched. Learn more about HarperDB clustering here.

Operation is restricted to super_user roles only

operation (required) - must always be update_node
node_name (required) - the node name of the remote node you are updating
subscriptions (required) - The relationship created between nodes. Must be an object array and include schema, table, subscribe and publish:
- schema - the schema to replicate from
- table - the table to replicate from
- subscribe - a boolean which determines if transactions on the remote table should be replicated on the local table
- publish - a boolean which determines if transactions on the local table should be replicated on the remote table
- start_time (optional) - How far back to go to get transactions from node being added. Must be in UTC YYYY-MM-DDTHH:mm:ss.sssZ format

Body

{
    "operation": "update_node",
    "node_name": "ec2-18-223-239-195",
    "subscriptions": [
        {
            "schema": "dev",
            "table": "dog",
            "subscribe": true,
            "publish": false,
            "start_time": "2022-09-02T20:06:35.993Z"
        }
    ]
}

Response: 200

{
    "message": "Successfully updated 'ec2-3-22-181-22'"
}

Set Node Replication

A more adeptly named alias for add and update node. This operation behaves as a PATCH/upsert, meaning it will insert or update the specified replication configurations while leaving other table replication configuration untouched. The database (aka schema) parameter is optional, it will default to data.

Operation is restricted to super_user roles only

operation (required) - must always be set_node_replication
node_name (required) - the node name of the remote node you are updating
subscriptions (required) - The relationship created between nodes. Must be an object array and table, subscribe and publish:
- database (optional) - the database to replicate from
- table (required) - the table to replicate from
- subscribe (required) - a boolean which determines if transactions on the remote table should be replicated on the local table
- publish (required) - a boolean which determines if transactions on the local table should be replicated on the remote table

Body

{
    "operation": "set_node_replication",
    "node_name": "node1",
    "subscriptions": [
        {
            "table": "dog",
            "subscribe": true,
            "publish": true
        }
    ]
}

Response: 200

{
    "message": "Successfully updated 'ec2-3-22-181-22'"
}

Cluster Status

Returns an array of status objects from a cluster. A status object will contain the clustering node name, whether or not clustering is enabled, and a list of possible connections. Learn more about HarperDB clustering here.

Operation is restricted to super_user roles only

operation (required) - must always be cluster_status

Body

{
    "operation": "cluster_status"
}

Response: 200

{
    "node_name": "ec2-18-221-143-69",
    "is_enabled": true,
    "connections": [
        {
            "node_name": "ec2-3-22-181-22",
            "status": "open",
            "ports": {
                "clustering": 12345,
                "operations_api": 9925
            },
            "latency_ms": 13,
            "uptime": "30d 1h 18m 8s",
            "subscriptions": [
                {
                    "schema": "dev",
                    "table": "dog",
                    "publish": true,
                    "subscribe": true
                }
            ]
        }
    ]
}

Cluster Network

Returns an object array of enmeshed nodes. Each node object will contain the name of the node, the amount of time (in milliseconds) it took for it to respond, the names of the nodes it is enmeshed with and the routes set in its config file. Learn more about HarperDB clustering here.

Operation is restricted to super_user roles only

operation (required)- must always be cluster_network
timeout (optional) - the amount of time in milliseconds to wait for a response from the network. Must be a number
connected_nodes (optional) - omit connected_nodes from the response. Must be a boolean. Defaults to false
routes (optional) - omit routes from the response. Must be a boolean. Defaults to false

Body

{
    "operation": "cluster_network"
}

Response: 200

{
    "nodes": [
        {
            "name": "local_node",
            "response_time": 4,
            "connected_nodes": ["ec2-3-142-255-78"],
            "routes": [
                {
                    "host": "3.142.255.78",
                    "port": 9932
                }
            ]
        },
        {
            "name": "ec2-3-142-255-78",
            "response_time": 57,
            "connected_nodes": ["ec2-3-12-153-124", "ec2-3-139-236-138", "local_node"],
            "routes": []
        }
    ]
}

Remove Node

Removes a HarperDB instance and associated subscriptions from the cluster. Learn more about HarperDB clustering here.

Operation is restricted to super_user roles only

operation (required) - must always be remove_node
name (required) - The name of the node you are de-registering

Body

{
    "operation": "remove_node",
    "node_name": "ec2-3-22-181-22"
}

Response: 200

{
    "message": "Successfully removed 'ec2-3-22-181-22' from manifest"
}

Configure Cluster

Bulk create/remove subscriptions for any number of remote nodes. Resets and replaces any existing clustering setup. Learn more about HarperDB clustering here.

Operation is restricted to super_user roles only

operation (required) - must always be configure_cluster
connections (required) - must be an object array with each object containing node_name and subscriptions for that node

Body

{
    "operation": "configure_cluster",
    "connections": [
        {
            "node_name": "ec2-3-137-184-8",
            "subscriptions": [
                {
                    "schema": "dev",
                    "table": "dog",
                    "subscribe": true,
                    "publish": false
                }
            ]
        },
        {
            "node_name": "ec2-18-223-239-195",
            "subscriptions": [
                {
                    "schema": "dev",
                    "table": "dog",
                    "subscribe": true,
                    "publish": true
                }
            ]
        }
    ]
}

Response: 200

{
    "message": "Cluster successfully configured."
}

Purge Stream

Will purge messages from a stream

Operation is restricted to super_user roles only

operation (required) - must always be purge_stream
database (required) - the name of the database where the streams table resides
table (required) - the name of the table that belongs to the stream
options (optional) - control how many messages get purged. Options are:
- keep - purge will keep this many most recent messages
- seq - purge all messages up to, but not including, this sequence

Body

{
    "operation": "purge_stream",
    "database": "dev",
    "table": "dog",
    "options": {
      "keep": 100
    }
}

PreviousUsers and Roles NextCustom Functions

Last updated 7 months ago