Install HarperDB

This documentation contains information for installing HarperDB locally. Note that if you’d like to get up and running quickly, you can try a managed instance with HarperDB Cloud. HarperDB is a cross-platform database; we recommend Linux for production use, but HarperDB can run on Windows and Mac as well, for development purposes. Installation is usually very simple and just takes a few steps, but there are a few different options documented here.

HarperDB runs on Node.js, so if you do not have it installed, you need to do that first (if you have installed, you can skip to installing HarperDB, itself). Node.js can be downloaded and installed from their site. For Linux and Mac, we recommend installing and managing Node versions with NVM, which has instructions for installation, but generally NVM can be installed with:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

And then logout and login, and then install Node.js using nvm. We recommend using LTS, but support all currently maintained Node versions (which is currently version 14 and newer, and make sure to always uses latest minor/patch for the major version):

nvm install 18

Install and Start HarperDB

Then you can install HarperDB with NPM and start it:

npm install -g harperdb
harperdb

HarperDB will automatically start after installation.

If you are setting up a production server on Linux, we have much more extensive documentation on how to configure volumes for database storage, set up a systemd script, configure your operating system for use a database server in our linux installation guide.

With Docker

If you would like to run HarperDB in Docker, install Docker Desktop on your Mac or Windows computer. Otherwise, install the Docker Engine on your Linux server.

Once Docker Desktop or Docker Engine is installed, visit our Docker Hub page for information and examples on how to run a HarperDB container.

Offline Install

If you need to install HarperDB on a device that doesn't have an Internet connection, you can choose your version and download the npm package and install it directly (you’ll still need Node.js and NPM):

Download Install Package

Once you’ve downloaded the .tgz file, run the following command from the directory where you’ve placed it:

npm install -g harperdb-X.X.X.tgz harperdb install

For more information visit the HarperDB Command Line Interface guide.

Installation on Less Common Platforms

HarperDB comes with binaries for standard AMD64/x64 or ARM64 CPU architectures on Linux, Windows (x64 only), and Mac (including Apple Silicon). However, if you are installing on a less common platform (Alpine, for example), you will need to ensure that you have build tools installed for the installation process to compile the binaries (this is handled automatically), including:

• Go: version 1.19.1

• GCC

• Make

• Python v3.7, v3.8, v3.9, or v3.10

If you wish to install locally or already have a configured server, see the basic Installation Guide

The following is a recommended way to configure Linux and install HarperDB. These instructions should work reasonably well for any public cloud or on-premises Linux instance.

These instructions assume that the following has already been completed:

1. Linux is installed

2. Basic networking is configured

3. A non-root user account dedicated to HarperDB with sudo privileges exists

4. An additional volume for storing HarperDB files is attached to the Linux instance

5. Traffic to ports 9925 (HarperDB Operations API,) 9926 (HarperDB Custom Functions,) and 9932 (HarperDB Clustering) is permitted

For this example, we will use an AWS Ubuntu Server 22.04 LTS m5.large EC2 Instance with an additional General Purpose SSD EBS volume and the default “ubuntu” user account.

(Optional) LVM Configuration

Logical Volume Manager (LVM) can be used to stripe multiple disks together to form a single logical volume. If striping disks together is not a requirement, skip these steps.

Find disk that already has a partition

used_disk=$(lsblk -P -I 259 | grep "nvme.n1.*part" | grep -o "nvme.n1")

Create array of free disks

declare -a free_disks
mapfile -t free_disks < <(lsblk -P -I 259 | grep "nvme.n1.*disk" | grep -o "nvme.n1" | grep -v "$used_disk")

Get quantity of free disks

free_disks_qty=${#free_disks[@]}

Construct pvcreate command

cmd_string=""
for i in "${free_disks[@]}"
do
cmd_string="$cmd_string /dev/$i"
done

Initialize disks for use by LVM

pvcreate_cmd="pvcreate $cmd_string"
sudo $pvcreate_cmd

Create volume group

vgcreate_cmd="vgcreate hdb_vg $cmd_string"
sudo $vgcreate_cmd

Create logical volume

sudo lvcreate -n hdb_lv -i $free_disks_qty -l 100%FREE hdb_vg

Configure Data Volume

Run lsblk and note the device name of the additional volume

lsblk

Create an ext4 filesystem on the volume (The below commands assume the device name is nvme1n1. If you used LVM to create logical volume, replace /dev/nvme1n1 with /dev/hdb_vg/hdb_lv)

sudo mkfs.ext4 -L hdb_data /dev/nvme1n1

Mount the file system and set the correct permissions for the directory

mkdir /home/ubuntu/hdb
sudo mount -t ext4 /dev/nvme1n1 /home/ubuntu/hdb
sudo chown -R ubuntu:ubuntu /home/ubuntu/hdb
sudo chmod 775 /home/ubuntu/hdb

Create a fstab entry to mount the filesystem on boot

echo "LABEL=hdb_data /home/ubuntu/hdb ext4 defaults,noatime 0 1" | sudo tee -a /etc/fstab

Configure Linux and Install Prerequisites

If a swap file or partition does not already exist, create and enable a 2GB swap file

sudo dd if=/dev/zero of=/swapfile bs=128M count=16
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo "/swapfile swap swap defaults 0 0" | sudo tee -a /etc/fstab

Increase the open file limits for the ubuntu user

echo "ubuntu soft nofile 500000" | sudo tee -a /etc/security/limits.conf
echo "ubuntu hard nofile 1000000" | sudo tee -a /etc/security/limits.conf

Install Node Version Manager (nvm)

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

Load nvm (or logout and then login)

. ~/.nvm/nvm.sh

Install Node.js using nvm (read more about specific Node version requirements)

nvm install <the node version>

Install and Start HarperDB

Here is an example of installing HarperDB with minimal configuration.

npm install -g harperdb
harperdb start \
  --TC_AGREEMENT "yes" \
  --ROOTPATH "/home/ubuntu/hdb" \
  --OPERATIONSAPI_NETWORK_PORT "9925" \
  --HDB_ADMIN_USERNAME "HDB_ADMIN" \
  --HDB_ADMIN_PASSWORD "password"

Here is an example of installing HarperDB with commonly used additional configuration.

npm install -g harperdb
harperdb start \
  --TC_AGREEMENT "yes" \
  --ROOTPATH "/home/ubuntu/hdb" \
  --OPERATIONSAPI_NETWORK_PORT "9925" \
  --HDB_ADMIN_USERNAME "HDB_ADMIN" \
  --HDB_ADMIN_PASSWORD "password" \
  --OPERATIONSAPI_NETWORK_HTTPS "true" \
  --CUSTOMFUNCTIONS_NETWORK_HTTPS "true" \
  --CLUSTERING_ENABLED "true" \
  --CLUSTERING_USER "cluster_user" \
  --CLUSTERING_PASSWORD "password" \
  --CLUSTERING_NODENAME "hdb1"

HarperDB will automatically start after installation. If you wish HarperDB to start when the OS boots, you have two options

You can set up a crontab:

(crontab -l 2>/dev/null; echo "@reboot PATH=\"/home/ubuntu/.nvm/versions/node/v18.15.0/bin:$PATH\" && harperdb start") | crontab -

Or you can create a systemd script at /etc/systemd/system/harperdb.service

Pasting the following contents into the file:

[Unit]
Description=HarperDB

[Service]
Type=simple
Restart=always
User=ubuntu
Group=ubuntu
WorkingDirectory=/home/ubuntu
ExecStart=/bin/bash -c 'PATH="/home/ubuntu/.nvm/versions/node/v18.15.0/bin:$PATH"; harperdb'

[Install]
WantedBy=multi-user.target

And then running the following:

systemctl daemon-reload
systemctl enable harperdb

For more information visit the HarperDB Command Line Interface guide and the HarperDB Configuration File guide.