HarperDB uses Basic Auth and JSON Web Tokens (JWTs) to secure our HTTP requests. In the context of an HTTP transaction, basic access authentication is a method for an HTTP user agent to provide a user name and password when making a request.
** You do not need to log in separately. Basic Auth is added to each HTTP request like create_schema, create_table, insert etc… via headers. **
A header is added to each HTTP request. The header key is “Authorization” the header value is “Basic <<your username and password buffer token>>”
In the below code sample, you can see where we add the authorization header to the request. This needs to be added for each and every HTTP request for HarperDB.
Note: This function uses btoa. Learn about btoa here.
HarperDB was set up to require very minimal configuration to work out of the box. There are, however, some best practices we encourage for anyone building an app with HarperDB.
HarperDB allows for managing cross-origin HTTP requests. By default, HarperDB enables CORS for all domains if you need to disable CORS completely or set up an access list of domains you can do the following:
Open the harperdb-config.yaml file this can be found in <ROOTPATH>, the location you specified during install.
In harperdb-config.yaml there should be 2 entries under operationsApi.network
: cors and corsAccessList.
cors
To turn off, change to: cors: false
To turn on, change to: cors: true
corsAccessList
The corsAccessList
will only be recognized by the system when cors
is true
To create an access list you set corsAccessList
to a comma-separated list of domains.
i.e. corsAccessList
is http://harperdb.io,http://products.harperdb.io
To clear out the access list and allow all domains: corsAccessList
is [null]
HarperDB provides the option to use an HTTP or HTTPS and HTTP/2 interface. The default port for the server is 9925.
These default ports can be changed by updating the operationsApi.network.port
value in <ROOTPATH>/harperdb-config.yaml
By default, HTTPS is turned off and HTTP is turned on. It is recommended that you never directly expose HarperDB's HTTP interface through a publicly available port. HTTP is intended for local or private network use.
You can toggle HTTPS and HTTP in the settings file. By setting operationsApi.network.https
to true/false. When https
is set to false
, the server will use HTTP (version 1.1). Enabling HTTPS will enable both HTTPS/1.1 and HTTPS/2.
HarperDB automatically generates a certificate (certificate.pem), a certificate authority (ca.pem) and a private key file (privateKey.pem) which live at <ROOTPATH>/keys/
.
You can replace these with your own certificates and key.
Changes to these settings require a restart. Use operation harperdb restart
from HarperDB Operations API.
HarperDB uses role-based, attribute-level security to ensure that users can only gain access to the data they’re supposed to be able to access. Our granular permissions allow for unparalleled flexibility and control, and can actually lower the total cost of ownership compared to other database solutions, since you no longer have to replicate subsets of your data to isolate use cases.
HarperDB utilizes a Role-Based Access Control (RBAC) framework to manage access to HarperDB instances. A user is assigned a role that determines the user’s permissions to access database resources and run core operations.
Role permissions in HarperDB are broken into two categories – permissions around database manipulation and permissions around database definition.
Database Manipulation: A role defines CRUD (create, read, update, delete) permissions against database resources (i.e. data) in a HarperDB instance.
At the table-level access, permissions must be explicitly defined when adding or altering a role – i.e. HarperDB will assume CRUD access to be FALSE if not explicitly provided in the permissions JSON passed to the add_role
and/or alter_role
API operations.
At the attribute-level, permissions for attributes in all tables included in the permissions set will be assigned based on either the specific attribute-level permissions defined in the table’s permission set or, if there are no attribute-level permissions defined, permissions will be based on the table’s CRUD set.
Database Definition: Permissions related to managing schemas, tables, roles, users, and other system settings and operations are restricted to the built-in super_user
role.
Built-In Roles
There are three built-in roles within HarperDB. See full breakdown of operations restricted to only super_user roles here.
super_user
- This role provides full access to all operations and methods within a HarperDB instance, this can be considered the admin role.
This role provides full access to all Database Definition operations and the ability to run Database Manipulation operations across the entire database schema with no restrictions.
cluster_user
- This role is an internal system role type that is managed internally to allow clustered instances to communicate with one another.
This role is an internally managed role to facilitate communication between clustered instances.
structure_user
- This role provides specific access for creation and deletion of data.
When defining this role type you can either assign a value of true which will allow the role to create and drop schemas & tables. Alternatively the role type can be assigned a string array. The values in this array are schemas and allows the role to only create and drop tables in the designated schemas.
User-Defined Roles
In addition to built-in roles, admins (i.e. users assigned to the super_user role) can create customized roles for other users to interact with and manipulate the data within explicitly defined tables and attributes.
Unless the user-defined role is given super_user
permissions, permissions must be defined explicitly within the request body JSON.
Describe operations will return metadata for all schemas, tables, and attributes that a user-defined role has CRUD permissions for.
Role Permissions
When creating a new, user-defined role in a HarperDB instance, you must provide a role name and the permissions to assign to that role. Reminder, only super users can create and manage roles.
role
name used to easily identify the role assigned to individual users.
Roles can be altered/dropped based on the role name used in and returned from a successful add_role
, alter_role
, or list_roles
operation.
permissions
used to explicitly defined CRUD access to existing table data.
Example JSON for add_role
request
Setting Role Permissions
There are two parts to a permissions set:
super_user
– boolean value indicating if role should be provided super_user access.
If super_user
is set to true, there should be no additional schema-specific permissions values included since the role will have access to the entire database schema. If permissions are included in the body of the operation, they will stored within HarperDB, but ignored, as super_users have full access to the database.
permissions
: Schema tables that a role should have specific CRUD access to should be included in the final, schema-specific permissions
JSON.
For user-defined roles (i.e. non-super_user roles, blank permissions will result in the user being restricted from accessing any of the database schema.
Table Permissions JSON
Each table that a role should be given some level of CRUD permissions to must be included in the tables
array for its schema in the roles permissions JSON passed to the API (see example above).
Important Notes About Table Permissions
If a schema and/or any of its tables are not included in the permissions JSON, the role will not have any CRUD access to the schema and/or tables.
If a table-level CRUD permission is set to false, any attribute-level with that same CRUD permission set to true will return an error.
Important Notes About Attribute Permissions
If there are attribute-specific CRUD permissions that need to be enforced on a table, those need to be explicitly described in the attribute_permissions
array.
If a non-hash attribute is given some level of CRUD access, that same access will be assigned to the table’s hash_attribute
, even if it is not explicitly defined in the permissions JSON.
See table_name1’s permission set for an example of this – even though the table’s hash attribute is not specifically defined in the attribute_permissions array, because the role has CRUD access to ‘attribute1’, the role will have the same access to the table’s hash attribute.
If attribute-level permissions are set – i.e. attribute_permissions.length > 0 – any table attribute not explicitly included will be assumed to have not CRUD access (with the exception of the hash_attribute
described in #2).
See table_name1’s permission set for an example of this – in this scenario, the role will have the ability to create, insert and update ‘attribute1’ and the table’s hash attribute but no other attributes on that table.
If an attribute_permissions
array is empty, the role’s access to a table’s attributes will be based on the table-level CRUD permissions.
See table_name2’s permission set for an example of this.
The __createdtime__
and __updatedtime__
attributes that HarperDB manages internally can have read perms set but, if set, all other attribute-level permissions will be ignored.
Please note that DELETE permissions are not included as a part of an individual attribute-level permission set. That is because it is not possible to delete individual attributes from a row, rows must be deleted in full.
If a role needs the ability to delete rows from a table, that permission should be set on the table-level.
The practical approach to deleting an individual attribute of a row would be to set that attribute to null via an update statement.
The table below includes all API operations available in HarperDB and indicates whether or not the operation is restricted to super_user roles.
Keep in mind that non-super_user roles will also be restricted within the operations they do have access to by the schema-level CRUD permissions set for the roles.
You may have gotten an error like, Error: Must execute as <<username>>
.
This means that you installed HarperDB as <<user>>
. Because HarperDB stores files natively on the operating system, we only allow the HarperDB executable to be run by a single user. This prevents permissions issues on files.
For example if you installed as user_a, but later wanted to run as user_b. User_b may not have access to the hdb files HarperDB needs. This also keeps HarperDB more secure as it allows you to lock files down to a specific user and prevents other users from accessing your files.
Schemas and Tables | Restricted to Super_Users |
---|---|
NoSQL Operations | Restricted to Super_Users |
---|---|
SQL Operations | Restricted to Super_Users |
---|---|
Bulk Operations | Restricted to Super_Users |
---|---|
Users and Roles | Restricted to Super_Users |
---|---|
Clustering | Restricted to Super_Users |
---|---|
Custom Functions | Restricted to Super_Users |
---|---|
Registration | Restricted to Super_Users |
---|---|
Jobs | Restricted to Super_Users |
---|---|
Logs | Restricted to Super_Users |
---|---|
Utilities | Restricted to Super_Users |
---|---|
Token Authentication | Restricted to Super_Users |
---|---|
describe_all
describe_schema
describe_table
create_schema
X
drop_schema
X
create_table
X
drop_table
X
create_attribute
drop_attribute
X
insert
update
upsert
delete
search_by_hash
search_by_value
search_by_conditions
select
insert
update
delete
csv_data_load
csv_file_load
csv_url_load
import_from_s3
list_roles
X
add_role
X
alter_role
X
drop_role
X
list_users
X
user_info
add_user
X
alter_user
X
drop_user
X
cluster_set_routes
X
cluster_get_routes
X
cluster_delete_routes
X
add_node
X
update_node
X
cluster_status
X
remove_node
X
configure_cluster
X
custom_functions_status
X
get_custom_functions
X
get_custom_function
X
set_custom_function
X
drop_custom_function
X
add_custom_function_project
X
drop_custom_function_project
X
package_custom_function_project
X
deploy_custom_function_project
X
registration_info
get_fingerprint
X
set_license
X
get_job
search_jobs_by_start_date
X
read_log
X
read_transaction_log
X
delete_transaction_logs_before
X
read_audit_log
X
delete_audit_logs_before
X
delete_records_before
X
export_local
X
export_to_s3
X
system_information
X
restart
X
restart_service
X
get_configuration
X
configure_cluster
X
create_authentication_tokens
refresh_operation_token
HarperDB uses token based authentication with JSON Web Tokens, JWTs.
This consists of two primary operations create_authentication_tokens
and refresh_operation_token
. These generate two types of tokens, as follows:
The operation_token
which is used to authenticate all HarperDB operations in the Bearer Token Authorization Header. The default expiry is one day.
The refresh_token
which is used to generate a new operation_token
upon expiry. This token is used in the Bearer Token Authorization Header for the refresh_operation_token
operation only. The default expiry is thirty days.
The create_authentication_tokens
operation can be used at any time to refresh both tokens in the event that both have expired or been lost.
Users must initially create tokens using their HarperDB credentials. The following POST body is sent to HarperDB. No headers are required for this POST operation.
A full cURL example can be seen here:
An example expected return object is:
The operation_token
value is used to authenticate all operations in place of our standard Basic auth. In order to pass the token you will need to create an Bearer Token Authorization Header like the following request:
operation_token
expires at a set interval. Once it expires it will no longer be accepted by HarperDB. This duration defaults to one day, and is configurable in harperdb-config.yaml. To generate a new operation_token
, the refresh_operation_token
operation is used, passing the refresh_token
in the Bearer Token Authorization Header. A full cURL example can be seen here:
This will return a new operation_token
. An example expected return object is:
The refresh_token
also expires at a set interval, but a longer interval. Once it expires it will no longer be accepted by HarperDB. This duration defaults to thirty days, and is configurable in harperdb-config.yaml. To generate a new operation_token
and a new refresh_token
the create_authentication_tokensoperation
is called.
Token timeouts are configurable in harperdb-config.yaml with the following parameters:
operationsApi.authentication.operationTokenTimeout
: Defines the length of time until the operation_token expires (default 1d).
operationsApi.authentication.refreshTokenTimeout
: Defines the length of time until the refresh_token expires (default 30d).
A full list of valid values for both parameters can be found here.