(Experimental) Plugins
The new, experimental plugin API is an iteration of the existing extension system. It simplifies the API by removing the need for multiple methods (start
, startOnMainThread
, handleFile
, setupFile
, etc.) and instead only requires a single handleApplication
method. Plugins are designed to be more extensible and easier to use, and they are intended to replace the concept of extensions in the future.
Similar to the existing extension API, a plugin must specify an pluginModule
option within config.yaml
. This must be a path to the plugin module source code. The path must resolve from the root of the module directory. For example: pluginModule: plugin.js
.
If the plugin is being written in something other than JavaScript (such as TypeScript), ensure that the path resolves to the built version, (i.e. pluginModule: ./dist/index.js
)
It is also recommended that all extensions have a package.json
that specifies JavaScript package metadata such as name, version, type, etc. Since plugins are just JavaScript packages, they can do anything a JavaScript package can normally do. It can be written in TypeScript, and compiled to JavaScript. It can export an executable (using the bin property). It can be published to npm. The possibilities are endless!
The key to a plugin is the handleApplication()
method. It must be exported by the pluginModule
, and cannot coexist with any of the other extension methods such as start
, handleFile
, etc. The component loader will throw an error if both are defined.
The handleApplication()
method is executed sequentially across all worker threads during the component loading sequence. It receives a single, scope
argument that contains all of the relevant metadata and APIs for interacting with the associated component.
The method can be async and it is awaited by the component loader.
However, it is highly recommended to avoid event-loop-blocking operations within the handleApplication()
method. See the examples section for best practices on how to use the scope
argument effectively.
Configuration
As plugins are meant to be used by applications in order to implement some feature, many plugins provide a variety of configuration options to customize their behavior. Some plugins even require certain configuration options to be set in order to function properly.
As a brief overview, the general configuration options available for plugins are:
files -
string
|string[]
|FilesOptionsObject
- optional - A glob pattern string or array of strings that specifies the files and directories to be handled by the plugin's defaultEntryHandler
instance.urlPath -
string
- optional - A base URL path to prepend to the resolvedfiles
entries handled by the plugin's defaultEntryHandler
instance.timeout -
number
- optional - The timeout in milliseconds for the plugin's operations. If not specified, the system default is 30 seconds. Plugins may override the system default themselves, but this configuration option is the highest priority and takes precedence.
File Entries
Just like extensions, plugins support the files
and urlPath
options for file entry matching. The values specified for these options are used for the default EntryHandler
instance created by the scope.handleEntry()
method. As the reference documentation details, similar options can be used to create custom EntryHandler
instances too.
The files
option can be a glob pattern string, an array of glob pattern strings, or a more expressive glob options object.
The patterns cannot contain
..
or start with/
.The pattern
.
or./
is transformed into**/*
automatically.Often, it is best to omit a leading
.
or./
in the glob pattern.
The urlPath
option is a base URL path that is prepended to the resolved files
entries.
It cannot contain
..
.If it starts with
./
or is just.
, the name of the plugin will be automatically prepended to it.
Putting this all together, to configure the static built-in extension to serve files from the web
directory but at the /static/
path, the config.yaml
would look like this:
static:
files: 'web/**/*'
urlPath: '/static/'
Keep in mind the urlPath
option is completely optional.
As another example, to configure the graphqlSchema built-in extension to serve only *.graphql
files from within the top-level of the src/schema
directory, the config.yaml
would look like this:
graphqlSchema:
files: 'src/schema/*.graphql'
As detailed, the files
option also supports a more complex object syntax for advanced use cases.
For example, to match files within the web
directory, and omit any within web/images
, you can use a configuration such as:
static:
files:
source: 'web/**/*'
ignore: 'web/images/**'
If you're transitioning from the extension system, the
files
option object no longer supports anonly
field. Instead, use theentryEvent.entryType
or the specificentryEvent.eventType
fields inonEntryEventHandler(entryEvent)
method or any of the specificEntryHandler
events.
Timeouts
The default timeout for all plugins is 30 seconds. If the method does not complete within this time, the component loader will throw an error and unblock the component loading sequence. This is to prevent the component loader from hanging indefinitely if a plugin fails to respond or takes too long to execute.
The plugin module can export a defaultTimeout
variable (in milliseconds) that will override the system default.
For example:
export const defaultTimeout = 60_000; // 60 seconds
Additionally, users can specify a timeout
option in their application's config.yaml
file for a specific plugin. This option takes precedence over the plugin's defaultTimeout
and the system default.
For example:
customPlugin:
package: '@harperdb/custom-plugin'
files: 'foo.js'
timeout: 45_000 # 45 seconds
Example: Statically hosting files
This is a functional example of how the handleApplication()
method and scope
argument can be used to create a simple static file server plugin. This example assumes that the component has a config.yaml
with the files
option set to a glob pattern that matches the files to be served.
This is a simplified form of the static built-in extension.
export function handleApplication(scope) {
const staticFiles = new Map();
scope.options.on('change', (key, value, config) => {
if (key[0] === 'files' || key[0] === 'urlPath') {
// If the files or urlPath options change, we need to reinitialize the static files map
staticFiles.clear();
logger.info(`Static files reinitialized due to change in ${key.join('.')}`);
}
});
scope.handleEntry((entry) => {
if (entry.entryType === 'directory') {
logger.info(`Cannot serve directories. Update the files option to only match files.`);
return;
}
switch (entry.eventType) {
case 'add':
case 'change':
// Store / Update the file contents in memory for serving
staticFiles.set(entry.urlPath, entry.contents);
break;
case 'unlink':
// Remove the file from memory when it is deleted
staticFiles.delete(entry.urlPath);
break;
}
});
scope.server.http(
(req, next) => {
if (req.method !== 'GET') return next(req);
// Attempt to retrieve the requested static file from memory
const staticFile = staticFiles.get(req.pathname);
return staticFile
? {
statusCode: 200,
body: staticFile,
}
: {
statusCode: 404,
body: 'File not found',
};
},
{ runFirst: true }
);
}
In this example, the entry handler method passed to handleEntry
will manage the map of static files in memory using their computed urlPath
and the contents
. If the config file changes (and thus a new default file or url path is specified) the plugin will clear the file map as well to remove artifacts. Furthermore, it uses the server.http()
middleware to hook into the HTTP request handling.
This example is heavily simplified, but it demonstrates how the different key parts of scope
can be used together to provide a performant and reactive application experience.
Function: handleApplication(scope: Scope): void | Promise<void>
handleApplication(scope: Scope): void | Promise<void>
Parameters:
scope -
Scope
- An instance of theScope
class that provides access to the relative application's configuration, resources, and other APIs.
Returns: void | Promise<void>
This is the only method a plugin module must export. It can be async and is awaited by the component loader. The scope
argument provides access to the relative application's configuration, resources, and other APIs.
Class: Scope
Scope
Extends
EventEmitter
Event: 'close'
'close'
Emitted after the scope is closed via the close()
method.
Event: 'error'
'error'
error -
unknown
- The error that occurred.
Event: 'ready'
'ready'
Emitted when the Scope is ready to be used after loading the associated config file. It is awaited by the component loader, so it is not necessary to await it within the handleApplication()
method.
scope.close()
scope.close()
Returns: this
- The current Scope
instance.
Closes all associated entry handlers, the associated scope.options
instance, emits the 'close'
event, and then removes all other listeners on the instance.
scope.handleEntry([files][, handler])
scope.handleEntry([files][, handler])
Parameters:
files -
FilesOption
|FileAndURLPathConfig
|onEntryEventHandler
- optionalhandler -
onEntryEventHandler
- optional
Returns: EntryHandler
- An instance of the EntryHandler
class that can be used to handle entries within the scope.
The handleEntry()
method is the key to handling file system entries specified by a files
glob pattern option in config.yaml
. This method is used to register an entry event handler, specifically for the EntryHandler
'all'
event. The method signature is very flexible, and allows for the following variations:
scope.handleEntry()
(with no arguments) Returns the defaultEntryHandler
created by thefiles
andurlPath
options in theconfig.yaml
.scope.handleEntry(handler)
(wherehandler
is anonEntryEventHandler
) Returns the defaultEntryHandler
instance (based on the options withinconfig.yaml
) and uses the providedhandler
for the'all'
event.scope.handleEntry(files)
(wherefiles
isFilesOptions
orFileAndURLPathConfig
) Returns a newEntryHandler
instance that handles the specifiedfiles
configuration.scope.handleEntry(files, handler)
(wherefiles
isFilesOptions
orFileAndURLPathConfig
, andhandler
is anonEntryEventHandler
) Returns a newEntryHandler
instance that handles the specifiedfiles
configuration and uses the providedhandler
for the'all'
event.
For example:
export function handleApplication(scope) {
// Get the default EntryHandler instance
const defaultEntryHandler = scope.handleEntry();
// Assign a handler for the 'all' event on the default EntryHandler
scope.handleEntry((entry) => {
/* ... */
});
// Create a new EntryHandler for the 'src/**/*.js' files option with a custom `'all'` event handler.
const customEntryHandler = scope.handleEntry(
{
files: 'src/**/*.js',
},
(entry) => {
/* ... */
}
);
// Create another custom EntryHandler for the 'src/**/*.ts' files option, but without a `'all'` event handler.
const anotherCustomEntryHandler = scope.handleEntry({
files: 'src/**/*.ts',
});
}
And thus, if the previous code was used by a component with the following config.yaml
:
customPlugin:
files: 'web/**/*'
Then the default EntryHandler
instances would be created to handle all entries within the web
directory.
scope.requestRestart()
scope.requestRestart()
Returns: void
Request a Harper restart. This does not restart the instance immediately, but rather indicates to the user that a restart is required. This should be called when the plugin cannot handle the entry event and wants to indicate to the user that the Harper instance should be restarted.
This method is called automatically by the scope
instance if the user has not defined an scope.options.on('change')
handler or if an event handler exists and is missing a necessary handler method.
scope.resources
scope.resources
Returns: Map<string, Resource>
- A map of the currently loaded Resource instances.
scope.server
scope.server
Returns: server
- A reference to the server global API.
scope.options
scope.options
Returns: OptionsWatcher
- An instance of the OptionsWatcher
class that provides access to the application's configuration options. Emits 'change'
events when the respective plugin part of the component's config file is modified.
For example, if the plugin customPlugin
is configured by an application with:
customPlugin:
files: 'foo.js'
And has the following handleApplication(scope)
implementation:
export function handleApplication(scope) {
scope.options.on('change', (key, value, config) => {
if (key[0] === 'files') {
// Handle the change in the files option
scope.logger.info(`Files option changed to: ${value}`);
}
});
}
Then modifying the files
option in the config.yaml
to bar.js
would log the following:
Files option changed to: bar.js
scope.logger
scope.logger
Returns: logger
- A scoped instance of the logger
class that provides logging capabilities for the plugin.
It is recommended to use this instead of the logger
global.
scope.name
scope.name
Returns: string
- The name of the plugin as configured in the config.yaml
file. This is the key under which the plugin is configured.
scope.directory
scope.directory
Returns: string
- The directory of the application. This is the root directory of the component where the config.yaml
file is located.
Interface: FilesOption
FilesOption
string
|string[]
|FilesOptionsObject
Interface: FilesOptionsObject
FilesOptionsObject
source -
string
|string[]
- required - The glob pattern string or array of strings.ignore -
string
|string[]
- optional - An array of glob patterns to exclude from matches. This is an alternative way to use negative patterns. Defaults to[]
.
Interface: FileAndURLPathConfig
FileAndURLPathConfig
files -
FilesOption
- required - A glob pattern string, array of glob pattern strings, or a more expressive glob options object determining the set of files and directories to be resolved for the plugin.urlPath -
string
- optional - A base URL path to prepend to the resolvedfiles
entries.
Class: OptionsWatcher
OptionsWatcher
Extends
EventEmitter
Event: 'change'
'change'
key -
string[]
- The key of the changed option split into parts (e.g.foo.bar
becomes['foo', 'bar']
).value -
ConfigValue
- The new value of the option.config -
ConfigValue
- The entire configuration object of the plugin.
The 'change'
event is emitted whenever an configuration option is changed in the configuration file relative to the application and respective plugin.
Given an application using the following config.yaml
:
customPlugin:
files: 'web/**/*'
otherPlugin:
files: 'index.js'
The scope.options
for the respective plugin's customPlugin
and otherPlugin
would emit 'change'
events when the files
options relative to them are modified.
For example, if the files
option for customPlugin
is changed to web/**/*.js
, the following event would be emitted only within the customPlugin
scope:
scope.options.on('change', (key, value, config) => {
key; // ['files']
value; // 'web/**/*.js'
config; // { files: 'web/**/*.js' }
});
Event: 'close'
'close'
Emitted when the OptionsWatcher
is closed via the close()
method. The watcher is not usable after this event is emitted.
Event: 'error'
'error'
error -
unknown
- The error that occurred.
Event: 'ready'
'ready'
config -
ConfigValue
|undefined
- The configuration object of the plugin, if present.
This event can be emitted multiple times. It is first emitted upon the initial load, but will also be emitted after restoring a configuration file or configuration object after a 'remove'
event.
Event: 'remove'
'remove'
The configuration was removed. This can happen if the configuration file was deleted, the configuration object within the file is deleted, or if the configuration file fails to parse. Once restored, the 'ready'
event will be emitted again.
options.close()
options.close()
Returns: this
- The current OptionsWatcher
instance.
Closes the options watcher, removing all listeners and preventing any further events from being emitted. The watcher is not usable after this method is called.
options.get(key)
options.get(key)
Parameters:
key -
string[]
- The key of the option to get, split into parts (e.g.foo.bar
is represented as['foo', 'bar']
).
Returns: ConfigValue
| undefined
If the config is defined it will attempt to retrieve the value of the option at the specified key. If the key does not exist, it will return undefined
.
options.getAll()
options.getAll()
Returns: ConfigValue
| undefined
Returns the entire configuration object for the plugin. If the config is not defined, it will return undefined
.
options.getRoot()
options.getRoot()
Returns: Config
| undefined
Returns the root configuration object of the application. This is the entire configuration object, basically the parsed form of the config.yaml
. If the config is not defined, it will return undefined
.
Interface: Config
Config
[key: string]
ConfigValue
An object representing the config.yaml
file configuration.
Interface: ConfigValue
ConfigValue
string
|number
|boolean
|null
|undefined
|ConfigValue[]
|Config
Any valid configuration value type. Essentially, the primitive types, an array of those types, or an object comprised of values of those types.
Class: EntryHandler
EntryHandler
Extends: EventEmitter
Created by calling scope.handleEntry()
method.
Event: 'all'
'all'
entry -
FileEntry
|DirectoryEntry
- The entry that was added, changed, or removed.
The 'all'
event is emitted for all entry events, including file and directory events. This is the event that the handler method in scope.handleEntry
is registered for. The event handler receives an entry
object that contains the entry metadata, such as the file contents, URL path, and absolute path.
An effective pattern for this event is:
async function handleApplication(scope) {
scope.handleEntry((entry) => {
switch (entry.eventType) {
case 'add':
// Handle file addition
break;
case 'change':
// Handle file change
break;
case 'unlink':
// Handle file deletion
break;
case 'addDir':
// Handle directory addition
break;
case 'unlinkDir':
// Handle directory deletion
break;
}
});
}
Event: 'add'
'add'
entry -
AddFileEvent
- The file entry that was added.
The 'add'
event is emitted when a file is created (or the watcher sees it for the first time). The event handler receives an AddFileEvent
object that contains the file contents, URL path, absolute path, and other metadata.
Event: 'addDir'
'addDir'
entry -
AddDirEvent
- The directory entry that was added.
The 'addDir'
event is emitted when a directory is created (or the watcher sees it for the first time). The event handler receives an AddDirEvent
object that contains the URL path and absolute path of the directory.
Event: 'change'
'change'
entry -
ChangeFileEvent
- The file entry that was changed.
The 'change'
event is emitted when a file is modified. The event handler receives a ChangeFileEvent
object that contains the updated file contents, URL path, absolute path, and other metadata.
Event: 'close'
'close'
Emitted when the entry handler is closed via the entryHandler.close()
method.
Event: 'error'
'error'
error -
unknown
- The error that occurred.
Event: 'ready'
'ready'
Emitted when the entry handler is ready to be used. This is not automatically awaited by the component loader, but also is not required. Calling scope.handleEntry()
is perfectly sufficient. This is generally useful if you need to do something after the entry handler is absolutely watching and handling entries.
Event: 'unlink'
'unlink'
entry -
UnlinkFileEvent
- The file entry that was deleted.
The 'unlink'
event is emitted when a file is deleted. The event handler receives an UnlinkFileEvent
object that contains the URL path and absolute path of the deleted file.
Event: 'unlinkDir'
'unlinkDir'
entry -
UnlinkDirEvent
- The directory entry that was deleted.
The 'unlinkDir'
event is emitted when a directory is deleted. The event handler receives an UnlinkDirEvent
object that contains the URL path and absolute path of the deleted directory.
entryHandler.name
entryHandler.name
Returns: string
- The name of the plugin as configured in the config.yaml
file. This is the key under which the plugin is configured.
The name of the plugin.
entryHandler.directory
entryHandler.directory
Returns: string
The directory of the application. This is the root directory of the component where the config.yaml
file is located.
entryHandler.close()
entryHandler.close()
Returns: this
- The current EntryHandler
instance.
Closes the entry handler, removing all listeners and preventing any further events from being emitted. The handler can be started again using the entryHandler.update()
method.
entryHandler.update(config)
entryHandler.update(config)
Parameters:
config -
FilesOption
|FileAndURLPathConfig
- The configuration object for the entry handler.
This method will update an existing entry handler to watch new entries. It will close the underlying watcher and create a new one, but will maintain any existing listeners on the EntryHandler instance itself.
This method returns a promise associated with the ready event of the updated handler.
Interface: BaseEntry
BaseEntry
stats -
fs.Stats
|undefined
- The file system stats for the entry.urlPath -
string
- The recommended URL path of the entry.absolutePath -
string
- The absolute path of the entry.
The foundational entry handle event object. The stats
may or may not be present depending on the event, entry type, and platform.
The urlPath
is resolved based on the configured pattern (files:
option) combined with the optional urlPath
option. This path is generally useful for uniquely representing the entry. It is used in the built-in components such as jsResource
and static
.
The absolutePath
is the file system path for the entry.
Interface: FileEntry
FileEntry
Extends BaseEntry
contents -
Buffer
- The contents of the file.
A specific extension of the BaseEntry
interface representing a file entry. We automatically read the contents of the file so the user doesn't have to bother with FS operations.
There is no DirectoryEntry
since there is no other important metadata aside from the BaseEntry
properties. If a user wants the contents of a directory, they should adjust the pattern to resolve files instead.
Interface: EntryEvent
EntryEvent
Extends BaseEntry
eventType -
string
- The type of entry event.entryType -
string
- The type of entry, either a file or a directory.
A general interface representing the entry handle event objects.
Interface: AddFileEvent
AddFileEvent
Extends EntryEvent
, FileEntry
eventType -
'add'
entryType -
'file'
Event object emitted when a file is created (or the watcher sees it for the first time).
Interface: ChangeFileEvent
ChangeFileEvent
Extends EntryEvent
, FileEntry
eventType -
'change'
entryType -
'file'
Event object emitted when a file is modified.
Interface: UnlinkFileEvent
UnlinkFileEvent
Extends EntryEvent
, FileEntry
eventType -
'unlink'
entryType -
'file'
Event object emitted when a file is deleted.
Interface: FileEntryEvent
FileEntryEvent
AddFileEvent
|ChangeFileEvent
|UnlinkFileEvent
A union type representing the file entry events. These events are emitted when a file is created, modified, or deleted. The FileEntry
interface provides the file contents and other metadata.
Interface: AddDirEvent
AddDirEvent
Extends EntryEvent
eventType -
'addDir'
entryType -
'directory'
Event object emitted when a directory is created (or the watcher sees it for the first time).
Interface: UnlinkDirEvent
UnlinkDirEvent
Extends EntryEvent
eventType -
'unlinkDir'
entryType -
'directory'
Event object emitted when a directory is deleted.
Interface: DirectoryEntryEvent
DirectoryEntryEvent
AddDirEvent
|UnlinkDirEvent
A union type representing the directory entry events. There are no change events for directories since they are not modified in the same way as files.
Function: onEntryEventHandler(entryEvent: FileEntryEvent | DirectoryEntryEvent): void
onEntryEventHandler(entryEvent: FileEntryEvent | DirectoryEntryEvent): void
Parameters:
entryEvent -
FileEntryEvent
|DirectoryEntryEvent
Returns: void
This function is what is passed to the scope.handleEntry()
method as the handler for the 'all'
event. This is also applicable to a custom .on('all', handler)
method for any EntryHandler
instance.
Last updated