Embedded javascript plugins (#1393)

This commit is contained in:
WithoutPants
2021-05-26 14:17:53 +10:00
committed by GitHub
parent cc5ec650ae
commit 9b57fbbf50
108 changed files with 30179 additions and 91 deletions

View File

@@ -0,0 +1,147 @@
# Embedded Plugins
Embedded plugins are executed within the stash process using a scripting system.
## Supported script languages
Stash currently supports Javascript embedded plugins using [otto](https://github.com/robertkrimen/otto).
# Javascript plugins
## Plugin input
The input is provided to Javascript plugins using the `input` global variable, and is an object based on the structure provided in the `Plugin input` section of the [Plugins](/help/Plugins.md) page. Note that the `server_connection` field should not be necessary in most embedded plugins.
## Plugin output
The output of a Javascript plugin is derived from the evaluated value of the script. The output should conform to the structure provided in the `Plugin output` section of the [Plugins](/help/Plugins.md) page.
There are a number of ways to return the plugin output:
### Example #1
```
(function() {
return {
Output: "ok"
};
})();
```
### Example #2
```
function main() {
return {
Output: "ok"
};
}
main();
```
### Example #3
```
var output = {
Output: "ok"
};
output;
```
## Logging
See the `Javascript API` section below on how to log with Javascript plugins.
# Plugin configuration file format
The basic structure of an embedded plugin configuration file is as follows:
```
name: <plugin name>
description: <optional description of the plugin>
version: <optional version tag>
url: <optional url>
exec:
- <path to script>
interface: [interface type]
tasks:
- ...
```
The `name`, `description`, `version` and `url` fields are displayed on the plugins page.
## exec
For embedded plugins, the `exec` field is a list with the first element being the path to the Javascript file that will be executed. It is expected that the path to the Javascript file is relative to the directory of the plugin configuration file.
## interface
For embedded plugins, the `interface` field must be set to one of the following values:
* `js`
# Task configuration
Tasks are configured using the following structure:
```
tasks:
- name: <operation name>
description: <optional description>
defaultArgs:
argKey: argValue
```
A plugin configuration may contain multiple tasks.
The `defaultArgs` field is used to add inputs to the plugin input sent to the plugin.
# Javascript API
## Logging
Stash provides the following API for logging in Javascript plugins:
| Method | Description |
|--------|-------------|
| `log.Trace(<string>)` | Log with the `trace` log level. |
| `log.Debug(<string>)` | Log with the `debug` log level. |
| `log.Info(<string>)` | Log with the `info` log level. |
| `log.Warn(<string>)` | Log with the `warn` log level. |
| `log.Error(<string>)` | Log with the `error` log level. |
| `log.Progress(<float between 0 and 1>)` | Sets the progress of the plugin task, as a float, where `0` represents 0% and `1` represents 100%. |
## GQL
Stash provides the following API for communicating with stash using the graphql interface:
| Method | Description |
|--------|-------------|
| `gql.Do(<query/mutation string>, <variables object>)` | Executes a graphql query/mutation on the stash server. Returns an object in the same way as a graphql query does. |
### Example
```
// creates a tag
var mutation = "\
mutation tagCreate($input: TagCreateInput!) {\
tagCreate(input: $input) {\
id\
}\
}";
var variables = {
input: {
'name': tagName
}
};
result = gql.Do(mutation, variables);
log.Info("tag id = " + result.tagCreate.id);
```
## Utility functions
Stash provides the following API for utility functions:
| Method | Description |
|--------|-------------|
| `util.Sleep(<milliseconds>)` | Suspends the current thread for the specified duration. |

View File

@@ -0,0 +1,108 @@
# External Plugins
External plugins are executed by running an external binary.
## Plugin interfaces
Stash communicates with external plugins using an interface. Stash currently supports RPC and raw interface types.
### RPC interface
The RPC interface uses JSON-RPC to communicate with the plugin process. A golang plugin utilising the RPC interface is available in the stash source code under `pkg/plugin/examples/gorpc`. RPC plugins are expected to provide an interface that fulfils the `RPCRunner` interface in `pkg/plugin/common`.
RPC plugins are expected to accept requests asynchronously.
When stopping an RPC plugin task, the stash server sends a stop request to the plugin and relies on the plugin to stop itself.
### Raw interface
Raw interface plugins are not required to conform to any particular interface. The stash server will send the plugin input to the plugin process via its stdin stream, encoded as JSON. Raw interface plugins are not required to read the input.
The stash server reads stdout for the plugin's output. If the output can be decoded as a JSON representation of the plugin output data structure then it will do so. If not, it will treat the entire stdout string as the plugin's output.
When stopping a raw plugin task, the stash server kills the spawned process without warning or signals.
## Logging
External plugins may log to the stash server by writing to stderr. By default, data written to stderr will be logged by stash at the `error` level. This default behaviour can be changed by setting the `errLog` field in the plugin configuration file.
Plugins can log for specific levels or log progress by prefixing the output string with special control characters. See `pkg/plugin/common/log` for how this is done in go.
# Plugin configuration file format
The basic structure of an external plugin configuration file is as follows:
```
name: <plugin name>
description: <optional description of the plugin>
version: <optional version tag>
url: <optional url>
exec:
- <binary name>
- <other args...>
interface: [interface type]
errLog: [one of none trace, debug, info, warning, error]
tasks:
- ...
```
The `name`, `description`, `version` and `url` fields are displayed on the plugins page.
## exec
For external plugins, the `exec` field is a list with the first element being the binary that will be executed, and the subsequent elements are the arguments passed. The execution process will search the path for the binary, then will attempt to find the program in the same directory as the plugin configuration file. The `exe` extension is not necessary on Windows systems.
> **⚠️ Note:** The plugin execution process sets the current working directory to that of the stash process.
Arguments can include the plugin's directory with the special string `{pluginDir}`.
For example, if the plugin executable `my_plugin` is placed in the `plugins` subdirectory and requires arguments `foo` and `bar`, then the `exec` part of the configuration would look like the following:
```
exec:
- my_plugin
- foo
- bar
```
Another example might use a python script to execute the plugin. Assuming the python script `foo.py` is placed in the same directory as the plugin config file, the `exec` fragment would look like the following:
```
exec:
- python
- {pluginDir}/foo.py
```
## interface
For external plugins, the `interface` field must be set to one of the following values:
* `rpc`
* `raw`
See the `Plugin interfaces` section above for details on these interface types.
The `interface` field defaults to `raw` if not provided.
## errLog
The `errLog` field tells stash what the default log level should be when the plugin outputs to stderr without encoding a log level. It defaults to the `error` level if no provided. This field is not necessary if the plugin outputs logging with the appropriate encoding. See the `Logging` section above for details.
# Task configuration
Tasks are configured using the following structure:
```
tasks:
- name: <operation name>
description: <optional description>
defaultArgs:
argKey: argValue
execArgs:
- <arg to add to the exec line>
```
A plugin configuration may contain multiple tasks.
The `defaultArgs` field is used to add inputs to the plugin input sent to the plugin.
The `execArgs` field allows adding extra parameters to the execution arguments for this task.

View File

@@ -1,6 +1,6 @@
# Plugins
Stash supports the running external tasks via plugins. Plugins are implemented by calling an external binary.
Stash supports the running tasks via plugins. Plugins can be implemented using embedded Javascript, or by calling an external binary.
> **⚠️ Note:** Plugin support is still experimental and is likely to change.
@@ -16,74 +16,11 @@ Loaded plugins can be viewed in the Plugins page of the Settings. After plugins
Plugins provide tasks which can be run from the Tasks page.
> **⚠️ Note:** It is currently only possible to run one task at a time. No queuing is currently implemented.
# Creating plugins
# Plugin configuration file format
See [External Plugins](/help/ExternalPlugins.md) for details for making external plugins.
The basic structure of a plugin configuration file is as follows:
```
name: <plugin name>
description: <optional description of the plugin>
version: <optional version tag>
url: <optional url>
exec:
- <binary name>
- <other args...>
interface: [interface type]
errLog: [one of none trace, debug, info, warning, error]
tasks:
- ...
```
## Plugin process execution
The `exec` field is a list with the first element being the binary that will be executed, and the subsequent elements are the arguments passed. The execution process will search the path for the binary, then will attempt to find the program in the same directory as the plugin configuration file. The `exe` extension is not necessary on Windows systems.
> **⚠️ Note:** The plugin execution process sets the current working directory to that of the stash process.
Arguments can include the plugin's directory with the special string `{pluginDir}`.
For example, if the plugin executable `my_plugin` is placed in the `plugins` subdirectory and requires arguments `foo` and `bar`, then the `exec` part of the configuration would look like the following:
```
exec:
- my_plugin
- foo
- bar
```
Another example might use a python script to execute the plugin. Assuming the python script `foo.py` is placed in the same directory as the plugin config file, the `exec` fragment would look like the following:
```
exec:
- python
- {pluginDir}/foo.py
```
## Plugin interfaces
The `interface` field currently accepts one of two possible values: `rpc` and `raw`. It defaults to `raw` if not provided.
Plugins may log to the stash server by writing to stderr. By default, data written to stderr will be logged by stash at the `error` level. This default behaviour can be changed by setting the `errLog` field.
Plugins can log for specific levels or log progress by prefixing the output string with special control characters. See `pkg/plugin/common/log` for how this is done in go.
### RPC interface
The RPC interface uses JSON-RPC to communicate with the plugin process. A golang plugin utilising the RPC interface is available in the stash source code under `pkg/plugin/examples/gorpc`. RPC plugins are expected to provide an interface that fulfils the `RPCRunner` interface in `pkg/plugin/common`.
RPC plugins are expected to accept requests asynchronously.
When stopping an RPC plugin task, the stash server sends a stop request to the plugin and relies on the plugin to stop itself.
### Raw interface
Raw interface plugins are not required to conform to any particular interface. The stash server will send the plugin input to the plugin process via its stdin stream, encoded as JSON. Raw interface plugins are not required to read the input.
The stash server reads stdout for the plugin's output. If the output can be decoded as a JSON representation of the plugin output data structure then it will do so. If not, it will treat the entire stdout string as the plugin's output.
When stopping a raw plugin task, the stash server kills the spawned process without warning or signals.
See [Embedded Plugins](/help/EmbeddedPlugins.md) for details for making embedded plugins.
## Plugin input
@@ -116,7 +53,7 @@ Plugins may accept an input from the stash server. This input is encoded accordi
}
```
The `server_connection` field contains all the information needed for a plugin to access the parent stash server.
The `server_connection` field contains all the information needed for a plugin to access the parent stash server, if necessary.
## Plugin output
@@ -130,23 +67,3 @@ Plugin output is expected in the following structure (presented here as JSON for
```
The `error` field is logged in stash at the `error` log level if present. The `output` is written at the `debug` log level.
## Task configuration
Tasks are configured using the following structure:
```
tasks:
- name: <operation name>
description: <optional description>
defaultArgs:
argKey: argValue
execArgs:
- <arg to add to the exec line>
```
A plugin configuration may contain multiple tasks.
The `defaultArgs` field is used to add inputs to the plugin input sent to the plugin.
The `execArgs` field allows adding extra parameters to the execution arguments for this task.