Ciaren

Writing a Ciaren plugin

Writing a Ciaren plugin

A plugin can contribute to the catalog (nodes/connectors/model types/engines/ exporters/validators), declare capabilities and permissions, ship executable nodes, connectors, and ML model types that run end-to-end (via NodeRuntime, ConnectorRuntime, and ModelProvider builders), and subscribe to lifecycle/execution events.

A plugin is a small Python package that implements the Plugin contract and registers one or more providers. It depends only on the Ciaren plugin API (app.plugin_api, which will publish separately as ciaren-plugin-api) — never on Ciaren's private internals.

See a complete, runnable example in examples/plugins/hello-node-plugin/.

1. Implement Plugin

from app.plugin_api import (
    NodeProvider, NodeSpec, Plugin, PluginMetadata, PortSpec, ServiceRegistry,
)

class _MyNodes(NodeProvider):
    def nodes(self) -> list[NodeSpec]:
        return [
            NodeSpec(
                id="acme.greeting",
                label="Add Greeting",
                category="columns",
                description="Adds a greeting column.",
                provider="acme.hello",
                inputs=(PortSpec(id="in"),),
                outputs=(PortSpec(id="out"),),
                default_config={"name": "world"},
                capabilities=("node.greeting",),
            )
        ]

class AcmePlugin(Plugin):
    def metadata(self) -> PluginMetadata:
        return PluginMetadata(id="acme.hello", name="Acme Hello", version="0.1.0-alpha.1")

    def register(self, registry: ServiceRegistry) -> None:
        registry.register_node_provider(_MyNodes())

Other provider interfaces you can register: ConnectorProvider (executable connectors →), ModelProvider (ML model types →), StorageProvider, ExecutionProvider, ExporterProvider, ValidatorProvider, AIProvider, AuthProvider, LicenseProvider.

Make the node executable (NodeRuntime)

A NodeSpec only describes a node. To make it run, ship a NodeRuntime and return it from the provider's node_implementations(), keyed by node id. The runtime works on pandas frames; Ciaren bridges to the active engine (pandas/polars) automatically, so a single runtime runs on both.

from app.plugin_api import NodeRuntime

class GreetingRuntime(NodeRuntime):
    def execute(self, inputs, config):
        df = inputs["in"].copy()
        df[config["column"]] = f"Hello, {config.get('name', 'world')}!"
        return {"out": df}

    # Optional: makes "Export Python" work for this node.
    def to_python_code(self, input_vars, output_vars, config):
        col, name = config["column"], config.get("name", "world")
        return f"{output_vars['out']} = {input_vars['in']}.assign(**{{{col!r}: {f'Hello, {name}!'!r}}})"

class _MyNodes(NodeProvider):
    def nodes(self): ...
    def node_implementations(self):
        return {"acme.greeting": GreetingRuntime()}

Once registered the node executes in runs and previews, passes graph validation, and (if to_python_code is implemented) appears in both the pandas and polars exports — exactly like a built-in.

Give the node a real sidebar form (config_schema)

Declare the node's form on its spec and the editor renders it — labeled inputs, selects, checkboxes, column pickers — no frontend code:

NodeSpec(
    id="acme.greeting",
    ...,
    config_schema={"fields": [
        {"key": "column", "label": "New column", "type": "string", "required": True},
        {"key": "name", "label": "Greet who", "type": "string", "default": "world"},
        {"key": "shout", "label": "Uppercase", "type": "boolean", "default": False},
    ]},
)

Field types: string, number, integer, boolean, select (+ options), string_list, and column/column_list (resolved against the columns arriving on the node's wire). Without a schema, the editor falls back to fields inferred from default_config, so every plugin node stays configurable.

Host services in the runtime (NodeContext)

Ciaren actually invokes execute_with_context(inputs, config, context); the default implementation delegates to execute, so simple runtimes never notice. Override it when you need the context:

  • context.in_preview — True during editor previews on sampled data; skip training/persisting and return a cheap placeholder.
  • context.models — the MLflow-backed ModelStore for train-style nodes (see ML Model Plugins).
  • context.permissions — the permissions the user actually granted.
  • context.license_token — this plugin's own signed license token (raw JSON, or ""). Forward it to your server to build a thin-client paid node whose logic and license check run server-side; see thin-client plugins.

Node categories

NodeSpec.category controls where the node lands in the editor palette. Use a built-in category (input, clean, columns, reshape, analytics, quality, chart, ml, output, or plugins) to slot it into that section. Unknown values are normalized to plugins, so the node still renders and runs normally without the frontend needing a new palette section.

React to events

A plugin can subscribe to lifecycle and execution hooks via registry.events inside register(). Subscribers are error-isolated (a raising hook is logged and skipped) and run synchronously in registration order.

from app.plugin_api import Hook

class AuditPlugin(Plugin):
    def metadata(self): ...
    def register(self, registry):
        registry.events.subscribe(Hook.after_graph_execute, self._log_run)

    def _log_run(self, *, flow_id, run_id, status, **_):
        print(f"[audit] flow {flow_id} run {run_id}: {status}")

Emitted today (app.plugin_api.Hook): plugin_enabled, plugin_disabled, before_graph_execute, after_graph_execute, before_node_execute, after_node_execute, and export_requested. Graph-level and export hooks fire for every run/export; node-level hooks fire in the in-process (thread) execution path — in process mode a worker can't reach parent subscribers, so prefer graph-level hooks for cross-mode behaviour.

Reserved (defined for a stable namespace but not emitted yet — don't rely on them firing): plugin_installed (install runs in the CLI, a separate process), project_created / project_opened / project_saved, graph_loaded, and graph_validated.

2. Add a manifest

ciaren-plugin.json at the plugin directory root — see plugin-manifest.md. The loader validates it and checks compatibility on two independent axes before importing your code: the Ciaren app version (ciaren specifier) and the plugin-contract version (api_version vs the backend's PLUGIN_API_VERSION). An incompatible plugin on either axis is rejected up front and reported in /api/plugins/diagnostics — it never runs. The contract version bumps only when app.plugin_api changes, independently of your plugin's own version; while it is pre-1.0 (alpha) it makes no backward-compatibility promise, so target the exact version the backend reports and rebuild when it bumps. See Contract versioning.

You don't have to hand-write it. Because your Plugin already declares the id, version, permissions, nodes, and categories, generate the manifest from the code so the two can't drift:

ciaren-plugin manifest ./my-plugin              # writes ciaren-plugin.json
ciaren-plugin manifest ./my-plugin --out -      # or print it to stdout

This still ships and validates the manifest before any code runs — it only removes the duplicated, drift-prone copy.

3. Make it discoverable

Installed package — declare an entry point in pyproject.toml:

[project.entry-points."ciaren.plugins"]
acme = "acme_hello.plugin:AcmePlugin"

Local directory (no install) — drop the plugin directory under a path on CIAREN_PLUGINS_DIR (or ~/.ciaren/plugins), with its ciaren-plugin.json declaring an entrypoint.

4. Verify

ciaren serve
curl localhost:8055/api/plugins
curl localhost:8055/api/plugins/diagnostics   # shows isolated load errors
curl localhost:8055/api/catalog/nodes          # your node appears here

Rules

  • Depend only on app.plugin_api; never import Ciaren internals.
  • Ids must be unique. A plugin cannot shadow a core node id — the registry rejects the collision and rolls the whole plugin back.
  • A failing or incompatible plugin is isolated: it shows up under /api/plugins/diagnostics, it does not crash the app.
PreviousBuild Your First Plugin (10 minutes)NextML Model Plugins