Configuration

Flowgen reads a single YAML file (passed via --config). Top-level sections:

flows:             # Flow discovery (required).
cache:             # Distributed cache backend (optional).
resources:         # External resource loading (optional).
http_server:       # Webhook / metrics HTTP server (optional).
mcp_server:        # MCP server for tools/resources/prompts (optional).
ai_gateway:        # OpenAI-compatible LLM gateway (optional).
web:               # Admin web UI (optional).
retry:             # Default retry policy for every task (optional).
event_buffer_size: # Per-edge channel capacity (optional).
telemetry:         # OpenTelemetry export (optional).

Only flows is required. Every other section has working defaults.

Minimal config

flows:
  path: /etc/flowgen/flows/

Discovers every .yaml/.yml/.json file under the path and runs them with an in-memory cache, no resources directory, no HTTP server, and no telemetry.

Full reference

flows:
  path: /etc/flowgen/flows/
  cache:
    enabled: true
    prefix: flowgen.flows
    db_name: flowgen_system

cache:
  enabled: true
  type: nats
  credentials_path: /etc/nats/credentials.json
  url: "{{env.NATS_URL}}"
  db_name: flowgen_cache
  history: 10
  tombstone_ttl: "1h"

resources:
  path: /etc/flowgen/resources/

http_server:
  enabled: true
  port: 3000
  path: /api/flowgen/workers/v1
  credentials_path: /etc/flowgen/credentials/http.json
  auth:
    type: jwt
    secret: "shared-secret"

mcp_server:
  enabled: true
  port: 3001
  path: /mcp/v1
  credentials_path: /etc/flowgen/credentials/mcp.json

ai_gateway:
  enabled: true
  port: 3002
  path: /v1
  credentials_path: /etc/flowgen/credentials/ai.json

web:
  enabled: true
  port: 8080
  path: /flowgen

retry:
  max_attempts: 10
  initial_backoff: "1s"

event_buffer_size: 10000

telemetry:
  enabled: true
  otlp_endpoint: http://localhost:4317
  service_name: flowgen
  metrics_export_interval: "60s"

flows

Flow discovery. Filesystem and distributed cache can be used independently or together; flowgen merges flows from both sources at startup.

Both sources can be active at the same time. Use this for gradual migration: a small set of bootstrap flows mounted from disk (typically a sync flow that pulls user flows from Git into the cache), with the bulk of user flows loaded from the cache. On a name collision the filesystem entry wins, so a locally mounted bootstrap flow cannot be silently overridden by a stale cache entry.

Cache-mode flow loading is useful when flows are provisioned dynamically — for example, a control plane writes flow YAML into NATS KV and every flowgen replica picks them up.

cache

Distributed cache backend. When omitted, flowgen uses an in-memory cache (single-node only).

If NATS is configured but unreachable, flowgen falls back to in-memory automatically and logs a warning. See Caching.

resources

External resource loading for SQL queries, prompts, scripts, schemas. See Resources.

When resources is omitted, only inline content is supported — any task that uses { resource: ... } will fail at startup with a clear error.

http_server

HTTP server that hosts every http_endpoint route plus health/metrics endpoints.

mcp_server

MCP server for exposing mcp_tool, mcp_resource, and mcp_prompt tasks to LLM clients.

ai_gateway

OpenAI-compatible LLM gateway that serves every registered llm_proxy flow.

web

Embedded admin dashboard for inspecting loaded flows and resources at runtime.

The UI lists loaded flows with description, tags, status, and last-run timestamp; click a row to preview the raw source YAML with syntax highlighting. A Resources tab browses everything under the configured resources.path and renders .md files as HTML plus code files (SQL, YAML, JSON, Rhai, JS/TS, Bash, Python) with Prism.

retry

Sets the default retry policy for every task. Individual tasks can override via their own retry field.

See Retry for the two retry patterns (circuit breaker for processors, infinite reconnect for subscribers).

event_buffer_size

Each pair of connected tasks shares a bounded mpsc channel. event_buffer_size sets the channel capacity (in events). When the channel fills, the upstream task blocks until the downstream task drains a slot — this is how backpressure propagates through the flow. No events are dropped.

The default (10,000) is sufficient for most workloads. The buffer only needs to absorb the gap between producer and consumer processing rates; downstream throughput is determined by task processing speed, not channel depth. Increase it if you observe producer stalls in flows with very bursty fan-out patterns and fast consumers.

telemetry

OpenTelemetry export over OTLP/gRPC. See Telemetry.

When telemetry is omitted entirely, no OTLP exporter starts but tracing logs still go to stderr.

Running

flowgen --config /etc/flowgen/config.yaml

Flowgen validates the config at startup. Any field with a typo, missing required value, or invalid type produces an error before any flow runs.