Python API¶
Everything below is importable from cereyan unless a module path is shown. This page is rendered from the docstrings in python/cereyan/ by mkdocstrings; edit the docstrings, not this page.
Decorators and the App¶
flow
¶
flow(fn: Callable | None = None, *, name: str | None = None, description: str | None = None, tags: Iterable[str] = (), group: str | None = None, run_name: str | Callable[..., str] | None = None, isolated: bool = False, log_prints: bool = False, app=None, **options)
Register a function as a flow on app or on the default App.
Use as @flow or @flow(...). Calling the returned Flow runs it:
offline, the run is recorded in the local store; while a server is up, the run
is handed to the server and its logs stream back. Parameters come from the
function's type hints and are coerced before the body runs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fn
|
Callable
|
The function to wrap; supplied by the decorator syntax. |
None
|
name
|
str | None
|
Flow name; defaults to the function name. A flow's identity is
|
None
|
description
|
str | None
|
Shown in the UI; defaults to the function's docstring. |
None
|
tags
|
Iterable[str]
|
Tags copied onto every run. |
()
|
group
|
str | None
|
The group this flow is listed under in the UI; defaults to the flow's project. Groups are a flat axis, not a level inside the project: flows in different projects declaring the same group form one group, and a group named after a project merges with the flows that default to it. |
None
|
run_name
|
str | Callable[..., str] | None
|
A |
None
|
isolated
|
bool
|
Run every run in a fresh engine process instead of the warm pool. |
False
|
log_prints
|
bool
|
Tee |
False
|
app
|
App | None
|
The |
None
|
schedule
|
Cron | Interval | RRule | list | None
|
A |
required |
schedules
|
Cron | Interval | RRule | list | None
|
Same as |
required |
retries
|
int
|
How many times a failed run is retried. |
required |
retry_delay
|
float | list[float] | exponential
|
Wait before each retry: seconds, a list of per-attempt seconds,
or |
required |
timeout_seconds
|
float | None
|
End the run as Failed with sub-state |
required |
crash_retries
|
int | None
|
How many times a run whose engine died is rerun before it is
marked Failed; defaults to |
required |
priority
|
int
|
Dispatch order among queued runs, higher first; never preempts a running run. A negative value also raises the engine's OS niceness on Linux and macOS. |
required |
max_concurrent
|
int | None
|
Cap on runs of this flow in Pending or Running at once, implemented as a resource named after the flow; unlimited by default. |
required |
on_overlap
|
str
|
What a new run does when |
required |
resources
|
dict[str, float] | None
|
Named resources and the amount each run holds, for example
|
required |
after
|
str | tuple | list[str] | None
|
Upstream dependency: a flow name, |
required |
batch_key
|
str | None
|
With a list in |
required |
disable_after
|
tuple[int, int, int] | None
|
|
required |
runner
|
ThreadRunner | ProcessRunner | None
|
A |
required |
bulk_complete
|
Callable | None
|
|
required |
on_completion
|
Iterable[Callable]
|
Hooks |
required |
on_failure
|
Iterable[Callable]
|
Hooks called after a run fails. |
required |
on_crashed
|
Iterable[Callable]
|
Hooks called after a run crashes. |
required |
on_cancellation
|
Iterable[Callable]
|
Hooks called after a run is cancelled. |
required |
Returns:
| Type | Description |
|---|---|
Flow
|
The flow wrapping |
Flow
¶
Flow(fn: Callable, *, name: str | None = None, description: str | None = None, tags: Iterable[str] = (), group: str | None = None, run_name: str | Callable[..., str] | None = None, isolated: bool = False, log_prints: bool = False, schedule: Any = None, schedules: Any = None, retries: int = 0, retry_delay: Any = 0, timeout_seconds: float | None = None, crash_retries: int | None = None, priority: int = 0, max_concurrent: int | None = None, on_overlap: str = 'enqueue', resources: dict[str, float] | None = None, after: Any = None, batch_key: str | None = None, disable_after: tuple | None = None, runner: Any = None, bulk_complete: Callable | None = None, on_completion: Iterable[Callable] = (), on_failure: Iterable[Callable] = (), on_crashed: Iterable[Callable] = (), on_cancellation: Iterable[Callable] = ())
A registered flow: the wrapped function plus its options, parameters, and identity.
Instances are created by flow; call one to start a run. Attributes mirror the
decorator options, and parameters and schema describe the parameters derived
from the function's type hints.
task
¶
Register a function as a task: each call inside a flow becomes a task run.
Use as @task or @task(...). Outside a flow the function runs as-is. Inside a
flow each call is recorded as a task run with a dynamic key such as load-0,
and submit and map run calls concurrently on the flow's runner.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fn
|
Callable
|
The function to wrap; supplied by the decorator syntax. |
None
|
name
|
str | None
|
Task name; defaults to the function name. |
required |
description
|
str | None
|
Shown in the UI; defaults to the function's docstring. |
required |
tags
|
Iterable[str]
|
Tags recorded on every task run. |
required |
log_prints
|
bool
|
Tee |
required |
retries
|
int
|
How many times a failed task run is retried. |
required |
retry_delay
|
float | list[float] | exponential
|
Wait before each retry: seconds, a list of per-attempt seconds,
or |
required |
timeout_seconds
|
float | None
|
Fail the task run with sub-state |
required |
output
|
Target | Callable[..., Target] | None
|
A Target, or a callable over the task's arguments returning one; when the target exists the task run ends Skipped without executing. |
required |
cache
|
CachePolicy | None
|
|
required |
cache_expires
|
timedelta | None
|
A |
required |
persist_result
|
bool
|
Store the return value under |
required |
serializer
|
str
|
|
required |
resources
|
dict[str, float] | None
|
Named resources and the amount each task run holds, for example
|
required |
on_completion
|
Iterable[Callable]
|
Hooks |
required |
on_failure
|
Iterable[Callable]
|
Hooks called after a task run fails. |
required |
on_cancellation
|
Iterable[Callable]
|
Hooks called after a task run is cancelled. |
required |
Returns:
| Type | Description |
|---|---|
Task
|
The task wrapping |
Task
¶
Task(fn: Callable, *, name: str | None = None, description: str | None = None, tags: Iterable[str] = (), log_prints: bool = False, retries: int = 0, retry_delay: Any = 0, timeout_seconds: float | None = None, output: Any = None, cache: CachePolicy | None = None, cache_expires: timedelta | None = None, persist_result: bool = False, serializer: str = 'pickle', resources: dict[str, float] | None = None, on_completion: Iterable[Callable] = (), on_failure: Iterable[Callable] = (), on_cancellation: Iterable[Callable] = ())
A registered task: the wrapped function plus its options.
Instances are created by task. Calling one inside a flow records a task run;
submit and map return Future objects instead of waiting.
App
¶
Registry of flows and routes. Its name is the project every flow belongs to.
task
¶
Decorator equivalent to @task; tasks are not tied to an App, this exists for symmetry.
register
¶
register(flow: Flow) -> None
Add a flow to this App; raises FlowRegistrationError when another flow of the same name is registered.
route
¶
Register a custom HTTP route handled by the decorated function.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
method
|
str
|
HTTP method. |
required |
path
|
str
|
Path template with |
required |
rule
¶
rule(on=None, flow=None, tags=None, states=None, unless=None, within=None, at=None, tz=None, **guards)
Register a code rule whose action calls the decorated fn(event, run).
A reactive rule fires when an event matching on happens. A
proactive rule fires when the event named by unless does not
happen: either within seconds of the on event (event-armed),
or by each tick of the cron expression at in timezone tz
(clock-armed, no on).
Event and state names are checked here, so a rule that could never fire
raises at import rather than sitting silent. Use the constants in
cereyan.events and cereyan.states to get them right the first time;
a custom event name outside the reserved prefixes is always accepted.
A value in states matches the run's state type or its sub-state
name, so states=["Scheduled"] also covers Late and AwaitingRetry.
See cereyan.rules.register for the full argument list.
serve
¶
Serve the flows and routes registered so far, blocking until stopped.
Host and port given here rank below the CLI flags and environment and
above cereyan.toml.
get_default_app
¶
get_default_app(source_file: str | None = None) -> App
The default App, created on first use and named after the directory of the module that first needed it.
The current run¶
cereyan.runtime is a live view of the run executing in this thread. Each attribute is None outside a run, so a task called directly from a test reads None rather than raising.
| Attribute | Type | Carries |
|---|---|---|
cereyan.runtime.run |
RunContext |
id, external_id, name, flow, parameters, and flow_name and project |
cereyan.runtime.task_run |
TaskRunContext |
id (the external UUID), name, task_key, dynamic_key |
cereyan.runtime.flow |
Flow |
The flow of the current run |
from cereyan import flow, task, runtime
@task
def record() -> str:
return f"{runtime.run.name}/{runtime.task_run.dynamic_key}"
@flow
def report() -> str:
return f"{runtime.flow.name}#{runtime.run.id}: {record()}"
assert runtime.run is None
assert report().endswith("/record-0") # "report#1: <run name>/record-0"
Schedules and retry delays¶
Cron
dataclass
¶
Cron(cron: str, timezone: str | None = None, day_or: bool = True, catchup: str = 'skip', catchup_max: int = 100, key: str | None = None)
A cron schedule evaluated by wall clock in timezone.
Interval
dataclass
¶
Interval(interval: float | timedelta, anchor: datetime | None = None, timezone: str | None = None, catchup: str = 'skip', catchup_max: int = 100, key: str | None = None)
Fire every interval (seconds or timedelta) from anchor.
Intervals under a day are elapsed time; longer intervals keep their local wall-clock time across DST changes.
to_json
¶
The schedule as the server stores it; raises ValueError for a non-positive interval.
RRule
dataclass
¶
RRule(rrule: str, timezone: str | None = None, catchup: str = 'skip', catchup_max: int = 100, key: str | None = None)
An iCalendar recurrence rule set; must include DTSTART.
exponential
dataclass
¶
Retry delay growing as base * 2**attempt with optional jitter.
delay
¶
Seconds to wait before retry number attempt (0-based), capped at maximum.
Targets, caching and results¶
Target
¶
Bases: Protocol
Anything with exists(): the protocol a task's output= must satisfy.
LocalTarget
¶
A file on the local filesystem written atomically.
open
¶
Open the file; write modes write to a temporary path that replaces the target on close, so readers never see a partial file.
CachePolicy
¶
Bases: Flag
INPUTS and SOURCE are the members of CachePolicy, exported at the top level so cache=INPUTS + SOURCE reads naturally.
Concurrency¶
Future
¶
ThreadRunner
¶
Bases: BaseRunner
Runs submitted tasks on a bounded set of threads with the run context propagated. A task that waits on a child while every worker is busy gets a warning and a temporary extra worker so the wait cannot deadlock.
ProcessRunner
¶
Bases: BaseRunner
Runs each submitted task in a spawned process; arguments and results must be picklable and the task function importable. Timeouts terminate the worker.
Logging, events and artifacts¶
get_run_logger
¶
A logger that carries the current run and task-run in its records.
emit_event
¶
Record an event. Inside a run the resource defaults to that run; outside a run the event goes straight to the store or, when a server is up, to its API.
Any name is allowed except one under a
reserved prefix (see cereyan.events.RESERVED_PREFIXES) that the engine does not
emit, which is always a mistake rather than a custom event.
Raises:
| Type | Description |
|---|---|
ValueError
|
When the name is empty, not a string, or shadows the engine's own namespace. |
EventName
¶
Bases: str
An engine event name; a str carrying the exact wire name.
Attributes:
| Name | Type | Description |
|---|---|---|
resource |
str
|
The resource kind the event hangs off ( |
when |
str
|
One sentence on when the engine records it. |
payload_fields |
tuple[str, ...]
|
The payload keys the emit site sets. |
EventGroup
¶
The events under one reserved prefix, as attributes.
events.run.failed is "run.failed" and events.run.any is
"run.*", which matches every event under the prefix.
StateName
¶
Bases: str
A state type or named sub-state; a str carrying the wire name.
Attributes:
| Name | Type | Description |
|---|---|---|
state_type |
str
|
For a sub-state, the type it belongs to; for a state type, itself. |
is_sub_state |
bool
|
|
create_markdown
¶
Attach a markdown artifact to the current run or task run.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
Markdown source, rendered on the run page. |
required |
key
|
str | None
|
Optional stable key; artifacts sharing a key form a history across runs. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The artifact id. |
Raises:
| Type | Description |
|---|---|
CereyanError
|
Outside a run, or when the artifact exceeds 1 MB. |
create_table
¶
create_table(rows: list[dict] | list[list], key: str | None = None, columns: list[str] | None = None) -> str
Attach a table artifact.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rows
|
list[dict] | list[list]
|
A list of dicts (columns inferred from the keys, in first-seen order) or a
list of lists (then pass |
required |
key
|
str | None
|
Optional stable key. |
None
|
columns
|
list[str] | None
|
Column names; required for list-of-lists rows. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The artifact id. |
create_progress
¶
Attach a progress bar artifact.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
percent
|
float
|
Completion from 0 to 100; values outside are clamped. |
required |
key
|
str | None
|
Optional stable key; pass the same key to |
None
|
label
|
str | None
|
Text shown next to the bar. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The artifact id. |
update_progress
¶
Record a new value for the progress artifact with key.
Each update is a new artifact row under the same key, so the history is kept.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
The key given when the progress artifact was created. |
required |
percent
|
float
|
Completion from 0 to 100. |
required |
label
|
str | None
|
Text shown next to the bar. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The artifact id. |
create_link
¶
Attach a link artifact.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
str
|
The link target. |
required |
text
|
str | None
|
Link text; defaults to the URL. |
None
|
key
|
str | None
|
Optional stable key. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The artifact id. |
create_image
¶
create_image(url_or_bytes: str | bytes, key: str | None = None, media_type: str = 'image/png') -> str
Attach an image artifact from a URL or raw bytes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url_or_bytes
|
str | bytes
|
An image URL, or the image bytes to embed as a data URI. |
required |
key
|
str | None
|
Optional stable key. |
None
|
media_type
|
str
|
MIME type used when bytes are given. |
'image/png'
|
Returns:
| Type | Description |
|---|---|
str
|
The artifact id. |
Variables¶
Variable
¶
Named JSON values shared by every project on the machine, optionally encrypted.
Names match [a-z0-9][a-z0-9_./-]* and values are limited to 64 KB. Offline, values
go straight to the store; while a server holds the store, they go through its API.
Secrets are encrypted with the key kept at <home>/secret.key and are masked in
the API and the UI.
get
staticmethod
¶
Read a variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
The variable name. |
required |
default
|
Any
|
Returned when the variable does not exist. |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The stored value, decrypted for secrets. |
set
staticmethod
¶
set(name: str, value: Any, tags: list[str] | None = None, secret: bool = False, overwrite: bool = True) -> None
Create or update a variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
The variable name. |
required |
value
|
Any
|
Any JSON-serialisable value (dates and dataclasses are converted). |
required |
tags
|
list[str] | None
|
Tags shown on the Variables page. |
None
|
secret
|
bool
|
Encrypt the value at rest and mask it in the API and the UI. |
False
|
overwrite
|
bool
|
When |
True
|
Raises:
| Type | Description |
|---|---|
ValueError
|
On an invalid name or a value over 64 KB. |
CereyanError
|
When |
unset
staticmethod
¶
Delete a variable.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
The variable name. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
|
Human input¶
wait_for_input
¶
Return the answer given to this run, pausing it when there is none yet.
Inside a served run the first call transitions the run to Paused with
the prompt and ends the attempt; the engine is free while the run waits.
POST /api/runs/{id}/resume (or the Resume button, or the MCP
resume_run tool) stores the answer and schedules a new attempt, which
reruns the flow from the top and gets the answer from this call. Tasks
marked cache=INPUTS are skipped on the replay.
Outside a served run the answer is read from the terminal, or an error is raised when stdin is not interactive.
Client¶
The module-level functions use the server recorded in server.json of the runtime home; Client targets any server explicitly.
run
¶
Create a run of a registered flow on the live server and return it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
flow
|
str
|
Flow name. |
required |
project
|
str | None
|
Project name; required when the name exists in several projects. |
None
|
**parameters
|
Any
|
Flow parameters. |
{}
|
Raises:
| Type | Description |
|---|---|
ServerUnavailable
|
When no server is running for the home. |
CereyanError
|
When the flow is unknown or ambiguous. |
list_runs
¶
The items of GET /api/runs on the live server with the given filters.
list_flows
¶
GET /api/flows on the live server, optionally filtered by project.
Client
¶
Client(base_url: str = '', timeout: float = 30.0, token: str | None = None, socket_path: str | None = None)
HTTP client for a running server, on the standard library only.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
base_url
|
str
|
The server URL, for example |
''
|
timeout
|
float
|
Seconds to wait for a response. |
30.0
|
token
|
str | None
|
API token; defaults to |
None
|
socket_path
|
str | None
|
Connect over this Unix socket instead of TCP. |
None
|
Most methods return the decoded JSON of the matching endpoint (see the HTTP API
reference) and raise ApiError on an error status or
ServerUnavailable when the server cannot be reached.
flows
¶
GET /api/flows, optionally filtered by project.
runs
¶
GET /api/runs with the given filters (flow, project, state_type, tags, limit, cursor, ...); returns the page.
logs
¶
logs(run_id: int, after: int = 0, level: str | None = None, search: str | None = None, limit: int = 1000) -> dict
GET /api/runs/{id}/logs after sequence after, optionally filtered by level and search.
resume
¶
POST /api/runs/{id}/resume: answer a Paused run with input and schedule its next attempt.
cancel
¶
POST /api/runs/{id}/cancel: cancel a queued run at once or ask a running one to stop.
delete_run
¶
DELETE /api/runs/{id}: remove a finished run and its task runs, logs, and artifacts.
counts
¶
GET /api/counts: run counts by state for the dashboard, optionally per project.
backfill
¶
backfill(flow_id: int, parameter: str, start: str, end: str, *, interval: Any = '1d', concurrency: int = 1, extra_parameters: dict | None = None, reverse: bool = False) -> dict
POST /api/flows/{id}/backfill: create one run per step of parameter from start to end.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
flow_id
|
int
|
The flow's id. |
required |
parameter
|
str
|
The date or datetime parameter to step. |
required |
start
|
str
|
First value, ISO formatted. |
required |
end
|
str
|
Last value, inclusive. |
required |
interval
|
Any
|
Step as seconds or a duration such as |
'1d'
|
concurrency
|
int
|
How many of the backfill's runs may execute at once. |
1
|
extra_parameters
|
dict | None
|
Fixed values for other parameters. |
None
|
reverse
|
bool
|
Create the newest value first. |
False
|
Returns:
| Type | Description |
|---|---|
dict
|
The backfill status with its id, tag, and counts. |
backfill_status
¶
GET /api/backfills/{id}: counts of the backfill's runs by state.
cancel_backfill
¶
POST /api/backfills/{id}/cancel: cancel the backfill's remaining runs.
schedules
¶
GET /api/flows/{id}/schedules: the flow's schedules with their next fire times.
upcoming
¶
GET /api/flows/{id}/upcoming: the runs materialised ahead for the flow.
events
¶
GET /api/events newest first, filtered by name prefix kind and other filters; with after returns events following that sequence number in ascending order.
rules
¶
GET /api/rules: every rule with its match clause, actions, guards, and fire counts.
submit
¶
submit(project: str, flow: str, parameters: dict | None = None, *, name: str | None = None, tags: list[str] | None = None, module: str | None = None, source_dir: str | None = None, description: str | None = None, parameter_schema: dict | None = None, options: dict | None = None, flow_tags: list[str] | None = None, flow_group: str | None = None, created_by: str = 'client') -> dict
POST /api/runs: create a run with explicit project and flow, registering the flow when needed.
This is the low-level call used by the offline handoff and by run; pass
module and source_dir so the server can import a flow it has not seen.
run
¶
run(flow: str, project: str | None = None, *, name: str | None = None, tags: list[str] | None = None, **parameters: Any) -> dict
Create a run of a registered flow by name. Returns the run.
find_server
¶
find_server(home: str | None = None) -> Client | None
A client for the live server of home, or None when there is none; raises AuthRequired when a token is needed.
Custom routes¶
Request
¶
Response
¶
Response(body: bytes | str = b'', status: int = 200, headers: dict[str, str] | None = None, media_type: str | None = None)
An explicit response: body bytes or str, status, headers.
HTTPError
¶
Errors¶
CereyanError
¶
Bases: Exception
Base class for errors raised by cereyan.
ParameterError
¶
StoreLocked
¶
Bases: RuntimeError
The store is locked by another process.
TransitionRejected
¶
Bases: RuntimeError
A state transition was rejected by the rules.
ServerUnavailable
¶
Bases: CereyanError
No live server was found for the home directory.
ApiError
¶
AuthRequired
¶
Bases: CereyanError
The server requires a token the client does not have (or a wrong one).