Key Concepts
MediaMoth coordinates media-processing work through pipelines, jobs, service metadata, and event-driven projections. This page defines those concepts and how they relate when a pipeline runs.
Media
A media record is metadata about content, not the stored file itself. It includes a title, year, video properties such as resolution and chapter range, and custom tags. Input and output files are addressed separately with file-source URIs.
Pipelines
A pipeline is an event-sourced workflow definition. Its nodes form a directed acyclic graph: independent branches may run in parallel, while a node with predecessors waits until every predecessor finishes.
Workflow Service supports creating, updating, and deleting pipelines. A pipeline can also own reusable parameter templates.
Nodes
A node connects a stable node ID, a label, a worker service ID, and the IDs of its next nodes. Workflow Service validates graph structure and checks that referenced services exist.
Node IDs matter after execution too. Job Service stores node metadata on concrete jobs so search, rerun, correction, and backfill operations can relate historical work to the pipeline definition.
Services and schemas
A processing service registers through health events and advertises:
- a service ID and version;
- a parameter JSON Schema;
- a UI Schema;
- its health and worker availability.
The schema is generated from the service's typed Go parameters. Clients can therefore render forms for new worker versions without maintaining a separate parameter model.
Current processing services include:
- Video Service for FFmpeg and HandBrake conversion;
- Chapter Inserter for fixed and ML-detected chapter boundaries;
- VHS Decode for RF-capture decoding;
- TBC Decode for exporting time-base-corrected captures.
Jobs
A job is one execution of a pipeline for selected media. Job Service converts pipeline nodes into a job sequence, resolves smart tags, snapshots the resolved tag map, assigns execution IDs, and queues the root nodes.
Workers report start, progress, completion, and failure. When a node finishes, Job Service queues downstream nodes whose predecessors are all complete.
The API also supports cancellation, explicit retry/requeue, pipeline reruns, streamed queue submission, and corrections for historical job metadata.
Events and projections
Command services persist state changes as events. Query and Search services read projections derived from those events. Kafka carries cross-service events and worker status; PostgreSQL stores service state and River queues; Elasticsearch backs job, service, pipeline, and media search.
If a projection is missing or its implementation changes, an administrative command can replay stored events. See Projection rebuilds.
How the pieces connect
- Media metadata supplies built-in and custom smart tags.
- A pipeline selects registered services and orders their nodes.
- A pipeline template may prefill node parameters.
- Job Service resolves parameters and creates a concrete job sequence.
- Worker services validate their typed parameters and enqueue River work.
- Status events advance the sequence and update projections and search indexes.