Skip to main content

Glossary

TermDefinition
Agent (WME agent, daghandler)Go service on every worker (port 8090) that receives DAG files, clones processor repositories and controls processor containers.
AirflowWorkflow scheduler that runs WME's generated DAGs using the CeleryExecutor.
Asset / Asset categoryGeneric catalogue objects; a worker is an asset of category Worker.
Celery queueThe queue a worker listens on (WORKER_NAME). Each processor's DAG task is sent to its worker's queue.
Compose applicationProcessor runtime mode: a Docker Compose app cloned from GitHub.
Data interface typeTemplate for a kind of endpoint (kafka, elasticsearch, …) with its connection parameters.
Data kindDescription of a payload: shape, cardinality and schema.
Derived environment variableVariable injected into a processor from a connected resource: <INTERFACE>_<KEY>_<DIRECTION>.
Digital resource (DS / DR)A concrete data source or sink — an interface type plus values.
DraftA workflow proposed by the Workflow Assistant, reviewed and saved in the Lab.
FlowerCelery monitoring UI/API; WME imports workers from it.
Initialize ResourcesAdmin action that seeds a user's catalogue.
Logstash PipelineWME-managed processor type that generates a Logstash pipeline from its inputs, outputs and filter.
ObservationNormalized record of an event seen on a resource, stored in the observations index.
OrphanA WME-labelled container (or Logstash pipeline) whose processor no longer exists.
Processor definition (PD)Catalogue description of a processing component: runtime, interfaces, parameters.
Processor / processor manifest (PM)An instance of a definition in a workflow, with a worker and values.
Processor orchestrator (PO)Backend name for a workflow.
ProvisioningCloning a Compose processor's repository onto every worker.
Single containerProcessor runtime mode: one image run with docker run.
Teardown DAGThe second DAG generated per workflow; stops and removes its containers.
WorkerHost that runs processors: Celery worker + WME agent.
Worker StationThe worker a processor is assigned to.
WorkflowA graph of processors and resources with scheduling settings; deployed through Airflow.
Workflow template (WT)Reusable graph for creating workflows.