Dependency-aware executionAsync Workflow Orchestrator
A DAG workflow engine with validated dependencies, concurrent async execution, plugin tasks, resumable state, and run-level observability.
System routeRead left to right
- 01Workflow spec
- 02DAG validate
- 03Schedule
- 04Execute
- 05Resume
01 / Context and problemThe operational problem
Multi-step jobs need more than a task list: dependencies must be valid, failures must propagate clearly, and interrupted work must remain inspectable.
OwnershipBuilt the graph validator, execution manager, plugin protocol, persistent task state, FastAPI control plane, YAML examples, CLI, JSONL logging, tests, CI, and Docker runtime.
02 / Technical challengeWhat made it non-trivial
Concurrency, retries, timeouts, cancellation, dependency failure, and resume behavior had to produce a coherent, persistent task history.
SQLite is the default local state store.
Built-in plugins are deliberately small demonstration tasks.
The public project does not claim a hosted production service.
03 / System designThe architecture decision
The engine validates the graph before execution, schedules topological levels under a bounded semaphore, resolves typed task plugins, persists every transition, and exposes trigger, inspect, cancel, resume, and graph endpoints.
01Python02asyncio03FastAPI04SQLAlchemy05Pydantic06YAML07Pytest08Docker
04 / ReliabilityDesigned for the failure path
- 01
Cycle and missing-dependency validation
- 02
Bounded concurrent execution
- 03
Retries and per-task timeout limits
- 04
Cancellation tokens and dependency-aware skipping
- 05
Persistent workflow and task state
- 06
Per-run JSONL operational logs
05 / Result and evidenceWhat this system proves
A compact, installable workflow engine that demonstrates graph reasoning, stateful async execution, API design, packaging, and strict CI quality gates.
Tayyab can design orchestration behavior where state transitions and failure semantics are first-class concerns.
06 / LessonsWhat carries forward
01Validate the complete graph before any side effect starts.
02Persist state around task execution so interruption is observable.
03Plugin boundaries keep the scheduler independent from task implementation.