Data Pipeline
**Stream Processing as a Service (SPaaS)**，allowing them to focus on business application logic while platform provides the scale, operations, and domain expertise.
Fast prototyping of stream processing solutions，提供规范
Provide tooling to enable users to self-serve
failure as first class citizen
some important aspects that impacted our design:
- Assume unreliable network
- Trust underlying runtime infrastructure, but design automatic healing capabilities
- Enforce job level isolation for multi-tenants support
- Reduce blast radius when failure arise
- Design for automatic reconciliation if any components drifts from desired state or even if disaster failure occurs
- Handle & propagate back pressure correctly
Source of Truth Store is a durable, persistent storage that keeps all the desired state information.
关于状态的存储