Apache Pulsar架构原理

消息系统的发展历程

大数据实时数据流处理流程

云原生架构特点

1.存储和计算分离

2.节点对等 

3.独立扩展 

4.灵活扩容 

5.快速容错

企业级流存储层：Apache BookKeeper

BookKeeper分布式日志/流存储具有特性：

1.低延时、高吞吐、持久化

2.强一致

3.高可用

4.I/O隔离

企业级特性：大集群、多租户

原生的跨地域复制

Pulsar介绍：

Apache Pulsar is a cloud-native event streaming platform

Connect, Store, and Process real-time event streams

Connect: Pulsar clients, IO connectors, Protocol handlers

Store: Apache BookKeeper, Tiered storage

Process:

Pulsar Functions - Lightweight serverless computation  

Spark / Flink - Unified data processing

Presto - Interactive queries

Tenant & Namespace 

Pulsar Architecture

Apache BookKeeper

BookKeeper — 稳定的 IO 质量

Namespace Bundles

Load Manager - Assign bundles to brokers

Topic Lookup

HTTP Topic Lookup

Topic Lookup with Proxies

Geo-Replication

Geo-replication w/o Global ZK

Aggregation Pattern