README.md

Awesome Apache Pulsar

Apache® Pulsar™ is an open-source, distributed messaging and streaming platform built for the cloud.

🏠 Official website

📚 Documentation

📝 Release notes

🗣 Community

• GitHub Discussions is a right place to ask a question or suggest an idea.

• StackOverflow is also a good place to ask a question. Please use the apache-pulsar tag.

• 🐞 GitHub Issues is a right place to submit a bug report.

• Slack is a good place to ask a quick question or reach specific people via DM.

  If you want to ask a question, please consider one of the resources listed above. Slack isn't indexable by Google and it's history is limited to 90 days in free plan.

• Full list of community resources

🛠️ Web UIs

• Pulsar Manager - official management console for Apache Pulsar. open-source
• Dekaf for Apache Pulsar - ☕️ proprietary with free plan
• Streamvisor - proprietary with free plan
• Pulsar Express - simple Web Interface for Apache Pulsar. open-source
• Pulsar Admin Console - web-based UI that administrates topics, namespaces, sources, sinks, and various aspects of Apache Pulsar features. open-source

📘 Books

• Apache Pulsar in Action by David Kjerrumgaard. 400 pages. Released in 2021.
• Mastering Apache Pulsar: Cloud Native Event Streaming at Scale by Jowanza Joseph. 240 pages. Released in 2022.

🎓 Courses

• StreamNative Academy
  • APC-100 Introduction to Apache Pulsar
  • DEV-200 Practical Apache Pulsar Application Development.
  • 🔜 Operator Fundamentals.
• Learn Apache Pulsar.
• Handling Streaming Data with Apache Pulsar 2.

🪪 Certifications

• CERT-100 Apache Pulsar Developer Certification Level 1: Fundamentals.

Get Pulsar

Managed solutions

• StreamNative Cloud
• Enterprise-ready Apache Pulsar by DataStax

Self-managed

• Download binaries
• 🐳 Official Docker container
• Official Helm Chart
• Helm charts by StreamNative
• Helm chart by DataStax

🛟 Support providers

• StreamNative
• DataStax
• Dattell

Articles

Why Pulsar?

• Apache Pulsar vs. Apache Kafka 2022 Benchmark by Matteo Merli and Penghui Li.
• Streaming War and How Apache Pulsar is Acing the Battle by Shivji Kumar Jha and Sachidananda Maharana.
• StreamNative: Enabling Real-time Messaging and Streaming for the Cloud by Sijie Guo and Matteo Merli.
• David Kjerrumgaard, Pulsar In Action Author, Talks All Things Pulsar by Carolyn King.
• A More Accurate Perspective on Pulsar’s Performance Compared to Kafka by Penghui Li and Sijie Guo.
• Benchmarking Pulsar and Kafka - The Full Benchmark Report - 2020 by Penghui Li and Sijie Guo.
• Pulsar vs Kafka - Part 2 - Adoption, Use Cases, Differentiators, and Community by Carolyn King, Addison Higham, and Sijie Guo.
• Apache Pulsar Adoption: Why Companies Use the Streaming and Messaging Platform by Carolyn King, Addison Higham, and Sijie Guo.
• A Guide to Apache Pulsar: Compare Features and Architecture to Apache Kafka by Carolyn King, Sijie Guo, and Addison Higham.
• One Storage System for both Real-time and Historical Data Analysis - An Apache Pulsar story by Yijie Shen.

Case studies

• Handling 100K Consumers with One Pulsar Topic by Hongjie Zhai from NTT.
• WeChat: Using Apache Pulsar to Support the High Throughput Real-time Recommendation Service by Shen Liu from WeChat.
• How Proxima Beta Implemented CQRS and Event Sourcing on Top of Apache Pulsar and ScyllaDB by Lei Shi from Proxima Beta, part of Tencent Interactive Entertainment Group Global (IEG Global).
• 600K Topics Per Cluster: Stability Optimization of Apache Pulsar at Tencent Cloud by Xiaolong Ran from Tencent Cloud.
• Client Optimization: How Tencent Maintains Apache Pulsar Clusters with over 100 Billion Messages Daily by Mingyu Bao and Dawei Zhang from Tencent TEG Data Platform Department.
• How Apache Pulsar is Helping Iterable Scale its Customer Engagement Platform by Greg Methvin from Iterable.
• Powering Federated Learning at Tencent with Apache Pulsar by Chao Zhang from Tencent Data Platform Department.
• How Apache Pulsar Helps Streamline Message System and Reduces O&M Costs at Tuya Smart by Yong Zhang from Tuya Smart.
• Why we moved from Apache Kafka to Apache Pulsar by Simba Khadder from streamsql.io.
• Apache Pulsar Helps Tencent Process Tens of Billions of Financial Transactions Efficiently with Virtually No Data Loss by Dezhi Liu from Tencent.
• How Orange Financial combats financial fraud in over 50M transactions a day using Apache Pulsar by Vincent Xie from Orange Financial.
• Powering Tencent Billing Platform with Apache Pulsar by Dezhi Liu from Tencent.
• Apache Pulsar Adoption Story in ActorCloud (IoT Platform) by Rocky Jin.
• Build a Priority-based Push Notification System Using Apache Pulsar at GeTui by Zi Xiang from Getui.
• Simplifying Zhaopin’s Event Center with Apache Pulsar by Penghui Li.

For beginners

• Understanding Pulsar in 10 Minutes: A Guide for Kafka Users by Elliot West.

Adopting Pulsar

• How to Migrate from RabbitMQ to Apache Pulsar by David Kjerrumgaard.

Application development

• Spring into Pulsar Part 3: Building An Application with the New Spring Library for Apache Pulsar by Tim Spann.
• Spring into Pulsar Part 2: Spring-based Microservices for Multiple Protocols with Apache Pulsar by Tim Spann.
• Spring into Pulsar by Tim Spann.
• Using Cloud Native Buildpacks to Improve the Function Image Building Capability of Function Mesh by Tian Fang.
• What the FLiP is the FLiP Stack? by Tim Spann.
• Streaming Real-Time Chat Messages into Scylla with Apache Pulsar by Tim Spann.
• Apache Pulsar Client Application Best Practices by Ioannis Polyzos.
• Integrating Apache Pulsar with BigQuery by Ioannis Polyzos.
• Developing Event-Driven Microservices with Apache Pulsar: Part I by Tim Spann.
• Building Edge Applications With Apache Pulsar by Tim Spann.
• Streaming Data Pipelines with Pulsar IO by Ioannis Polyzos.
• Scalable Stream Processing with Pulsar’s Key_Shared Subscription by David Kjerrumgaard.
• Exactly-Once Semantics with Transactions in Pulsar by Penghui Li.
• Migrate to Serverless with Pulsar Functions by Axel Sirota.
• Pulsar Functions Deep Dive by Sanjeev Kulkarni.
• Announcing MQTT-on-Pulsar: Bringing Native MQTT Protocol Support to Apache Pulsar by Penghui Li and Xiaolong Ran.
• Announcing AMQP-on-Pulsar: bring native AMQP protocol support to Apache Pulsar by Penghui Li, Ran Gao, and Zongtang Hu.
• How to trace Pulsar messages with OpenTracing and Jaeger by Penghui Li.
• Use Apache SkyWalking to Trace Apache Pulsar Messages by Penghui Li.
• Use Apache Pulsar as Streaming Table with 8 Lines of Code by Yijie Shen.

Operations

• Pulsar Operators Tutorial Part 3: Create and Deploy a Containerized Pulsar Client by Yuwei Sung.
• Pulsar Operators Tutorial Part 2: Manage Pulsar Custom Resources with ArgoCD by Yuwei Sung.
• Pulsar Operators Tutorial Part 1: Create an Apache Pulsar Cluster on Kubernetes by Yuwei Sung.
• Using Cloud Native Buildpacks to Improve the Function Image Building Capability of Function Mesh by Tian Fang.
• Failure Is Not an Option: It Is a Given. by David Kjerrumgaard.
• Auto-Scaling Pulsar Functions in Kubernetes Using Custom Metrics by Ran Gao.
• Pulsar Isolation Part IV: Single Cluster Isolation by Ran Gao.
• Pulsar Isolation Part III: Separate Pulsar Clusters Sharing a Single BookKeeper Cluster by Ran Gao.
• Pulsar Isolation Part II: Separate Pulsar Clusters by Ran Gao and Yu Liu.
• Pulsar Isolation Part I: Taking an In-Depth Look at How to Achieve Isolation in Pulsar by Penghui Li and Yu Liu.
• 10 Useful Pulsarctl Commands to Manage Your Cluster by Aniket Bhattacharyea.
• Function Mesh - Simplify Complex Streaming Jobs in Cloud by Neng Lu and Rui Fu.
• Taking a Deep-Dive into Apache Pulsar Architecture for Performance Tuning by Penghui Li and Devin Bost.
• A Practical Guide to Enterprise-Grade Security in Apache Pulsar by Teng Fu.

How Pulsar works

• Improving Regular Expression-Based Subscriptions in Pulsar Consumers by Andras Beni.
• A Deep Dive into Transaction Coordinators in Apache Pulsar by Penghui Li and Bo Cong.
• A Deep-dive of Transactions in Apache Pulsar by Penghui Li.
• A Deep Dive into the Topic Data Lifecycle in Apache Pulsar by Hang.
• Deep Dive into Data Placement Policies by Hang.
• Deep Dive into Message Chunking in Pulsar by Zike Yang.
• Tweaking the BookKeeper Protocol - Unbounded Ledgers by Jack Vanlightly.
• Offset Implementation in Kafka-on-Pulsar by Yunze Xu.
• A Deep Dive into Transaction Buffer in Apache Pulsar by Xiangying Meng.
• Achieving Broker Load Balancing with Apache Pulsar by Heesung Sohn and Kai Wang.

Announcements

• Announcing the Hudi Sink Connector for Apache Pulsar by Yong Zhang. 🗓️ Jan 26, 2023
• Announcing the Iceberg Sink Connector for Apache Pulsar by Hang. 🗓️ Dec 14, 2022
• Announcing the Flink-Pulsar SQL Connector by Yufei Zhang. 🗓️ Sep 29, 2022
• The Beauty of Apache Pulsar: Individual Message Acknowledgment by Asaf Mesika. 🗓️ Sep 22, 2022
• Announcing Spring for Apache Pulsar by Alexander Preuss. 🗓️ Sep 21, 2022
• Announcing the Flink-Pulsar Sink Connector by Yufei Zhang. 🗓️ Aug 30, 2022
• Announcing the Delta Lake Sink Connector for Apache Pulsar by Hang. 🗓️ Aug 17, 2022
• Introducing Pulsar Resources Operator for Kubernetes by Eric Shen and Fushu Wang. 🗓️ Aug 15, 2022
• What’s New in Apache Pulsar 2.9.3 by Jun Ma and Qiang Zhao. 🗓️ Jul 27, 2022
• Announcing the Google Cloud Pub/Sub Connector for Apache Pulsar by Zixuan Liu. 🗓️ Jun 24, 2022
• What’s New in Apache Pulsar 2.10.1 by Penghui Li and Jun Ma. 🗓️ Jun 12, 2022
• What’s New in Apache Pulsar 2.10 by Penghui Li and Dave Duggins. 🗓️ Apr 12, 2022
• What’s New in Apache Pulsar 2.9.2 by Ran Gao and Yu Liu. 🗓️ Apr 12, 2022
• Cloudera and StreamNative Announce the Integration of Apache NiFi and Apache Pulsar by David Kjerrumgaard. 🗓️ Mar 9, 2022
• Moving Toward a ZooKeeper-Less Apache Pulsar by David Kjerrumgaard. 🗓️ Jan 25, 2022
• What’s New in Apache Pulsar 2.7.4 by Yu Liu. 🗓️ Dec 14, 2021
• Pulsar Hits 10,000 GitHub Stars Milestone by Alice Bi. 🗓️ Dec 7, 2021
• What’s New in Apache Pulsar 2.8.1 by Yu Liu and Hang. 🗓️ Sep 23, 2021
• What’s New in Apache Pulsar 2.7.3 by Bo Cong and Yu Liu. 🗓️ Aug 11, 2021
• Apache Pulsar Launches 2.8: Unified Messaging and Streaming With Transactions by Matteo Merli and Sijie Guo. 🗓️ Jun 15, 2021
• Pulsar Hits Its 400th Contributor & Passes Kafka in Monthly Active Contributors by Carolyn King and Alice Bi. 🗓️ Jun 14, 2021
• Pulsar User Survey 2021 Highlights by Carolyn King. 🗓️ Jun 11, 2021
• 2021 Apache Pulsar User Report Announcement by Sijie Guo and Carolyn King. 🗓️ Jun 10, 2021
• Matteo Merli, Apache Pulsar PMC Chair, Joins StreamNative as CTO by Carolyn King. 🗓️ Jun 8, 2021
• What’s New in Apache Pulsar 2.6.4 by Yong Zhang. 🗓️ Jun 8, 2021
• Announcing AMQP 1.0 Connector for Apache Pulsar by Ran Gao and Yu Liu. 🗓️ Apr 26, 2021
• Call for Beta Users: Function Mesh Now Available for Pulsar Functions by Carolyn King. 🗓️ Apr 5, 2021
• Announcing AWS SQS Connector for Apache Pulsar by Yu Liu and Rui Fu. 🗓️ Mar 17, 2021
• What's New in Apache Pulsar 2.7.0 by Penghui Li. 🗓️ Dec 25, 2020
• What's New in Pulsar Flink Connector 2.7.0 by Jianyun Zhao and Jennifer Huang. 🗓️ Dec 24, 2020
• Cloud-Native Apache Pulsar 2.7 Supports Transactions and Azure Blob Storage Offloader by Penghui Li and Jennifer Huang. 🗓️ Dec 3, 2020
• Apache Pulsar 2.6.2 by Xiaolong Ran. 🗓️ Nov 20, 2020
• Introducing Cloud Storage Sink Connector - Streaming Data From Apache Pulsar to Cloud Objects by Jianyun Zhao. 🗓️ Oct 20, 2020
• Pulsar Flink Connector 2.5.0 by Jianyun Zhao. 🗓️ Sep 17, 2020
• Apache Pulsar Celebrates 300th contributor by Jennifer Huang and Carolyn King. 🗓️ Aug 24, 2020
• Apache Pulsar 2.6.1 by Xiaolong Ran. 🗓️ Aug 21, 2020
• Apache Pulsar 2.6.0 by Penghui Li. 🗓️ Jun 18, 2020
• Apache Pulsar 2.5.2 by Jia Zhai. 🗓️ May 20, 2020
• Apache Pulsar 2.5.1 by Guangning E. 🗓️ Apr 23, 2020
• Announcing Kafka-on-Pulsar: bring native Kafka protocol support to Apache Pulsar by Sijie Guo, Jia Zhai, and Pierre Zemb. 🗓️ Mar 24, 2020
• Announcing: The Apache Pulsar 2020 User Survey Report by Sijie Guo. 🗓️ Mar 17, 2020
• Pulsar Milestone Celebration — 200 Contributors! by Yu Liu. 🗓️ Dec 20, 2019
• What's New in Apache Pulsar 2.4.2 by Xiaolong Ran. 🗓️ Dec 4, 2019
• Introduction to Pulsarctl by Yong Zhang. 🗓️ Nov 6, 2019
• StreamNative open sourced and contributed Apache Pulsar Manager to ASF by Guangning E. 🗓️ Sep 24, 2019
• What's New in Apache Pulsar 2.4.0 by Sijie Guo. 🗓️ Jul 9, 2019

Uncategorized

• How to Build a Distributed Database with Apache BookKeeper — Part 3 by Enrico Olivelli.
• How to Build a Distributed Database with Apache BookKeeper — Part 2 by Enrico Olivelli.
• How to Build a Distributed Database with Apache BookKeeper - Part 1 by Enrico Olivelli.

Events

Meetup groups

• 🇺🇸 New York City Apache Pulsar Meetup
• 🇺🇸 SF Bay Area Apache Pulsar Meetup

Pulsar Summit

• 🔜 Pulsar Virtual Summit Europe 2023

• Pulsar Summit Asia 2022

  • Pulsar Summit Asia 2022 Recap by Sherlock Xu.
  • Opening Remarks - Day 2. Sherlock Xu.
  • Keynote - Event-Driven Applications Done Right. Matteo Merli.
  • Handling 100K Consumers with One Topic: Practices and Technical Details. Hongjie Zhai.
  • Awesome Pulsar in Yum China. Chauncey Yan.
  • Building Modern Data Streaming Apps. Timothy Spann.
  • Apache Pulsar Development 101 with Python. Timothy Spann.
  • Towards a ZooKeeper-less Pulsar, etcd, etcd, etcd. David Kjerrumgaard.
  • Introducing TableView: Pulsar's database table abstraction.David Kjerrumgaard.
  • Turn Data in Pulsar to Real-time Charts and Alerts via Streaming SQL. Gang Tao, Zhimin Liang.
  • Streaming wars and How Apache Pulsar is acing the battle. Shivji Kumar Jha, Sachidananda Maharana.
  • Taking Jakarta JMS to new generation Messaging Systems. Enrico Olivelli, Mary Grygleski.
  • Keeping on top of hybrid cloud usage with Pulsar. Shivji Kumar Jha, Tarun Annapareddy.
  • Make querying from Pulsar easier: Introduce Flink Pulsar SQL Connector. Yufei Zhang.
  • Opening Remarks - Day 1. Chloe Wang. 🇨🇳
  • Apache Pulsar - A Cloud Native, Unified Messaging and Streaming System for Modern Data Infrastructure. Jia Zhai. 🇨🇳
  • What You Should Know About Apache Pulsar in 2022. Penghui Li. 🇨🇳
  • Pulsar + Envoy: Building an OTO Marketing Platform for Different Business Scenarios on Microservices. Jason Jiang. 🇨🇳
  • Stability Optimization of Apache Pulsar at Huawei Mobile Services. Lin Lin. 🇨🇳
  • Pulsar in Smart Education: How NetEase Youdao Put Pulsar into Practice for Complex Business Scenarios. Jiaqi Shen. 🇨🇳
  • Running Trillions of Messages: How Apache Pulsar Supports Big Data Business at Tencent. Dawei Zhang. 🇨🇳
  • The Exploration and Practice of Apache Pulsar in vivo. Limin Quan, Jianbo Chen. 🇨🇳
  • Apache Pulsar in Volcano Engine E-MapReduce: Integration and Scenarios. Xin Liang. 🇨🇳
  • Migrate Cross-cluster Tenant Based on Geo Replication. Mingze Han. 🇨🇳
  • Support Large Traffic in WeChat: The Practice of Apache Pulsar in Real Time Recommendation. Shen Liu. 🇨🇳
  • Apache Pulsar + KubeEdge: Managing Edge Devices with Low Latency and Persistent Storage. Ryan Zhao, Chenlin Liu. 🇨🇳
  • Migrating from RabbitMQ to Apache Pulsar: Using AMQP-on-Pulsar(AoP) in E-commerce Industry. Yifei Ming. 🇨🇳
  • Apache Pulsar in the edge-device with thousands of topics:Pulsar Functions in Smart Cities. Yuanpeng Liang. 🇨🇳
  • How to achieve better team integration and data governance by Apache Pulsar. Lei Shi. 🇨🇳
  • A Deep Dive into Pulsar's Geo-replication for High Availability. Jialing Wang. 🇨🇳
  • A New Way of Managing Pulsar with Infrastructure as Code. Max Xu, Fushu Wang. 🇨🇳
  • Deep Dive into Pulsar Lifecycle. Xiaolong Ran. 🇨🇳
  • Deep Dive into Message Chunking in Pulsar. Zike Yang. 🇨🇳
  • Dive into the differences between Apache Pulsar and Apache Kafka. Teng Fu. 🇨🇳
  • Native Pulsar support in Tencent TianQiong StarRocks. Guangxu Cheng. 🇨🇳
  • Pulsar + Flink + Camel: How Vertice Built its CMDB-based Real-time Data platform. Wei Wang. 🇨🇳
  • Build A Real-Time Data Warehouse Based on Apache Pulsar. Zike Yang, Dezhi Liu. 🇨🇳
  • Simplify Pulsar Functions Development with SQL. Rui Fu. 🇨🇳
  • RisingWave: Cloud-Native Streaming SQL over Pulsar. Tao Wu. 🇨🇳
  • Pulsar + TiDB: Real Time GitHub Data Analysis in OSSInsight. Qizhi Wang. 🇨🇳
  • Using Pulsar in Apache ShenYu to Store and Publish Access Logs. Bowen Li. 🇨🇳

• Pulsar Summit San Francisco 2022

  • Pulsar Summit SF 2022: Community Event Recap by Kristin Crosier.
  • Sizzle Reel
  • Opening Remarks. Carolyn King.
  • Event-Driven Applications Done Right. Sijie Guo, Metteo Merli.
  • Beam + Pulsar Powerful Stream Processing at Scale. Byron Ellis.
  • Data Democracy: Journey to User-Facing Analytics. Xiang Fu.
  • Pulsar @ Scale. 200M RPM and Thousand of Instances. Ignacio Alvarez.
  • Pulsar's Journey in Yahoo!: On prem, Cloud and Hybrid. Ludwig Pummer, Rajan Dhabalia.
  • Understanding Broker Rebalancing. Heesung Sohn.
  • Message Redelivery: An Unexpected Journey. David Kjerrumgaard.
  • Validating Apache Pulsar’s Behavior Under Failure Conditions. Lari Hotari.
  • Towards a ZooKeeper-less Pulsar, etcd, etcd, etcd. Matteo Merli.
  • Blue-Green Deploys with Pulsar & Envoy in an Event-Driven Microservice Ecosystem. Kai Levy, Zach Walsh.
  • Building Reliable Lakehouses with Apache Pulsar and Delta Lake. Nick Karpov.
  • Unlocking the Power of Lakehouse Architectures with Apache Pulsar and Apache Hudi. Alexey Kudinkin, Addison Higham.
  • Cross the Streams! Creating Streaming Data Pipelines with Apache Flink + Pulsar. Caito Scherr.
  • Is Using KoP (Kafka-on-Pulsar) a Good Idea?. Ricardo Ferreira.
  • Simplify Pulsar Functions Development with SQL. Rui Fu.
  • Distributed Database Design Decisions to Support High Performance Event Streaming. Peter Corless.
  • Building an Asynchronous Application Framework with Python and Pulsar. Zac Bentley.

• Pulsar Summit Asia 2021

  • Apache Pulsar. What we Learned from Building a Modern Messaging and Streaming System for Cloud. Sijie Guo.
  • Supporting the entire lifecycle of Streaming Data. Metteo Merli.
  • Designing Pulsar Clients to Run in Isolation. Sourabh Agrawal, Shivji Kumar Jha.
  • Apache Pulsar with MQTT for Edge Computing. Tim Spann.
  • Opening Remarks. 🇨🇳
  • The Future of Unified Messaging and Streaming with Apache Pulsar. Jia Zhai. 🇨🇳
  • What's New in Apache Pulsar 2.9. Penghui Li. 🇨🇳
  • Apache Pulsar with K8S, One Messaging and Streaming Solution for Industry. Xiaohaosheng Jia, Eric Shen. 🇨🇳
  • Enabling Hybrid Cloud Strategies with Apache Pulsar. 🇨🇳
  • Log System As Backbone – How We Built the World’s Most Advanced Vector Database on Pulsar. Xiaofan Luan. 🇨🇳
  • Building Smart Parking SaaS System with Apache Pulsar to Make Business Data-intelligence. Song Li. 🇨🇳
  • The Evolution History of RoP (RocketMQ-on-Pulsar). Xialong Ran, Mingze Han. 🇨🇳
  • Function Mesh for Apache Pulsar, the Way for Simple Streaming Solutions. Rui Fu. 🇨🇳
  • How Tencent Applies Apache Pulsar to Apache InLong. Chao Zhang. 🇨🇳
  • KoP (Kafka on Pulsar): Improvements Made in KoP 2.9.0. Yunze Xu. 🇨🇳
  • MoP (MQTT on Pulsar) - a Powerful Tool for Apache Pulsar in IoT. Jiwei Guo. 🇨🇳
  • PowerFL meets Pulsar: A Novel Communication Framework for Privacy-Preserving Computation. Huaming Rao. 🇨🇳
  • Pulsar in the Lakehouse: Overview of Apache Pulsar and Delta Lake Connector. Ke Xie. 🇨🇳
  • Design, Development, and Use of the New Flink-based Pulsar Connector. Yufan Sheng. 🇨🇳
  • RabbitMQ on Pulsar's Practice in Tencent Cloud. Yuqiang Lin. 🇨🇳
  • The Evolution of Trillion-level Real-time Messaging System in BIGO. Zhanpeng Wu. 🇨🇳
  • The Practice of Apache Pulsar for Logging in China Mobile. Jialing Wang. 🇨🇳
  • A New Transport Plugin for Apache SkyWalking Using Apache Pulsar. Haochao Zhuang. 🇨🇳

• Pulsar Virtual Summit Europe 2021 by Alice Bi.

  • Highlights from the Pulsar Virtual Summit Europe 2021
  • Supporting the Entire Lifecycle of Streaming Data. Metteo Merli.
  • A Foundation Backbone for Clever Cloud. Quentin Adam.
  • Pulsar in the Lakehouse: Apache Pulsar with Apache Spark and Delta Lake. Ryan Zhu, Addison Higham.
  • Pulsar Watermarking. Eron Wright.
  • Writing Custom Sink Connectors for Pulsar I/O. Ricardo Ferreia.
  • Using the JMS 2.0 API with Apache Pulsar. Enrico Olivelli.
  • No Surprises Geo Replication. Gaurav Ashok.
  • Using the FLiPN Stack for Edge AI (Flink, NiFi, Pulsar). Timothy Spann.
  • Interactive Analytics on Pulsar with Pulsar SQL. Alex Sirota.
  • Application of Apache Pulsar in Tencent Billing and Tencent Advertising. Mingyu Bao.
  • Deep Dive into the Pulsar Binary Protocol. Christophe Bornet.
  • Opening Remarks. Carolyn King.

• Pulsar Summit North America 2021

  • Highlights from The Pulsar Virtual Summit North America 2021 by Carolyn King.
  • Keynote - Why Unified Messaging and Streaming Is the Future. Matteo Merli, Sijie Guo.
  • Exactly Once Made Easy Transactional Messaging in Apache Pulsar. Sijie Guo, Addison Higham.
  • How Pulsar Stores Your Data. Shivji Kumar Jha.
  • Building the Next-Generation Platform on Pulsar at Intuit. Madhavan Narayanan, Sajith Sebasian, Amit Kaushal, Gokul Sarangapani.
  • Replicated Subscriptions: Taking Geo-Replication to the Next Level. Matteo Merli.
  • Scaling Apache Pulsar to 10 Petabytes/Day. Karthik Ramasamy.
  • Take Kafka on Pulsar to Production at Internet Scale: Improvement Made for Pulsar 2.8.0. Yunze Xu.
  • Manage Pulsar Cluster Lifecycles with Kubernetes Operators. Yang Yang.
  • Why Micro Focus Chose Pulsar for Data Ingestion. Srikanth Natarajan.
  • How Narvar Uses Pulsar to Power the Post-Purchase Experience. Ankush Goyal.
  • Pushing Pulsar Performance to the Limits. Jack Vanlightly, Ivan Kelly.
  • Down to the Rabbit Hole with Pulsar I/O. Ricardo Ferreira.
  • Apache BookKeeper State Store: A Durable Key-Value Store. Prashant Kumar.
  • Exclusive Producer: Using Pulsar to Build Distributed Applications. Matteo Merli.
  • How Big Is Too Big? Message Size in Pulsar. Anton Rodriguez.
  • Reduce Redundant Producers from Partitioned Producer. Yuri Muzushima.
  • Advanced Stream Processing with Flink and Pulsar. Till Rogrmann, Addison Higham.

• Pulsar Summit Asia 2020

  • The Rise of Apache Pulsar Community. Jennifer Huang.
  • Convergence of Messaging, Streaming, and Storage. Sijie Guo.
  • Introduce to Apache Pulsar 2.7.0: New Features and Updates. Jia Zhai & Penghui Li.
  • Pulsar-as-a-Service: Introducing StreamNative Cloud. Yang Yang.
  • Function Mesh - Complex Streaming Jobs in a Simple Way. Neng Lv, Sijie Guo.
  • Apache Pulsar at Yahoo! Japan - adoption, operational experiences and future. Nozomi Kurihara.
  • Running a secure Apache Pulsar Cluster. Shivji Kumar Jha.
  • CloudNative BookKeeper: What's new in BP-41: Bookie Network Address Change Tracking. Enrico Olivelli.
  • A Tale about Streaming Analytics using Pulsar and Elastic. Ricardo Ferreira.
  • Databases in the Microservices World. Rob Richardson.
  • Continuous Intelligence from Pulsar Event Streams. Simon Crosby.
  • What's New in BKVM 3.0 - BookKeeper Visual Manager. Matteo Minardi, Enrico Olivelli.
  • Migrate to Serverless with Pulsar Functions. Axel Sirota.
  • Connecting millions of users at Chatroulette. Tamer Abdulazim, Gabriel Volpe.
  • How Splunk is using Pulsar IO. Jerry Peng.
  • Apache Pulsar, To Kafka and Beyond!. William McLane.
  • Structured Data Stream with Pulsar. Shivji Kumar Jha.
  • Pulsar for Kafka People. Jesse Anderson.
  • Running Apache Pulsar on Tencent Cloud: New Challenges, Discussion, Practice. Lin Lin. 🇨🇳
  • Getting Started with Apache Pulsar: a Contributor's Journey. Hang Chen & Penghui Li. 🇨🇳
  • Apache Pulsar Helps Dada Group Build Real-Time Retail Services in China. Qinghua Zhang, Mingming Wan. 🇨🇳
  • Apache Pulsar in Huawei Cloud IoT Platform: Why and How to Configure Apache Pulsar. 🇨🇳
  • Tuya Cloud Builds Word-leading AI+IoT Platform with Apache Pulsar. Tianci Gao. 🇨🇳
  • Practices of the New Generation of Securities Trading Counters Powered with Apache Pulsar. 🇨🇳
  • Processing Billions of Messages Daily with Apache Pulsar in Kingsoft Cloud Log Service. 🇨🇳
  • Powering Newland Billing System with Apache Pulsar. Xiaohua Zhou. 🇨🇳
  • A Daredevil' Story: Apache Pulsar in Zhaopin.com. Shunli Gao. 🇨🇳
  • Federated Learning Sparkles with Apache Pulsar at Tencent Angel PowerFL. Chao Zhang. 🇨🇳
  • China Mobile Cloud Powered by Apache Pulsar: the Way to Cloud. Fangbin Sun. 🇨🇳
  • Powering Zhaopin's Real-time Search Recommendations with Apache Pulsar SQL. Shiming Li. 🇨🇳
  • When the Energy Internet meets Apache Pulsar. Rui Fu. 🇨🇳
  • How BIGO builds Real-Time Message System with Apache Pulsar and Flink. Hang Chen. 🇨🇳
  • Transactional Event Streaming with Apache Pulsar . Bo Cong. 🇨🇳
  • Pulsar Manager - Your Powerful Tool to Manage Apache Pulsar. Guangning E. 🇨🇳
  • Life with Apache Pulsar in Su Ning E-Commerce. Xinglin Chen. 🇨🇳
  • Real-time IoT Data Management using Apache IoTDB and Apache Pulsar. Xiangdong Huang. 🇨🇳
  • Benchmarking Pulsar vs. Kafka on AWS: Process & Results. Penghui Li. 🇨🇳
  • KoP: the bridge from Kafka to Pulsar. Yunze Xu. 🇨🇳
  • When TiDB meets Apache Pulsar - a Comprehensive Solution for Real-time Big Data. Yilin Zhao. 🇨🇳
  • Apache Pulsar + Flink: the Path to Unified Batch and Streaming. Jia Zhai. 🇨🇳
  • Pulsar Functions: a Lightweight Computing Framework for Apache Pulsar Based on Golang. 🇨🇳
  • Deploying Apache Pulsar with dLocal in Kubernetes. Alex Zheng. 🇨🇳
  • Integration of Apache APISIX and Apache Pulsar. Ming Wen. 🇨🇳

• Pulsar Summit 2020

  • Opening Keynote. Carolyn, Matteo, Sijie.
  • Why Splunk Chose Pulsar. Karthik Ramasamy.
  • Scaling Customer Engagement with Apache Pulsar. Greg Methvin.
  • Getting Pulsar Spinning: How we decided on and deployed Apache Pulsar in under a year. Addison Higham.
  • How Splunk Mission Control leverages various Pulsar subscription types. Pranav Dharma.
  • Securing your Pulsar Cluster with Vault. Chris Kellogg.
  • Kafka on Pulsar: Bringing native Kafka protocol support to Pulsar. Pierre Zemb, Sijie Guo.
  • Event propagation across microservices using Pulsar and CDC. Maheedhar Gunturu, Eyal Gutkind.
  • The first step to Multi AZ architecture for Apache BookKeeper. Anup Ghatage, Ankit Jain.
  • Introducing HerdDB : a distributed JVM embeddable database built upon Apache BookKeeper. Enrico Olivelli.
  • Pulsar Storage on BookKeeper: Seamless Evolution. Joe Francis, Rajan Dhabalia.
  • Pulsar Functions Deep Dive. Sanjeev Kulkarni.
  • Streaming, Fast and Slow. Caito Scherr.
  • Ten reasons to choose Apache Pulsar over Apache Kafka for Event Sourcing (CQRS). Robert van Mölken.
  • Finding your pulse for a global enterprise communications nervous system. William McLane.
  • Unified Data Processing with Apache Flink and Apache Pulsar. Seth Wiesman.
  • Building a Messaging Solutions for OVHcloud with Apache Pulsar. Pierre Zemb.
  • Using Apache Pulsar to Provide Real-Time IoT Analytics on the Edge. David Kjerrumgaard.
  • Easily Build a Smart Pulsar Stream Processor. Simon Crosby.
  • Architecting Production IoT Analytics. Paige Roberts.
  • Large scale log pipeline using Apache Pulsar. Nozomi Kurihara.
  • Pulsar unikernel FaaS, the new Pulsar functions runners?. Alexandre Duval, Geoffroy Couprie.
  • Interactive querying of streams using Apache Pulsar. Jerry Peng.
  • Stream or segment: What is the best way to access your events in Pulsar. Neng Lu.
  • Pulsar Architectural Patterns for CI/CD Automation and Self-Service. Devin Bost.
  • How Apache Pulsar Helps Tencent Process Tens of Billions of Transactions Efficiently. Ningguo Chen.
  • Five years of operating a large scale globally replicated Pulsar installation. Francis, Ludwig Pummer.
  • Pulsar for Kafka People. Jesse Anderson.
  • Testing Event-Driven Applications over Pulsar. Venky Ganti.
  • Feature Stores: Building Machine Learning Infrastructure on Apache Pulsar. Simba Khadder.
  • Unify Storage Backend for Batch and Streaming Computation with Apache Pulsar. Vincent Xie.

📺 Videos

StreamNative YouTube channel

• StreamNative Academy Introduction to Pulsar

  • Pulsar Benefits
  • Common Use Cases
  • How Pulsar uses the Pub-Sub Model
  • Introduction to Pulsar Subscriptions
  • Introduction to Pulsar API
  • Introduction to Multi-Tenancy in Pulsar
  • What is Apache Pulsar?
  • Benefits of using Apache Pulsar
  • Multi-layered Architecture

• StreamNative Academy

  • Welcome to StreamNative Academy
  • Introduction to Apache Pulsar
  • How to Get Started with an Apache Pulsar Cluster
  • How to Get Started with Apache Pulsar Producers
  • How to Get Started with Apache Pulsar Consumers
  • How to Deploy Apache Pulsar Using Helm Charts
  • How to Build and Deploy a Source Connector with Apache Pulsar
  • How to Integrate Apache Pulsar with BiqQuery
  • Intro to Apache Pulsar 101 (old version)

• Apache Pulsar Webinars

  • How to Build an Event Sourcing App with Apache Pulsar. 🗓️ Mar 29, 2022.
  • Pulsar and Nifi for Cloud Data Lakes. 🗓️ Mar 11, 2022.
  • Build an Event-Driven Architecture with Pulsar. 🗓️ Jan 14, 2022.
  • "Apache Pulsar in Action" by David Kjerrumgaard - Book Launch. 🗓️ Dec 10, 2021.
  • StreamNative Cloud Workshop. 🗓️ Nov 4, 2021.
  • Developing Event-driven Microservices using Apache Pulsar - Part 3. 🗓️ Oct 2, 2021.
  • Developing Event-driven Microservices using Apache Pulsar - Part 2. 🗓️ Sep 16, 2021.
  • Developing Event-driven Microservices using Apache Pulsar - Part 1. 🗓️ Sep 10, 2021.
  • How Splunk Built a Real-Time Stream Processing Engine with Apache Pulsar x Flink. 🗓️ May 3, 2021.
  • Apache Flink x Pulsar Virtual Meetup: Streaming SQL at Uber and Facebook - Day Two. 🗓️ Mar 18, 2021.
  • Apache Flink x Pulsar Virtual Meetup: Streaming SQL at Uber and Facebook - Day One. 🗓️ Mar 18, 2021.
  • TiDB + Pulsar, Event Streaming Architecture in Action. 🗓️ Mar 4, 2021.
  • Watch Your Streams: Implementing OpenTelemetry with Apache Pulsar. 🗓️ Jan 26, 2021.
  • Low-Latency Stream Processing with Jet. 🗓️ Oct 27, 2020.
  • Managing Real-Time Data Teams. 🗓️ Aug 28, 2020.
  • How to Operate Pulsar in Production. 🗓️ Jul 28, 2020.
  • Lessons From Managing A Pulsar Cluster by Nutanix. 🗓️ May 26, 2020.
  • Common architectural patterns on Apache Pulsar. 🗓️ Apr 28, 2020.
  • Introducing Kafka-on-Pulsar: bring native Kafka protocol support to Apache Pulsar. 🗓️ Mar 31, 2020.

• TGI Pulsar is a weekly live video streaming about Apache Pulsar and its ecosystem.

  • #23 June Project Updates on Apache Pulsar. 🗓️ Jun 9, 2021.
  • #22 May Project Updates on Apache Pulsar. 🗓️ May 13, 2021.
  • #21 April Updates on Apache Pulsar. 🗓️ Apr 16, 2021.
  • #20 March Updates on Apache Pulsar. 🗓️ Mar 11, 2021.
  • #19 February Updates on Apache Pulsar. 🗓️ Feb 15, 2021.
  • #18 From Single-AZ to Multi-AZ and Geo-replication. 🗓️ Aug 7, 2020.
  • #17 Pulsar Multi-Tenancy Best Practices. 🗓️ Jul 31, 2020.
  • #16 Backlog and StorageSize. 🗓️ Jul 24, 2020.
  • #15 What's New In Pulsar 2.6.0?. 🗓️ Jul 10, 2020.
  • #14 Pulsar Schema and Flink Integration. 🗓️ Jun 26, 2020.
  • #13 Pulsar Schema and Presto SQL. 🗓️ Jun 5, 2020.
  • #12 Introduction of Pulsar Schema. 🗓️ May 29, 2020.
  • #11 BookKeeper AutoRecovery. 🗓️ May 22, 2020.
  • #10 BookKeeper LAC Protocol. 🗓️ May 15, 2020.
  • #9 Introduction of Apache BookKeeper. 🗓️ May 8, 2020.
  • #8 What is New in Pulsar 2.5.1?. 🗓️ May 1, 2020.
  • #7 Cluster & Geo-replication. 🗓️ Apr 24, 2020.
  • #6 Lifecycle of a Pulsar message. 🗓️ Apr 17, 2020.
  • #5 Take a deep-dive into Pulsar architecture for performance tuning. 🗓️ Apr 10, 2020.
  • #4 Deep dive into authentication and authorization. 🗓️ Apr 3, 2020.
  • #3 Secure a Pulsar cluster with TLS. 🗓️ Mar 27, 2020.
  • #2 Proxy and Kubernetes Deployment. 🗓️ Mar 20, 2020.
  • #1 Install a Pulsar cluster. 🗓️ Mar 13, 2020.

  Monthly updates

  • Apache Pulsar Updates & Pulsar Schema Registry. Matteo Merli. 🗓️ Sep 10, 2021.

  Pulsar Office hours

  • 🗓️ Feb 17, 2021
  • 🗓️ Mar 17, 2021
  • 🗓️ Apr 30, 2021
  • 🗓️ May 20, 2021

DataStax Developers YouTube channel

• Bring Streaming to Cassandra with Apache Pulsar
• 5 Minutes About Apache Pulsar
  • Real-time data pipeline with Astra and Decodable.co
  • Extended Functions SDK for Java
  • Getting Started with the Elastic Search Sink
  • Getting Started with Pulsar SQL
  • Astra Streaming Connectors
  • Migrating from JMS to Pulsar with Starlight for JMS
  • Creating Your First Pulsar I/O Sink
  • Getting Started with the Cassandra Sink
  • Creating a Pulsar I/O Source Connector
  • Multi-Topic Processing with Java Functions
  • Creating a Python Function
  • Message Deduplication
  • Working with Avro Schemas
  • Simple Producer
  • Simple Consumer
  • 10 Reasons for Pulsar
  • Migrating from Kafka to Pulsar with Starlight for Kafka
  • What is a Partitioned Topic
  • Managing Geo-replicated Clusters with Pulsar Admin Console
  • Geo-replication Using Pulsar Standalone Clusters
  • How BookKeeper keeps your data safe
  • Pulsar Shell
  • Json Web Token (JWT) Client Authentication
  • Exclusive Producer Access Mode
  • Proving Pulsar Performance with OpenMessaging Benchmark
  • In-Order Processing using Partitioned Topics and Failover Subscriptions
  • Getting Started with a File Source
  • Reactive Applications with Spring Boot
  • Getting Started with Astra Streaming
  • OKTA Integration using OpenID Connect
  • Functions in the Kubernetes Runtime
  • Geo-replication
  • Getting Started with DataStax Admin Console
  • Your First Transaction
  • Getting Started with Pulsar using Kubernetes
  • Consumer Receive Modes
  • Key Shared Subscriptions
  • Creating a Java Function
  • Shared Subscriptions
  • Failover Subscriptions