Insights from the Apache Beam Summit 2023

Apache Beam is an open-source, popular streaming data processing framework. These are my thoughts, reflections, and insights from the 2023 edition of the Beam Summit. This was the 6th year of Beam summits, and it was the biggest event yet with 500+ Beamers in attendance at a brand-new waterfront venue in New York City with amazing views. The total number of attendees was 1000+ with the virtual component. You can read the Beam Summit 2023 Event Report to learn more about the event.

My top personal impressions were:

1. The community is larger and friendlier than ever. Lots of new companies using Beam, and lots of new faces in attendance.
2. Beam is in production everywhere. People choose it because it is easy to use, easy to operate, and easy to learn.
3. Beam is in the center of Data & AI. When it comes to AI, Beam comes with batteries included. There are robust, fast, easy-to-use (just a few lines) tools with efficient GPU support out of the box. There are also lots of examples, solutions, and turnkey transforms (e.g., pre-processing, post-processing, and inference) to get anyone started on AI OPs as quickly as possible.
4. Streaming analytics is growing exponentially, and Beam is at the center of streaming first experiences.
5. Managed Beam platforms are spreading and growing as well. Data teams are choosing to standardize on Beam. Deploying Beam on Kubernetes is a common pattern, and there are out-of-the-box solutions for doing that for Beam.
6. ML, SQL, and GPUs are everywhere, and Beam users are using those at scale today.
7. People are excited about multi-language support, a unique Beam feature, for mixing & matching languages/libraries/teams easily. Multi-language transforms are in production, at multiple companies, and solving real-world problems.
8. People invest in Beam to future-proof their careers and their data infrastructure. Beam runs portably on different clouds, languages, and data processing systems.
9. The Beam community is focused on improving user experience by strategically investing in ML, usability, and streaming.
10. 10/10 amazing community, and an amazing organization, thanks to all organizers, speakers, and attendees!

I hope to see more people like you in person at next year’s summit. This is an amazing community to be part of, and if you are interested you can learn more about how you can contribute.