Cloudflare uses Kafka to decouple microservices and communicate the creation, change or deletion of various resources via a common data format in a fault-tolerant manner. This decoupling is one of many factors that enables Cloudflare engineering teams to work on multiple features and products concurrently.
We learnt a lot about Kafka on the way to one trillion messages, and built some interesting internal tools to ease adoption as well as improve resiliency that will be explored in this talk.
Interview:
What's the focus of your work these days?
Andrea: I work as a Systems Engineer at Cloudflare. I research and help prioritize what's needed to make other developers' lives easier. I design, implement and maintain platforms used by other teams to do their job effectively. For example, we maintain SDKs for Kafka and platforms that allow emitting alerts, emails audit logs and so on.
What's the motivation for your talk at QCon London 2023?
We wanted to share more about what we do for productivity, organizational culture, and also I've never spoken to such a big conference before, so it's an amazing chance for me to become more confident.
How would you describe your main persona and target audience for this session?
I think there will be a lot of people that have a lot of experience, senior engineering, CTOs, tech leads.
Is there anything specific that you'd like people to walk away with after watching your session?
A better understanding of how to set up for success when deciding to integrate with Kafka, especially in a big company where there's a lot of traffic. Many of the points in our presentation also apply to different worlds, different platforms and systems. I think everyone can get something from it.
Speaker
Andrea Medda
Senior Systems Engineer @Cloudflare
Andrea is a Senior Systems Engineer at Cloudflare. He loves using cutting edge technology and approaches to solve real customer problems.
Andrea loves Golang and distributed systems. He has a pet friend Maru that loves to pop by in his meetings.
Find Andrea Medda at:
Speaker
Matt Boyle
Engineering Manager @Cloudflare
Matthew Boyle is an experienced technical leader in the field of distributed systems, specializing in using Go.
He has worked at huge companies such as Cloudflare and General Electric, as well as exciting high-growth startups such as Curve and Crowdcube.
Matt has been writing Go for production since 2018 and often shares blog posts and fun trivia about Go over on Twitter (@MattJamesBoyle).