We’ve all been there. It’s 2am, you’re knee-deep in a ClickHouse query that refuses to cooperate, the Helm chart values file looks like it was written in ancient Sumerian, and your AI assistant confidently suggests a Karpenter configuration option that hasn’t existed since version 0.27. The AI is brilliant, creative, and occasionally a magnificent liar when it comes to niche technical documentation.
Back in December 2024, I wrote about gcp-kafka-auth-handler, a utility I created to bridge the authentication gap between Apache Beam Dataflow and GCP Managed Kafka. Since then, the project has evolved significantly as part of our broader multi-cloud journey. Today, I’m pleased to announce that the library has been renamed to kafka-auth-handler and now supports both GCP and AWS MSK.
Managing CI/CD pipelines across multiple repositories can quickly become unwieldy. Each project needs versioning, container builds, deployments, and releases—often with subtle variations that lead to duplicated workflow code. This post introduces an open deployment framework built entirely on GitHub Actions, designed to bring consistency and reusability to cloud-native deployments.
When working with Google Cloud Platform’s Managed Service for Apache Kafka, you’ll quickly discover that authentication can be surprisingly challenging, especially when using Apache Beam Dataflow pipelines. In this post, I’ll share a utility I created called gcp-kafka-auth-handler that bridges this gap.
When architecting cloud-based solutions, one key principle I follow is to isolate resources within their respective regions and avoid sharing or replicating them across regions. This approach consistently provides a more secure and compliant framework for business continuity. Recently, AWS has introduced replication capabilities for various resources. In this post, I will delve into AWS Key Management Service (KMS) and assess whether adopting replication for KMS keys offers tangible benefits.