Cloud Architecture Notepad

Notes about AWS, GCP, K8s, AI and Data

A simple Go program that uses concurrency
A simple Go program that uses concurrency

Go supports multiple concurrency models, this diagram shows what actually want to implement, it parallelises 3 long running operations, generally these are calls to external services, for testing purposes these external call are mocked with different random delays (1-50 ms) and there is a restriction the whole service execution time cannot last more than 20ms.

Using aws-cli and oathtool
Using aws-cli and oathtool

Using two factor authentication with the AWS CLI is sometimes a pain, you need to get a new token every N minutes then parse the result of this operation and create the corresponding environment variables, I’ve installed the oath-toolkit and configured the AWS CLI to get these OTP dynamically.

Installing Minikube and Hyperkit
Installing Minikube and Hyperkit

I’ve got some problems with Minikube and Virtualbox, everytime I close my laptop it stops working and I need to restart, delete, start, reboot, etc. it’s not very stable for this use case. I’ve installed the HyperKit driver and configured Minikube to use this instead the default Virtualbox.

Creating a fault-tolerant NAT instance on AWS
Creating a fault-tolerant NAT instance on AWS

With the new NAT Gateway this looks like something of the past, but recently I had to configure a NAT instance in our deployment. This Cloudformation configuration creates an autoscaling group with 1 instance allows recovery. The following diagram shows the resources created by the stack.

Latest Posts

Kafka Auth Handler Goes Multi-Cloud
Kafka Auth Handler Goes Multi-Cloud

Back in December 2024, I wrote about gcp-kafka-auth-handler, a utility I created to bridge the authentication gap between Apache Beam Dataflow and GCP Managed Kafka. Since then, the project has evolved significantly as part of our broader multi-cloud journey. Today, I’m pleased to announce that the library has been renamed to kafka-auth-handler and now supports both GCP and AWS MSK.

Building an Open Deployment Framework with GitHub Actions
Building an Open Deployment Framework with GitHub Actions

Managing CI/CD pipelines across multiple repositories can quickly become unwieldy. Each project needs versioning, container builds, deployments, and releases—often with subtle variations that lead to duplicated workflow code. This post introduces an open deployment framework built entirely on GitHub Actions, designed to bring consistency and reusability to cloud-native deployments.

GCP Managed Kafka Authentication Handler
GCP Managed Kafka Authentication Handler

When working with Google Cloud Platform’s Managed Service for Apache Kafka, you’ll quickly discover that authentication can be surprisingly challenging, especially when using Apache Beam Dataflow pipelines. In this post, I’ll share a utility I created called gcp-kafka-auth-handler that bridges this gap.