About

Creating fault-tolerant, highly available and maintainable workloads

Abstract

My role involves translating the technical requirements of a project into its architectural and design framework, which essentially shapes the final product. Over the years, I’ve specialised in creating cloud solutions that are fault-tolerant, highly available, deployed globally, and managed by small teams. I aim to share my experiences and some of the solutions I’ve designed and implemented.

Recently, I’ve been exploring certain technologies that have captured my attention. Firstly, Golang has piqued my interest. Coming from a strong background in low-latency Java services, this modern language offers a new realm of possibilities to develop exceptionally fast services. It combines the power of C/C++ with a more user-friendly development approach. Additionally, there’s been a revolution in software distribution with tools like Packer or Docker, eliminating the need for specific versions of virtual machines or interpreters. This significantly enhances the reliability of software production.

I perceive this as a cyclic evolution. Previously, C/C++ was recognised as the language for high-speed services, yet its maintenance, learning curve, and compilation posed challenges. Presently, we’re in an era where the internet and platforms like Github collaborate to facilitate easier software sharing. Go leverages this environment, producing a binary executable that can run on any supported operating system.

Another significant technology stack I’ve been involved with is Kubernetes (K8s). Having worked extensively in cloud environments, I’ve witnessed the substantial benefits of implementing Kubernetes, ranging from cost efficiencies to accelerated software development. Currently, I’m engaged in migrating a sizable platform comprising various workloads, including APIs, UIs, batch processing, and low-latency services. I intend to document and share this ongoing journey.

I hope that this blog proves beneficial to someone navigating similar technological landscapes.

Latest Posts

MCP Servers - Teaching AI to Read the Manual (So You Don't Have To)
MCP Servers - Teaching AI to Read the Manual (So You Don't Have To)

We’ve all been there. It’s 2am, you’re knee-deep in a ClickHouse query that refuses to cooperate, the Helm chart values file looks like it was written in ancient Sumerian, and your AI assistant confidently suggests a Karpenter configuration option that hasn’t existed since version 0.27. The AI is brilliant, creative, and occasionally a magnificent liar when it comes to niche technical documentation.

Kafka Auth Handler Goes Multi-Cloud
Kafka Auth Handler Goes Multi-Cloud

Back in December 2024, I wrote about gcp-kafka-auth-handler, a utility I created to bridge the authentication gap between Apache Beam Dataflow and GCP Managed Kafka. Since then, the project has evolved significantly as part of our broader multi-cloud journey. Today, I’m pleased to announce that the library has been renamed to kafka-auth-handler and now supports both GCP and AWS MSK.

Building an Open Deployment Framework with GitHub Actions
Building an Open Deployment Framework with GitHub Actions

Managing CI/CD pipelines across multiple repositories can quickly become unwieldy. Each project needs versioning, container builds, deployments, and releases—often with subtle variations that lead to duplicated workflow code. This post introduces an open deployment framework built entirely on GitHub Actions, designed to bring consistency and reusability to cloud-native deployments.