IDP

Confessions of a recovering GitOps addict

There’s a moment in every tech trend’s lifecycle when the magic starts to wear off. It’s like realizing the artisanal, organic, free-range coffee you’ve been paying eight dollars for just tastes like… coffee. For me, and many others in the DevOps trenches, that moment has arrived for GitOps.

We once hailed it as the silver bullet, the grand unifier, the one true way. Now, I’m here to tell you that the romance is over. And something much more practical is taking its place.

The alluring promise of a perfect world

Let’s be honest, we all fell hard for GitOps. The promise was intoxicating. A single source of truth for our entire infrastructure, nestled right in the warm, familiar embrace of Git. Pull Requests became the sacred gates through which all changes must pass. CI/CD pipelines were our holy scrolls, and tools like ArgoCD and Flux were the messiahs delivering us from the chaos of manual deployments.

It was a world of perfect order. Every change was audited, every state was declared, and every rollback was just a git revert away. It felt clean. It felt right. It felt… professional. For a while, it was the hero we desperately needed.

The tyranny of the pull request

But paradise had a dark side, and it was paved with endless YAML files. The first sign of trouble wasn’t a catastrophic failure, but a slow, creeping bureaucracy that we had built for ourselves.

Need to update a single, tiny secret? Prepare for the ritual. First, the offering: a Pull Request. Then, the prayer for the high priests (your colleagues) to grant their blessing (the approval). Then, the sacrifice (the merge). And finally, the tense vigil, watching ArgoCD’s sync status like it’s a heart monitor, praying it doesn’t flatline.

The lag became a running joke. Your change is merged… but has it landed in production? Who knows! The sync bot seems to be having a bad day. When everything is on fire at 2 AM, Git is like that friend who proudly tells you, “Well, according to my notes, the plan was for there not to be a fire.” Thanks, Git. Your record of intent is fascinating, but I need a fire hose, not a historian.

We hit our wall during what should have been a routine update.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: auth-service
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: auth-service-container
        image: our-app:v1.12.4
        envFrom:
        - secretRef:
            name: production-credentials

A simple change to the production-credentials secret required updating an encrypted file, PR-ing it, and then explaining in the commit message something like, “bumping secret hash for reasons”. Nobody understood it. Infrastructure changes started to require therapy sessions just to get merged.

And then, the tools fought back

When a system creates more friction than it removes, a rebellion is inevitable. And the rebels have arrived, not with pitchforks, but with smarter, more flexible tools.

First, the idea that developers should be fluent in YAML began to die. Internal Developer Platforms (IDPs) like Backstage and Port started giving developers what they always wanted: self-service with guardrails. Instead of wrestling with YAML syntax, they click a button in a portal to provision a database or spin up a new environment. Git becomes a log of what happened, not a bottleneck to make things happen.

Second, we remembered that pushing things can be good. The pull-based model was trendy, but let’s face it: push is immediate. Push is observable. We’ve gone back to CI pipelines pushing manifests directly into clusters, but this time they’re wearing body armor.

# This isn't your old wild-west kubectl apply
# It's a command wrapped in an approval system, with observability baked in.
deploy-cli --service auth-service --env production --approve

The change is triggered precisely when we want it, not when a bot feels like syncing. Finally, we started asking a radical question: why are we describing infrastructure in a static markup language when we could be programming it? Tools like Pulumi and Crossplane entered the scene. Instead of hundreds of lines of YAML, we’re writing code that feels alive.

import * as aws from "@pulumi/aws";

// Create an S3 bucket with versioning enabled.
const bucket = new aws.s3.Bucket("user-uploads-bucket", {
    versioning: {
        enabled: true,
    },
    acl: "private",
});

Infrastructure can now react to events, be composed into reusable modules, and be written in a language with types and logic. YAML simply can’t compete with that.

A new role for the abdicated king

So, is GitOps dead? No, that’s just clickbait. But it has been demoted. It’s no longer the king ruling every action; it’s more like a constitutional monarch, a respected elder statesman.

It’s fantastic for auditing, for keeping a high-level record of intended state, and for infrastructure teams that thrive on rigid discipline. But for high-velocity product teams, it’s become a beautifully crafted anchor when what we need is a motor.

We’ve moved from “Let’s define everything in Git” to “Let’s ship faster, safer, and saner with the right tools for the job.”

Our current stack is a hybrid, a practical mix of the old and new:

  • Backstage to abstract away complexity for developers.
  • Push-based pipelines with strong guardrails for immediate, observable deployments.
  • Pulumi for typed, programmable, and composable infrastructure.
  • Minimal GitOps for what it does best: providing a clear, auditable trail of our intentions.

GitOps wasn’t a mistake; it was the strict but well-meaning grandparent of infrastructure management. It taught us discipline and the importance of getting approval before touching anything important. But now that we’re grown up, that level of supervision feels less like helpful guidance and more like having someone watch over your shoulder while you type, constantly asking, “Are you sure you want to save that file?” The world is moving on to flexibility, developer-first platforms, and code you can read without a decoder ring. If you’re still spending your nights appeasing the YAML gods with Pull Request sacrifices for trivial changes… you’re not just living in the past, you’re practically a fossil.

AWS IDP Short reference Architecture

Organizations must be agile and innovative to stay competitive in today’s software development era, which has led to changes in how applications are built, deployed, and managed.

This necessitates the transformation of static CI/CD setups into modern Internal Developer Platforms (IDPs) that provide developers with the tools needed to innovate and move quickly.

While every platform looks different, specific common patterns emerge. To help simplify things, I’ve consolidated the platform designs of dozens of setups into standard patterns based on real-world experiences, which have been proven to work effectively. By adopting these patterns, organizations can create IDPs that keep them ahead of the competition and deliver innovative applications faster.

This diagram provides an overview of one reference architecture for a dynamic IDP using AWS EKS, RDS, Backstage, Humanitec, GitHub Actions, Terraform, and several other technologies.

AWS IDP Architecture

Design principles.

01 Focus on the user. The most important customers of a developer platform
are developers. Developers need to be heavily involved in the design,
prioritization of features, and testing to ensure the platform is fit for purpose
and fully self-service. 

02 Run your platform team like a start-up. Establish a small central team that
owns the platform and is responsible for marketing it, ensuring it is easily consumable and fulfills developers’ needs.

03 Build golden paths vs cages. Developers should be free to choose their
abstraction level. While your IDP should provide a set of golden paths for developers to follow, it should never force their use.

04 Drive standardization by design. Enabling self-service means platform
engineers must define how to vend resources and configuration. This ensures
every resource is built securely, compliant, and well-architected.

05 Implement Dynamic Configuration Management. Dynamic Configuration Management significantly reduces config complexity and enforces standardization by continuously generating app and infrastructure configs with every single deployment. This allows you to enforce policies and standards with every git-push.

06 Let developers decide on their platform interface. The developer
platform should never break your developer’s workflow or force them to use a
specific interface. To support this, a code-based workflow by default works
best, with the option to use a UI, CLI, or API.

07 Keep code as the single source of truth. This ensures everyone is working
from the same version, reducing the risk of errors.

08 Assume a brownfield scenario. Use tools that have already been
productized and adopted by the organization (such as backlog management,
CI/CD toolchain, and container platform) and let the existing teams pursue
integration through plug-ins into the platform. Where applicable, organizations
should use open-source tooling and a cloud-native approach.