Why Argo CD Wasn't Enough: Real GitOps Pain and the Tools That Fixed It
JosephCheng

JosephCheng @josephcc

About: I write what I build — GitOps, Kubernetes, MLOps, and everything in between. Say hi or reach out: joseph.mycena@gmail.com

Location:
Taiwan
Joined:
Apr 9, 2025

Why Argo CD Wasn't Enough: Real GitOps Pain and the Tools That Fixed It

Publish Date: May 8
2 0

This is the first post in the "Building a Real-World GitOps Setup" series.

👉 Part 2: From Kro RGD to Full GitOps: How I Built a Clean Deployment Flow with Argo CD

👉 Part 3: Designing a Maintainable GitOps Repo Structure: Managing Multi-Service and Multi-Env with Argo CD + Kro

👉 Part 4: GitOps Promotion with Kargo: Image Tag → Git Commit → Argo Sync

👉 Part 5: Implementing a Modular Kargo Promotion Workflow: Extracting PromotionTask from Stage for Maintainability

👉 Part 6: Designing a Maintainable GitOps Architecture: How I Scaled My Promotion Flow from a Simple Line to a System That Withstands Change


This series isn't about feature overviews or polished diagrams. It's a real journey - how I started with Argo CD, ran into scaling pains, and ended up building a workflow around Kro and Kargo to simplify the chaos.


The Starting Point: Argo CD as Our First GitOps Tool

Our team uses GitLab to manage our projects. Since Argo CD supports Git as a source of truth, it was naturally the first GitOps tool we adopted.
It's stable, visual, and easy to get started with - at least at the beginning.
But as the number of services grew, so did the complexity.


The Problem: Too Many YAML Files, Too Much Maintenance

As we added more services, I started to feel like managing YAML was getting out of hand.
Each service had its own Deployment, Service, and ConfigMap. Updating an image tag or tweaking an environment variable meant editing three different files and submitting three PRs just to change a single port.

That felt wrong.

Isn't there a way to change one file and have everything else update accordingly?

I wanted to stop maintaining three YAML files just to represent "this service is now version X."

What I really needed was:

  • A single instance.yaml per service
  • Define tag, port, and env vars in one place
  • Use that to generate the actual K8s manifests
  • Let Argo CD keep syncing, while keeping the repo clean

The Solution Appears: Discovering Kro

At this point, I started looking for tools that could turn high-level service definitions into manifests automatically.
That's when I found Kro

Here's what I was trying to simplify:

Before:

frontend/
├── deployment.yaml
├── service.yaml
└── configmap.yaml
Enter fullscreen mode Exit fullscreen mode
backend/
├── deployment.yaml
├── service.yaml
└── configmap.yaml
Enter fullscreen mode Exit fullscreen mode

After:

frontend/
└── instance.yaml → Kro → generated manifests
Enter fullscreen mode Exit fullscreen mode

Example instance.yaml:

spec:
  values:
    deployment:
      tag: 0320.1
      port: 3000
      image: frontend
    config:
      LOG_LEVEL: debug
      API_URL: https://api.example.com
Enter fullscreen mode Exit fullscreen mode

Just editing this one file could trigger all necessary changes.
It felt clean, declarative, and scalable.


Why I Chose Kro Instead of Writing Raw YAML

At this point, I didn’t want to manage dozens of YAML files by hand anymore.

What I really needed was a way to describe the intent of a service — not duplicate the same boilerplate across deployments, services, and configmaps.

I didn’t want to turn my GitOps repo into a config sprawl.

I wanted something:

  • Declarative, but still readable
  • Focused on service logic, not YAML mechanics
  • Easy to compose and integrate with Argo CD
  • Able to reason about dependencies between resources

That’s when Kro clicked.

With a single instance.yaml per service, I could:

  • Define tag, port, and config in one place
  • Use a ResourceGraphDefinition to render full Kubernetes manifests
  • Let Argo CD do the syncing, while I focused on intent

And one design choice I found especially elegant:

Kro builds a Directed Acyclic Graph (DAG) from your defined resources, which ensures:

  • All referenced values are valid and resolvable
  • Dependencies (like Service → Deployment → ConfigMap) follow a safe apply order
  • There's no cyclic reference that might cause unpredictable state

This structure gave me both clarity and safety — I could think in terms of service logic, while Kro guaranteed a consistent deployment process behind the scenes.

Kro also validates this structure as part of its ResourceGraphDefinition processing — catching circular references, invalid types, and broken dependencies early.


TL;DR - The GitOps Workflow in One Diagram

The full GitOps flow: instance files drive Kro, image tags trigger Kargo, and Argo CD keeps the cluster in sync.
A workflow where instance files define the service setup, and everything else flows from that.


So What's Next?

This post was all about motivation and setup.

Next, I'll show you how I wrote my first ResourceGraphDefinition (RGD)-the template Kro uses to render Deployments, Services, and ConfigMaps. I'll walk through its structure, the pitfalls I encountered, and how I got it to render real, production-ready manifests.

If you've ever patched the same deployment.yaml three times just to bump an image tag, this series is for you.

Follow along as I break down the entire GitOps stack, one layer at a time.
Thanks for reading 🙇

🧠 If you're also trying to simplify GitOps or reduce YAML fatigue, I'd love to hear how you're approaching it.

Comments 0 total

    Add comment