Implementing Secure Breakglass Access for ArgoCD with Vault, External Secrets, and Terraform

Overview

This article outlines a secure and automated approach to implementing breakglass access for ArgoCD. Breakglass access refers to emergency administrative access that can be used when standard authentication methods fail or are temporarily unavailable. This solution integrates HashiCorp Vault, the External Secrets Operator (ESO), and Terraform to securely provision and manage credentials while maintaining flexibility and minimizing operational overhead.

Goals

Automate provisioning of a breakglass user for ArgoCD.
Store credentials securely using Vault.
Enable ArgoCD to dynamically access credentials via External Secrets.
Allow breakglass provisioning to be toggled per environment.
Avoid manual secret management or multi-repo coordination.

Architecture Components

HashiCorp Vault: Serves as the single source of truth for secrets.
External Secrets Operator (ESO): Syncs secrets from Vault to Kubernetes.
Terraform: Automates the provisioning of secrets and configuration.
ArgoCD: The GitOps tool that requires controlled emergency access support.

Solution Breakdown

1. Secret Generation and Storage in Vault

Terraform is used to:

Generate a random password for the breakglass account.
Hash the password using bcrypt.
Store both the plaintext and hashed versions in Vault under a dedicated path (e.g., secrets/platform/argocd/breakglass).

Example structure:

{
  "accounts.breakglass.username": "breakglass",
  "accounts.breakglass.password": "<bcrypt-hashed-password>",
  "argocd-login-password": "<plaintext-password>"
}

2. Conditional Provisioning with Terraform

A toggle (create_break_glass_access) is used to control whether the breakglass secret is provisioned in a specific environment.

When true: credentials are created and pushed to Vault.
When false: an empty password is stored, effectively disabling login.

This avoids needing separate manual steps to deactivate access.

3. Integration with External Secrets

The External Secrets Operator is configured to sync the Vault secret into a Kubernetes secret (argocd-secret), which ArgoCD reads for authentication.

Example ExternalSecret config:

- secretKey: accounts.breakglass.username
  remoteRef:
    key: platform/argocd/breakglass
    property: accounts.breakglass.username
- secretKey: accounts.breakglass.password
  remoteRef:
    key: platform/argocd/breakglass
    property: accounts.breakglass.password

This ensures secrets are injected into the argocd-secret, enabling login without exposing credentials in Git.

Challenge: ConfigMap vs. Secret for ArgoCD Accounts

One of the key design decisions involved how to configure the accounts.breakglass.enabled flag in ArgoCD. While ArgoCD expects user enablement via its argocd-cm ConfigMap, syncing this flag dynamically via ESO is not supported out-of-the-box.

Resolution:
The system avoids relying on this flag by using a controlled approach:

If the Terraform flag is false, no valid credentials are stored in Vault.
As a result, the synced Kubernetes secret contains an empty password, rendering login ineffective even if the account is enabled in the ConfigMap.

This minimizes operational complexity by removing the need to manage multiple sources of truth.

Policy Integration

In environments where fine-grained access control is required, a Vault policy is added to allow only specific service accounts or roles (e.g., breakglass, platform-admins) to retrieve the secret.

Example policy:

path "secrets/data/platform/argocd/breakglass" {
  capabilities = ["read"]
}

Environment-Specific Controls

The solution supports per-environment deployment using configuration files (e.g., dev.tfvars, preprod.tfvars), allowing each environment to independently enable or disable breakglass access.

This makes it easy to:

Enable access in preprod or staging for testing.
Keep access disabled in production unless needed.
Control access lifecycles via GitOps workflows.

Benefits

No need to hardcode or expose credentials in repositories.
Seamless integration with ArgoCD using existing mechanisms (secrets).
Unified control via Terraform.
Supports dynamic toggling without manual intervention.

Conclusion

Implementing a robust break-glass mechanism for ArgoCD using HashiCorp Vault, External Secrets Operator (ESO), and Terraform significantly enhances the security and maintainability of Kubernetes-based environments. By automating the generation, storage, and syncing of emergency credentials, this solution eliminates manual intervention while ensuring credentials remain protected, auditable, and environment-controlled.

This design also simplifies operations during critical scenarios by reducing the number of steps required to enable or disable access. Integrating with Vault provides centralized secret management, while ESO ensures seamless syncing to Kubernetes. By keeping the setup modular and driven by infrastructure-as-code, organizations can adopt this pattern across multiple environments with minimal duplication and high confidence.

This approach can serve as a template for other sensitive access workflows, ensuring security is never compromised even under pressure.

Eunice js @eunice-js