🚀 Kubernetes ConfigMaps and Secrets: Automating Updates with Generators 🛠️

📋 Overview of the Problem

In Kubernetes, ConfigMaps 📄 and Secrets 🔒 are used to manage configuration data and sensitive information (e.g., passwords) for applications running in pods. However, a significant issue arises when the data in a ConfigMap or Secret is updated: the dependent Kubernetes deployment does not automatically update or redeploy 😕. This means that pods continue to use outdated configuration values until a manual intervention (e.g., kubectl rollout restart) is performed. ConfigMap Generators ⚙️ and Secret Generators 🔑 (typically used with tools like Kustomize) address this issue by automating the process of updating deployments when configurations change.

🔍 Key Concepts Explained

1. ConfigMaps 📄

Definition: A ConfigMap is a Kubernetes resource used to store non-sensitive configuration data in key-value pairs or as files. It decouples configuration from application code, making applications portable and easier to manage.
Use Case: ConfigMaps are used to provide environment variables, configuration files, or command-line arguments to containers in a pod.
Example:

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: db-credentials
  data:
    password: password1

This ConfigMap (db-credentials) stores a key-value pair where the key is password and the value is password1.
- How It’s Used in a Deployment: A deployment can reference a ConfigMap to set environment variables for a container:

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-deployment
  spec:
    replicas: 1
    selector:
      matchLabels:
        app: nginx
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
        - name: nginx
          image: nginx
          env:
          - name: DB_PASSWORD
            valueFrom:
              configMapKeyRef:
                name: db-credentials
                key: password

Here, the DB_PASSWORD environment variable is set to the value of the password key in the db-credentials ConfigMap (password1).

2. Secrets 🔒

Definition: Secrets are similar to ConfigMaps but are designed to store sensitive data, such as passwords, API keys, or certificates. Secrets are base64-encoded by default (though not encrypted unless additional measures are taken).
Use Case: Secrets are used for sensitive configurations, like database credentials, that should not be exposed in plain text.
Example:

  apiVersion: v1
  kind: Secret
  metadata:
    name: db-secret
  type: Opaque
  data:
    password: cGFzc3dvcmQx # base64-encoded "password1"

This Secret (db-secret) stores a base64-encoded password.
- How It’s Used in a Deployment: Similar to ConfigMaps, a deployment can reference a Secret to set environment variables:

  env:
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: db-secret
        key: password

3. The Problem: No Automatic Redeployment on ConfigMap/Secret Updates 🚫

Issue: When a ConfigMap or Secret is updated (e.g., changing password1 to password2), the deployment referencing it does not automatically redeploy. As a result, running pods continue to use the old configuration values.
Why This Happens:
- A Kubernetes deployment’s pod spec references a ConfigMap/Secret by its name and key. If the ConfigMap/Secret’s name and key remain unchanged (only the value changes), the deployment’s pod spec is unaffected.
- Kubernetes does not detect changes to the content of a ConfigMap/Secret as a reason to redeploy pods.
Example Scenario:
1. A ConfigMap (db-credentials) is created with password: password1.
2. A deployment (nginx-deployment) references this ConfigMap to set the DB_PASSWORD environment variable.
3. The ConfigMap is updated to password: password2 using kubectl apply -f configmap.yaml.
4. The deployment remains unchanged because its pod spec still references the same ConfigMap name (db-credentials) and key (password).
5. Running pods continue to use DB_PASSWORD=password1 until a manual kubectl rollout restart deployment nginx-deployment is executed.
Verification:
- Check the ConfigMap: kubectl describe configmap db-credentials shows password: password2.
- Check the pod’s environment variable: kubectl exec <pod-name> -- printenv | grep DB shows DB_PASSWORD=password1.
- After a manual restart (kubectl rollout restart deployment nginx-deployment), the new pod shows DB_PASSWORD=password2.

4. ConfigMap Generators ⚙️

Definition: A ConfigMap Generator is a feature in Kustomize (a Kubernetes configuration management tool) that creates ConfigMaps dynamically and ensures that changes to configuration data trigger redeployments.
How It Works:
- When a ConfigMap Generator is defined, Kustomize creates a ConfigMap with a unique name by appending a random suffix (e.g., db-cred-abc123) to the base name (e.g., db-cred).
- The deployment’s pod spec is updated to reference this unique ConfigMap name.
- When the ConfigMap’s data is updated, Kustomize generates a new ConfigMap with a new random suffix (e.g., db-cred-xyz789) and updates the deployment to reference the new ConfigMap.
- Since the deployment’s pod spec changes (due to the new ConfigMap name), Kubernetes triggers a redeployment automatically.

Example:

Kustomization File (kustomization.yaml):

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: db-cred
  literals:
  - password=password1
resources:
- deployment.yaml

Deployment File (deployment.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        env:
        - name: DB_PASSWORD
          valueFrom:
            configMapKeyRef:
              name: db-cred
              key: password

What Happens:
Kustomize generates a ConfigMap named db-cred-abc123 with password: password1.
The deployment is updated to reference db-cred-abc123.
When the kustomization.yaml is updated to password=password2, Kustomize generates a new ConfigMap (e.g., db-cred-xyz789) and updates the deployment to reference db-cred-xyz789.
The deployment’s pod spec changes, triggering a redeployment, and the new pods use DB_PASSWORD=password2.

Key Benefit: No manual kubectl rollout restart is needed. The redeployment is automatic because the ConfigMap name changes.

5. Secret Generators 🔑

Definition: Secret Generators are similar to ConfigMap Generators but create Secrets instead. They follow the same principles to ensure automatic redeployment when sensitive data changes.
How It Works:
- A Secret Generator creates a Secret with a unique name (e.g., db-secret-abc123).
- The deployment references this Secret.
- When the Secret’s data is updated, a new Secret with a new random suffix is created, and the deployment is updated, triggering a redeployment.
Example:
- Kustomization File:
```
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
secretGenerator:
- name: db-secret
  literals:
  - password=password1
resources:
- deployment.yaml
```
- Deployment File:
```
env:
- name: DB_PASSWORD
  valueFrom:
    secretKeyRef:
      name: db-secret
      key: password
```
- What Happens:
- Kustomize generates a Secret named db-secret-abc123.
- The deployment references db-secret-abc123.
- When the Secret’s data is updated to password=password2, a new Secret (db-secret-xyz789) is created, and the deployment is updated, triggering a redeployment.

6. Providing Files in Generators 📂

ConfigMap Generator with Files:

Instead of literals, a ConfigMap Generator can reference a file whose content becomes the value of a key.
Example:

configMapGenerator:
- name: nginx-config
  files:
  - nginx.conf

File (nginx.conf):

  server {
    listen 80;
    server_name example.com;
  }

Resulting ConfigMap:

  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: nginx-config-abc123
  data:
    nginx.conf: |
      server {
        listen 80;
        server_name example.com;
      }

The file name (nginx.conf) becomes the key, and the file’s content becomes the value.

Secret Generator with Files:

Similarly, a Secret Generator can reference a file, and the content is base64-encoded in the Secret.
Example:

secretGenerator:
- name: nginx-secret
  files:
  - secret.txt

File (secret.txt): my-secret-data
Resulting Secret:

  apiVersion: v1
  kind: Secret
  metadata:
    name: nginx-secret-abc123
  type: Opaque
  data:
    secret.txt: bXktc2VjcmV0LWRhdGE= # base64-encoded "my-secret-data"

7. Handling Stale ConfigMaps/Secrets 🗑️

Problem: Each time a ConfigMap or Secret is updated, a new one is created with a new random suffix, leaving the old ones behind. Over time, this results in many stale ConfigMaps/Secrets that are no longer used.
Example:
- Initial ConfigMap: db-cred-abc123
- After update: db-cred-xyz789 (new), db-cred-abc123 (stale)
- After another update: db-cred-pqr456 (new), db-cred-xyz789 (stale), db-cred-abc123 (stale)
Solution: Pruning Stale Objects:
- Kustomize provides a --prune flag with kubectl apply to delete unused objects.
- To identify objects for pruning, assign a common label to all generated ConfigMaps/Secrets.
- Example:
```
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: db-cred
  literals:
  - password=password1
  options:
    labels:
      appconfig: myconfig
- name: redis-cred
  literals:
  - password=password1
  options:
    labels:
      appconfig: myconfig
resources:
- deployment.yaml
```
- All generated ConfigMaps have the label appconfig: myconfig.
- Apply with Pruning:
```
kubectl apply -k k8s/overlays/prod --prune -l appconfig=myconfig
```
- This command:
  1. Applies the new configuration, creating new ConfigMaps (e.g., db-cred-pqr456, redis-cred-stu789).
  2. Deletes any objects with the label appconfig: myconfig that are no longer referenced (e.g., db-cred-abc123, db-cred-xyz789).
- Result: Only the latest ConfigMaps (db-cred-pqr456, redis-cred-stu789) remain.
Alternative Solutions:
- Use Kubernetes garbage collection mechanisms (e.g., setting an owner reference on ConfigMaps/Secrets so they are deleted when the deployment is deleted).
- Schedule periodic cleanup jobs to remove stale objects based on labels or age.

🧠 Deep Insights

Why Random Suffixes? 🔄
- The random suffix ensures that each ConfigMap/Secret is treated as a new resource. This is critical because Kubernetes only triggers a redeployment when the pod spec changes. By changing the ConfigMap/Secret name, the deployment’s pod spec is modified, forcing a redeployment.
Kustomize’s Role 🛠️:
- Kustomize is a declarative configuration management tool that simplifies generating and managing Kubernetes resources. ConfigMap and Secret Generators are part of Kustomize’s ability to automate configuration updates.
- Kustomize integrates with kubectl (e.g., kubectl apply -k) and is built into Kubernetes since version 1.14.
ConfigMaps vs. Secrets ⚖️:
- Both serve similar purposes but differ in sensitivity and encoding:
  - ConfigMaps: Plain text, non-sensitive data.
  - Secrets: Base64-encoded, sensitive data (though base64 is not encryption, so additional security measures like encryption at rest are needed).
- Generators treat them identically in terms of automation, but Secrets require careful handling due to their sensitive nature.
Pruning Importance 🧹:
- Without pruning, stale ConfigMaps/Secrets accumulate, increasing cluster clutter and potentially exposing old sensitive data (in the case of Secrets).
- Labels provide a clean way to track and manage generated resources.
Real-World Application 🌍:
- In production, ConfigMap/Secret Generators are critical for applications with frequently changing configurations (e.g., database credentials, API keys).
- They reduce operational overhead by eliminating manual redeployment steps and ensure applications always use the latest configuration.

🚶‍♂️ Example Workflow

Let’s walk through a complete example to solidify the concepts:

Initial Setup 🏗️:

Kustomization File (kustomization.yaml):

 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 configMapGenerator:
 - name: db-cred
   literals:
   - password=password1
   options:
     labels:
       appconfig: myconfig
 resources:
 - deployment.yaml

Deployment File (deployment.yaml):

 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: nginx-deployment
 spec:
   replicas: 1
   selector:
     matchLabels:
       app: nginx
   template:
     metadata:
       labels:
         app: nginx
     spec:
       containers:
       - name: nginx
         image: nginx
         env:
         - name: DB_PASSWORD
           valueFrom:
             configMapKeyRef:
               name: db-cred
               key: password

Apply:
```
 kubectl apply -k .
```
Result:
- ConfigMap: db-cred-abc123 with password: password1.
- Deployment references db-cred-abc123.
- Pod has DB_PASSWORD=password1.

Update Configuration 🔧:

Update kustomization.yaml:

 configMapGenerator:
 - name: db-cred
   literals:
   - password=password2
   options:
     labels:
       appconfig: myconfig

Apply with Pruning:

 kubectl apply -k . --prune -l appconfig=myconfig

Result:
- New ConfigMap: db-cred-xyz789 with password: password2.
- Deployment updated to reference db-cred-xyz789.
- Old ConfigMap (db-cred-abc123) deleted due to pruning.
- New pod has DB_PASSWORD=password2.

Verify ✅:

Check ConfigMap:

 kubectl describe configmap db-cred-xyz789

Output: password: password2

Check Pod:

 kubectl exec <new-pod-name> -- printenv | grep DB

Output: DB_PASSWORD=password2

🎯 Summary

ConfigMaps 📄 and Secrets 🔒 store configuration and sensitive data, respectively, but updates to their content do not trigger redeployments.
ConfigMap/Secret Generators ⚙️🔑 (via Kustomize) solve this by:
- Creating uniquely named ConfigMaps/Secrets with random suffixes.
- Updating deployments to reference the new names, triggering automatic redeployments.
- Supporting literals or files for flexible configuration.
Pruning 🗑️ is essential to clean up stale ConfigMaps/Secrets using labels and the --prune flag.
This approach automates configuration updates, reduces manual intervention, and ensures applications use the latest configurations in Kubernetes.

omkar shelke @omkara18

🚀 Automating Kubernetes ConfigMaps and Secrets with Kustomization Generators 🛠️