Migrating from Ingress to Gateway API: Lessons from the Trenches

Recently, I carried out a migration from Kubernetes Ingress to Gateway API for one of our production workloads. This post is a distilled summary of the lessons learned, the approach taken, and key insights you might find helpful if you’re considering or planning a similar transition.

The Kubernetes Gateway API is not just an incremental improvement over Ingress. It’s a rethinking of how routing and traffic control should work in modern, scalable Kubernetes environments. It introduces a more expressive and extensible model, enabling:

Separation of concerns: Decouple infrastructure and application layer routing more cleanly.
Rich routing capabilities: HTTP routing, header matching, path rewriting, traffic splitting, and more.
Better role-based access control: Via resource boundaries between Gateway, GatewayClass, and HTTPRoute.
Future-ready extensibility: Designed to support gRPC, mesh-style routing, and more advanced features.

Why We Decided to Migrate?

While Ingress has served us well, we started facing challenges:

Tight coupling of routing rules and infrastructure.
Difficulty in managing multi-tenant workloads.
Need for more granular routing policies.
Preparing our infrastructure for future scalability and extensibility.

The Kubernetes community is also gradually pushing Gateway API as the eventual successor to Ingress, making this a good time to get ahead.

Planning the Migration: What to Consider
Before jumping into resource definitions, it's crucial to answer the following:

Are your current ingress controllers supported by Gateway API?
Will you replace Ingress or run both during transition?
Are there any custom annotations, backendConfigs, or load balancer features your current setup depends on?
How is your DNS managed?
What certificate management approach are you using pre-shared certs, cert-manager, or cloud provider-managed certs?

We decided on a one-time conversion for our primary ingress resources, and then gradually onboarded other services to Gateway API.

Tools That Helped
For bootstrapping new Gateway API resources from existing Ingress configurations, can use:

migrating from-ingress guide is really a good place to start with.

🔧 ingress2gateway a CLI tool that helps convert Ingress resources into Gateway API format.

⚠️ Note: It currently only supports the ingress-nginx provider, so it’s not a universal solution but very handy as a starting point.

🔧 Architecture Comparison
Before (Ingress-Based)

[External LB] --> [Ingress Controller] --> [Service] --> [Pods]
           |                            |
        [DNS A Record]             [BackendConfig]

After (Gateway API-Based)

[External LB] --> [Gateway] --> [HTTPRoute] --> [Service] --> [Pods]
           |                                 |
        [DNS A Record]             [Certificate Mgmt]

Real-World Example: Migrating an External Ingress
Original Setup

Ingress: Exposed via external load balancer
DNS: A record pointing to the Ingress IP
BackendConfig: Defined for Cloud Armor or timeouts
Service: Type NodePort or ClusterIP
Deployment/Pod: Backend app running on port 8080

Migration Process
Create a GatewayClass:


apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: external-gateway
spec:
  controllerName: k8s.io/gateway-nginx

2. Define a Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
  namespace: default
spec:
  gatewayClassName: external-gateway
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    hostname: "example.com"

3. Replace Ingress with HTTPRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: my-app-route
  namespace: default
spec:
  parentRefs:
  - name: my-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: "/"
    backendRefs:
    - name: my-service
      port: 8080

DNS Consideration: If your Gateway exposes a new load balancer IP, update the A record in DNS to point to the new IP.
Certificates: You can now move toward managed certificates using cert-manager or your cloud provider-managed certs via Gateway annotations or controller-specific config.
Can as well create a BackendConfig, Gateway policy for SSL, healthcheckpolicy etc as per required configuration.

Challenges Faced

Some annotations from Ingress didn’t map 1:1 (e.g., backendConfig.cloud.google.com) — needed equivalent Gateway API controller support.
DNS propagation during switch can cause short-term disruption if not carefully planned. Consider updating ttl accordingly.
ingress2gateway tool required post-editing of YAML to match specific load balancer or controller nuances.

Conclusion
Migrating from Ingress to Gateway API is more than just replacing YAML manifests. It’s about rethinking how your applications are exposed, routed, and secured. While it requires upfront effort, the long-term benefits in maintainability and scalability are worth it.

If you're working with a supported provider like ingress-nginx, tooling like ingress2gateway can help kickstart your journey.

🔑 Key Takeaways
✅ Gateway API enables better separation of concerns and more robust routing policies.

🔍 Analyze current Ingress dependencies (certs, annotations, configs) before migrating.

🛠 Use ingress2gateway as a scaffold, not a final product.

🌐 Don’t forget DNS! Plan your switch carefully.

📈 Gateway API is future-forward, migrating early positions your infrastructure for what's coming next.

📬 Got questions or want to share your migration experience? Drop a comment or reach out. Let’s grow together!

upendragurugubelli @mrupguru