What is an API Gateway? A practical guide for developers

An API gateway is a single server that sits between clients and a collection of back-end services, accepting every API request, routing it to the correct service, and returning the response. It exists because, as an application grows from one monolith into many small services, clients need a single, stable address to talk to — and the operator needs a single place to enforce authentication, rate limits, and observability.

Why you need an API gateway

In a single-service application, the client connects directly to the one server. When you break that application into multiple services (a pattern often called microservices), the client would otherwise have to know the address of every service, handle authentication separately for each, retry failed calls, and parse different response shapes. That quickly becomes unworkable.

The API gateway solves several problems at once:

The trade-off is that the gateway itself becomes a critical component. If it goes down, requests stop reaching the back end. This is why most production deployments run the gateway as a redundant cluster (multiple instances behind a load balancer) and keep its logic thin.

Core responsibilities

Not every gateway does every job, but most handle some combination of these.

Routing. The gateway maps each incoming request to a specific back-end service based on the path, host, or header. A request to https://api.example.com/users/42 might go to the users service, while https://api.example.com/orders/77 goes to the orders service. Clients see a single URL; routing happens server-side.

Authentication and authorization. Instead of every service verifying API keys or JWT tokens, the gateway validates credentials once. If the token is valid, the gateway forwards the request (often with headers like X-User-Id added) so the back end knows who is calling. The OpenID and OAuth 2.0 standards describe the delegation patterns most gateways implement.

Rate limiting and throttling. To protect back-end services from overload, the gateway counts requests per client (by API key, IP, or user ID) and rejects or delays excess requests. This prevents a single noisy client from exhausting shared resources.

Request and response transformation. The gateway can rewrite paths, add or strip headers, convert JSON to XML, translate a REST-style request into a gRPC call, or aggregate multiple back-end calls into a single client response. Aggregation is especially common in mobile apps, where the device would otherwise make dozens of round trips.

Caching. Some responses are identical for many clients (a public product catalog, a static configuration blob). The gateway can cache these and serve them directly, offloading the back end entirely for the cache-hit case.

Observability. Because every request passes through the gateway, it is the natural place to collect metrics (latency, error rates, request volume), emit structured logs, and report on traffic patterns. Most gateways expose a Prometheus-compatible metrics endpoint.

Transport security. The gateway terminates TLS, centralizes certificate management, and (in some setups) requires mutual TLS (mTLS) so that calling services must present a client certificate in addition to a token.

The request lifecycle

A typical request through an API gateway follows this path:

1. The client sends GET https://api.example.com/users/42 with an Authorization: Bearer <token> header.
2. The gateway terminates TLS and parses the request.
3. The gateway validates the token. If invalid, it returns 401 Unauthorized immediately — the back end never sees the request.
4. The gateway applies rate-limit counters for the calling client. If over the quota, it returns 429 Too Many Requests.
5. The gateway looks up the route (in this case: /users/* → users-service) and forwards the request to a healthy back-end instance.
6. The back-end service processes the request and returns a response.
7. The gateway records latency, increments counters, and returns the response to the client.

For a well-tuned gateway, steps 2–5 take a few milliseconds. The back-end call dominates the total time.

Common patterns

Single gateway, one team

The simplest pattern: one gateway routes everything to many services. Most teams start here. It works well until traffic exceeds what one gateway instance can handle.

Gateway per region

For globally distributed users, you may run a gateway in every region (for example, one in Frankfurt, one in Singapore, one in Virginia). Each region routes to back-end services in or near its region, reducing latency.

Gateway per Bounded Context

In larger organizations, a single shared gateway can become a deployment bottleneck (every team needs every other team's changes to ship). Splitting into one gateway per bounded context (e.g., a "Payments Gateway" and a "Catalog Gateway") trades operational complexity for team independence.

BFF (Back-end for Front-end)

When a web app, iOS app, and Android app need different response shapes, you can run a separate gateway for each. Each BFF aggregates and reshapes back-end calls to fit its client's needs.

Edge gateway

When the gateway lives in a CDN edge network (Cloudflare, Fastly, AWS CloudFront), the back ends are protected by a layer that is geographically close to users and absorbs attacks before they reach your infrastructure. Cloudflare API Gateway, for example, provides JWT validation, schema validation, and OWASP Top 10 protections at the edge.

API gateway vs reverse proxy vs load balancer

These terms overlap and the boundaries are fuzzy. A useful summary:

In practice, the same software (Nginx, Envoy, HAProxy, Kong) often plays all three roles at once. The distinction is in the configuration and the features enabled, not necessarily the binary.

Popular implementations

When you probably do not need one

A gateway adds a network hop, a new component to operate, and a new failure mode. For a single-service application, this is unnecessary. You only need a gateway once you have multiple services, multiple clients with different needs, or a need for centralized rate limiting / auth that you would otherwise repeat in every service.

A reasonable rule of thumb: start with a reverse proxy (Nginx, Caddy) and add gateway features (auth, rate limiting) only when you actually need them. Reach for a full API gateway when you have at least three back-end services or when you must enforce policies consistently across teams.

What to look for when choosing a gateway

1. Performance overhead. A gateway that adds 50 ms to every request is a problem; one that adds 1–2 ms is fine. Benchmark before committing.
2. Plugin ecosystem. Kong and APISIX have rich plugin libraries; Nginx requires you to wire most features yourself.
3. Dynamic configuration. Can you change routes and policies without reloading? For high-traffic services, a config reload that drops connections is unacceptable.
4. Observability. Does it emit Prometheus metrics, OpenTelemetry traces, and structured logs out of the box?
5. Authentication options. JWT, OAuth 2.0, mTLS, API keys, custom — what does it support natively?
6. Where it runs. In your cluster, at the edge, or as a managed service? Each has trade-offs in latency, control, and operational burden.

FAQ

Is an API gateway the same as a service mesh?

No. An API gateway handles north-south traffic — requests from outside your system (clients, mobile apps, partners) to your services. A service mesh (Istio, Linkerd) handles east-west traffic — requests between your services. They solve different problems but the underlying proxies (Envoy, Nginx) are often the same.

Does an API gateway replace my load balancer?

Usually not. A gateway can include load-balancing features, but for raw TCP/UDP traffic or for distributing across many instances of a single service, a dedicated load balancer is still common. Many architectures use both: a load balancer in front of the gateway, then the gateway in front of the services.

Can an API gateway add latency?

Yes — every request now goes through an extra hop. A well-tuned gateway adds 1–5 ms. A poorly configured one (deep plugin chains, synchronous external auth calls, no connection pooling) can add tens to hundreds of milliseconds. Measure under load, not just at idle.

Should the API gateway also handle caching?

For some responses, yes. Caching public or near-static data at the gateway is a common win, especially for mobile clients. But for highly dynamic or per-user data, caching at the gateway can introduce correctness bugs (stale data) and is usually better handled by the back-end service or a dedicated cache (Redis, Varnish).

What happens if the API gateway goes down?

If the gateway is a single instance, the entire API goes down with it. Production deployments run multiple gateway instances behind a load balancer, with health checks so a failed instance is removed automatically. The gateway should be stateless (no local session data) so any instance can handle any request.

Verdict

An API gateway is a useful piece of architecture once you have multiple back-end services and want consistent cross-cutting policies. Start simple: a reverse proxy with rate limiting and a single auth check is enough for many teams. Reach for a full gateway — Kong, Envoy, Cloudflare API Gateway, or a managed service — once you have a clear pain point that it solves: too many duplicated auth checks, inconsistent rate limits, the need for request aggregation, or a need to enforce schema validation centrally. A gateway is a tool, not a goal; the goal is to keep your services focused on business logic and your clients talking to one stable, secure endpoint.