How to monitor a Node.js application with Prometheus and Grafana

A production Node.js service is not "done" when it boots — it is done when you can see what is happening inside it. Prometheus is the open-source standard for collecting numeric time-series metrics from instrumented applications, and Grafana is the open-source standard for turning those metrics into dashboards and alerts. Combined, they form the backbone of modern observability for Node.js services. This guide builds the full stack from scratch: instrumenting an Express API with the official prom-client library, running Prometheus and Grafana in Docker, scraping the application, and creating dashboards with PromQL.

What Prometheus and Grafana do

Prometheus is a systems monitoring and alerting toolkit originally built at SoundCloud in 2012 and now a graduated Cloud Native Computing Foundation project. It collects metrics as time-series data, where each data point is a numeric value recorded at a specific timestamp and tagged with key-value pairs called labels. Prometheus uses a pull model: the server periodically scrapes an HTTP endpoint that your application exposes, instead of receiving pushes from your code.

Grafana is an open-source analytics and visualization platform. It connects to Prometheus (and dozens of other data sources), lets you write queries in PromQL (Prometheus's query language), and renders the results as graphs, gauges, tables, and alerts on dashboards. In short: Prometheus stores the numbers, Grafana shows them to humans.

Both are battle-tested at scale: Prometheus scrapes millions of time series per server, and Grafana dashboards sit behind on-call screens at companies of every size. For a Node.js service, the typical setup looks like this:

Node.js app --(/metrics)--> Prometheus --(PromQL)--> Grafana
                                  |
                                  v
                              Alertmanager -> Slack / Email / Pager

Instrumenting the application

The most widely used Node.js client is siimon/prom-client, an Apache-2.0 library that supports counters, gauges, histograms, and summaries. Install it next to Express:

npm install prom-client express

Create a metrics module that registers default Node.js metrics (event-loop lag, memory, GC, active handles) and exposes a few custom application metrics:

// src/metrics.js
const client = require("prom-client");

const register = new client.Registry();
client.collectDefaultMetrics({ register });

// Custom: total HTTP requests, labelled by method/route/status
const httpRequestsTotal = new client.Counter({
  name: "http_requests_total",
  help: "Total HTTP requests",
  labelNames: ["method", "route", "status"],
  registers: [register],
});

// Custom: request latency histogram
const httpRequestDurationSeconds = new client.Histogram({
  name: "http_request_duration_seconds",
  help: "HTTP request duration in seconds",
  labelNames: ["method", "route", "status"],
  buckets: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10],
  registers: [register],
});

module.exports = {
  register,
  httpRequestsTotal,
  httpRequestDurationSeconds,
};

Wire the metrics into an Express app. The key principle: a middleware measures every request, then exposes /metrics as a plain HTTP endpoint that Prometheus can scrape:

// src/app.js
const express = require("express");
const {
  register,
  httpRequestsTotal,
  httpRequestDurationSeconds,
} = require("./metrics");

const app = express();

app.use((req, res, next) => {
  const end = httpRequestDurationSeconds.startTimer();
  res.on("finish", () => {
    // Use req.route?.path to avoid label cardinality explosion
    const route = req.route?.path || req.path;
    const labels = {
      method: req.method,
      route,
      status: String(res.statusCode),
    };
    end(labels);
    httpRequestsTotal.inc(labels);
  });
  next();
});

app.get("/", (_req, res) => res.json({ ok: true }));
app.get("/users/:id", (req, res) => res.json({ id: req.params.id }));

// Scrape endpoint — Prometheus polls this URL
app.get("/metrics", async (_req, res) => {
  res.set("Content-Type", register.contentType);
  res.end(await register.metrics());
});

app.listen(3000, () => console.log("listening on 3000"));

Two important guardrails from the Prometheus best-practices guide. First, never use a raw URL path like /users/42 as a label value — every user ID becomes a new time series and the database grows forever. Use req.route.path (the pattern /users/:id) instead. Second, register collectDefaultMetrics once at startup so event-loop lag, memory, GC, and Node.js version are recorded automatically.

You can verify the endpoint locally before going further:

$ curl -s http://localhost:3000/metrics | head
# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total{method="GET",route="/",status="200"} 3
# HELP process_cpu_user_seconds_total Total user CPU time spent in seconds
# TYPE process_cpu_user_seconds_total counter
process_cpu_user_seconds_total 0.42

Running Prometheus and Grafana with Docker Compose

The simplest way to run the rest of the stack is a single Compose file. Create monitoring/compose.yaml:

services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prom-data:/prometheus
    ports:
      - "9090:9090"
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    volumes:
      - grafana-data:/var/lib/grafana
    ports:
      - "3001:3000"
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=admin
    depends_on:
      - prometheus
    restart: unless-stopped

volumes:
  prom-data:
  grafana-data:

The Prometheus configuration tells the server what to scrape and how often. Create monitoring/prometheus.yml:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "nodejs-app"
    metrics_path: /metrics
    static_configs:
      - targets: ["host.docker.internal:3000"]
        labels:
          service: "api"

host.docker.internal is the special hostname Docker Desktop (and Docker on Linux 20.10+) provides so a container can reach the host machine. On plain Linux without that, replace it with the host's LAN IP. Start the stack:

$ docker compose up -d

Then point Prometheus at your application: if you are running the Express app on the host on port 3000, Prometheus will scrape it every 15 seconds. Verify on the Prometheus UI at http://localhost:9090/targets — the nodejs-app job should appear in green "UP" state.

Adding the Prometheus data source in Grafana

Grafana listens on http://localhost:3001 (mapped from container port 3000). The first login uses admin / admin and asks you to set a new password. To add Prometheus as a data source:

1. Click the cogwheel in the sidebar and choose Data Sources.
2. Click Add data source and select Prometheus.
3. Set URL to http://prometheus:9090 (the Compose service name resolves inside the default network).
4. Click Save & Test — Grafana will tell you the data source is working.

The Prometheus data source has shipped with Grafana since 2.5.0 in 2015, so no plugin install is needed.

Building your first dashboard

Grafana dashboards are made of panels. Each panel runs a PromQL query against the data source and renders a visual. The metrics you registered earlier let you build the four panels every API dashboard needs.

1. Request rate (per-second, by status code). Counts how many requests are landing on your service and groups them by HTTP status. PromQL uses rate to convert a counter into requests-per-second:

sum by (status) (rate(http_requests_total[1m]))

2. Error rate (5xx fraction). What fraction of responses are server errors? A rising line here wakes someone up:

sum(rate(http_requests_total{status=~"5.."}[5m]))
  /
sum(rate(http_requests_total[5m]))

3. Latency percentiles. A histogram is observed into buckets; histogram_quantile reconstructs a percentile across the buckets. The 95th and 99th percentiles tell you what most users and the slowest users actually experience:

histogram_quantile(0.95,
  sum by (le) (rate(http_request_duration_seconds_bucket[5m]))
)

4. Event-loop lag. Default Node.js metrics include nodejs_eventloop_lag_seconds. A long event loop is the most common reason a Node.js service "feels slow" even when CPU is low:

nodejs_eventloop_lag_seconds

Add a graph panel for each query, choose the Prometheus data source, and lay the four panels out in a 2×2 grid. The result is a real-time picture of your service: traffic, errors, latency, and runtime health.

Alerting with Prometheus

Dashboards show what happened. Alerts tell you something needs attention right now. Prometheus evaluates alert rules — PromQL expressions with thresholds — on the same evaluation_interval (15s by default). Create monitoring/alerts.yml:

groups:
  - name: api
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m]))
            /
          sum(rate(http_requests_total[5m]))
            > 0.05
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Error rate above 5% for 2m"
          description: "API is returning errors on more than 5% of requests."

      - alert: HighLatencyP95
        expr: |
          histogram_quantile(0.95,
            sum by (le) (rate(http_request_duration_seconds_bucket[5m]))
          ) > 1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "p95 latency over 1s"

The for: 2m clause requires the condition to hold for two minutes before the alert fires — without it, a single bad request would page someone. Add rule_files: ["alerts.yml"] to prometheus.yml and reload Prometheus with curl -X POST http://localhost:9090/-/reload. Alerts in firing state appear at http://localhost:9090/alerts. To turn those alerts into Slack messages or pages, point Prometheus at an Alertmanager instance with the appropriate webhook configuration.

Production hardening

The setup above is enough to learn the stack. For a real deployment, three things matter most.

Retention. Prometheus stores data on local disk. The default retention is 15 days. For longer history, use a remote-write adapter to ship data to Grafana Cloud, Thanos, or a managed service. Prometheus is designed for reliability during incidents, not for cheap long-term storage.

High-cardinality labels. A label like user_id or full URL path creates a new time series for every value and is the single most common Prometheus outage. Always prefer coarse labels (route, status, method) over raw request data. The route from req.route.path is the safe choice.

Security. Lock down /metrics behind an internal network, a reverse proxy with basic auth, or a service mesh. Do not expose it to the public internet — it leaks the shape of your traffic and can be abused for fingerprinting. Prometheus's own ports should also be inside a private network.

FAQ

Why use Prometheus instead of a SaaS like Datadog?

Prometheus is open source, vendor-independent, and free. It runs as a single static binary with no external dependencies, so you keep working during the kind of partial outages where hosted services sometimes degrade. The trade-off is that you operate the storage and dashboards yourself. For small teams that prefer managed, the same metrics flow into Grafana Cloud, Datadog, or New Relic with minimal change.

Do I need Grafana? Can I just use Prometheus's built-in UI?

Prometheus ships with a basic expression browser at /graph for ad-hoc queries. For ongoing work — dashboards, alerting visualizations, sharing with non-engineers — Grafana is the standard. You can also import hundreds of community dashboards from grafana.com/dashboards to skip writing panels by hand.

What is the difference between a counter, gauge, and histogram?

A counter is a value that only goes up (total requests, total errors). A gauge is a value that can go up or down (current memory use, active connections). A histogram samples observations into buckets so you can compute percentiles and averages later (request duration, response size). For request rate, use a counter; for "how much memory is in use right now", use a gauge; for "what was the 95th-percentile latency over the last 5 minutes", use a histogram.

How do I avoid running out of disk on Prometheus?

The two levers are the --storage.tsdb.retention.time flag (default 15 days) and the --storage.tsdb.retention.size flag. For long-term storage, the recommended pattern is Prometheus locally for fast recent queries plus a remote-write target (Thanos, Mimir, Grafana Cloud) for the historical archive.

Can I monitor more than just Node.js with this stack?

Yes. Prometheus is language-agnostic — any service that exposes a /metrics endpoint in the Prometheus format can be scraped. For databases, queues, and operating systems, use exporters: node_exporter for Linux host metrics, redis_exporter for Redis, postgres_exporter for PostgreSQL. The Grafana dashboard stays the same; you just add more jobs to prometheus.yml.