What a SQL query does

A query reads from one or more tables and returns a result set of rows and columns. It does not change the data; that is the job of INSERT, UPDATE, and DELETE. A query can be nested inside another query (a subquery) and can reference a previous query in the same FROM clause (a derived table or inline view). Since SQL:1999 you can also name a subquery with a WITH clause — a common table expression (CTE) — and CTEs can reference themselves recursively, which is how SQL implements tree and graph traversals.

Three properties distinguish a query from a procedure:

• Declarative: you describe the desired result, not the steps to produce it. The database's query optimizer chooses a plan.
• Set-based: the result is a set (or multi-set) of rows, not a stream of records to be processed one at a time. The same row may appear more than once unless you ask for DISTINCT.
• Closed under composition: the output of a query is itself a table, so you can pipe one query into another, nest them, or join the result of one query against a real table.

The SELECT statement

The SELECT statement has several optional clauses. The full syntax (in the order they are evaluated) is:

SELECT [DISTINCT] select_list
FROM   table_source
[WHERE predicate]
[GROUP BY columns]
[HAVING group_predicate]
[ORDER BY columns [ASC | DESC]]
[LIMIT n | FETCH FIRST n ROWS ONLY];

Each clause has a distinct job:

• SELECT — the columns or expressions to return. SELECT * returns every column. SELECT count(*) is an aggregate expression, not a column reference.
• FROM — the table or tables the data comes from. With more than one table and no join condition, you get a Cartesian product.
• WHERE — filters rows before grouping. The predicate is evaluated for each row, and only rows where it is TRUE survive.
• GROUP BY — collapses rows that share the same values in the listed columns into a single group. Combined with aggregate functions, this is how you compute sums, counts, and averages per category.
• HAVING — filters groups after GROUP BY. Aggregate functions are allowed here but not in WHERE.
• ORDER BY — sorts the result. Without it, row order is undefined.
• LIMIT / FETCH FIRST — caps the number of rows returned. LIMIT is non-standard (MySQL, PostgreSQL, SQLite); FETCH FIRST n ROWS ONLY is the SQL:2008 standard, supported by DB2, PostgreSQL, SQL Server 2012+, Oracle 12c, and H2.

A simple example, adapted from the Wikipedia SELECT page:

SELECT title, price, price * 0.06 AS sales_tax
FROM   Book
WHERE  price > 100.00
ORDER BY title;

The expression price * 0.06 AS sales_tax projects a computed column — a value the database evaluates on the fly. The alias sales_tax gives it a name in the result set.

Filtering with WHERE

WHERE is applied row by row before any grouping. The predicate uses three-valued logic: a comparison can be TRUE, FALSE, or UNKNOWN (when either side is NULL). Rows that are not TRUE are dropped, which means NULL values silently disappear from comparisons. To find rows where a column is or is not null, you use IS NULL and IS NOT NULL — = NULL is never true.

A filter can combine conditions with AND, OR, and NOT, and it can use pattern matching (LIKE), set membership (IN), and range checks (BETWEEN). All of these are evaluated by the same engine and typically use an index when one is available; the role of a database index is to let the engine find matching rows without scanning the entire table.

Joining tables

A JOIN clause combines columns from two or more tables into a single result set. It corresponds to a join operation in relational algebra: the engine matches rows from each table according to a predicate, then stitches the matched columns together. The standard join types are:

Using the canonical example from the Wikipedia JOIN page, with employees and departments:

SELECT e.LastName, e.DepartmentID, d.DepartmentName
FROM   employee e
INNER JOIN department d ON e.DepartmentID = d.DepartmentID;

This returns every employee with their department name. Williams, who has a NULL DepartmentID, is excluded because NULL never matches anything (not even NULL itself). The USING shorthand is available when the join columns share a name and you do not need to qualify them:

SELECT * FROM employee INNER JOIN department USING (DepartmentID);

USING is not supported by SQL Server or Sybase. NATURAL JOIN is a stricter version that auto-matches every column with the same name — concise but fragile, because adding a column to either table silently changes the join.

A self-join joins a table to itself, which is the standard way to model hierarchies (employees and their managers, threads and their parent posts) in a flat schema.

Aggregating with GROUP BY

GROUP BY collapses rows that share the same value in the listed columns, then lets you apply aggregate functions — count(*), sum(amount), avg(price), min(date), max(score) — to each group. The classic example, again from the SELECT page, counts authors per book:

SELECT b.title, count(*) AS Authors
FROM   Book b
JOIN   Book_author ba ON b.isbn = ba.isbn
GROUP BY b.title;

The order of evaluation matters: WHERE runs first (row filter), then GROUP BY (collapse), then aggregate functions compute one value per group, then HAVING (group filter), then SELECT (project), then ORDER BY, then LIMIT.

To filter on the result of an aggregate, use HAVING, not WHERE. WHERE count(*) > 5 is a syntax error in standard SQL because the aggregate does not exist yet when WHERE is evaluated. The correct pattern:

SELECT g.GroupName, count(*) AS NumberOfMembers
FROM   users u
INNER JOIN groups g ON g.Userid = u.Userid
GROUP BY GroupName
HAVING count(*) > 5;

The ISO standard evaluation order for a SELECT is, in sequence: evaluate FROM (build a virtual table from the Cartesian product and the ON clause), apply WHERE, apply GROUP BY, apply HAVING, evaluate the SELECT list, apply DISTINCT, then apply ORDER BY. Modern optimizers may reorder operations internally, but the result must match this sequence.

Subqueries and common table expressions

A subquery is a query nested inside another. The most common placement is in the WHERE clause, where it can feed a value to a comparison:

SELECT isbn, title, price
FROM   Book
WHERE  price < (SELECT AVG(price) FROM Book)
ORDER BY title;

A correlated subquery references a column from the outer query and is re-evaluated for each outer row. Correlated subqueries are expressive but often slow; a JOIN with the same intent is usually faster.

A common table expression (CTE) names a subquery so it can be referenced more than once, which is essential for recursive queries:

WITH RECURSIVE subordinates AS (
  SELECT id, name, manager_id FROM employees WHERE id = 1
  UNION ALL
  SELECT e.id, e.name, e.manager_id
  FROM   employees e
  INNER JOIN subordinates s ON e.manager_id = s.id
)
SELECT * FROM subordinates;

This walks the employee hierarchy starting from id 1 and is a common pattern for organizational trees, threaded comments, and graph data stored in tables.

How a query is executed

When you submit a query, the database parses the SQL, validates it against the schema, and hands the result to a query optimizer. The optimizer's job is to pick a query plan — the actual sequence of operations the engine will run — that produces the correct result as cheaply as possible. The optimizer has two main degrees of freedom:

• Join order. Because joins are commutative and associative, the mathematical result is the same no matter which order the engine joins tables. The cost, however, can differ by orders of magnitude. A bad plan on a 1M-row table joined to a 1M-row table can take hours; the right plan finishes in milliseconds. Optimizers estimate intermediate row counts using table and index statistics, then choose the cheapest plan.
• Join algorithm. Three fundamental algorithms exist: nested loop join (good when one side is small or has an index on the join key), sort-merge join (good when both sides are already sorted on the join key), and hash join (good for large unsorted inputs that fit in memory). Modern databases may also use bitmap join indexes (Teradata, Oracle) for low-cardinality columns.

The actual operations a database performs — sequential scans, index lookups, sorts, hash aggregations, parallel workers — are recorded in an EXPLAIN or EXPLAIN ANALYZE plan, which is the first thing to look at when a query is slow.

Common SQL pitfalls

A few mistakes that catch almost everyone:

• Forgetting that NULL never matches. WHERE DepartmentID = NULL returns zero rows; you need WHERE DepartmentID IS NULL.
• Confusing WHERE and HAVING. WHERE runs before grouping and cannot reference aggregates; HAVING runs after grouping and almost always does.
• Implicit cross joins. Listing several tables in FROM separated by commas without a WHERE clause produces a Cartesian product, which is rarely what you want and can be catastrophically expensive.
• Non-deterministic order without ORDER BY. Without an explicit sort, the database is free to return rows in any order, and that order can change between runs.
• Vendor-specific syntax. LIMIT 10 (MySQL/PostgreSQL/SQLite), FETCH FIRST 10 ROWS ONLY (SQL:2008, DB2/PostgreSQL/SQL Server 2012+/Oracle 12c), TOP 10 (SQL Server, Access), and WHERE ROWNUM <= 10 (Oracle) all do the same thing, but the syntax differs. Writing portable SQL means preferring FETCH FIRST.
• Subtle inner-join NULLs. As the JOIN page notes, inner joins on columns that may contain NULL silently drop rows. For nullable foreign-key columns, a left outer join is usually safer.

FAQ

What is a SQL query?

A SQL query is a declarative statement sent to a relational database that asks for a result set of rows and columns, typically written with the SELECT keyword. The statement describes what data you want, not how to retrieve it; the database engine chooses the execution plan. Queries are read-only and have no persistent effect on the data.

What is the difference between WHERE and HAVING?

WHERE filters individual rows before grouping and cannot reference aggregate functions. HAVING filters groups produced by GROUP BY and is the only place aggregates can appear in a filter clause. Use WHERE for column conditions like price > 100, and HAVING for conditions on the result of an aggregate like count(*) > 5.

What is the difference between INNER JOIN and LEFT JOIN?

INNER JOIN returns only rows where the join condition matches in both tables. LEFT OUTER JOIN returns all rows from the left table, with the right table's columns populated when a match exists and NULL when it does not. The rule of thumb: if you might be missing matching rows on the right side but still want every left-side row, use LEFT JOIN. If unmatched rows are bugs, use INNER JOIN.

How does GROUP BY work?

GROUP BY collapses rows that share the same value in the listed columns into a single group, then lets aggregate functions like count(*), sum(amount), and avg(price) compute one value per group. The collapsed rows are no longer individually addressable in the result; only the grouping columns and the aggregate values appear. Filter on the aggregate with HAVING, not WHERE.

What is a relational database?

A relational database is a database whose logical schema is based on the relational model introduced by Edgar F. Codd in his 1970 paper "A Relational Model of Data for Large Shared Data Banks". In this model:

• A table (relation) is a set of rows that share the same structure.
• A row (tuple) is one record, a single instance of what the table describes.
• A column (attribute) is a named field with a defined data type (integer, text, date, etc.).

Every row in a relation must be unique, and uniqueness is enforced by a primary key — one or more columns whose values never repeat inside the table. References to rows in other tables are stored as foreign keys, and referential integrity rules keep those references valid.

The language used to define, query, and manipulate these tables is SQL (Structured Query Language). SQL is declarative: a query states what data to retrieve, and the database engine decides how to retrieve it. The key distinction from earlier file systems and from many NoSQL stores is that the relational model enforces a structured schema before data is inserted. Types, constraints, and keys are declared up front, and the database rejects writes that violate them.

How tables and relationships work

Three kinds of relationships can exist between tables:

• One-to-one — each row in table A corresponds to at most one row in table B (for example, a users table and a users_private table that holds sensitive data split off for access control).
• One-to-many — each row in table A can relate to many rows in table B, but each row in B relates to only one row in A. The classic example is a customers table and an orders table: one customer can place many orders, but each order belongs to one customer.
• Many-to-many — rows in A can relate to many rows in B and vice versa. Students enroll in many courses; each course has many students. Many-to-many relationships are implemented with a third junction table whose rows pair primary keys from both sides.

The mechanics look like this:

CREATE TABLE customers (
  id    INTEGER PRIMARY KEY,
  name  TEXT NOT NULL,
  email TEXT UNIQUE
);

CREATE TABLE orders (
  id         INTEGER PRIMARY KEY,
  customer_id INTEGER REFERENCES customers(id),
  amount     NUMERIC(10,2) NOT NULL,
  created_at TIMESTAMP NOT NULL
);

-- Insert a customer
INSERT INTO customers (id, name, email) VALUES (1, 'Alice', 'alice@example.com');

-- Insert an order that references that customer
INSERT INTO orders (id, customer_id, amount, created_at)
VALUES (100, 1, 49.99, '2026-09-04 10:00:00');

-- Retrieve each order with the customer's name
SELECT o.id, c.name, o.amount, o.created_at
FROM   orders o
JOIN   customers c ON c.id = o.customer_id
WHERE  c.email = 'alice@example.com';

The REFERENCES clause declares the foreign key; the database engine refuses to insert an order whose customer_id does not already exist in customers. The JOIN clause combines rows from both tables based on the key match. Tools such as dbt build layered SQL models on top of an RDBMS like postgres, and cloud services like supabase expose a managed postgres instance with auth and APIs layered on top.

Normalization: reducing redundancy

Normalization is the process of restructuring a relational schema to minimize redundancy and avoid certain update anomalies. Edgar F. Codd defined a series of normal forms, each one stricter than the last:

• First Normal Form (1NF) — every column holds an atomic (indivisible) value, there are no repeating groups, and each row is unique. A column that stores a comma-separated list of phone numbers violates 1NF; the fix is a separate phone_numbers table.
• Second Normal Form (2NF) — the table is in 1NF and every non-key column depends on the whole primary key, not just part of it. This matters only for tables with composite primary keys. A (student_id, course_id) → instructor dependency violates 2NF if instructor depends only on course_id.
• Third Normal Form (3NF) — the table is in 2NF and has no transitive dependencies: non-key columns do not depend on other non-key columns. If course_id → department → department_head, then department_head should move to a departments table.
• Boyce–Codd Normal Form (BCNF) — a stricter version of 3NF that handles the rare cases 3NF misses, where a non-key attribute determines part of a candidate key.

The practical trade-off is between write-time integrity and read-time performance. A fully normalized schema removes duplication, so updating a value (a customer's address, say) happens in exactly one place. But heavy normalization means more joins at query time, which can be slow on large datasets. Many production systems therefore denormalize selectively — for example, storing a cached total_orders count on a customer row to avoid a COUNT(*) join on every page load — while keeping the rest of the schema normalized. Analytics warehouses built on platforms like snowflake typically denormalize further, using wide star or snowflake schemas for query speed.

ACID transactions

A relational database is almost always paired with ACID transactions: the atomicity, consistency, isolation, and durability guarantees covered in detail in our transactions guide. In practice this means:

• A set of related writes either all happen or none of them do.
• The database never leaves constraints in a broken state.
• Concurrent transactions don't see each other's uncommitted changes.
• Committed changes survive crashes and power loss.

Consider a bank transfer:

BEGIN;
  UPDATE accounts SET balance = balance - 100 WHERE id = 1;   -- debit
  UPDATE accounts SET balance = balance + 100 WHERE id = 2;   -- credit
COMMIT;

If the second UPDATE fails (network error, constraint violation, system crash), the BEGIN/COMMIT pair is not completed and the database engine rolls the whole transaction back. Account 1 is not debited. This is the property that makes relational databases the default choice for finance, e-commerce inventory, and any system where partial writes are unacceptable.

SQL: the query language

SQL evolved from SEQUEL, a language developed by Donald D. Chamberlin and Raymond F. Boyce at IBM in the early 1970s for their System R prototype. The first commercially available RDBMS was Oracle V2, released in 1979. SQL was standardized by ANSI in 1986 and by ISO in 1987, and the standard has been revised several times since — most recently as SQL:2023.

Conceptually SQL is grounded in relational algebra, which defines a small set of operators on relations: selection (filter rows), projection (pick columns), join (combine tables by key), union (merge compatible rows), and so on. Real SQL also includes grouping, aggregation, and subqueries.

The four core data-manipulation statements are:

• SELECT — retrieve rows from one or more tables.
• INSERT — add new rows.
• UPDATE — modify existing rows.
• DELETE — remove rows.

Schema definition uses CREATE TABLE, ALTER TABLE, and DROP TABLE, plus constraint clauses such as PRIMARY KEY, FOREIGN KEY, NOT NULL, UNIQUE, and CHECK. SQL is the common surface across virtually all RDBMSs, including postgres, MySQL, SQLite, SQL Server, and Oracle.

Popular relational database systems

A few systems dominate the landscape:

• PostgreSQL — open-source RDBMS widely considered the most feature-complete. Advanced types, indexing, JSON support, extensions, and strict standards conformance. Often run on its own, on top of a managed provider like supabase, or as the engine behind a cloud warehouse product.
• MySQL — the most widely deployed open-source RDBMS, especially in web stacks. Owned by Oracle; the MariaDB fork is also popular.
• SQLite — a serverless, embedded library. Not a separate process — the engine is linked into the application. The most widely deployed database engine in the world by install count, found in phones, browsers, and countless desktop apps.
• Microsoft SQL Server — a commercial RDBMS with deep Windows / Azure integration and a long history in enterprise IT.
• Oracle Database — the original commercial RDBMS lineage (from Oracle V2 in 1979); still dominant in large enterprise deployments.

Most modern analytics and transformation tooling targets these engines. dbt compiles SQL into version-controlled models that run on any of them; snowflake is a cloud-native SQL data warehouse that separates storage from compute.

When to use a relational database

A relational database is the right choice when:

• The data has a clear, stable structure with relationships between entities (users, orders, products, invoices, and the joins between them).
• You need ACID transactions — money movement, inventory, anything where partial writes are unacceptable.
• The workload needs complex joins across multiple tables.
• Data integrity, defined constraints, and a single source of truth matter more than schema flexibility.

NoSQL alternatives (document, key-value, graph, columnar) win when the data is unstructured, the workload is write-heavy at massive scale and can tolerate eventual consistency, or the access patterns don't fit a tabular model. In practice most production systems use both: a relational database for the system of record, plus specialized stores on the side.

FAQ

What is a relational database?

A relational database is a database that stores data in tables of rows and columns, enforces a predefined schema with typed columns, and uses primary and foreign keys to express relationships between tables. The standard language for working with these databases is SQL.

What is normalization in a relational database?

Normalization is the process of restructuring tables so that each piece of data lives in exactly one place, which avoids update, insertion, and deletion anomalies. It is graded into normal forms — 1NF, 2NF, 3NF, BCNF — each with stricter rules about keys and dependencies.

What is the difference between SQL and NoSQL?

SQL databases (postgres, MySQL, Oracle, SQL Server, SQLite) use a structured tabular schema and SQL to query it, and typically provide ACID transactions. NoSQL databases (document, key-value, wide-column, graph) use flexible schemas, scale horizontally, and usually trade strict consistency for availability under the BASE model.

When should you use a relational database?

Use a relational database when the data has a clear structure with relationships between entities, when ACID transactions are required, when you need joins across multiple tables, and when data integrity matters more than schema flexibility. For unstructured or rapidly changing data at very large scale, a NoSQL store is often a better fit.

What is object storage

Object storage (also called object-based storage or blob storage) is a data storage architecture that manages data as "blobs" or "objects", as opposed to other storage architectures like file systems, which manage data as a file hierarchy, and block storage, which manages data as blocks within sectors and tracks.

Each object is typically associated with a variable amount of metadata, and a globally unique identifier. Object storage can be implemented at multiple levels, including the device level (object-storage device), the system level, and the interface level. In each case, object storage seeks to enable capabilities not addressed by other storage architectures, like interfaces that are directly programmable by the application, a namespace that can span multiple instances of physical hardware, and data-management functions like data replication and data distribution at object-level granularity.

The term blob was coined by Jim Starkey while working at Digital Equipment Corporation, and was later adopted for the Rdb/VMS database. The humorous backronym "binary large object" was retroactively applied to it. The architectural research that shaped modern object storage was led by Garth Gibson at Carnegie Mellon in 1995, in a project called Network-Attached Secure Disks (NASD), which promoted the idea of splitting namespace operations from bulk reads and writes to optimize scale. Gibson later co-founded Panasas to commercialize these ideas, and one of his collaborators, Howard Gobioff, went on to co-design the Google File System.

How objects differ from files and blocks

A traditional file system organises data in a tree of folders and files. Each file has a path, a fixed set of metadata (name, size, timestamps, permissions), and a set of bytes. To find a file, you walk the tree. A block storage system breaks data into fixed-size blocks, each addressed by a logical block number, and reassembles them on read. Block storage is what an operating system mounts as a local disk.

An object is a different kind of unit. Instead of a path, each object has a unique identifier (often a URL-like key) and a rich, application-defined set of metadata attributes. The container that holds objects is usually called a bucket (in S3) or a container (in Azure Blob Storage). Within a bucket there is no folder hierarchy: all objects live in a flat namespace, and the application gives each object a key. The combination of a key, a value (the bytes), and a set of attributes is what object storage calls an "object".

Object storage also allows the addressing and identification of individual objects by more than just file name and file path. Object storage adds a unique identifier within a bucket, or across the entire system, to support much larger namespaces and eliminate name collisions.

Rich metadata and custom attributes

File systems have a small, fixed set of metadata: filename, creation date, type, owner. Object storage explicitly separates file metadata from data to support additional capabilities. Object storage provides for full function, custom, object-level metadata, which can be used to:

• capture application-specific or user-specific information for better indexing
• support data-management policies (for example, a policy to drive object movement from one storage tier to another)
• centralise management across many nodes and clusters
• optimise metadata storage (encapsulated, database, or key-value) and caching/indexing independently from the data storage

This is the part that makes object storage useful for unstructured data: you can tag a video with its encoding, duration, and content rating; you can tag a backup with the database it came from and the time of the snapshot; and the storage system can use those tags to drive lifecycle policy without the application having to re-read the object.

Programmatic access through REST APIs

Object storage provides programmatic interfaces to allow applications to manipulate data. At the base level, this includes create, read, update, and delete (CRUD) functions for basic read, write, and delete operations. Some object storage implementations go further, supporting object versioning, object replication, life-cycle management, and movement of objects between different tiers and types of storage. Most API implementations are REST-based, allowing the use of many standard HTTP calls.

That is why the same HTTP verbs that power the web (GET, PUT, POST, DELETE) are also the operations you use to read, write, and delete objects. From an application's point of view, an object store is just an HTTP service.

How Amazon S3 implements object storage

Amazon S3 is the service that made object storage the default. Launched in the United States on March 14, 2006, and in Europe in November 2007, S3 is built on the same scalable storage infrastructure that runs Amazon's e-commerce network.

The basic storage units of Amazon S3 are objects which are organised into buckets. Each object is identified by a unique, user-assigned key. Buckets can be managed using the console, programmatically with the AWS SDK, or the REST API. Objects can be up to five terabytes in size. If an object is larger than 5 GB, it must be divided into chunks prior to uploading, and any object larger than 5 GB must be uploaded via the S3 multipart upload API.

Requests are authorised using an access control list associated with each object bucket, and S3 supports versioning, which is disabled by default. The AWS authentication mechanism also allows the creation of authenticated URLs valid for a specified amount of time, so a private object can be shared temporarily without making the bucket public.

S3 offers nine different storage classes with different levels of durability, availability, and performance requirements. The default is S3 Standard, for frequently accessed data. Other classes target infrequent access, archival, and lower-cost long-term retention. S3 can also be used to replace static web-hosting infrastructure: an S3 bucket can be configured with an index document and an error document, and every object can be served as a public HTTP URL. As of 2025, S3 stores 500 trillion objects across hundreds of exabytes and serves 200 million requests per second, peaking at about one petabyte per second in bandwidth.

Object storage vs key-value storage

The border between an object store and a key-value store is blurred, with key-value stores being sometimes loosely referred to as object stores. They are similar in two respects: the object identifier (the equivalent of a key) can be an arbitrary string, and the data can be of an arbitrary size.

The key differences are these. Object stores allow one to associate a limited set of attributes (metadata) with each piece of data; the combination of a key, value, and set of attributes is the object. Object stores are optimised for large amounts of data (hundreds of megabytes or even gigabytes), whereas key-value stores are typically used for relatively small values (kilobytes). Finally, object stores usually offer weaker consistency guarantees, such as eventual consistency, whereas key-value stores often offer strong consistency. This is the trade-off that lets object stores scale to trillions of objects across many regions.

The S3-compatible ecosystem

The broad adoption of Amazon S3 and related tooling has given rise to competing services based on the S3 API. These services use the standard programming interface but are differentiated by their underlying technologies and business models. A standard interface enables better competition from rival providers and allows economies of scale in implementation.

Several storage providers such as Cloudian, Backblaze B2, and Wasabi offer S3-compatible storage with options of on-premises and private cloud deployments. Several open-source implementations of S3-compatible object-store servers can be self-hosted, such as Garage. IBM Cloud Object Storage also offers a cloud-based S3-compatible option. Cloud providers like Google Cloud Storage (May 2010) and Microsoft Azure Blob Storage have object products of their own; the S3 API has effectively become the de facto interface that most clients are written against.

Where object storage is used

Object storage systems allow retention of massive amounts of unstructured data in which data is written once and read once (or many times). Object storage is used for purposes such as storing videos and photos, songs on streaming services, and files in online collaboration services. The vast majority of cloud storage available in the market leverages an object-storage architecture.

Concretely, object storage is the right choice when the workload is dominated by large, write-once-read-many (or read-many-times) files: media assets, backups, log archives, software binaries, data lakes, and the static files behind a web or mobile application. Cloud functions read configuration and templates from object storage; CDNs source content from object storage; data lake analytics query Parquet files in object storage; and machine learning training pipelines read datasets from object storage.

When object storage is the wrong choice

One of the limitations with object storage is that it is not intended for transactional data, as object storage was not designed to replace NAS file access and sharing; it does not support the locking and sharing mechanisms needed to maintain a single, accurately updated version of a file. Editing a single byte in an object means re-uploading the whole object, and the object does not provide the random-access read/write pattern a database or a block device needs.

If your application needs random writes, low-latency reads of small records, or a shared filesystem with locking, you want a block device (for example, an EBS volume), a relational database, or a network file system, not an object store.

FAQ

What is the difference between object storage and a file system?

A file system organises data in a hierarchy of folders and files, with a fixed set of metadata per file. Object storage organises data as flat objects in buckets; each object has a unique key, the bytes, and a rich set of application-defined metadata attributes, and there are no folders.

Is Amazon S3 the same as object storage?

Amazon S3 is one specific implementation of object storage, and the one that made the architecture mainstream. Many other services (Google Cloud Storage, Azure Blob Storage, Backblaze B2, Wasabi, Cloudflare R2, and self-hosted options like MinIO) implement the same general model and often expose the same S3 API.

Can I modify part of an object?

No. Object storage treats each object as an opaque blob. To change any byte, the application has to upload a new object (typically with the same key) and replace the old one. This is the price of the architecture's simplicity and scale; it is also why databases, which need in-place updates, are built on top of block storage rather than object storage.

How large can an object be?

In Amazon S3, an object can be between 0 bytes and 5 TB. Objects larger than 5 GB must be uploaded using the multipart upload API. Other object services have their own limits, but the order of magnitude is the same.

How is object storage different from block storage?

Block storage breaks data into fixed-size blocks addressed by number and reassembles them on read; it is what an operating system mounts as a local disk and is the right choice for databases, boot volumes, and any workload that needs random read/write access. Object storage treats each file as an opaque object addressed by key; it is cheaper per gigabyte at scale, but you cannot edit part of an object in place.

Sources

• Wikipedia — Object storage (en.wikipedia.org/wiki/Object_storage, CC BY-SA 4.0)
• Wikipedia — Amazon S3 (en.wikipedia.org/wiki/Amazon_S3, CC BY-SA 4.0)

How microservices decompose a system

The hardest part of microservices is not writing the services — it is deciding where the boundaries go. A bad boundary creates coupling between services that defeats the whole point; a good boundary isolates change and lets teams work independently.

Bounded Context and Domain-Driven Design

The primary tool for drawing boundaries is Domain-Driven Design's Bounded Context pattern. DDD recognizes that a large domain model cannot be unified across a whole organisation — different teams use different vocabulary, and forcing everyone onto one schema causes conflicts and slowdowns. A Bounded Context defines a clear boundary within which a single model, called a Ubiquitous Language, applies. Microservices typically map one-to-one onto a Bounded Context: the "orders" service owns the "order" model and nothing outside it can read or write that data directly.

Martin Fowler describes this as the key strategic insight of DDD: instead of one massive unified model, you build several smaller models, each coherent within its boundary, with explicit rules for how the boundaries relate to each other.

Wikipedia adds that this one-to-one mapping between Bounded Context and microservice is "typically ideal as it maintains clear boundaries, reduces coupling, and enables independent deployment and scaling."

The Strangler Fig Pattern

When you have a working monolith and want to migrate to microservices, the Strangler Fig pattern makes it gradual. Coined by Martin Fowler after the rainforest vine that grows around a host tree, it means: build new functionality as separate services, route traffic to them using an API gateway, and slowly migrate existing behaviour from the monolith until the monolith can be switched off. Microsoft Learn describes it as supporting "incremental migration from a legacy system by gradually replacing specific pieces of functionality with new services. Consumers continue to use the same interface, unaware that the migration is taking place."

Granularity: finding the right size

Microservices are fine-grained by definition, but how fine? Wikipedia warns that very fine-grained services are easier to test and deploy independently but suffer lower performance and reduced reliability due to more network hops. Coarse-grained services are more robust but harder to test in isolation. The right size depends on team structure, fault isolation needs, and scalability requirements — there is no universal formula.

How services communicate

Smart endpoints and dumb pipes

The microservice community follows the principle of smart endpoints and dumb pipes. Services own their domain logic and act as filters — they receive a request, apply logic, and produce a response. The infrastructure between them, whether an HTTP bus or a message queue, stays simple. Martin Fowler puts it simply: "Applications built from microservices aim to be as decoupled and as cohesive as possible."

Synchronous and asynchronous patterns

Wikipedia identifies the three main approaches: synchronous HTTP/REST, asynchronous messaging, and GraphQL. Synchronous calls are simple but couple services in time — if the callee is slow, the caller waits. Asynchronous messaging (RabbitMQ, Kafka, Cloudflare Queues) decouples in time: the producer sends a message and moves on; consumers pick it up when ready.

Resiliency in distributed systems

Microsoft Learn lists the three patterns every microservice developer must handle: Retry (a transient failure should be retried a few times before giving up), Circuit Breaker ("too many failed requests can cause a bottleneck, as pending requests accumulate in the queue... The Circuit Breaker pattern can prevent a service from repeatedly trying an operation that is likely to fail"), and Distributed Tracing ("a single transaction might span multiple services. That can make it hard to monitor the overall performance and health of the system").

Data in a microservices world

Database-per-service

The defining data principle of microservices is that each service owns its own database. This lets every service choose the right data store for its workload — a search service might use Elasticsearch, an orders service PostgreSQL, a recommendation service Redis. Wikipedia summarises the benefit: services "can be monitored and scaled independently" because they run in independent processes.

Eventual consistency

When each service owns its own database, a business transaction that touches three services cannot use a single local transaction. Instead, the system tolerates temporary inconsistency. Wikipedia defines eventual consistency as a model "used in distributed computing to achieve high availability. An eventually consistent system ensures that if no new updates are made to a given data item, eventually all read accesses to that item will return the last updated value."

The trade-off is expressed as BASE versus ACID. Wikipedia contrasts the two: "Eventually-consistent services are often classified as providing BASE semantics (basically-available, soft-state, eventual consistency), in contrast to traditional ACID (atomicity, consistency, isolation, durability)." Critics note that eventual consistency provides only a liveness guarantee without safety guarantees — any intermediate value can be read before convergence.

The Saga pattern

When a business operation spans several services, you replace a distributed ACID transaction with a Saga. Microsoft Learn defines it precisely: "The Saga pattern manages transactions by breaking them into a sequence of local transactions. Each local transaction: completes its work atomically within a single service; updates the service's database; initiates the next transaction via an event or message. If a local transaction fails, the saga performs a series of compensating transactions to reverse the changes."

Two approaches exist. Choreography distributes control — each service reacts to events from the previous one. Orchestration centralises control through a single orchestrator that tells each service what to do. Microsoft Learn notes that choreography works well for simple workflows ("no single point of failure; responsibilities distributed") but becomes hard to follow for complex flows and makes integration testing difficult. Orchestration suits complex workflows better but introduces a central point of failure.

Common challenges

Microservices introduce complexity that does not exist in a monolith. Wikipedia categorises the main concerns:

• Information barriers — services cannot query each other's data directly, so reports and analytics require additional data-sharing mechanisms.
• Network cost — "Inter-service calls over a network have a higher cost in terms of network latency and message processing time than in-process calls within a monolithic service process."
• Testing complexity — testing one service in isolation is straightforward, but testing interactions between services requires contracts, mocks, or a full integration environment.
• Two-phase commit is an anti-pattern — "Two-phased commits are regarded as an anti-pattern in microservices-based architectures, resulting in a tighter coupling of all the participants within the transaction." The alternative is Saga.
• Timeout antipattern — "Short timeouts may fail legitimate requests prematurely, while long timeouts can result in slow error responses."

Martin Fowler adds that the sheer number of services requires "decentralized continuous delivery and DevOps with holistic service monitoring" to operate effectively. Microservices also need a Bulkhead pattern — Microsoft Learn describes it as isolating "critical resources, such as connection pools, memory, and CPU, for each workload or service. This isolation prevents a single workload or service from consuming all resources and prevents cascading failures."

When microservices make sense

This is the most important question, and Martin Fowler's Microservice Premium gives the clearest guidance. His central thesis: "The microservices approach is all about handling a complex system, but in order to do so the approach introduces its own set of complexities." The costs are real — automated deployment, monitoring, failure handling, eventual consistency.

His primary guideline: "don't even consider microservices unless you have a system that's too complex to manage as a monolith. The majority of software systems should be built as a single monolithic application. Do pay attention to good modularity within that monolith, but don't try to separate it into separate services."

The complexity that drives teams toward microservices can come from large teams, multi-tenancy, many user interaction models, independent business-function evolution, or sheer size — a monolith that is "too big to modify and deploy." Until you hit that wall, a well-modularised monolith is faster to build and easier to operate.

FAQ

How does microservice architecture differ from service-oriented architecture (SOA)?

Microservices are a specific form of SOA that emphasises extreme decentralisation. Where traditional SOA often uses an enterprise service bus (ESB) as a smart intermediary, microservices prefer "dumb pipes" and smart endpoints. Microservices also favour independent deployment of each service and often different programming languages per service, whereas SOA tends toward more integration at the bus. The microservices movement grew partly as a reaction to heavyweight ESB-centric SOA.

What is the difference between microservices and a modular monolith?

A modular monolith is one deployable unit with enforced internal boundaries — modules or packages that separate concerns. A microservices architecture has the same separation but also runs each module as an independent process with its own database. The modular monolith is simpler to develop and deploy; the microservices architecture allows independent scaling and different technology stacks per module. If you can enforce module boundaries in a monolith, you can migrate to microservices later without rewriting the logic.

How do you handle data consistency across microservices?

The pragmatic answer is eventual consistency via the Saga pattern or event-driven updates. Instead of a single ACID transaction, you accept that services will be briefly out of sync and use compensating transactions or event subscriptions to bring them back in sync. For read-heavy scenarios, CQRS (Command Query Responsibility Segregation) separates the write model from the read model, letting read services maintain their own query-optimised data copies without blocking writes.

When should you not use microservices?

Avoid microservices for simple CRUD applications, small teams, or early-stage products where you are still discovering the domain model. The overhead of managing many services — CI/CD pipelines, service discovery, distributed tracing, separate databases — is only worth it when the scale or team complexity genuinely requires it. Fowler's Microservice Premium is the test: if your monolith is not already causing problems, the migration cost will outweigh the benefits.

Sources

• Wikipedia — Microservices (CC BY-SA 4.0)
• Martin Fowler — Microservices
• Martin Fowler — Bounded Context
• Martin Fowler — Strangler Fig Application
• Microsoft Learn — Saga Pattern
• Microsoft Learn — Microservice Design Patterns
• Wikipedia — Eventual consistency (CC BY-SA 4.0)
• Martin Fowler — Microservice Premium
• Microsoft Learn — Interservice Communication

What can be lazy-loaded

Lazy loading applies to any resource that is non-critical for the first render. The MDN performance guide groups the candidates into several categories.

Each of these mechanisms shifts work off the critical path, but only when the resource is truly non-critical for the first paint. Lazy-loading the LCP image, for example, can delay the most important render and hurt perceived performance.

The native loading attribute

Modern browsers ship a built-in lazy-loading mechanism for images and iframes: the loading HTML attribute. It accepts two values: lazy defers the resource until it is near the viewport, while eager (the default) loads it normally. Adding the attribute is enough; no JavaScript library is required, and lazy-loading still works when JavaScript is disabled (though the load is only deferred, not skipped, when JS is off).

<img src="photo.jpg" loading="lazy" alt="…" width="600" height="400">
<iframe src="https://example.com" loading="lazy" width="600" height="400"></iframe>

Browsers determine how far ahead of the viewport to fetch by using an effective-connection-type heuristic. On a slow 2G connection the threshold is tighter (the browser fetches only when the user is very close), while on a fast 4G connection the threshold is much wider, so images are usually already loaded by the time the user reaches them. Chrome's thresholds have been tuned over time; the July 2020 update, for example, widened them on 4G to better match developer expectations.

The loading attribute is independent of fetch priority. An image marked loading="lazy" fetchpriority="high" is still delayed while it is off-screen, then fetched with high priority once it is near the viewport. The combination is rarely needed because the browser would have prioritized that image anyway. For above-the-fold images, use fetchpriority="high" (without loading="lazy") to mark them as the LCP candidate.

The Intersection Observer API

The loading attribute is built on top of the Intersection Observer API, which is the general-purpose primitive for "tell me when this element enters the viewport". Before Intersection Observer, sites had to wire up scroll, resize, and orientationchange event handlers and call getBoundingClientRect() on the main thread for every element being observed. On a long infinite-scroll page with many observers, that work compounded into noticeable jank.

Intersection Observer runs the visibility check off the main thread. The browser fires a callback when an observed element intersects a root element (the viewport by default) or a specified ancestor, or when the intersection ratio crosses a threshold. The API exposes the root, rootMargin (a CSS-like offset to grow or shrink the trigger area), and a list of threshold ratios. A typical lazy-load setup watches an element with a rootMargin of 200px so the image starts loading a little before the user scrolls it into view.

Code splitting and dynamic imports

JavaScript bundles can be split so that only the code needed for the current view is fetched up front. Entry-point splitting separates code by route, while dynamic splitting uses the import() expression to load a module on demand. A practical example is a route that only loads a chart library when the user navigates to a "Reports" tab; users who never visit that tab never download the chart code at all.

Scripts declared with type="module" are deferred by default, which is another form of lazy loading. The browser fetches them in parallel with HTML parsing but only executes them after the document has been parsed. async scripts execute as soon as they finish downloading, which is useful for independent functionality (analytics, ads) but can interfere with the order of execution.

When lazy loading helps and when it hurts

Lazy loading is a net win for resources that are clearly below the fold and unlikely to be needed immediately, but it can backfire on a few patterns. Lazy-loading the LCP image pushes the most important render later, which is the opposite of what you want. Lazy-loading above-the-fold images forces the browser to wait for two fetches (the lazy trigger, then the image) where one eager fetch would have sufficed. Lazy-loading content in a carousel the user is likely to scroll through often wastes time on the first interaction, because the user sees an empty placeholder while the first slide is being fetched.

Always pair loading="lazy" images with explicit width and height attributes (or a CSS aspect-ratio container). Without reserved dimensions, the image slot is zero-sized until the file loads, and the surrounding content snaps downward as the image paints in — a classic source of Cumulative Layout Shift. The same lesson applies to iframes: a width and height (or styled container) prevents the page from jumping when the iframe finally appears.

FAQ

What is the difference between loading="lazy" and the Intersection Observer API?

loading="lazy" is the browser's built-in lazy-loading attribute for images and iframes; the browser handles the visibility detection internally. The Intersection Observer API is a general-purpose JavaScript primitive that lets you detect when any element enters or exits the viewport, which you can use to lazy-load arbitrary resources (background images, video posters, custom markup) and to drive patterns like infinite scroll.

Does lazy loading hurt SEO?

Lazy loading itself does not hurt SEO as long as the content becomes available to crawlers and the page does not have to wait on user interaction. Googlebot renders pages with a headless browser that executes JavaScript, and Google has confirmed that the loading attribute is safe for crawl. The real SEO risk is performance: if lazy-loading the LCP image delays the most important render, the page scores worse on Core Web Vitals and may rank lower. Always keep the LCP image eager.

Should I lazy-load the first image on a page?

No. The first (and largest) image in the initial viewport is typically the LCP element, and delaying it hurts perceived performance. Mark it with fetchpriority="high" and let it load eagerly. Save loading="lazy" for images that are clearly below the fold and unlikely to be visible on first paint.

Can I lazy-load CSS?

Yes, but the common techniques are different from loading="lazy". You can scope non-critical CSS to a media query (@media (min-width: 800px)), preload it with <link rel="stylesheet" media="print" onload="this.media='all'">, or split CSS by route. CSS that is render-blocking for the first paint should not be deferred.

Why does my lazy-loaded image still load immediately?

The browser has already decided the image is close enough to the viewport that deferring would not save anything. The loading attribute is a hint, not a hard promise. On fast connections with a generous rootMargin, the image starts fetching as soon as the parser reaches it.

Summary

Lazy loading defers non-critical resources until they are needed, shrinking the critical rendering path and reducing the data a page consumes on first load. The browser's loading attribute handles images and iframes; the Intersection Observer API covers anything else; import() and code splitting cover JavaScript. Always reserve dimensions on lazy-loaded media to avoid layout shift, and never lazy-load the LCP image. Used carefully, lazy loading is one of the highest-leverage performance techniques on the modern web.

What is edge computing?

Edge computing runs computer programs close to where requests are made, rather than in a distant data center. Karim Arabi, during an IEEE keynote in 2014, described it as computing that occurs outside the cloud, at the network's edge — particularly for applications that need immediate data processing.

The defining characteristic is physical proximity to the user. A centralized cloud data center might be on another continent; an edge node may be in the same city or even on the same local network. This proximity determines the latency advantage: round-trip times from a nearby edge node are measured in single-digit milliseconds, while the same request to a distant cloud region may take ten times longer.

In cloud gaming, for example, edge nodes — sometimes called "gamelets" — are typically within one or two network hops from the client, which is fast enough for real-time interactive gaming. In contrast, a centralized cloud game server might add 100–200 ms of round-trip latency, making fast-paced games unplayable.

How it works

A conventional web application follows this path: a user's request travels over the internet to a central cloud region, the server processes it, and the response travels back. With edge computing, the request is intercepted and handled by a server that sits between the user and the origin cloud.

Modern platforms make this practical. Cloudflare Workers, for example, runs your code across Cloudflare's global network of over 200 data centers — you write a single JavaScript function and it executes on the nearest node to each incoming request, without you managing any servers. The platform provides key-value storage (Cloudflare KV) for fast, edge-cached reads and built-in observability.

The architecture typically involves:

• Edge nodes — geographically distributed servers that run the application code.
• The origin — a central cloud environment that holds the authoritative data store and business logic.
• The network path — requests first hit an edge node; if the data is cached, it is served immediately; otherwise the edge node forwards the request to the origin and caches the response for subsequent requests.

This is sometimes called a tiered architecture: the edge tier serves cached or static content, while the origin tier handles personalized, dynamic, or write operations that cannot be cached.

Benefits of edge computing

Lower latency

The closer the computation is to the user, the faster the response. For applications where every millisecond matters — video streaming, online gaming, financial trading, real-time communications — edge computing can mean the difference between a smooth experience and a frustrating one. Research cited on Wikipedia notes that human perception for tasks such as facial recognition falls in the 370–620 ms range; a well-designed edge platform can respond faster than this threshold, which is critical for augmented reality and other perception-sensitive applications.

Reduced bandwidth

Sending all raw data from edge devices to a central cloud wastes bandwidth. If a user's device performs video analysis locally and sends only the recognized text to the cloud, it transmits a fraction of the data it would otherwise send. This is a significant advantage for IoT deployments, where devices may be on constrained networks or have limited data plans.

Reliability

An edge node can continue serving cached content even if the connection to the central cloud is disrupted. A voice assistant, for example, may continue providing local services to nearby users during an internet outage. The distributed nature of edge systems also means that the failure of a single node does not take down the entire service — traffic is routed to the nearest healthy node.

Privacy

Processing data at the edge can keep sensitive information local rather than transmitting it to a remote cloud. User data stays on or near the device, which shifts data ownership partially from service providers toward end users.

Challenges

Security

The distributed nature of edge computing introduces new security concerns. Data travels between distributed nodes over the public internet, which requires encryption mechanisms that operate independently of the central cloud. Edge nodes themselves may be resource-constrained devices with limited capacity for cryptographic operations. A shift from a centralized trust model to a decentralized one is required, and managing security updates across hundreds of geographically distributed nodes is more complex than updating a single cloud server.

Scalability

Scaling a distributed edge network must account for the heterogeneity of devices — different performance levels, energy constraints, dynamic connectivity conditions, and reliability variations compared to a centralized cloud environment. Adding a new node to the edge network requires coordination, and security mechanisms on constrained devices can introduce latency in inter-node communication.

Operational complexity

Running code across hundreds of edge locations is operationally different from managing a handful of cloud servers. Debugging issues that appear only in specific geographic regions, managing deployments across heterogeneous infrastructure, and maintaining consistent configuration require specialized tooling. "Serverless" is a misnomer — servers are still there; the management complexity is shifted from the operations team to the development team, which must reason about distributed execution, partial failures, and eventual consistency.

Applications

Edge computing is used across a wide range of real-world scenarios:

• Internet of Things (IoT) — sensor data from connected devices is processed at the edge, reducing the volume of data that must be transmitted to the cloud. Smart grid systems, connected cars, and industrial automation all rely on edge processing for real-time responsiveness.
• Cloud gaming — game logic executes on edge nodes close to players, keeping round-trip latency below the threshold of human perception.
• Smart cities — traffic cameras, environmental sensors, and utility monitors process data locally and only forward exceptions or summaries to a central system.
• Augmented and virtual reality — object recognition and scene analysis run on the edge device or a nearby node to minimize the delay between movement and visual feedback.
• Edge AI — machine learning models run directly on edge devices or nearby servers, enabling real-time inference without round-tripping to a cloud API. This is sometimes called "on-device AI" or "local AI."
• Content delivery — the original form of edge computing, still widely used: CDN servers cache and serve images, videos, and static assets from locations near the user.

Edge computing vs cloud computing

Edge computing and cloud computing are not competing approaches — they complement each other. The cloud provides virtually unlimited compute and storage for batch processing, complex analytics, and durable data storage. The edge provides low-latency, high-availability execution for real-time, user-facing operations. The most resilient architectures use both: edge for immediate responses, cloud for authoritative data and complex processing.

FAQ

What is edge computing?

Edge computing is a distributed computing model that places computation and data storage physically close to the devices and users that need them, rather than running everything in a centralized cloud data center. It reduces latency by eliminating the round trip to a distant server and reduces bandwidth by processing data locally.

What is the difference between edge computing and cloud computing?

Cloud computing runs applications in centralized data centers that can be far from users, offering virtually unlimited compute and storage at the cost of higher latency. Edge computing distributes computation across many geographically dispersed nodes close to users, minimizing latency and bandwidth at the cost of greater operational complexity and a decentralized trust model.

What are the main advantages of edge computing?

The three primary advantages are latency (responses in single-digit milliseconds rather than tens or hundreds), bandwidth (data is processed locally rather than transmitted to the cloud), and reliability (edge nodes can continue serving cached content even when the connection to the cloud is interrupted). Privacy is a secondary benefit: sensitive data can be processed at the edge rather than sent to a remote server.

When should you use edge computing?

Use edge computing when your application needs response times faster than a round trip to a centralized cloud can provide — for real-time interactions such as gaming, AR, video streaming, IoT sensor processing, or financial trading. It is also valuable when bandwidth is constrained (IoT devices on limited networks) or when reliability during internet outages matters. For batch processing, complex analytics, and operations that benefit from virtually unlimited centralized compute, cloud computing remains the better choice.

Why normalization matters

Codd stated the objectives of normalization beyond 1NF as: freeing the collection of relations from undesirable insertion, update, and deletion dependencies; reducing the need to restructure relations when new types of data are introduced; making the relational model more informative to users; and making relations neutral to the query statistics that change over time.

The motivation is concrete. Tables that are not sufficiently normalized suffer three classic anomalies:

• Insertion anomaly. Facts cannot be recorded at all unless other, unrelated facts are also present. In a single "Faculty and Their Courses" table, a newly hired professor who has not yet been assigned to teach any course cannot be inserted, because the Course Code column is part of the row and would have to be set to NULL.
• Update anomaly. The same fact appears on multiple rows, and a change has to be applied everywhere. If a professor's office address lives in every row of an "Employees' Skills" table (one row per skill), then a single address change must touch all of that professor's skill rows. If the update is partial, the table gives conflicting answers to "what is this person's address?".
• Deletion anomaly. Deleting one fact inadvertently deletes another. In the Faculty/Courses table, removing the last course a professor teaches also removes the professor from the table — unless the course column is set to NULL, which is its own kind of anomaly.

A fully normalized database can be extended to accommodate new types of data with minimal changes to existing structure, so applications interacting with the database are minimally affected. Normalized relations also tend to mirror real-world concepts and their interrelationships more closely than denormalized ones.

The normal forms at a glance

A normal form is a rule that a table must satisfy. Each form is strictly stronger than the previous one, and the forms are progressive: a table in 3NF is automatically in 2NF and 1NF; a table in BCNF is automatically in 3NF; and so on. The full list, ordered from least to most normalized, is:

• UNF — Unnormalized form (repeating groups or composite values in a column).
• 1NF — First normal form.
• 2NF — Second normal form.
• 3NF — Third normal form.
• EKNF — Elementary key normal form (rarely discussed; sits between 3NF and BCNF).
• BCNF — Boyce–Codd normal form.
• 4NF — Fourth normal form.
• 5NF — Fifth normal form.
• DKNF — Domain-key normal form.
• 6NF — Sixth normal form.

The informal meaning of "the table is normalized" in everyday database work is almost always "the table is in 3NF". C. J. Date has argued that only a database in 5NF is truly "normalized", but the practical convention is 3NF.

First normal form (1NF)

A table is in 1NF when every field contains a single, atomic value — no sets of values, no nested records, no repeating groups. A column like Subject containing MySQL, Database, Design for one row violates 1NF because the column contains multiple values.

The fix is to extract the repeated values into a separate table. If a Book table has a Subject column that holds several subject names, the subjects are moved into a Title – Subject table with one row per (book, subject) pair. The original Book table loses the Subject column, and the relation between books and subjects is now in a separate, properly atomic table.

This is the level at which SQL itself imposes structure: a relational table is, by definition, a set of rows, and a column cannot legally contain a sub-relation.

Second normal form (2NF)

A table is in 2NF when it is in 1NF and every non-key column depends on the whole of every candidate key, not just part of it. 2NF only matters when the candidate key is composite — a primary key made of two or more columns.

Consider a Book table with a composite key of {Title, Format} and a Price column. Price depends on Title and Format (the same title in hardcover costs more than the same title in e-book). But other columns, like Author, Pages, or Publisher, depend on Title alone. Because those non-key columns depend on only part of the candidate key, the table is not in 2NF.

The standard fix is to split: keep Title, Author, Pages, Publisher, … in the Book table (keyed on Title alone), and move Price into a separate Price table keyed on {Title, Format}. After the split, every non-key column in each table depends on the whole key of its own table.

In databases where every table is keyed on a single column (a synthetic id, for example), 2NF violations are usually impossible by construction — a non-key column trivially depends on a single-column key. This is one reason synthetic keys are so common in modern schemas: they make 2NF concerns moot.

Third normal form (3NF)

A table is in 3NF when it is in 2NF and no non-key column depends on another non-key column. This is the rule that removes transitive dependencies.

After the 2NF split, the Book table has Title as its key, but Author Nationality depends on Author, and Author depends on Title. That is a transitive dependency: Title → Author → Author Nationality. The same transitive chain exists for Publisher Country (depends on Publisher, which depends on Title) and for Genre Name (depends on Genre ID, which depends on Title).

To reach 3NF, those transitively-dependent columns are pulled out into their own tables:

• Author (Author, Nationality)
• Publisher (Publisher, Country)
• Genre (Genre ID, Name)

The Book table is left with Title, Author, Pages, Thickness, Publisher, Genre ID — only columns that depend on Title directly. After the split, changing a publisher's country is a one-row update in the Publisher table, not a sweep across the Book table; a new genre can be added without inserting a new book; and a book can be inserted before its author is recorded by leaving Author NULL (or empty), without losing the book record itself.

Informally, a relational database is often described as "normalized" if it meets 3NF, and most 3NF relations are free of insertion, update, and deletion anomalies.

Boyce–Codd normal form (BCNF)

BCNF is a slightly stricter version of 3NF. A table is in BCNF if and only if, for every one of its functional dependencies X → Y, either X → Y is trivial (Y is a subset of X) or X is a superkey for the table. Since BCNF is strictly stronger than 3NF, every BCNF table is automatically in 3NF, but not every 3NF table is in BCNF.

The classical example is a "Today's court bookings" table for a tennis club with one hard court (Court 1) and one grass court (Court 2). Each row stores the booking plus a Rate type (SAVER, STANDARD, PREMIUM-A, PREMIUM-B), where each rate type implies a specific court and a specific member/non-member flag. The candidate keys of the table are {Court, Start time}, {Court, End time}, {Rate type, Start time}, and {Rate type, End time}. All four attributes are prime (each belongs to some candidate key), so 2NF and 3NF are both satisfied.

The table is not in BCNF because of the dependency Rate type → Court. Rate type is not a superkey (a single rate type can be in many rows), and Court is not a subset of Rate type. The fix is to split the table into a Rate types table (keyed on Rate type, with the implied Court and member flag) and a Today's bookings table that no longer carries the redundant Rate type → Court information. Both resulting tables are in BCNF.

BCNF has a known cost: not every non-BCNF table can be decomposed into BCNF tables and preserve all the functional dependencies of the original. Beeri and Bernstein showed in 1979 that a set of functional dependencies such as {AB → C, C → B} cannot be represented by any BCNF schema. In practice, this is rare, but it explains why some teams stop at 3NF even when BCNF is theoretically reachable.

BCNF also has a computational footnote: it is NP-complete, given a database schema in 3NF, to determine whether the schema actually violates BCNF. The check is cheap for any one schema but the worst case is hard.

Beyond 3NF: 4NF, 5NF, 6NF

The higher normal forms address redundancy that 3NF and BCNF do not catch.

Fourth normal form (4NF) removes redundancy caused by independent multivalued dependencies in the same table. The Wikipedia 4NF example has a Franchisee – Book – Location table where each franchisee offers certain books in certain locations; the combination is not constrained — a franchisee may carry any subset of books in any subset of locations. The table has no non-key attributes and is therefore in BCNF, but the same (Franchisee ID, Title) pair appears with every location, and the same (Franchisee ID, Location) pair appears with every title. Decomposing into Franchisee – Book and Franchisee – Location removes the redundancy and the table is in 4NF.

Fifth normal form (5NF) removes redundancy that surfaces only when a table can be reconstructed losslessly from a join of three or more projections, but not from any two of them. The classic example, again from the Wikipedia article, involves a supplier / book / franchisee relationship where the constraint is that a supplier supplies a title to a franchisee if and only if the supplier supplies the title, the title is supplied to the franchisee, and the franchisee is supplied by the supplier. Without that constraint, the three-way join produces spurious rows; with the constraint encoded, the table cannot be decomposed into two tables without losing information, and is already in 5NF.

Sixth normal form (6NF) is the strictest in practical use: "a table is in 6NF when the row contains the Primary Key, and at most one other attribute". Every non-key column is isolated in its own table, joined back to the key when needed. This is rarely used in OLTP because of the proliferation of tables and the need to update many tables for a single conceptual change, but in data warehouses — Sybase IQ, Microsoft SQL Server columnstore indexes, and similar systems — an internal 6NF representation is exactly what a columnar store does. Each column is stored separately, queries that scan a single column read only that column's data, and range filters on a low-cardinality column benefit from the compression that isolated storage enables.

Codd's progressive property still holds: 6NF implies 5NF, 5NF implies 4NF, 4NF implies BCNF, BCNF implies 3NF, 3NF implies 2NF, and 2NF implies 1NF. The practical sweet spot for most transactional systems is 3NF or BCNF; analytics workloads often deliberately denormalize back toward 1NF, or even use a star schema, which knowingly violates 2NF/3NF for read performance.

Denormalization: deliberately breaking the rules

Denormalization is the strategy of adding redundant copies of data to a previously normalized schema in order to improve read performance, at the cost of write performance and consistency overhead. Wikipedia frames it as a read-vs-write trade-off: a normalized design often has to join many tables to answer a single question, and joins are expensive; denormalization adds precomputed or copied data so a question can be answered from one table.

There are two broad approaches:

• DBMS-managed denormalization. The database engine stores redundant copies and keeps them in sync. SQL Server's indexed views and Oracle/PostgreSQL's materialized views are the standard examples. The logical schema stays normalized; the physical layout does not.
• Designer-managed denormalization. The database administrator adds the redundancy directly to the schema (extra columns that could be derived from a join, replicated lookup values, precomputed counts). The DBMS does not keep these copies in sync; the application or a trigger has to, which adds a new class of bugs to manage.

A denormalized data model is not the same as a data model that was never normalized. Denormalization only helps if it is applied on top of a schema that was at least 3NF, with the resulting anomalies and constraints accounted for. The data-warehouse use case is the canonical one: heavy analytical reads, infrequent writes, and the operational discipline to keep the denormalized views consistent.

FAQ

What does it mean for a database to be normalized?

A database is normalized when its tables satisfy the rules of the normal forms — most commonly up to 3NF or BCNF. A normalized schema stores each fact in one place, so updates do not have to be made in many rows, and inserts or deletes cannot accidentally lose unrelated facts. The opposite is denormalized, where redundant copies of the same fact are stored in multiple places to make reads faster.

What is the difference between 1NF, 2NF, and 3NF?

1NF requires every column to hold a single, atomic value — no sets, no nested records. 2NF requires that, on top of 1NF, every non-key column depends on the whole of every candidate key (no partial key dependencies). 3NF requires that, on top of 2NF, no non-key column depends on another non-key column (no transitive dependencies). Each form is strictly stronger than the previous, so a 3NF table is automatically in 2NF and 1NF.

Why is 3NF usually the stopping point?

Most OLTP schemas aim for 3NF because most insertion, update, and deletion anomalies disappear once transitive dependencies are removed, and 3NF is much cheaper to reason about and maintain than BCNF or the higher forms. Higher normal forms (4NF, 5NF, 6NF) handle subtler kinds of redundancy — multivalued dependencies, join dependencies, isolated columns — that rarely appear in transactional data models, so the additional complexity is not usually worth it for an OLTP workload.

What is the difference between 3NF and BCNF?

BCNF is a stricter form of 3NF: a table is in BCNF if every non-trivial functional dependency starts with a superkey. A 3NF table can violate BCNF when there are overlapping candidate keys and a non-key dependency follows from one of them, as in the tennis-club "Rate type → Court" example. The fix is to split the table so the non-superkey determinant becomes a key in its own table. Some non-BCNF tables cannot be decomposed into BCNF and preserve all original functional dependencies; in those cases 3NF is the practical ceiling.

When should you denormalize a database?

Denormalize when the workload is read-heavy and the cost of repeated joins is the bottleneck, especially in reporting and analytics use cases. The most common pattern is to keep the normalized schema as the source of truth and expose denormalized views (materialized views, indexed views, or pre-aggregated tables) that the DBMS keeps in sync. Manual denormalization is also possible but adds consistency-management overhead: every redundant copy must be updated whenever the underlying data changes, or the database will quietly return inconsistent answers.

Is database normalization the same as SQL?

No. Normalization is a property of a schema (the table definitions, keys, and relationships). SQL is the language used to query a schema. You can run SQL against a denormalized schema, and you can use a non-SQL system to store a normalized schema. The two are independent — the choice of query language does not determine the schema's normal form, and the schema's normal form does not dictate the query language.

What a webhook actually is

A webhook is a method for one web application to alter its behavior based on events from another. You register a URL (the endpoint) on the source service, choose which events you care about, and the service sends an HTTP request — usually a POST with a JSON payload — to that URL whenever one of those events fires.

Webhooks are built on HTTP, so they integrate into web services without adding new infrastructure. You only need a public endpoint that can receive a POST request. Because they reuse the plain web stack, they are the closest thing to a universal way for one service to notify another in real time.

The term was coined by Jeff Lindsay in 2007 from the general programming concept of a hook — a place where you can attach custom code to run when something happens.

Webhooks vs polling vs a regular API

The key idea is the direction of the request.

• Polling — your application repeatedly asks the service "anything new?" on a schedule. You control timing, but you usually get no real-time notification: the freshest data appears at the next poll, and a lot of requests return no new events.
• A regular API (question-answer) — your application calls the service when you decide, such as a REST call to GET /users. The service answers on demand, but it will not reach out to you.
• A webhook (event-driven) — the service calls your endpoint the moment an event happens. You get real-time delivery and only receive data when there is actually something to process.

None is universally better. Webhooks give you real-time push and avoid wasted polling, but they require a reachable, secured endpoint and careful handling of retries and failures. For a deeper look at the request side, see the API explained guide and the HTTP methods explained guide.

How a webhook delivery works

A typical webhook flow has three stages:

1. Subscription — you create a webhook on the source service and give it the URL of your endpoint (for example https://api.example.com/webhooks/stripe). You select the events you want and, for many services, a shared secret.
2. Event — something happens on the source side: a charge succeeds, a pull request opens, a deploy finishes.
3. Delivery — the source makes an HTTP request to your endpoint. The request is almost always a POST with a JSON body describing the event, plus headers that identify the delivery. Your server returns a 2xx status to acknowledge it.

POST /webhooks/stripe HTTP/1.1
Host: api.example.com
Content-Type: application/json
Stripe-Signature: t=1724860000,v1=abcdef...

{
  "id": "evt_1QxY2example",
  "type": "payment_intent.succeeded",
  "data": { "object": { "amount": 4999, "currency": "usd" } }
}

The type (or a header) tells you what happened, and the body carries the details. Because the webhook is a plain HTTP POST, the same endpoint can serve events from many different providers — a single route can fan out by event type to the right handler.

Verifying webhook signatures

Because a webhook is an unauthenticated-style inbound HTTP request, anyone who discovers your endpoint could POST fake events to it. To know the request is genuine, verify its signature — this is the single most important thing you do with an inbound webhook.

Most providers sign the request body with a shared secret using HMAC. GitHub sends X-Hub-Signature-256, Stripe sends Stripe-Signature, Slack sends X-Slack-Signature, Shopify sends X-Shopify-Hmac-Sha256. Your server computes the expected HMAC over the raw body with the secret and compares it to the header using a timing-safe comparison.

const https = require("node:https");
const crypto = require("node:crypto");

const secret = process.env.WEBHOOK_SECRET; // never hardcode secrets

function verify(payload, header) {
  if (!header) return false;
  const expected = "sha256=" + crypto
    .createHmac("sha256", secret)
    .update(payload)
    .digest("hex");
  // timing-safe compare — never plain ===
  const a = Buffer.from(expected);
  const b = Buffer.from(header);
  return a.length === b.length && crypto.timingSafeEqual(a, b);
}

A few extra rules:

• Use HTTPS for your endpoint — never deliver or receive webhook payloads over plain HTTP.
• Check the timestamp — many providers include the delivery time so you can reject requests that are too old (a replay attack).
• Store secrets in environment variables or a secret manager, never in source code. For the surrounding pattern, see the environment variables guide.
• Log rejections — invalid signatures are often the first sign of probing or misconfiguration.

Handling retries and making your handler idempotent

Webhooks are delivered over HTTP, which can fail: a network blip, your server restarting, an endpoint that responds too slowly. Providers handle this by retrying. Stripe retries failed deliveries for up to three days, and Shopify retries repeatedly over a window. That means you may receive the same event more than once, in any order.

Two practices make this safe:

1. Respond fast — return a 2xx within a few seconds, then process the payload asynchronously. If your handler is slow, the provider may time out and retry, which produces duplicates.
2. Make the handler idempotent — processing the same event twice must give the same result as processing it once. The standard approach is to check a deduplication key (the event id or a delivery GUID header) and skip events you have already processed.

Retries also mean events can arrive out of order — a subscription updated event may arrive before the created event it depends on. Where order matters, rely on timestamps or sequence numbers instead of arrival order. For critical integrations, consider a dead-letter queue where persistently failing events go for manual review instead of being silently lost.

Webhooks vs other real-time options

Webhooks are not the only way to get real-time data:

• WebSockets keep an open, two-way connection for live streaming and interactive apps, but are not an event-delivery/retry system. See the WebSocket explained guide.
• Message queues and event streams (like Kafka) give durable, replayable, ordered delivery, at the cost of running more infrastructure.
• Webhooks are the simplest to consume — a plain HTTP POST — and pair well with a small retry/idempotency layer. Stripe, GitHub and Shopify all use this pattern.

A hybrid is common and pragmatic: use the webhook as a real-time notification and keep a polling or API endpoint as a recovery path to backfill any event you might have missed. For the request/response API side of the same story, see the REST API with Node.js and Express guide.

FAQ

What is the difference between a webhook and an API?

An API is a request/response interface your application calls on demand — you ask, the service answers. A webhook is the reverse: the service calls your endpoint when an event happens, delivering the data to you without you asking. Both use HTTP. Roughly: an API lets you pull, a webhook lets the service push.

How do I test a webhook without waiting for a real event?

Most providers include a test/demo panel that sends a sample or simulated event to your endpoint. For local development you can use a public tunnel to expose your local server (GitHub, Stripe and many others accept a tunnel URL), or use a tool that can send a crafted POST with the correct body, headers and signature so you can exercise your verification and handler logic.

What happens if my webhook endpoint is down?

The provider detects the failed delivery and retries according to its policy — for example Stripe retries for up to three days with exponential backoff. In the meantime you may miss events, which is why an idempotent handler plus an API or polling recovery path is recommended for anything where missed events matter.

Is a webhook secure?

A webhook is only as secure as your verification. Anyone who knows your endpoint URL could POST to it, so you must verify the signature of every request and reject anything that does not match, use HTTPS, check timestamps against replay, and keep the shared secret out of source code. With those in place, webhooks are a safe, standard way to receive events.

Do I need a webhook or just an API?

If your application actively initiates the interaction (fetch a user, submit a form) you need an API. If you want to react to events that happen on another service in real time (a payment, a deploy, a product update) without polling, a webhook is the right tool. Many integrations use both: webhooks for push notification, an API for backfilling and reconciliation.

Verdict

Webhooks are the standard building block for event-driven integrations, and they are simple to consume because they are plain HTTP POSTs. Use them when you want a service to push real-time events to you. To build them safely, verify every signature with a timing-safe HMAC comparison, use HTTPS, check timestamps, keep secrets out of code, return 2xx fast and process asynchronously, and make your handler idempotent so retries and out-of-order deliveries do not corrupt your state. When delivery is critical, pair the webhook with an API or queue recovery path so a missed event is never silently lost.

What the Web Crypto API is

The Web Crypto API surfaces low-level cryptographic operations through the SubtleCrypto interface. Its entry point is window.crypto.subtle (or self.crypto.subtle in a Worker). The API is deliberately low-level — it does not hold your hand. You choose the algorithm, manage the key material, and handle the raw bytes. This is both its power and its risk: it is easy to use correctly for standard patterns and easy to misuse in subtle ways.

The API requires a secure context — it only works on pages served over HTTPS (or localhost). Any page loaded over plain HTTP throws a DOMException.

The operations it provides fall into three groups:

• Hashing — produce a fixed-length digest from arbitrary data (digest()).
• Signing — create or verify a digital signature with a key pair (sign(), verify()).
• Encryption — encrypt or decrypt with a symmetric key (encrypt(), decrypt()).
• Key management — generate, derive, import, export, and wrap keys (generateKey(), deriveKey(), importKey(), exportKey(), wrapKey(), unwrapKey()).

All methods return Promises and operate on ArrayBuffer or TypedArray data, not strings. You encode strings with TextEncoder and decode responses with TextDecoder.

Key concepts

Keys are objects, not strings

Cryptographic keys in the Web Crypto API are CryptoKey objects. They carry metadata — the algorithm, the key type (secret, private, or public), and whether they are extractable. You never pass raw bytes as a key; you always create or import a CryptoKey first.

Symmetric vs asymmetric

• Symmetric algorithms use the same key to encrypt and decrypt (AES). The key must be kept secret.
• Asymmetric algorithms use a key pair: a public key for encryption or signature verification, and a private key that stays on your side. RSA and ECDSA are asymmetric.

For most browser-based tasks — encrypting local data, signing a token, hashing a password — symmetric (AES) is simpler and faster. Use asymmetric when you need to share the encryption key or when the other party needs a public key for verification.

Timing-safe comparison

Never use === to compare two strings or byte sequences for equality in security-sensitive code. An attacker can measure how long the comparison takes and use the timing difference to deduce the secret. The Web Crypto API provides crypto.timingSafeEqual() for exactly this — always use it when comparing MACs, signatures, or any secret byte sequence.

Hashing data

Hashing turns any input into a fixed-length "fingerprint" in a one-way direction. You cannot recover the original from the hash. SHA-256 (256-bit output) is the most widely used browser-safe hash. SHA-384 and SHA-512 are also available. MD5 and SHA-1 are considered broken for security purposes.

A common use is deriving a key from a password, or creating a quick integrity check for a payload.

const encoder = new TextEncoder();
const data = encoder.encode("The quick brown fox");

const hashBuffer = await crypto.subtle.digest("SHA-256", data);
const hashArray = Array.from(new Uint8Array(hashBuffer));
const hashHex = hashArray.map(b => b.toString(16).padStart(2, "0")).join("");

console.log(hashHex);
// 4d156c9271fd4f5e99e27fc2e84d3f8f3c0e5b8f7d2a6e1c9b3d...

sha-256 is the string identifier for SHA-256. The output is an ArrayBuffer of 32 bytes (256 bits). Convert it to hex for readability, or keep it as bytes for further processing.

For hashing passwords before storing them, consider using PBKDF2 (via deriveKey) rather than a plain hash — it applies multiple iterations to slow down brute-force guesses. For a deeper look at hashing algorithms and their properties, see the SHA-2 article on Wikipedia.

Signing and verifying messages

Digital signatures let you prove that a message was created by the holder of a private key. The recipient uses the matching public key to verify. This is asymmetric cryptography: you sign with the private key, anyone can verify with the public key.

The Web Crypto API supports ECDSA (Elliptic Curve Digital Signature Algorithm) and RSASSA-PKCS1-v1_5. ECDSA is faster and produces smaller signatures — use it for new work.

// Generate a key pair
const keyPair = await crypto.subtle.generateKey(
  { name: "ECDSA", namedCurve: "P-256" },
  true, // extractable
  ["sign", "verify"]
);

const encoder = new TextEncoder();
const message = encoder.encode("I authorize this transfer");

// Sign
const sigBuffer = await crypto.subtle.sign(
  { name: "ECDSA", hash: "SHA-256" },
  keyPair.privateKey,
  message
);

// Export public key to share
const pubKeyBuffer = await crypto.subtle.exportKey("spki", keyPair.publicKey);

// Verify
const valid = await crypto.subtle.verify(
  { name: "ECDSA", hash: "SHA-256" },
  keyPair.publicKey,
  sigBuffer,
  message
);

console.log("Signature valid:", valid); // true

The signature is a fixed-length byte sequence tied to the message and the private key. If the message or the signature changes, verification fails. This pattern is used for signing JWTs, API tokens, and document attestations. For a practical guide on JWT signing in a Node.js API, see the JWT authentication guide.

Encrypting and decrypting data

Encryption turns readable data (plaintext) into unreadable data (ciphertext) using a key. The Web Crypto API supports three AES modes:

• AES-GCM — authenticated encryption (AEAD). The recommended mode. It detects tampering automatically. Requires a 12-byte initialization vector (IV) that must be unique per encryption (but does not need to be secret).
• AES-CBC — block cipher chaining. Requires a 16-byte IV. No built-in authentication — you must not rely on it alone for security-critical data.
• AES-CTR — counter mode. Requires an IV and a counter. Rarely needed for new work.

Always prefer AES-GCM. It is the only AES mode in Web Crypto that provides authenticated encryption out of the box, which protects against chosen-ciphertext attacks where an attacker asks the system to decrypt arbitrary messages and uses the result to learn about the key.

const encoder = new TextEncoder();
const plaintext = encoder.encode("Secret message");

// Generate a random 256-bit key
const key = await crypto.subtle.generateKey(
  { name: "AES-GCM", length: 256 },
  true,
  ["encrypt", "decrypt"]
);

// IV must be unique per encryption — use random bytes
const iv = crypto.getRandomValues(new Uint8Array(12));

const ciphertext = await crypto.subtle.encrypt(
  { name: "AES-GCM", iv },
  key,
  plaintext
);

// Store or transmit: key (kept secret), iv (can be stored alongside ciphertext), ciphertext
const decrypted = await crypto.subtle.decrypt(
  { name: "AES-GCM", iv },
  key,
  ciphertext
);

const decoder = new TextDecoder();
console.log(decoder.decode(decrypted)); // "Secret message"

Store the IV alongside the ciphertext — it is not secret, only unique. For a client-side application that encrypts local data (say, a note-taking app that stores data in IndexedDB), this pattern is all you need: generate a key, encrypt, store the key in localStorage or derive it from a password.

For storing encryption keys securely in a server-side application, see the environment variables guide — never hardcode secrets. For secret management at scale, HashiCorp Vault is purpose-built for this. See the Vault alternative page for a comparison.

Key derivation

Deriving a key lets you produce a cryptographic key from a password or another key material. The Web Crypto API supports PBKDF2 (Password-Based Key Derivation Function 2) and HKDF (HMAC-based Key Derivation Function). PBKDF2 is the right choice when you have a password and want a key.

const encoder = new TextEncoder();
const password = encoder.encode("user-password-123");

// Salt must be unique per password — store it alongside the derived key
const salt = crypto.getRandomValues(new Uint8Array(16));

const derivedKey = await crypto.subtle.deriveKey(
  { name: "PBKDF2", salt, iterations: 310000, hash: "SHA-256" },
  await crypto.subtle.importKey("raw", password, "PBKDF2", false, ["deriveKey"]),
  { name: "AES-GCM", length: 256 },
  true,
  ["encrypt", "decrypt"]
);

310,000 iterations is the current OWASP minimum for PBKDF2 with SHA-256 (as of 2023). The salt must be unique per user and stored in plaintext alongside the encrypted data — it is not secret, only makes precomputed attacks impossible.

Common mistakes to avoid

The Web Crypto API is low-level and easy to misuse. These are the patterns that cause real security incidents:

• Reusing an IV (nonce) with AES-GCM or AES-CTR — if the same IV is used twice with the same key, an attacker can recover both ciphertexts. Always generate a fresh random IV with crypto.getRandomValues() for every encryption operation.
• Using AES-CBC or AES-CTR without message authentication — without authentication, an attacker can flip bits in the ciphertext and the decryption will produce modified output with no error. Use AES-GCM to get authentication automatically, or implement HMAC over the ciphertext if you must use CBC/CTR.
• Hardcoding a key in source code — keys in code are in version control and logs. Derive them from a password, store them in localStorage (client-side), or retrieve them from a secret manager. For server-side patterns, see the environment variables guide.
• Using MD5 or SHA-1 for anything security-related — both are considered broken for collision resistance. SHA-256 is the minimum standard.
• Ignoring secure context requirements — the API throws in non-HTTPS contexts. Do not build features that silently degrade without noticing.
• Using === for secret comparison — always use crypto.timingSafeEqual() to compare MACs, signatures, and derived keys.

For a broader view of the surrounding ecosystem — API design, testing, monitoring — see the API explained guide.

FAQ

What is the difference between hashing and encryption?

Hashing is one-way: given data, you produce a fixed digest. You cannot recover the original from the digest. Encryption is two-way: given ciphertext and the correct key, you recover the plaintext. Use hashing for integrity checks and password storage; use encryption when the original data must be readable again.

Can I use Web Crypto in a Web Worker?

Yes. crypto.subtle is available in Workers via self.crypto.subtle. Running crypto operations in a Worker keeps them off the main thread and prevents the UI from stuttering during heavy operations like key derivation or large-file encryption.

Is Web Crypto secure for production use?

Yes, when used correctly. The underlying algorithms (AES-GCM, ECDSA, SHA-256) are well-vetted and implemented in browser-native code that is generally faster than JavaScript polyfills. The risk is in the usage — wrong algorithm choices, IV reuse, missing authentication, or hardcoded keys. The W3C specification itself carries a warning that the primitives are easy to misuse.

How does Web Crypto compare to Node.js crypto?

Node.js crypto module (and the node:crypto API since v15) provides roughly the same algorithms and often more. The Web Crypto API is the browser subset of those primitives, designed for the browser's threat model. Both use the same underlying native implementations on the system — Web Crypto in browsers, Node.js directly in server-side code. If you write code that works with window.crypto.subtle, the same patterns work with crypto.subtle in Node.js v15+.

Do I need a library for client-side encryption?

For the operations in this guide, no — the built-in API covers hashing, signing, and AES-GCM encryption without any dependency. Libraries like libsodium.js provide higher-level constructs (sealed boxes, password hashing with Argon2) that are harder to misuse, but for standard AES-GCM + ECDSA the Web Crypto API is sufficient and has no supply-chain risk.

Why servers compress text

Plain text is repetitive. The same words, HTML tags, CSS properties and JavaScript identifiers appear again and again. Compression algorithms replace that repetition with shorter references, so a 200 KB script might arrive as 50 KB and be rebuilt in the browser. Less data over the wire means a faster first paint, especially on slow or mobile connections. Compression is negotiated per request: the browser lists what it accepts in Accept-Encoding, and the server answers with the algorithm it chose in Content-Encoding.

How gzip works

gzip is the older, near-universal format. Technically it wraps the DEFLATE algorithm (defined in RFC 1951) with a header and a 32-bit CRC checksum (defined in RFC 1952). DEFLATE combines the LZ77 method — finding repeated strings and pointing back to earlier copies — with Huffman coding, which assigns shorter codes to common bytes. Because gzip is old and simple, every browser and server supports it, and it compresses quickly even for freshly generated, dynamic responses.

How Brotli works

Brotli (defined in RFC 7932 by Google engineers J. Alakuijala and Z. Szabadka) also combines LZ77 with Huffman coding, so it is built on the same ideas as DEFLATE. The difference is a large, pre-defined dictionary: Brotli ships with roughly 1,200 common words and phrases drawn from web text, HTML, CSS and JavaScript. When it meets a familiar fragment — a frequent tag, a property name, a library call — it can reference the dictionary instead of repeating the bytes. RFC 7932 describes its efficiency as "comparable to the best currently available general-purpose compression methods."

Brotli vs gzip: the practical difference

The Brotli dictionary pays off most on web content. Wikipedia summarizes independent comparisons showing that, relative to gzip, JavaScript compressed with Brotli is roughly 15% smaller, HTML around 20% smaller, and CSS around 16% smaller. Brotli also has 11 quality levels (0–11); higher levels compress more but take longer, so Brotli is usually applied to static assets that are pre-compressed once at build time. gzip compresses faster at every level, which keeps it the better choice for responses generated on the fly. In short: gzip is the safe, fast default; Brotli is the stronger choice for pre-built CSS, JS and HTML.

Browser support

gzip has been supported by every browser for decades. Brotli (br in the Content-Encoding header) is supported by all current versions of Chrome, Firefox, Safari and Edge. A server that supports both simply sends Brotli to browsers that ask for it and falls back to gzip for the rest.

FAQ

What is the difference between gzip and Brotli?

Both are lossless text compressors built on LZ77 and Huffman coding. Brotli adds a large pre-built dictionary of common web words and phrases, which lets it compress HTML, CSS and JavaScript more tightly — Wikipedia cites roughly 15–20% smaller files than gzip for typical web assets.

Is Brotli better than gzip?

For static, text-heavy assets such as CSS and JavaScript, Brotli usually produces smaller files than gzip. gzip compresses faster and is supported everywhere, so it remains a strong choice for dynamically generated responses. Many servers send Brotli to capable browsers and gzip as a fallback.

How does the browser know which compression to use?

The browser advertises supported algorithms in the Accept-Encoding request header (for example gzip, br); the server replies with the one it used in the Content-Encoding response header. The browser then decompresses the body before rendering.

Can compression make a website slower?

Compression itself adds a little CPU work on the server and a little on the client, but the saved bytes almost always outweigh that cost on text. The main caveat is Brotli's highest quality levels, which are slow; that is why Brotli is normally pre-applied to static files rather than computed per request.

Summary

gzip and Brotli both shrink web text using LZ77 and Huffman coding, but Brotli's built-in dictionary gives it an edge on HTML, CSS and JavaScript. gzip is the universal, fast default; Brotli is the stronger option for pre-compressed static assets, and every modern browser supports it. Pick a host or server that negotiates both, and your pages ship fewer bytes without changing a line of your code.

What JavaScript is

JavaScript is a high-level, dynamic, weakly-typed scripting language standardized as ECMAScript and described by MDN as a "lightweight interpreted (or just-in-time compiled) programming language with first-class functions." It is the default language of the browser, the runtime behind Node.js, and the lingua franca of the modern front-end ecosystem.

The defining properties matter because they shape what you can rely on:

• Dynamic typing — variables are not bound to a declared type; any variable can be reassigned to values of any type, and type checks happen at runtime.
• Weak typing with implicit coercion — JavaScript freely converts between types in operations, which is convenient but also the source of many subtle bugs.
• Prototype-based object orientation — objects inherit from other objects directly, and the language has grown class syntax on top of that model.
• Multi-paradigm — it supports object-oriented, functional, and imperative styles; the ecosystem uses all three.

JavaScript is also the only language every browser understands natively. That fact alone has made it the most widely deployed programming language in the world.

What TypeScript is

TypeScript is a free and open-source programming language developed and maintained by Microsoft. As Wikipedia describes it, TypeScript is "a high-level programming language that adds static typing with optional type annotations to JavaScript. It is designed for developing large applications. It transpiles to JavaScript." It is licensed under the Apache License 2.0 and is the language most often used today to write large Angular, VS Code, and a steadily growing slice of the React, Vue, and Node.js ecosystem.

The core idea is optional static types on top of JavaScript. A TypeScript file is a JavaScript file with extra annotations:

function greet(name: string): string {
  return "Hello, " + name;
}

The TypeScript compiler (tsc) reads these annotations, type-checks the program, and emits plain JavaScript that any browser or Node.js can run. Because the types are stripped, TypeScript never reaches the runtime — it is a developer-tooling layer that disappears at build time. The result is that all valid JavaScript is also valid TypeScript, which is why migration is incremental: rename .js to .ts and start adding annotations where they help.

TypeScript also adds language features that JavaScript did not have when TypeScript was created (enums, decorators, readonly modifiers, private modifiers, namespaces) and tracks ECMAScript proposals ahead of browser support.

The key differences

When JavaScript is the better choice

JavaScript is the right default in situations where the simplicity of "no build step, no types, run it" matters more than the static guarantees TypeScript provides:

• First language to learn — every web developer needs JavaScript, and learning it first means understanding the runtime that TypeScript ultimately compiles to.
• Small scripts and prototypes — a one-off Node.js tool, a five-line snippet in DevTools, a quick browser experiment; the build step is overhead with no payoff.
• Learning web fundamentals — DOM, fetch, events, the event loop. Adding type annotations on top of these concepts adds noise before the underlying ideas land.
• Tiny libraries or single-file components — when the surface is small, dynamic typing is rarely the source of bugs.
• Environments without a build pipeline — some legacy embed systems, microcontrollers, browser userscripts, and quick demos cannot or do not need a compilation step.

JavaScript is also the universal common ground. If you write a library you want anyone to consume, publishing plain JavaScript (or at least shipping a JavaScript build alongside TypeScript sources) is still the safest choice.

When TypeScript is the better choice

TypeScript earns its build step when the cost of runtime bugs and the cost of refactoring start to dominate. Common situations:

• Application code with a domain model — once a codebase models users, orders, products, or any non-trivial entity, types catch a long tail of "passed the wrong shape" mistakes that JavaScript would only catch by crashing at runtime.
• Large codebases with multiple contributors — type signatures double as documentation and as a check that one team's expectations match another team's implementation.
• Long-lived projects that will be refactored — renaming a property, changing a function signature, or splitting a module is dramatically safer when the compiler walks you through every callsite.
• Libraries meant to be consumed by other TypeScript users — types become the public API of the library; consumers get autocomplete and inline errors in their editor.
• Frameworks and tooling that already assume types — Angular has been TypeScript-first since v2; modern React, Vue, Node.js frameworks and ORMs are all designed with TypeScript in mind.
• Anywhere the team wants to catch mistakes at edit time — the single biggest quality-of-life argument for TypeScript is the red squiggly in the editor, before the code is ever run.

The cost of TypeScript is real: a tsconfig.json to maintain, a build step, occasional fights with the type system, and some time spent writing annotations. In a project above a few thousand lines or with more than one contributor, that cost is almost always worth paying.

The honest downsides

Both choices have real costs that the rosy side usually omits.

TypeScript adds a build step. Every change has to be compiled before it runs in a browser or on a server, which slows down inner-loop development if you do not set up a fast watcher. The type system is also not free to learn: generics, conditional types, and the way TypeScript infers shapes can be a project of their own. Some libraries ship with weak or missing types, and a few patterns (heavily dynamic code, certain meta-programming tricks) are awkward to express and end up needing any or unsafe casts.

JavaScript's downsides are the flip side of its simplicity. The compiler cannot warn you that you are calling a function with a string where it expects a number; the runtime will, often in production. Refactoring a large JavaScript codebase means grep, manual checking, and faith. Type coercion produces bugs ("1" + 1 is "11", not 2) that are obvious in hindsight but invisible in the moment. None of this is a reason to avoid JavaScript — it is the reason to be deliberate about the size of project you are willing to keep dynamic.

How to decide

There is no universal winner, and the two are not mutually exclusive:

• Start with JavaScript when you are new to programming or new to the web, or when the project is small enough that you can hold the whole thing in your head.
• Adopt TypeScript as soon as the project crosses the threshold where one of these becomes true: a domain model with several entities, more than one contributor, a long expected lifetime, or a need to refactor safely.
• Mix them — most teams use TypeScript for application code and JavaScript (or untyped TS) for tiny config files, throwaway scripts, and the like. The TypeScript compiler is happy to let .js files in.
• Migrate incrementally — TypeScript was designed for gradual adoption. You can flip the switch on one folder, then another, without rewriting the codebase.

If you are starting a brand-new project today, the default is increasingly TypeScript: most scaffolding tools (create-react-app, npm create vue, create-next-app) offer it out of the box, and the largest libraries in the ecosystem either are written in TypeScript or ship first-class types. If you are learning from zero, start with JavaScript so you understand the runtime, then add types when they start to feel like the missing piece.

FAQ

Is TypeScript a different language from JavaScript?

TypeScript is a different language in the sense that it has its own syntax, compiler, and tooling, but it is a strict superset of JavaScript. Every valid JavaScript program is also a valid TypeScript program — TypeScript only adds optional type annotations and a few language features on top. The TypeScript compiler emits plain JavaScript, so TypeScript code runs in the same runtimes as JavaScript (browsers, Node.js, Deno, Bun) with no special runtime support.

Do I need to learn JavaScript before TypeScript?

Yes. Because TypeScript compiles to JavaScript and the runtime is JavaScript, you need to understand JavaScript fundamentals — the event loop, prototypes, closures, the DOM, async/await, modules — to write useful TypeScript. The typical path is to spend a few weeks with plain JavaScript, then turn on TypeScript and use the types to deepen your understanding of the same code.

Does TypeScript replace JavaScript in browsers?

No. Browsers do not run TypeScript; they run JavaScript. TypeScript's role is at build time: the compiler checks the types, then emits the equivalent JavaScript, which is what actually ships. This is why every TypeScript project needs a build step before the code can be served to a browser.

Is TypeScript slower than JavaScript at runtime?

No. TypeScript has no runtime — types are erased at compile time, and the emitted JavaScript is the same code you would have written by hand (often better, because the compiler optimizes some patterns). Any runtime difference between a TypeScript and a hand-written JavaScript implementation of the same algorithm comes from the code itself, not from the language.

Verdict

JavaScript and TypeScript are not in competition; TypeScript is a layer on top of JavaScript that you can choose to use or not. Pick JavaScript when you want zero build steps, are writing something small, or are just learning the web platform. Pick TypeScript when the project has a domain worth modeling, a team that will grow, or a lifetime long enough to need safe refactoring. The two coexist on every project you will touch, and the right answer is rarely "always one, never the other" — it is "JavaScript now, TypeScript as soon as the code earns it."

About SSH key authentication

SSH (Secure Shell) uses public key cryptography for authenticating users. The protocol never sends your private key over the network; instead, it proves possession of the private key by signing a challenge. This eliminates password brute-force and phishing risks.

A typical SSH key pair consists of:

• Private key (~/.ssh/id_ed25519): Never shared; decrypts authentication challenges.
• Public key (~/.ssh/id_ed25519.pub): Shared with servers/GitHub; verifies signatures.

When you attempt to connect, the server encrypts a nonce with your public key; only your private key can decrypt it to prove identity.

Why Ed25519? Modern Ed25519 keys offer equivalent security to 3072-bit RSA with faster performance and smaller size. GitHub requires Ed25519 or RSA with SHA-2 signatures for keys uploaded after March 15, 2022.

Step 1 — Check for existing SSH keys

Before generating a new key, check whether you already have SSH keys in ~/.ssh/:

ls -la ~/.ssh/

Common filenames: id_rsa, id_ecdsa, id_ed25519. If you see both a private key (no extension) and a .pub file, a key pair already exists. Back up or reuse existing keys if preferred.

Step 2 — Generate a new Ed25519 key pair

Run ssh-keygen to create a new key pair. Replace your_email@example.com with your GitHub or account email.

ssh-keygen -t ed25519 -C "your_email@example.com"

You'll see prompts:

1. File to save the key: Press Enter to accept the default location (~/.ssh/id_ed25519).
2. Passphrase (optional but recommended): Enter a secure passphrase to encrypt your private key file. You'll need to enter this whenever the key is used unless loaded into ssh-agent.
3. Repeat passphrase: Confirm the passphrase.

The output shows the key fingerprint and randomart image — a visual representation of your key.

Legacy systems note: If your system doesn't support Ed25519 (very old OpenSSH), use ssh-keygen -t rsa -b 4096 -C "your_email@example.com" instead. Ensure the RSA key uses SHA-2 signatures (required by GitHub for newer keys).

Step 3 — Start ssh-agent and add your key

The ssh-agent is a background program that holds your private key decrypted in memory, so you don't enter the passphrase for every connection.

Start the agent and add your key:

# Start the agent in the background
eval "$(ssh-agent -s)"
# Add your private key to the agent
ssh-add ~/.ssh/id_ed25519

If you set a passphrase, you'll be prompted to enter it once. The agent will now supply your key for SSH authentication until you log out or kill the agent.

To make this persistent across sessions, add the eval "$(ssh-agent -s)" line to your shell profile (~/.bashrc, ~/.zshrc, etc.).

Step 4 — Copy the public key to GitHub

To authenticate with GitHub over SSH, add your public key to your GitHub account.

1. Copy the public key to your clipboard:

   # macOS
   pbcopy < ~/.ssh/id_ed25519.pub
   
   # Linux (requires xclip or wl-clipboard)
   xclip -selection clipboard < ~/.ssh/id_ed25519.pub
   # or
   wl-clipboard < ~/.ssh/id_ed25519.pub
   
   # WSL2
   clip.exe < ~/.ssh/id_ed25519.pub
   

2. In GitHub:

   • Click your profile picture → Settings.
   • In the left sidebar, click SSH and GPG keys.
   • Click New SSH key or Add SSH key.
   • Paste the key into the Key field.
   • Give it a descriptive title (e.g., "Personal Laptop Ed25519").
   • Select Key type as Authentication.
   • Click Add SSH key.

Verify: After adding, GitHub shows the key fingerprint and the date added.

Step 5 — Set up key-based login on a Linux server

To use SSH keys for server login (instead of passwords), copy your public key to the server's ~/.ssh/authorized_keys file.

Option A — Using ssh-copy-id (recommended)

If ssh-copy-id is available (installed with OpenSSH), run:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your_server_ip

Replace user with your username and your_server_ip with the server's address or hostname. You'll be prompted for the user's password one last time. The tool appends your public key to ~/.ssh/authorized_keys on the server.

Option B — Manual copy

If ssh-copy-id isn't available, manually append the key:

cat ~/.ssh/id_ed25519.pub | ssh user@your_server_ip 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys'

Again, replace user@your_server_ip with your target. You'll be prompted for the password.

Set correct permissions (critical)

On the server, ensure the .ssh directory and files have restrictive permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Incorrect permissions cause SSH to ignore the key file for security reasons.

Step 6 — Test your SSH connection

Verify that authentication works without a password prompt.

Test GitHub SSH

ssh -T git@github.com

You should see:

Hi <username>! You've successfully authenticated, but GitHub does not provide shell access.

If it's the first connection, GitHub shows its host key fingerprint. Verify it matches GitHub's published fingerprint, then type yes.

Test Linux server SSH

ssh user@your_server_ip

If you set a passphrase on the key, you'll be prompted to enter it (unless the key is loaded in ssh-agent). Otherwise, you should log in directly.

Step 7 — Optional: Add key to ssh-agent automatically

To avoid running ssh-add every session, add your key to the agent automatically.

Add these lines to your shell profile (~/.bashrc, ~/.zshrc):

# Start ssh-agent if not running
if [ -z "$SSH_AUTH_SOCK" ]; then
   eval "$(ssh-agent -s)"
   ssh-add ~/.ssh/id_ed25519
fi

This starts the agent on demand and loads your key.

Common mistakes to avoid

• Wrong permissions: ~/.ssh must be 700, authorized_keys must be 600. SSH ignores insecure permissions.
• Forgotten passphrase: If you set a passphrase and don't load the key into ssh-agent, you'll be prompted for it on every connection.
• Mixing key types: Ensure the public key you upload matches the private key you're using (Ed25519 vs RSA).
• Server not allowing pubkey auth: Check /etc/ssh/sshd_config on the server for PubkeyAuthentication yes and restart sshd after changes.
• Agent not running: If you see Agent admitted failure to sign using the key, the key isn't loaded in ssh-agent. Run ssh-add ~/.ssh/id_ed25519.

FAQ

How do I use different SSH keys for different accounts?

Use Host aliases in ~/.ssh/config. Example:

# GitHub personal account
Host github.com
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_ed25519_personal
   IdentitiesOnly yes

# Work account (different key)
Host github-work
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_ed25519_work
   IdentitiesOnly yes

Then use git@github-work:user/repo.git for work repos.

Can I use the same SSH key for GitHub and server login?

Yes. The same public key can be placed in multiple places: your GitHub account, server authorized_keys, or any service accepting SSH keys. However, for high-security separation, use different key pairs.

What if I lose my private key?

Generate a new key pair and repeat the setup process. Immediately remove the lost/public key from all services (GitHub, servers) to prevent unauthorized access.

Should I encrypt my private key with a passphrase?

Yes, if the key is stored on a laptop or shared machine. A passphrase encrypts the private key file so theft of the laptop doesn't grant immediate access. On a secure, isolated workstation you may skip it, but a passphrase is recommended.

How do I rotate SSH keys?

Generate a new key pair, add the new public key to all services, verify login works, then remove the old public key from GitHub and server authorized_keys files.

Summary

SSH key authentication provides secure, passwordless login for GitHub and Linux servers. By generating an Ed25519 key pair, using ssh-agent, and copying the public key to the right places, you replace vulnerable passwords with strong cryptography. Always verify connections, maintain proper file permissions, and rotate keys periodically for best security.

With these steps, you can authenticate to GitHub push/pull and SSH into servers without typing a password — while keeping your private key secret.

Where service workers fit

Service workers are a kind of web worker, which means they run off the main thread and have no DOM access. Unlike a regular web worker (which is a script you spawn and communicate with), a service worker is registered once, persists across page loads, and runs long after the page that registered it has been closed. The browser starts it on demand whenever an event in its scope fires.

The features a service worker unlocks are the building blocks of the modern app-like web:

• Offline support — serve cached HTML, CSS, JavaScript, and images when the network is unavailable.
• Faster repeat visits — short-circuit cached resources instead of going to the network.
• Background sync — replay a request when connectivity returns (e.g. send a queued message).
• Push notifications — wake the worker from a push event even when no tab is open.
• Request interception — rewrite headers, respond with synthesized data, or simulate network conditions.

This is why a service worker is the technical core of a progressive web app — the offline behavior and installability of a PWA are both implemented on top of one.

The basics

A service worker is a plain JavaScript file that lives on your origin. Three rules govern how you write it:

• It must be served from a secure origin. Browsers only register service workers on HTTPS, with one exception: http://localhost is treated as secure so local development works without certificates.
• It has no DOM access. No document, no window, no direct access to the page. Communication with the page goes through postMessage and the Clients API.
• It is fully asynchronous. APIs that are synchronous on the main thread, like synchronous XMLHttpRequest or localStorage, are not available. The worker is built around promises; the lifecycle events all take a event.waitUntil(promise) call to signal completion.

It also matters where the file lives. The browser scopes a service worker to the directory it is served from. A worker at /sw.js controls every page under /, a worker at /blog/sw.js controls only /blog/*, and so on. The convention is to put it at the root so it can act as a catch-all proxy.

The lifecycle

A service worker goes through a deliberately careful lifecycle. According to the web.dev service worker lifecycle guide by Jake Archibald, the intent of the lifecycle is to:

• Make offline-first possible.
• Allow a new service worker to get itself ready without disrupting the current one.
• Ensure an in-scope page is controlled by the same service worker (or no service worker) throughout.
• Ensure there is only one version of your site running at once.

The states a service worker moves through are:

1. Parsed — the browser has fetched sw.js and is checking whether it is new (byte-different from the currently installed version, or the first one ever).
2. Installing — the install event fires. This is where you typically pre-cache your app shell with caches.open(name).then(cache => cache.addAll([...])). event.waitUntil(promise) extends the install until the promise settles.
3. Installed (waiting) — the worker has installed successfully but the old version is still controlling open pages. The new worker waits until all pages using the old version are closed.
4. Activating — the activate event fires. This is where you typically clean up old caches from previous versions.
5. Activated — the worker is now in control of the page. From this point on it receives fetch and push events for any in-scope navigation.
6. Redundant — if a newer version installs and activates, the old worker is replaced and becomes redundant.

Two details from the MDN service worker API page shape how the lifecycle feels in practice:

• A page's fetches do not go through a service worker unless the page request itself went through a service worker. This means the first time you ship a worker, the current page keeps loading directly from the network; the worker only takes effect on the next navigation or after a refresh. If you need the new worker to control the current page immediately, call clients.claim() from inside activate.
• A new service worker does not interrupt an in-progress page. It waits for the page to close. This is intentional: it prevents two versions of your app from running at the same time, which is the only way to guarantee that IndexedDB, the cache storage, and your other persistent state have a single source of truth.

A minimal example

The shortest end-to-end demonstration is a cache-on-install, network-falling-back-to-cache fetch handler. From MDN's Using Service Workers guide:

// sw.js
const CACHE = 'app-shell-v1';

self.addEventListener('install', (event) => {
  event.waitUntil(
    caches.open(CACHE).then((cache) => cache.addAll([
      '/',
      '/styles.css',
      '/app.js',
    ]))
  );
});

self.addEventListener('activate', (event) => {
  // clean up old caches
  event.waitUntil(
    caches.keys().then((keys) =>
      Promise.all(keys.filter((k) => k !== CACHE).map((k) => caches.delete(k)))
    )
  );
});

self.addEventListener('fetch', (event) => {
  event.respondWith(
    fetch(event.request).catch(() => caches.match(event.request))
  );
});

Registration from a page looks like this:

if ('serviceWorker' in navigator) {
  navigator.serviceWorker.register('/sw.js');
}

A few patterns worth pulling out of the example:

• install is for filling the cache. Use it to pre-cache the app shell (HTML, CSS, the JS bundle, key images) so the first offline visit can render something useful.
• activate is for cleaning up old caches. Bump the cache name on each release (e.g. app-shell-v1 → app-shell-v2) and delete the previous one here.
• fetch is the proxy. event.respondWith(...) tells the browser "I will handle this request myself." Returning fetch(event.request).catch(() => caches.match(event.request)) is a common "network-first, fall back to cache" pattern.
• A service worker is JavaScript, but with a different scope. You can split the file into modules and import them statically; you just cannot use dynamic import().

Updating a deployed service worker

Updating a service worker is what most teams get wrong first. The lifecycle is designed to make updates safe but unobtrusive — you ship a new sw.js file, the browser downloads it in the background, runs install (with the new caches), and waits for the old page to close before activate.

A common update flow:

1. Bump a constant in sw.js (e.g. CACHE = 'app-shell-v2') and update the files in cache.addAll([...]).
2. The browser detects the byte difference on the next navigation and starts the new worker.
3. The new worker's install populates the new cache; the old worker continues serving the open page.
4. When every tab of the old page is closed, the new worker's activate runs and deletes the old cache.
5. The next navigation is controlled by the new worker.

If you need the new worker to take effect without waiting for users to close every tab, you can call self.skipWaiting() from install and clients.claim() from activate. The trade-off is that two versions of your app can briefly run side-by-side; the web.dev lifecycle guide walks through the cases where each approach is appropriate.

When to use a service worker

Service workers are the right tool when you need any of the following:

• Offline support for a published website — the canonical use case. PWAs, documentation sites, news readers, anything users may open on flaky connections.
• Faster repeat loads — serving the app shell and critical assets from the cache instead of the network. Pair with the cache and HTTP caching headers from your origin for the right balance.
• Background sync — the Background Sync API lets a worker replay a request after the device comes back online.
• Push notifications on the web — the Push API wakes the worker on receipt, and the Notifications API shows the actual message.
• A programmable network proxy — rewriting requests, adding headers, or stubbing responses in development.

When it is not the right tool:

• You only need to cache HTTP responses. A plain Cache-Control header on the response is cheaper and works without JavaScript.
• You need to do work on every page load. A service worker only handles events; if the work is on the critical rendering path it belongs in the page itself.
• You need access to the DOM. A service worker has none; build a regular web worker or put the code in the page.

FAQ

Do service workers have access to the DOM?

No. A service worker is a kind of web worker, which means it runs in its own thread with no access to document, window, localStorage, or any DOM API. The only way to talk to the page is postMessage from the worker to a controlled client, and message events from the page back to the worker. This is deliberate: it forces the worker to be non-blocking and keeps the page safe from anything the cached worker script might try to do to it.

Why does my new service worker not take effect immediately?

A page that loaded without a service worker does not start being controlled by one until the next navigation. MDN's Service Worker API page calls this out explicitly: "by default, a page's fetches won't go through a service worker unless the page request itself went through a service worker." The first registration only takes effect on the next page load, on the next open of the site, or after a refresh. If you need immediate control, call clients.claim() from inside the worker's activate event.

Can I use a service worker on plain HTTP?

No, except in two narrow cases. Service workers only register on secure contexts — origins served over HTTPS. The exception that matters for development is http://localhost: browsers treat localhost as a secure origin so you can test service workers without setting up a certificate. The other exception is browser-internal flags (Firefox has a developer-tools checkbox to allow service workers over HTTP) for debugging only; do not ship a public site over plain HTTP. Service workers over HTTP would be trivially exploitable by network attackers.

What is the difference between a service worker and a web worker?

A regular web worker is a script you spawn from a page with new Worker('foo.js'); it lives as long as the spawning page is open, communicates with the page over postMessage, and is destroyed when the page closes. A service worker is registered once with navigator.serviceWorker.register(...), persists independently of any page, runs events fired by the browser (fetch, push, sync, message), and can outlive every page that registered it. Both are off-main-thread, both have no DOM access, but a service worker is the durable, event-driven one.

Summary

A service worker is a small, persistent JavaScript file that lives on the user's device, sits between the page and the network, and lets the page work offline, load faster, and receive push events. The lifecycle — install, activate, then continuous fetch and other events — is the most subtle part of the API, and it is the part that catches most teams the first time. Once you understand that a new worker does not interrupt an in-progress page and that pages loaded without a worker do not start using one until the next navigation, the rest of the API falls into place: pre-cache on install, clean up old caches on activate, and respondWith from fetch to be the proxy the browser is waiting for.

What is serverless computing?

The term "serverless" comes from the fact that, as a developer, you never touch a server. According to the ISO/IEC 22123-2 cloud-computing standard, serverless computing is "a cloud service category where the customer can use different cloud capability types without the customer having to provision, deploy and manage either hardware or software resources, other than providing customer application code or providing customer data." It is a form of virtualized computing.

Despite the name, servers absolutely still exist — the cloud provider runs your code on them. "Serverless" is a misnomer in the sense that servers are still used to execute code; the difference is that you, the customer, never provision or maintain them. The provider handles capacity, patching, and availability.

Serverless is a broad ecosystem. It includes the cloud provider, Functions as a Service (FaaS), managed services, tools, and frameworks. It does not remove complexity so much as shift it: much operational work moves from an operations team to the development team, though operations still manages identity and access, networking, security policies, and cost optimization.

How Functions as a Service (FaaS) works

The core execution model behind most serverless applications is Functions as a Service. You write a single unit of business logic — a function — and deploy it to a FaaS platform such as AWS Lambda, Azure Functions, Google Cloud Functions, or Cloudflare Workers. The platform takes care of running it.

A typical mental model:

• You write and upload a function — just application code, no web server, no container orchestration.
• The platform handles invocation — an event (an HTTP request, a file upload, a message in a queue, a scheduled timer) triggers the function.
• The platform scales it automatically — from zero to millions of invocations, spinning instances up and down as demand changes. There is no capacity planning and no load balancer to configure.
• You pay per execution — billed on the actual time your code runs (CPU time), not for idle machines waiting around.

This is why serverless suits event-driven and bursty workloads: you only pay for the compute you actually use, and scale is handled for you.

Key benefits

• No infrastructure management. No servers to patch, no capacity to plan, no regional configuration. The provider owns availability and scaling.
• Automatic scaling. Functions scale from zero to millions of requests automatically, matching demand in real time.
• Pay-per-use pricing. You are billed for execution time, not for idle provisioned capacity. Quiet periods cost almost nothing.
• Faster delivery. Because deployment is just uploading code and wiring an event, teams can ship small units quickly.

Challenges and trade-offs

Serverless is not free of cost — it trades infrastructure concerns for a different set of problems.

• Monitoring and debugging are harder. Serverless applications are distributed and event-driven, and run in proprietary environments. Tracking an execution flow across many function invocations is harder than reading a single server's logs. Distributed tracing (for example AWS X-Ray), centralized logging, and OpenTelemetry are typical mitigations.
• Security has its own shape. OWASP notes that serverless applications are vulnerable to variations of traditional attacks plus serverless-specific ones, including "denial of wallet" — an attacker triggering many executions to inflate your bill. Insecure code and over-permissive permissions are common risks.
• Vendor lock-in. Applications are locked to a specific provider by default. Because of the high level of abstraction, you cannot configure the underlying environment, and a provider's managed services often only trigger that same provider's functions. Migrating a serverless workload to another cloud is difficult; designing from the start with multi-cloud in mind helps.
• Limits for sustained or heavy compute. High-performance or long-running workloads can hit resource limits (maximum memory, CPU, runtime). For sustained, predictable resource usage, bulk-provisioned servers are often more cost-effective than a per-execution model.

Anti-patterns to avoid

• Grain of sand. Making functions excessively small increases complexity, operational overhead, and performance inefficiency. There is an optimal level of modularization — split functions along real, stable boundaries, not into the smallest possible pieces.
• Lambda pinball / distributed monolith. When functions invoke each other in long fragmented chains, you get added latency, harder debugging, and reduced observability — effectively a distributed monolith. Prefer clear boundaries between functions and native service integrations over long chains of inter-function calls.

When should you use serverless?

Serverless is a good fit when:

• Your workload is event-driven — webhooks, queue messages, scheduled jobs, image or file processing.
• Your traffic is bursty or unpredictable — spikes that would wastefully idle always-on servers.
• You want to ship small units quickly without managing infrastructure.
• You can tolerate provider lock-in or design for portability from the start.

It is a weaker fit when:

• You need sustained, predictable, high-volume compute where a fixed server is cheaper.
• You run long-running or resource-heavy workloads close to platform limits.
• You have strict multi-cloud or portability requirements and cannot accept lock-in.

As a rule of thumb, reach for serverless when the platform's automatic scale and pay-per-use model align with an event-driven workload — and prefer fixed servers when you have steady load that would run 24/7 anyway.

FAQ

What does "serverless" actually mean?

Serverless means you, the developer, never provision or manage servers. The cloud provider runs your application code and handles capacity, scaling, and availability. The name is a slight misnomer — servers still exist — but they are entirely managed by the provider on your behalf.

What is Function as a Service (FaaS)?

FaaS is the core execution model of serverless. You deploy a single function to a platform (such as AWS Lambda, Azure Functions, Google Cloud Functions, or Cloudflare Workers), the platform invokes it on events, scales it automatically, and bills you for actual execution time rather than idle capacity.

Is serverless cheaper than traditional servers?

Not always. For event-driven or bursty workloads, pay-per-use is often cheaper because you stop paying for idle capacity. But for sustained, predictable, high-volume compute, a traditional provisioned server or VM can be more cost-effective.

What are the main downsides of serverless?

The main trade-offs are harder monitoring and debugging in distributed environments, a different security surface (including denial-of-wallet attacks), strong vendor lock-in that makes migration difficult, and resource limits that can hurt long-running or heavy compute workloads.

What REST is

REST (Representational State Transfer) is an architectural style introduced by Roy Fielding in his doctoral dissertation in 2000, built around the constraints that shaped the World Wide Web. An application that adheres to its constraints is called RESTful. In practice, when developers say "REST API," they usually mean an HTTP-based API organized around resources: each thing your system manages (users, orders, products) gets one or more URLs, and you act on it with HTTP methods.

The core idea is the uniform interface: your API exposes a stable set of resources identified by URLs, clients interact with them through standardized HTTP verbs, and responses contain representations of those resources.

The HTTP methods a REST API uses have well-defined semantics:

• GET requests a representation of the specified resource and should only retrieve data.
• POST submits an entity, often changing state or causing side effects.
• PUT replaces the target resource with the request content.
• PATCH applies partial modifications to a resource.
• DELETE deletes the specified resource.

These methods also have guarantees that matter for clients. GET, HEAD, and OPTIONS are safe (they do not change server state) and GET and HEAD are cacheable. PUT and DELETE are idempotent (repeating them has the same effect as doing them once). Understanding these properties is part of what makes a REST API predictable and cacheable at every layer, from the browser to a CDN.

What GraphQL is

GraphQL is a query language and runtime developed at Facebook, released as an open-source specification in 2015, and moved to the GraphQL Foundation under the Linux Foundation in 2018. Instead of many fixed endpoints, a GraphQL API exposes one endpoint and a typed schema. The schema describes every type and field the API can return, and clients send queries that ask for exactly the data they need.

A GraphQL query mirrors the shape of the result. If a client wants a user's name and only their latest posts, it writes that shape directly in the query, and the server returns JSON that follows it. The client drives the response; the server does not decide which fields to include. This fixes two problems common in REST:

• Over-fetching — a REST endpoint returns everything on a resource even when the client needs one field.
• Under-fetching — a REST client must make several requests (one per resource) or be given a bespoke endpoint to assemble a single view.

GraphQL supports reading data (queries), changing data (mutations), and real-time updates (subscriptions, often built on WebSockets). It is statically typed: the schema is the contract between client and server, and clients can validate queries against it before they run.

The key differences

When REST is the better choice

REST shines when its strengths map cleanly to your situation:

• Simple, resource-shaped APIs — if your data maps naturally to a handful of resources and clients mostly fetch whole objects, REST is uncomplicated and boring in the best way.
• Heavy caching and CDN use — because GET is cacheable and responses are per-URL, you can cache aggressively at the CDN, the proxy, and the browser, which is powerful for public, read-heavy content.
• Public APIs with unknown clients — the HTTP semantics (safe, idempotent, status codes) are well understood by every client and tool, and web infrastructure works with them out of the box.
• File uploads and simple integrations — multipart uploads, webhooks, and browser-native forms all work with REST without extensions.
• Long-lived, high-traffic services — a simple, stable REST surface is easy to cache and cheap to operate.

Many teams never outgrow REST. If your API has a few resources, consistent shapes, and modest client requirements, REST gives you caching, tooling, and simplicity for free.

When GraphQL is the better choice

GraphQL earns its complexity when REST's fixed shapes become the bottleneck:

• Many different clients, very different data needs — a mobile app wants small, specific payloads; a dashboard wants wide ones. GraphQL lets each client fetch exactly what it needs from one endpoint.
• Client-driven, nested data — when a view needs a user, their posts, and the comments on those posts, GraphQL assembles the whole graph in one query instead of making several round-trips.
• Rapidly evolving product — adding and deprecating fields in the schema is safer than contract changes across many endpoints.
• Strong typed contract as a team tool — one schema doubles as documentation, and clients can validate queries before they run.
• Multiple back-end services behind one API — resolvers can aggregate data from different sources and present it as a unified graph.

GraphQL makes the most sense when you control both client and server, clients actively, and the data shape is graph-like — which is exactly the situation where REST APIs tend to grow a pile of bespoke, one-off endpoints.

The honest downsides

Both styles have real costs, and choosing well means seeing the other side.

GraphQL is harder to cache. Because every query is different and all requests hit one endpoint, HTTP and CDN caching that works trivially for REST does not directly apply. Teams often add a caching layer (like Apollo or a CDN edge cache designed for GraphQL) and must think about per-query caching and the N+1 problem — where resolving a list triggers one query per item unless you batch and deduplicate with a data loader. A carelessly built GraphQL server can also produce heavier queries than intended, so query complexity limits and timeouts matter.

REST's weaknesses are the flip side of its simplicity. Clients are at the mercy of the server's chosen shapes: you over-fetch or under-fetch unless you create custom endpoints. Nested data often means several round-trips. Versioning is usually path-based, which leaves old endpoints around. And when different clients need very different views of the same resources, REST surfaces tend to drift from "one clean endpoint per resource" toward a graveyard of special-purpose endpoints.

How to decide

There is no universal winner — it depends on your clients, your team, and your operationally practical realities:

• Pick REST when you have a public or simple API, heavy read traffic, need CDN/browser caching, process file uploads, or value boring predictability.
• Pick GraphQL when you control the clients, they have diverse and nested data needs, your product evolves quickly, and you can invest in tooling and caching up front.
• Both can coexist — many teams run a REST API for public, cacheable read paths and a thin GraphQL layer for rich internal clients.

If you are unsure, start simple. REST is the safer default for most new APIs; adopt GraphQL when specific, repeated pain — over-fetching, many round-trips, or a broad multi-client product — makes the investment clearly worth it. The wrong choice is rarely permanent: the two styles can live side by side and you can migrate gradually.

FAQ

Is GraphQL a replacement for REST?

GraphQL is an alternative way to expose an API, not a drop-in replacement that is always better. It removes fixed per-resource endpoints and lets clients query exactly what they need from one typed endpoint. For many use cases REST is still a simpler and better-cached choice; for others GraphQL fixes real pain. Both can serve the same backend.

Can you use GraphQL and REST together?

Yes, and many teams do. A common pattern is keeping a REST API for public, cacheable, or simple read paths and adding a GraphQL layer for rich internal clients, or exposing both against the same data. They are not mutually exclusive, and you can migrate a service gradually rather than rewriting it.

Does GraphQL use HTTP?

GraphQL conventionally runs over HTTP with a single endpoint, usually accepting POST requests carrying the query, and it returns JSON. Under the hood it still uses HTTP, but it does not reuse standard HTTP caching and the per-resource URL semantics the way REST does, which is one reason caching is handled differently.

Which is faster: REST or GraphQL?

There is no inherent speed advantage — both run over HTTP. The practical difference is in the number of round-trips and the amount of data transferred. GraphQL can reduce both for nested, client-specific views. REST wins where HTTP caching means far fewer requests actually reach the server. Latency depends more on your server and caching than on the style itself.

Verdict

REST and GraphQL solve the same problem with opposite trade-offs, and the right answer depends on where you feel the pain. Choose REST when you want simple, resource-based endpoints plus free CDN and browser caching — the sensible default for most public and read-heavy APIs. Choose GraphQL when you control the clients, they need diverse and nested data, and the product evolves quickly enough that fixed shapes hurt. Start simple, adopt GraphQL deliberately when a concrete problem demands it, and remember the two styles coexist: you are not locked into one forever.

Why Node.js and Express?

Node.js is a server-side JavaScript runtime built on Chrome's V8 engine, designed for building scalable network applications. Its event-driven, non-blocking I/O model makes it efficient for data-intensive real-time applications. Express, first released in 2010, has become the de facto standard web framework for Node.js due to its simplicity, flexibility, and massive ecosystem. As of September 2024, Express 5.0.0 introduced modern async/await support and improved routing while maintaining backward compatibility for most middleware. The framework remains actively maintained, with the 4.x line receiving long-term support alongside the current 5.x releases.

Understanding REST principles is essential before coding. REST (Representational State Transfer) is an architectural style for networked applications that uses standard HTTP methods (GET, POST, PUT, DELETE) and status codes to manipulate resources identified by URLs. A RESTful API adheres to constraints like statelessness, cacheability, and a uniform interface, making it predictable and easy to consume.

Project Setup

Begin by initializing a new Node.js project and installing Express:

mkdir book-api && cd book-api
npm init -y
npm install express

For development convenience, also install nodemon for automatic restarts and dotenv for environment variable management:

npm install --save-dev nodemon dotenv

Create the project structure:

book-api/
├── src/
│   ├── controllers/
│   ├── middleware/
│   ├── routes/
│   ├── models/
│   └── app.js
├── .env
├── .gitignore
├── package.json
└── server.js

Basic Server Implementation

Create src/app.js to configure the Express application:

const express = require('express');
const app = express();

// Middleware to parse JSON bodies
app.use(express.json());

// Middleware to parse URL-encoded bodies (for form data)
app.use(express.urlencoded({ extended: true }));

// Basic route
app.get('/', (req, res) => {
  res.json({ message: 'Welcome to the Book API' });
});

module.exports = app;

Create src/server.js to start the server:

const app = require('./app');
const PORT = process.env.PORT || 3000;

app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

Update package.json to include a dev script:

"scripts": {
  "dev": "nodemon src/server.js",
  "start": "node src/server.js"
}

Run npm dev to start the server in development mode. Visiting http://localhost:3000 should return the welcome message.

Defining Routes and Controllers

REST APIs organize functionality around resources. For a book collection, typical endpoints include:

• GET /books — Retrieve all books
• GET /books/:id — Retrieve a specific book
• POST /books — Create a new book
• PUT /books/:id — Update a book
• DELETE /books/:id — Delete a book

Create src/controllers/bookController.js:

// In-memory store for demonstration (use a database in production)
let books = [];
let nextId = 1;

exports.getAllBooks = (req, res) => {
  res.json(books);
};

exports.getBookById = (req, res) => {
  const book = books.find(b => b.id === parseInt(req.params.id));
  if (!book) return res.status(404).json({ error: 'Book not found' });
  res.json(book);
};

exports.createBook = (req, res) => {
  const { title, author, publishedYear } = req.body;
  if (!title || !author) {
    return res.status(400).json({ error: 'Title and author are required' });
  }
  const newBook = {
    id: nextId++,
    title,
    author,
    publishedYear: publishedYear || null
  };
  books.push(newBook);
  res.status(201).json(newBook);
};

exports.updateBook = (req, res) => {
  const book = books.find(b => b.id === parseInt(req.params.id));
  if (!book) return res.status(404).json({ error: 'Book not found' });
  const { title, author, publishedYear } = req.body;
  if (title !== undefined) book.title = title;
  if (author !== undefined) book.author = author;
  if (publishedYear !== undefined) book.publishedYear = publishedYear;
  res.json(book);
};

exports.deleteBook = (req, res) => {
  const index = books.findIndex(b => b.id === parseInt(req.params.id));
  if (index === -1) return res.status(404).json({ error: 'Book not found' });
  books.splice(index, 1);
  res.status(204).send();
};

Create src/routes/bookRoutes.js:

const express = require('express');
const router = express.Router();
const bookController = require('../controllers/bookController');

router.get('/', bookController.getAllBooks);
router.get('/:id', bookController.getBookById);
router.post('/', bookController.createBook);
router.put('/:id', bookController.updateBook);
router.delete('/:id', bookController.deleteBook);

module.exports = router;

Register the routes in src/app.js:

const express = require('express');
const app = express();

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

const bookRoutes = require('./routes/bookRoutes');
app.use('/books', bookRoutes);

app.get('/', (req, res) => {
  res.json({ message: 'Welcome to the Book API' });
});

module.exports = app;

Essential Middleware

Production APIs benefit from middleware that enhances security, logging, and functionality.

CORS (Cross-Origin Resource Sharing)

If your API will be accessed from different domains (e.g., a frontend hosted separately), enable CORS:

npm install cors

const cors = require('cors');
app.use(cors()); // Allows all origins — configure restrictively in production

Security Headers

Use Helmet to set various HTTP headers for security:

npm install helmet

const helmet = require('helmet');
app.use(helmet());

Request Logging

Log incoming requests for debugging and monitoring:

app.use((req, res, next) => {
  console.log(`${req.method} ${req.path} - ${new Date().toISOString()}`);
  next();
});

Environment Variables

Store configuration like port, database URLs, and API keys in environment variables using dotenv. Create .env:

PORT=3000
NODE_ENV=development

Load them in src/server.js:

require('dotenv').config();
const app = require('./app');
const PORT = process.env.PORT || 3000;

app.listen(PORT, () => {
  console.log(`Server running in ${process.env.NODE_ENV} mode on port ${PORT}`);
});

Add .env to .gitignore to prevent committing sensitive data:

node_modules/
.env
.env.*

Error Handling

Implement centralized error handling to catch and format errors consistently. Add this middleware after all routes:

// 404 handler
app.use((req, res) => {
  res.status(404).json({ error: 'Route not found' });
});

// Global error handler
app.use((err, req, res, next) => {
  console.error(err.stack);
  const status = err.status || 500;
  res.status(status).json({
    error: {
      message: err.message || 'Internal Server Error',
      ...(process.env.NODE_ENV === 'development' && { stack: err.stack })
    }
  });
});

In controllers, pass errors to next(err) for asynchronous operations:

exports.createBook = async (req, res, next) => {
  try {
    // ... logic that might throw
  } catch (error) {
    next(error);
  }
};

Running in Production

For production, use a process manager like PM2 to keep the application running and handle clustering:

npm install -g pm2
pm2 start src/server.js --name book-api

Ensure NODE_ENV=production in your environment to disable development-specific logging and enable optimizations. Use a reverse proxy like Nginx for SSL termination, load balancing, and serving static files.

Testing the API

Test endpoints using tools like curl, HTTPie, or Postman:

# Get all books
curl http://localhost:3000/books

# Create a book
curl -X POST http://localhost:3000/books \
  -H "Content-Type: application/json" \
  -d '{"title":"The Great Gatsby","author":"F. Scott Fitzgerald","publishedYear":1925}'

# Get a specific book
curl http://localhost:3000/books/1

# Update a book
curl -X PUT http://localhost:3000/books/1 \
  -H "Content-Type: application/json" \
  -d '{"title":"Great Gatsby"}'

# Delete a book
curl -X DELETE http://localhost:3000/books/1

Common Best Practices

1. Use HTTP status codes correctly: 200 (OK), 201 (Created), 204 (No Content), 400 (Bad Request), 401 (Unauthorized), 403 (Forbidden), 404 (Not Found), 409 (Conflict), 500 (Internal Server Error).
2. Validate input: Always validate request data (body, params, query) before processing.
3. Keep controllers thin: Move business logic to service layers or models.
4. Use asynchronous handling: Prefer async/await with try/catch or promises; avoid blocking the event loop.
5. Secure sensitive data: Never hardcode secrets; use environment variables and secrets managers.
6. Enable compression: Use compression middleware (e.g., compression) to reduce response sizes.
7. Implement rate limiting: Protect against abuse with rate-limiter middleware.
8. Log effectively: Use structured logging (e.g., winston, pino) for production monitoring.
9. Document your API: Use OpenAPI/Swagger to generate interactive documentation.
10. Write tests: Implement unit and integration tests with frameworks like Jest and Supertest.

FAQ

What is the difference between Express 4.x and 5.x?

Express 5.x introduces native support for async/await in route handlers and middleware, eliminates certain deprecated APIs (like app.param(fn)), and improves error handling for asynchronous operations. The 4.x line remains under long-term support for stability-critical applications.

Do I need a database to build a REST API?

No — you can start with in-memory storage or flat files for learning or prototyping. For production, integrate a database (e.g., PostgreSQL, MongoDB) using an ORM (like Sequelize or Prisma) or a query builder.

How do I handle authentication in an Express API?

Common strategies include JSON Web Tokens (JWT), session-based authentication, or OAuth 2.0. Libraries like jsonwebtoken, passport, or express-jwt simplify implementation. Always use HTTPS in production to protect tokens.

Can I use TypeScript with Express?

Yes. Express officially supports TypeScript via @types/express. Initialize a TypeScript project with tsc --init, install type definitions, and rename .js files to .ts. Ensure esModuleInterop is enabled in tsconfig.json.

How should I structure a larger Express application?

As your API grows, adopt a layered structure: routes (handle HTTP), controllers (logic), services (business rules), models (data access), and middleware (cross-cutting concerns). Keep each file focused on a single responsibility.

Summary

Building a REST API with Node.js and Express involves setting up the server, defining RESTful routes, applying essential middleware, managing configuration via environment variables, handling errors gracefully, and following production best practices. By leveraging Express's minimalist core and Node.js's efficient runtime, developers can create scalable, maintainable APIs suitable for everything from small projects to enterprise systems. Remember to validate inputs, secure your endpoints, and monitor performance as your API evolves.

Sources

• MDN Web Docs: Express/Node introduction (CC-BY-SA)
• Express.js Documentation: Guide (MIT License)
• Node.js Documentation: About Node.js (MIT License)
• Wikipedia: Representational state transfer (CC-BY-SA)
• GitHub: Express repository (MIT License)

Why responsive design replaced the old approach

Before RWD became standard, developers talked about mobile web design. Sites served different experiences based on which device the visitor used — often through server-side browser sniffing that detected the browser type and served a stripped-down page to phones. Mobile devices used to be low-powered and some browsers couldn't handle CSS or full HTML, so this made sense at the time.

That's no longer the case. Modern phones handle the same technologies as desktop computers, and there are now many device shapes — laptops, tablets, phones, and watches. Instead of catering to a few screen sizes, you design defensively for common sizes plus unknowns. The goal stays the same as the old "mobile-friendly" approach: good layout, readable content, and reasonable performance on every device.

HTML is fluid by default

Plain HTML is inherently fluid. If you create a page with no CSS and resize the window, the browser automatically reflows the text to fit the viewport. The problem isn't wrapping — it's that a page's design breaks down without help: long lines of text on a wide monitor are hard to read, while a fixed-width design causes horizontal scrollbars on narrow screens and empty space on wide ones.

RWD solves this with three building blocks that Marcotte described in 2010: fluid grids, fluid images, and media queries.

Fluid grids and relative units

A fluid grid uses relative units (percentages, em, rem, viewport units like vw) instead of fixed pixel column widths, so the layout stretches and shrinks with the viewport. Modern CSS layout methods — Flexbox and CSS Grid — are inherently responsive and handle most of this for you, so you rarely hand-build percentage grids anymore.

Media queries: the condition-based switch

Media queries let you run tests about the device (for example, viewport width, orientation, or screen vs. print) and apply CSS selectively. They're initiated with the @media at-rule. Every stylesheet has an implicit media type of all unless you specify one, which means these two blocks are equivalent:

body { color: black; background-color: white; }

@media all { body { color: black; background-color: white; } }

You can target media types like print (useful to stop wasting ink on backgrounds) or screen, and you can add conditions on top — these are called media features. A common one is orientation:

@media (orientation: landscape) { /* landscape styles */ }
@media (orientation: portrait) { /* portrait styles */ }

For responsive design the most useful media feature is viewport width. min-width applies styles when the window is at least a certain width:

@media (min-width: 720px) { /* wide-screen styles */ }

max-width applies styles below a width, and you can write the same condition with range syntax:

@media (max-width: 400px) { /* narrow styles */ }
@media (width <= 400px) { /* equivalent */ }

You can combine conditions with and:

@media (min-width: 50em) and (max-width: 60em) { /* between 50em and 60em */ }

The point where a media feature condition becomes true is called a breakpoint. Best practice is to choose breakpoints based on your content — for instance, the width at which lines of text become uncomfortably long — rather than on specific device sizes, since popular device sizes change every release cycle. It's also recommended to use relative units rather than absolute sizes when defining breakpoints.

Mobile-first vs. desktop-first

A common approach is mobile-first: start with a single-column layout for narrow screens, then add media queries for wider screens only when you know there's enough room for a multi-column layout. This keeps the base experience simple and fast, and narrow devices never download unnecessary desktop-only layout rules.

The viewport meta tag

Mobile browsers once rendered desktop-width pages and squashed them to fit the phone screen, which made everything tiny. The viewport <meta> tag fixes this by telling the browser how wide the page should be. The standard declaration sets the page width to the device's CSS-pixel width:

<meta name="viewport" content="width=device-width, initial-scale=1">

The content attribute is a comma-separated list of key=value pairs, including width (or the special value device-width), height, initial-scale, maximum-scale, minimum-scale, and user-scalable. Avoid disabling zoom: setting user-scalable to no prevents people with low vision from reading the page, and WCAG requires at least 2× scaling (5× is the recommended target).

Responsive images

Text wraps automatically at the edge of the screen, but images have an intrinsic size and can overflow a narrow viewport. Several techniques keep them responsive:

Constrain with CSS. Use max-inline-size: 100% so an image never renders wider than its container, and block-size: auto so the browser preserves the aspect ratio while resizing:

img { max-inline-size: 100%; block-size: auto; }

Fit within an aspect ratio. If a design forces a different aspect ratio, object-fit: contain preserves the image's ratio (leaving empty space) and object-fit: cover crops it (filling the box). object-position changes where the crop focuses:

img { width: 100%; aspect-ratio: 2/1; object-fit: cover; object-position: top center; }

Hint the browser in HTML. Always include width and height attributes so the browser reserves the correct space and other content doesn't jump when the image loads (this prevents layout shift). Use loading="lazy" for images below the fold, and for a hero/LCP image use loading="eager" and optionally fetchpriority="high" to fetch it immediately:

<img src="hero.jpg" alt="Description" width="1200" height="800" loading="eager" fetchpriority="high">

Only set fetchpriority="high" on a truly vital image, because prioritizing one resource means the browser de-prioritizes another (such as a script or font file).

Deliver the right size. The srcset attribute complements (does not replace) src, giving the browser a set of candidate images and letting it choose the most appropriate one for the device and viewport.

Summary

Responsive web design is an approach, not a single technology: fluid grids and relative units let layouts adapt, media queries switch styles at content-driven breakpoints, the viewport meta tag makes mobile browsers treat pages at their real width, and fluid images with width/height hints prevent overflow and layout shift. The mindset is defensive — design for the known device sizes and be ready for the unknowns.

FAQ

What is the main difference between responsive and adaptive design?

Responsive design builds one fluid layout that responds to the screen using fluid grids, fluid images, and media queries. The older "mobile" approach — and some adaptive setups — served different layouts per device, sometimes via server-side browser detection. RWD renders well on any screen size with a single page, which is why it became the standard.

Do I need media queries for every breakpoint?

No. Start from a mobile-first single-column layout, then add media queries only where your content needs them — for example, where text lines get too long for comfortable reading. Breakpoints should be driven by content, not by the latest phone model, and mobile users should not download desktop-only layout rules.

Why does my mobile page look like a tiny desktop page?

That's the symptom before the viewport meta tag: mobile browsers rendered the desktop-width page and shrunk it, making text unreadably small. Adding <meta name="viewport" content="width=device-width, initial-scale=1"> makes the browser treat the page at the device's pixel width, so the layout reflows correctly on mobile.

How do responsive images prevent layout shift?

Images have an intrinsic size but text wraps, so a wide image can overflow or push content around. Setting the CSS rules above, giving every image width and height attributes so the browser reserves the space, and using loading="lazy" below the fold means the page keeps its layout instead of jumping when images load.

Should I disable pinch-to-zoom on mobile?

No. Setting user-scalable=no blocks zoom for people with low vision and is discouraged by accessibility guidance that requires at least 2× scaling. Keep the default zoom enabled and let the responsive layout do the work.

What rate limiting is and why you need it

In networking, rate limiting means controlling how many operations can be performed in a given period of time, usually to avoid overloading a system and causing performance degradation. A server might limit the number of requests it accepts from a single client in a time period, which not only optimizes overall performance but also mitigates attacks such as denial-of-service (DoS). Without rate limiting, a single misbehaving or malicious client can consume all of your API's resources and make it slow or unavailable for everyone else.

Rate limiting is commonly applied to public APIs, login and password-reset endpoints, and any route that triggers expensive work. It is often confused with throttling, which gradually slows a client down, and with debouncing, which coalesces rapid events. Rate limiting rejects requests outright once a ceiling is reached, which is the strictest of the three.

The 429 Too Many Requests response

When a client exceeds its allowed request rate, the server returns the HTTP status 429 Too Many Requests. According to RFC 6585, a Retry-After header may be included to indicate how long the client should wait before making a new request. The 429 status tells both the client and any tooling (caches, proxies, monitoring) that the request was deliberately refused because of rate limiting, not because the server is broken.

HTTP/1.1 429 Too Many Requests
Retry-After: 60

{
  "error": "rate_limit_exceeded",
  "message": "You have exceeded the limit of 100 requests per minute"
}

The client should read Retry-After and stop sending requests until that many seconds have passed, then retry. Well-behaved clients use this header to back off gracefully instead of hammering the API in a loop.

Rate limiting algorithms

There are several algorithms for deciding when to allow or block a request. Each trades simplicity, memory, and fairness differently.

Fixed window counter

The fixed window counter divides time into fixed buckets (for example, 100 requests per 60 seconds). Every request increments a counter for the current window; when the window expires, the counter resets. It is the simplest to implement and needs only one counter per key, but it has a known weakness: two consecutive windows are independent, so a client can make twice the limit in the instant around a window boundary. A burst of 100 requests at the end of one minute and 100 at the start of the next passes a 100-per-minute limit.

Sliding window log

The sliding window log records a timestamp for every request. To decide whether to allow a new request, the server counts how many timestamps fall inside the current moving window and rejects if the count exceeds the limit. It is the most accurate, because the window truly slides, but it stores every request timestamp, so memory grows with traffic. For a high-traffic endpoint this can become expensive.

Sliding window counter

The sliding window counter approximates a true rolling window with two integers: the count for the current window and the count for the previous one. It computes a weighted estimate of requests in the window by blending the two counters, giving smooth behavior that is close to a real sliding window at a small, constant memory cost. This makes it popular in production at scale, and it is what many library-based rate limiters use.

Token bucket

The token bucket models an explicit burst capacity plus a steady refill rate. A bucket holds up to capacity tokens; a token is added at a fixed rate, up to the capacity, and every request consumes one token. If the bucket is empty, the request is rejected. The token bucket allows short bursts (up to the full capacity) while enforcing a sustained average rate, which is ideal for APIs where clients legitimately burst, such as batch jobs or media streaming.

A common rule of thumb: token bucket when clients need to burst, sliding window counter when you want smooth, fair behavior under steady load, and fixed window when you need the simplest possible implementation.

Building a fixed window rate limiter in Node.js

You do not need a library to understand the mechanics. Here is a minimal fixed window limiter implemented as plain Node.js middleware for an Express app. It tracks one counter per IP address.

const WINDOW_MS = 60 * 1000; // 60 seconds
const MAX_REQUESTS = 100;

const hits = new Map(); // ip -> { count, resetAt }

function rateLimiter(req, res, next) {
  const ip = req.ip;
  const now = Date.now();
  let entry = hits.get(ip);

  if (!entry || now >= entry.resetAt) {
    entry = { count: 0, resetAt: now + WINDOW_MS };
    hits.set(ip, entry);
  }

  entry.count += 1;

  res.setHeader("X-RateLimit-Limit", String(MAX_REQUESTS));
  res.setHeader("X-RateLimit-Remaining", String(Math.max(0, MAX_REQUESTS - entry.count)));

  if (entry.count > MAX_REQUESTS) {
    res.setHeader("Retry-After", String(Math.ceil((entry.resetAt - now) / 1000)));
    return res.status(429).json({ error: "Too Many Requests" });
  }

  next();
}

app.use(rateLimiter);

This example stores state in a JavaScript Map in the process's memory. It works for a single Node.js instance, but it will not work correctly if you run multiple instances behind a load balancer, because each instance keeps its own counter. In a clustered or multi-instance deployment, use a shared store such as Redis so every instance reads the same counters.

Using a rate limiting library with Express

For production, a maintained library saves you from re-implementing edge cases such as IPv6 subnet handling, reliable header output, and pluggable stores. express-rate-limit is a widely used middleware for Express. Install it, configure a limiter, and attach it to your app or to specific routes.

import { rateLimit } from "express-rate-limit";

const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  limit: 100, // max 100 requests per window per IP
  standardHeaders: "draft-8", // emit the Ratelimit header
  legacyHeaders: false, // disable the older X-RateLimit- headers
});

app.use(limiter); // apply to the whole app
// or: app.use("/api/auth", limiter); // protect only sensitive routes

A shared store makes the limiter consistent across multiple instances:

import { RedisStore } from "rate-limit-redis";
// ... limiter = rateLimit({ store: new RedisStore(...), ... })

express-rate-limit supports custom stores like rate-limit-redis so the hit counts live in Redis instead of in a single process's memory. It also lets you customize the key generator (for example, keying on an authenticated user ID instead of the raw IP, which matters behind a proxy or NAT where many users share one IP).

Limiting a token bucket with a library

Libraries such as bottleneck and rate-limiter-flexible provide token bucket and other rate limiter classes that you can drive from your own code rather than as Express middleware. The general shape is: create a limiter with a capacity and a refill rate, then consume a token before doing work.

import { RateLimiterRedis } from "rate-limiter-flexible";

const limiter = new RateLimiterRedis({
  storeClient: redisClient,
  points: 100, // capacity (burst)
  duration: 60, // refill per 60 seconds
});

async function handle(req, res) {
  try {
    await limiter.consume(req.ip);
    // proceed with the request
  } catch {
    res.status(429).json({ error: "Too Many Requests", retryAfter: 60 });
  }
}

The consume call either resolves (allow) or rejects (limit exceeded). You decide what to do on rejection, typically returning 429 with a Retry-After header.

Where to apply rate limiting

Rate limiting is not one-size-fits-all. Different routes need different limits, and keying on the right identifier matters.

• Sensitive routes (login, password reset, signup) should have aggressive limits — for example, a handful of attempts per minute per IP or per account — because they are prime targets for credential stuffing and brute force.
• Read-only public routes can allow much higher limits.
• Key by user identity, not raw IP, when possible. Behind a proxy, NAT, or CDN many real users share a single visible IP, so limiting purely by IP can block legitimate users. If requests are authenticated, key on the user ID or API key.
• Servers behind a CDN such as Cloudflare can enforce rate limiting at the edge, before traffic reaches your origin. This offloads protection and lets you set custom limits on specific paths — a good complement to in-application limiting.

Best practices

• Always return 429 with a Retry-After header so well-behaved clients back off automatically and you do not silently drop traffic.
• Expose rate limit headers (X-RateLimit-Limit, X-RateLimit-Remaining, or the newer Ratelimit header) so clients and tooling can predict and respect the limit.
• Use a shared store for multi-instance deployments. In-memory counters are per-process and become inaccurate the moment you scale horizontally.
• Log and monitor 429 responses. A spike in 429s can reveal a bug in your client code, a misbehaving crawler, or an attack in progress.
• Pick the key carefully. Limiting by IP is simple but coarse; limiting by authenticated user or API key is far more fair behind proxies.
• Do not rate limit only as an afterthought — decide your limits, store, and headers up front so clients are built to respect them from the start.

FAQ

What is rate limiting in Node.js?

Rate limiting in Node.js is the practice of controlling how many requests a single client can make to your server in a period of time. When a client exceeds its allowance, the server returns the HTTP status 429 Too Many Requests. It is implemented either with a dedicated library or with a custom middleware that tracks request counts per client key (usually IP address or user ID).

What is the difference between rate limiting and throttling?

Rate limiting rejects requests once a client passes a maximum count in a time window, returning 429. Throttling instead slows a client down by introducing delay, allowing the request eventually but at a reduced pace. Debouncing is a third related technique that coalesces rapid repeated events into fewer calls. Rate limiting is the strictest of the three.

Which rate limiting algorithm should I use?

Use a token bucket when clients legitimately need to burst, a sliding window counter when you want smooth, fair behavior under steady load, and a fixed window counter when you need the simplest possible implementation. Sliding window log is most accurate but stores every request timestamp, so it suits low-traffic endpoints only. For most production Node.js APIs, a sliding window counter or token bucket is a good default.

Does rate limiting work with multiple Node.js instances?

Only if you use a shared store. An in-memory counter is local to a single process, so if you run several instances behind a load balancer each one would track its own counts, and the effective limit would be multiplied by the number of instances. Libraries such as express-rate-limit and rate-limiter-flexible accept a shared store such as Redis to keep counts consistent across all instances.

What protobuf is

Protocol Buffers are a free and open-source cross-platform data format developed by Google for serializing structured data. Google released protobuf publicly in 2008, though the internal project dates back to 2001. The design goals emphasized simplicity and performance — protobuf was built to be smaller and faster than XML, and it largely succeeded.

A schema for protobuf associates data types with field names, using integer tags to identify each field. The schema lives in a .proto file written in protobuf's Interface Description Language (IDL). You define your messages and services once, then run the protoc compiler to generate source code in your language of choice — C++, Java, Python, JavaScript, Go, PHP, Dart, Ruby, and many others are officially supported.

Once compiled, the generated code provides methods to serialize (marshall) structured data to a binary wire format and deserialize (unmarshall) it back into native language objects. The same .proto definition produces compatible code across all supported languages, so a Java program can serialize data that a Python program deserializes without either side knowing about the other.

The .proto file format

A protobuf message looks like this:

syntax = "proto3";

message Person {
  string name = 1;
  int32 age = 2;
  optional string email = 3;
}

Each field has a tag — a small integer written after the equals sign. The tag, not the field name, is what appears on the wire. When you serialize a Person, the output contains tag 1 for the name, tag 2 for the age, and so on. This is why protobuf messages are compact: field names never appear in the output, only their numeric identifiers.

Field tags are also the mechanism for forward and backward compatibility. If you add a new field with a new tag to a message, old code that does not know about that field simply ignores it during deserialization. Old data serialized by old code still decodes correctly in new code as long as field tags do not change. This tag-based evolution model lets you extend schemas over time without breaking existing clients or servers.

Protobuf supports several field types: string, int32, int64, float, double, bool, bytes, and nested message types. Collections use repeated (which maps to arrays or lists depending on the language). Maps use map<key_type, value_type>. Enums are supported with named values. The oneof construct lets a message hold at most one of a group of fields.

Binary wire format

The canonical protobuf serialization is binary. Unlike JSON or XML, a protobuf binary payload is not human-readable — but it is highly efficient. Field tags and wire types are encoded in a compact varint format; strings and bytes include their length prefix. A small integer tag takes 1 byte; a 32-bit integer takes 4 bytes; a 64-bit integer takes 8 bytes. No quotes, no field names, no whitespace.

Google also defines a text format for debugging and testing. This format is self-describing — it includes field names — but it loses the forward/backward compatibility guarantees of the binary format. The protobuf documentation explicitly recommends using the text format only for human editing and debugging, not for production data interchange.

Schema evolution and compatibility

Protobuf's compatibility story is one of its strongest features, but it comes with rules. Not all schema changes are safe:

Wire-safe changes (safe across the binary wire) include adding new fields, renaming fields (the wire only sees the tag, not the name), changing a field between 32-bit and 64-bit integer variants, and converting between optional and a oneof group. Old code and new code can communicate across these changes without data loss.

Wire-unsafe changes include changing a field's type (e.g., string to bytes), changing between a map and its equivalent repeated key-value message, or adding a required field. These break compatibility and should be avoided.

The ProtoJSON specification adds further nuance: when converting protobuf to JSON for API payloads, certain changes that are wire-safe in binary become unsafe in JSON because field names appear in the JSON output. Renaming a field that is also exposed in a JSON API will break clients that use the old name.

Protobuf vs JSON

JSON remains the dominant format for web APIs and wherever human-readability matters. Protobuf wins in specific scenarios:

• Performance-critical internal services: gRPC at Google powers nearly all inter-machine communication. Protobuf's binary encoding and code generation make it faster than JSON parsing, especially for high-throughput services.
• Cross-language contracts: defining a .proto schema once and generating native code for 10 languages eliminates the manual boilerplate of writing encode/decode logic in each language.
• Long-term data storage: protobuf's schema and tag-based evolution make it a solid choice for binary storage formats where schema migration matters.

JSON wins when:

• Human debugging is frequent: reading {"name":"Alice","age":30} requires no tooling; reading a protobuf binary requires the protoc decode plugin.
• Browser compatibility is required: browsers have no native protobuf support. JSON works everywhere over HTTP without code generation.
• Ecosystem tooling matters: JSON integrates with logging systems, observability platforms, and developer tools without friction.

ProtoJSON: protobuf on JSON

For environments where binary is not practical, protobuf defines ProtoJSON — a canonical mapping between protobuf messages and JSON. Field names map to camelCase JSON keys, enum values map to their string names by default, Timestamp and Duration use RFC 3339 strings, and BytesValue uses base64-encoded strings.

ProtoJSON is not simply JSON that happens to contain protobuf data — it is a deliberate specification with its own rules. int64 fields are quoted in JSON to avoid JavaScript's 53-bit integer precision limit. Well-known types like google.protobuf.Any use type URL strings. ProtoJSON is useful for REST APIs that want protobuf-style schema evolution but need JSON for client compatibility.

Limitations

Protobuf is not a universal solution. Key limitations:

• Not self-describing: a raw protobuf payload cannot be interpreted without its .proto schema. This differs from JSON or XML, which are human-readable without a schema. In practice this means schema files must be distributed alongside the code.
• Not streamable: protobuf assumes an entire message fits in memory. For streaming large datasets, you need to split data into multiple messages or choose a different format.
• No built-in compression: protobuf payloads compress well with gzip or zstd, but there is no default compression as there might be in some protocols.
• Not suitable for large scientific data: formats like FITS have less overhead for multi-dimensional floating-point arrays.
• No single specification: unlike JSON or XML, protobuf has no IETF standard. The canonical implementation lives at protobuf.dev, and while the format is stable, it is controlled by Google.

Protobuf and gRPC

Protocol Buffers are not only a data format — they are also the foundation of gRPC, Google's RPC framework. A gRPC service is defined in a .proto file with both message definitions and service declarations. The protoc compiler, with a gRPC plugin, generates both the data serialization code and the RPC client and server stubs.

This tight integration is why protobuf appears in this site's guide on REST vs gRPC and GraphQL vs gRPC. gRPC uses protobuf's binary efficiency and code generation to deliver low-latency, type-safe RPC calls over HTTP/2. If you are evaluating gRPC, you are implicitly adopting protobuf as your data format.

Verdict

Protocol Buffers are the right choice when you control both ends of the communication, need maximum performance and minimal payload size, or want a language-neutral data contract that evolves safely over time. The .proto schema as a single source of truth eliminates a class of encode/decode bugs that plague multi-language codebases.

Use JSON for public-facing APIs, browser clients, rapid prototyping, and anywhere human-readable payloads simplify debugging. Protobuf and JSON solve the same problem at opposite ends of the trade-off spectrum — choosing between them is a matter of whether your system values machine efficiency or human readability more.

FAQ

What is the difference between protobuf and JSON?

Protobuf serializes data to a compact binary format; JSON is human-readable text. Protobuf messages are smaller than equivalent JSON and parse faster, but you cannot read them without a .proto schema and the protoc compiler. JSON requires no schema and works natively in browsers.

Can protobuf be used in web browsers?

No. Browsers have no native protobuf support. You can use protobuf in a Node.js backend or in a web app that sends data to a server, but the browser itself cannot serialize or deserialize protobuf without a JavaScript library, and that library cannot use the protoc-generated code path that other languages use. For browser APIs, JSON is the practical choice.

How does protobuf achieve forward and backward compatibility?

Protobuf uses numeric field tags on the wire instead of field names. When you add a new field with a new tag, old code that does not recognize that tag simply skips it. Old data serialized by old code still decodes correctly because the old fields remain at their original tags. As long as you never reuse or remove field tags, the format is forward- and backward-compatible.

Is protobuf faster than JSON?

Yes, in most benchmarks. Binary encoding eliminates the overhead of parsing strings into numbers and handling Unicode. Code generation means serializers and deserializers are optimized at compile time, not interpreted at runtime. The performance advantage is most pronounced with large or frequent messages; for small, occasional payloads the difference is negligible.

What programming languages support protobuf?

Google officially supports C++, Java, Python, Go, Objective-C, C#, JavaScript, Ruby, PHP, and Dart. Community implementations exist for over 20 additional languages including Rust, Swift, Kotlin, MATLAB, and Ada. The official implementation at github.com/protocolbuffers/protobuf is the canonical reference.

What Prometheus and Grafana do

Prometheus is a systems monitoring and alerting toolkit originally built at SoundCloud in 2012 and now a graduated Cloud Native Computing Foundation project. It collects metrics as time-series data, where each data point is a numeric value recorded at a specific timestamp and tagged with key-value pairs called labels. Prometheus uses a pull model: the server periodically scrapes an HTTP endpoint that your application exposes, instead of receiving pushes from your code.

Grafana is an open-source analytics and visualization platform. It connects to Prometheus (and dozens of other data sources), lets you write queries in PromQL (Prometheus's query language), and renders the results as graphs, gauges, tables, and alerts on dashboards. In short: Prometheus stores the numbers, Grafana shows them to humans.

Both are battle-tested at scale: Prometheus scrapes millions of time series per server, and Grafana dashboards sit behind on-call screens at companies of every size. For a Node.js service, the typical setup looks like this:

Node.js app --(/metrics)--> Prometheus --(PromQL)--> Grafana
                                  |
                                  v
                              Alertmanager -> Slack / Email / Pager

Instrumenting the application

The most widely used Node.js client is siimon/prom-client, an Apache-2.0 library that supports counters, gauges, histograms, and summaries. Install it next to Express:

npm install prom-client express

Create a metrics module that registers default Node.js metrics (event-loop lag, memory, GC, active handles) and exposes a few custom application metrics:

// src/metrics.js
const client = require("prom-client");

const register = new client.Registry();
client.collectDefaultMetrics({ register });

// Custom: total HTTP requests, labelled by method/route/status
const httpRequestsTotal = new client.Counter({
  name: "http_requests_total",
  help: "Total HTTP requests",
  labelNames: ["method", "route", "status"],
  registers: [register],
});

// Custom: request latency histogram
const httpRequestDurationSeconds = new client.Histogram({
  name: "http_request_duration_seconds",
  help: "HTTP request duration in seconds",
  labelNames: ["method", "route", "status"],
  buckets: [0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10],
  registers: [register],
});

module.exports = {
  register,
  httpRequestsTotal,
  httpRequestDurationSeconds,
};

Wire the metrics into an Express app. The key principle: a middleware measures every request, then exposes /metrics as a plain HTTP endpoint that Prometheus can scrape:

// src/app.js
const express = require("express");
const {
  register,
  httpRequestsTotal,
  httpRequestDurationSeconds,
} = require("./metrics");

const app = express();

app.use((req, res, next) => {
  const end = httpRequestDurationSeconds.startTimer();
  res.on("finish", () => {
    // Use req.route?.path to avoid label cardinality explosion
    const route = req.route?.path || req.path;
    const labels = {
      method: req.method,
      route,
      status: String(res.statusCode),
    };
    end(labels);
    httpRequestsTotal.inc(labels);
  });
  next();
});

app.get("/", (_req, res) => res.json({ ok: true }));
app.get("/users/:id", (req, res) => res.json({ id: req.params.id }));

// Scrape endpoint — Prometheus polls this URL
app.get("/metrics", async (_req, res) => {
  res.set("Content-Type", register.contentType);
  res.end(await register.metrics());
});

app.listen(3000, () => console.log("listening on 3000"));

Two important guardrails from the Prometheus best-practices guide. First, never use a raw URL path like /users/42 as a label value — every user ID becomes a new time series and the database grows forever. Use req.route.path (the pattern /users/:id) instead. Second, register collectDefaultMetrics once at startup so event-loop lag, memory, GC, and Node.js version are recorded automatically.

You can verify the endpoint locally before going further:

$ curl -s http://localhost:3000/metrics | head
# HELP http_requests_total Total HTTP requests
# TYPE http_requests_total counter
http_requests_total{method="GET",route="/",status="200"} 3
# HELP process_cpu_user_seconds_total Total user CPU time spent in seconds
# TYPE process_cpu_user_seconds_total counter
process_cpu_user_seconds_total 0.42

Running Prometheus and Grafana with Docker Compose

The simplest way to run the rest of the stack is a single Compose file. Create monitoring/compose.yaml:

services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prom-data:/prometheus
    ports:
      - "9090:9090"
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    volumes:
      - grafana-data:/var/lib/grafana
    ports:
      - "3001:3000"
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=admin
    depends_on:
      - prometheus
    restart: unless-stopped

volumes:
  prom-data:
  grafana-data:

The Prometheus configuration tells the server what to scrape and how often. Create monitoring/prometheus.yml:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "nodejs-app"
    metrics_path: /metrics
    static_configs:
      - targets: ["host.docker.internal:3000"]
        labels:
          service: "api"

host.docker.internal is the special hostname Docker Desktop (and Docker on Linux 20.10+) provides so a container can reach the host machine. On plain Linux without that, replace it with the host's LAN IP. Start the stack:

$ docker compose up -d

Then point Prometheus at your application: if you are running the Express app on the host on port 3000, Prometheus will scrape it every 15 seconds. Verify on the Prometheus UI at http://localhost:9090/targets — the nodejs-app job should appear in green "UP" state.

Adding the Prometheus data source in Grafana

Grafana listens on http://localhost:3001 (mapped from container port 3000). The first login uses admin / admin and asks you to set a new password. To add Prometheus as a data source:

1. Click the cogwheel in the sidebar and choose Data Sources.
2. Click Add data source and select Prometheus.
3. Set URL to http://prometheus:9090 (the Compose service name resolves inside the default network).
4. Click Save & Test — Grafana will tell you the data source is working.

The Prometheus data source has shipped with Grafana since 2.5.0 in 2015, so no plugin install is needed.

Building your first dashboard

Grafana dashboards are made of panels. Each panel runs a PromQL query against the data source and renders a visual. The metrics you registered earlier let you build the four panels every API dashboard needs.

1. Request rate (per-second, by status code). Counts how many requests are landing on your service and groups them by HTTP status. PromQL uses rate to convert a counter into requests-per-second:

sum by (status) (rate(http_requests_total[1m]))

2. Error rate (5xx fraction). What fraction of responses are server errors? A rising line here wakes someone up:

sum(rate(http_requests_total{status=~"5.."}[5m]))
  /
sum(rate(http_requests_total[5m]))

3. Latency percentiles. A histogram is observed into buckets; histogram_quantile reconstructs a percentile across the buckets. The 95th and 99th percentiles tell you what most users and the slowest users actually experience:

histogram_quantile(0.95,
  sum by (le) (rate(http_request_duration_seconds_bucket[5m]))
)

4. Event-loop lag. Default Node.js metrics include nodejs_eventloop_lag_seconds. A long event loop is the most common reason a Node.js service "feels slow" even when CPU is low:

nodejs_eventloop_lag_seconds

Add a graph panel for each query, choose the Prometheus data source, and lay the four panels out in a 2×2 grid. The result is a real-time picture of your service: traffic, errors, latency, and runtime health.

Alerting with Prometheus

Dashboards show what happened. Alerts tell you something needs attention right now. Prometheus evaluates alert rules — PromQL expressions with thresholds — on the same evaluation_interval (15s by default). Create monitoring/alerts.yml:

groups:
  - name: api
    rules:
      - alert: HighErrorRate
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m]))
            /
          sum(rate(http_requests_total[5m]))
            > 0.05
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Error rate above 5% for 2m"
          description: "API is returning errors on more than 5% of requests."

      - alert: HighLatencyP95
        expr: |
          histogram_quantile(0.95,
            sum by (le) (rate(http_request_duration_seconds_bucket[5m]))
          ) > 1
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "p95 latency over 1s"

The for: 2m clause requires the condition to hold for two minutes before the alert fires — without it, a single bad request would page someone. Add rule_files: ["alerts.yml"] to prometheus.yml and reload Prometheus with curl -X POST http://localhost:9090/-/reload. Alerts in firing state appear at http://localhost:9090/alerts. To turn those alerts into Slack messages or pages, point Prometheus at an Alertmanager instance with the appropriate webhook configuration.

Production hardening

The setup above is enough to learn the stack. For a real deployment, three things matter most.

Retention. Prometheus stores data on local disk. The default retention is 15 days. For longer history, use a remote-write adapter to ship data to Grafana Cloud, Thanos, or a managed service. Prometheus is designed for reliability during incidents, not for cheap long-term storage.

High-cardinality labels. A label like user_id or full URL path creates a new time series for every value and is the single most common Prometheus outage. Always prefer coarse labels (route, status, method) over raw request data. The route from req.route.path is the safe choice.

Security. Lock down /metrics behind an internal network, a reverse proxy with basic auth, or a service mesh. Do not expose it to the public internet — it leaks the shape of your traffic and can be abused for fingerprinting. Prometheus's own ports should also be inside a private network.

FAQ

Why use Prometheus instead of a SaaS like Datadog?

Prometheus is open source, vendor-independent, and free. It runs as a single static binary with no external dependencies, so you keep working during the kind of partial outages where hosted services sometimes degrade. The trade-off is that you operate the storage and dashboards yourself. For small teams that prefer managed, the same metrics flow into Grafana Cloud, Datadog, or New Relic with minimal change.

Do I need Grafana? Can I just use Prometheus's built-in UI?

Prometheus ships with a basic expression browser at /graph for ad-hoc queries. For ongoing work — dashboards, alerting visualizations, sharing with non-engineers — Grafana is the standard. You can also import hundreds of community dashboards from grafana.com/dashboards to skip writing panels by hand.

What is the difference between a counter, gauge, and histogram?

A counter is a value that only goes up (total requests, total errors). A gauge is a value that can go up or down (current memory use, active connections). A histogram samples observations into buckets so you can compute percentiles and averages later (request duration, response size). For request rate, use a counter; for "how much memory is in use right now", use a gauge; for "what was the 95th-percentile latency over the last 5 minutes", use a histogram.

How do I avoid running out of disk on Prometheus?

The two levers are the --storage.tsdb.retention.time flag (default 15 days) and the --storage.tsdb.retention.size flag. For long-term storage, the recommended pattern is Prometheus locally for fast recent queries plus a remote-write target (Thanos, Mimir, Grafana Cloud) for the historical archive.

Can I monitor more than just Node.js with this stack?

Yes. Prometheus is language-agnostic — any service that exposes a /metrics endpoint in the Prometheus format can be scraped. For databases, queues, and operating systems, use exporters: node_exporter for Linux host metrics, redis_exporter for Redis, postgres_exporter for PostgreSQL. The Grafana dashboard stays the same; you just add more jobs to prometheus.yml.

OAuth is authorization, not authentication

The most common confusion is thinking OAuth is a login system. OAuth 2.0 is an authorization framework: it decides what a client is allowed to do with the user's resources. It does not, by itself, tell the client who the user is. To add identity, OpenID Connect (OIDC) builds an authentication layer on top of OAuth 2.0 and issues an ID token alongside the access token. When you log in with a third-party account, the workflow you experience is usually OAuth for authorization combined with OIDC for authentication.

The four roles

RFC 6749 defines four roles that every OAuth interaction involves:

• Resource owner — the entity that can grant access to a protected resource. When it is a person, this is the end-user.
• Resource server — the server that hosts the protected resources and accepts authorized requests using access tokens.
• Client — the application making requests on behalf of the resource owner, with its authorization. The client can be a web app, a mobile app, or a server-to-server service.
• Authorization server — the server that authenticates the resource owner, obtains authorization, and issues access and refresh tokens.

The resource owner's credentials are used only at the authorization server — never at the client. That separation is the core security win of OAuth.

Tokens: access tokens and refresh tokens

An access token is the credential a client presents to the resource server to access protected resources. It represents the authorization granted by the resource owner and is typically short-lived. A refresh token lets the client obtain a new access token after the current one expires, without forcing the user to log in again. Refresh tokens are sensitive: if they leak, an attacker can mint new access tokens indefinitely, so they must be stored securely and protected like secrets.

Bearer tokens, defined in RFC 6750, are the most common access token type. A bearer token is used by the client by putting it in the Authorization: Bearer <token> header of an HTTP request. Because anyone who holds the token can use it, bearer tokens require HTTPS to protect them in transit and must never be embedded in query strings, browser history, or logs.

The main grant types

RFC 6749 defines four grant types, each for a different client and use case:

• Authorization code — the recommended flow for web and native apps that can keep a secret. The client redirects the user to the authorization server, the user logs in and consents, and the server returns an authorization code to the client. The client then exchanges that code for tokens at the token endpoint.
• Implicit (deprecated) — originally designed for browser-only apps. It returned the access token directly in the URL fragment. Because the token leaks through browser history and referrers, RFC 9700 formally deprecates this flow.
• Resource owner password credentials (ROPC) — the client collects the user's password directly. The user's credentials end up in the client's hands, which defeats the point of delegation and breaks with MFA and passkeys, so RFC 9700 strongly discourages it.
• Client credentials — used for server-to-server communication where there is no user. The client authenticates with its own credentials and gets a token for a consented scope.

A fifth flow, the device authorization grant (RFC 8628), handles login on devices without browsers, like smart TVs and CLI tools.

The recommended flow: authorization code with PKCE

The current baseline, from RFC 9700, is the authorization code flow combined with Proof Key for Code Exchange (PKCE, RFC 7636):

1. The client creates a random secret called a code verifier and derives a code challenge from it (usually a SHA-256 hash).
2. It redirects the user to the authorization server's /authorize endpoint, sending the challenge, the requested scope, and a state value.
3. The user logs in and consents. The server redirects back to the client with an authorization code (and echoes the state).
4. The client exchanges the code for tokens at the /token endpoint, sending the code verifier along with it.
5. The server verifies that the verifier matches the original challenge, then issues an access token (and optionally a refresh token).

PKCE binds the authorization request to the token request. Even if an attacker intercepts the authorization code, they cannot exchange it because they do not hold the code verifier. This prevents authorization-code injection and some CSRF attacks, which is why PKCE is now recommended for all client types, not just mobile apps.

How to secure an OAuth implementation

Best current practice is codified in RFC 9700 (BCP 212, published January 2025). The key requirements are:

• Use the authorization code flow with PKCE for everything, including single-page and mobile apps. Do not use the implicit grant.
• Do not use the resource owner password credentials grant. Prefer the authorization code flow.
• Match redirect URIs exactly. The server compares the redirect_uri in each request against the exact registered value — no pattern matching, no wildcards, no prefix rules — so an attacker cannot redirect the authorization code to their own site.
• Use the state parameter as an anti-CSRF token and validate it on return.
• Protect tokens in transit and at rest. Bearer tokens travel over HTTPS only and are stored server-side in secure storage, never in URLs or logs.
• Keep refresh tokens separate and protected, and revoke them when no longer needed.
• Where supported, use sender-constrained tokens such as mutual TLS (RFC 8705) or DPoP (RFC 9449) so a stolen token cannot be replayed.

FAQ

What is the difference between OAuth and OpenID Connect?

OAuth 2.0 is an authorization framework that lets a client access a user's resources with limited, scoped tokens. It does not identify the user on its own. OpenID Connect (OIDC) is an authentication layer built on top of OAuth 2.0 that adds an ID token carrying the user's identity, so a client can both verify who the user is and access their resources.

What are the four roles in OAuth 2.0?

The resource owner (the user who grants access), the resource server (which hosts the protected resources), the client (the application requesting access on the user's behalf), and the authorization server (which authenticates the user and issues tokens). The user's credentials are only ever shared with the authorization server, never with the client.

What is the difference between an access token and a refresh token?

An access token is the short-lived credential a client presents to get protected resources. A refresh token is a longer-lived credential the client exchanges for new access tokens after the current one expires, letting a session continue without asking the user to log in again.

Why is the implicit flow deprecated?

The implicit flow returned the access token directly in the URL fragment, where it leaked through browser history, referrers, and logs, and it could not use refresh tokens. RFC 9700 (BCP 212) deprecates it in favor of the authorization code flow with PKCE, which modern browsers and CORS fully support.

Does OAuth allow a website to see my password?

No. With OAuth, you authenticate only at the authorization server. Your password never reaches the third-party client; the client receives a scoped token that lets it access only the resources you approved.

What a reverse proxy does

On the Internet, a forward proxy sits in front of a group of clients and forwards their requests to servers on their behalf. A reverse proxy does the opposite: it sits in front of one or more origin servers, intercepts requests from clients at the edge, and forwards them to the right back-end. To clients, the reverse proxy is the only server they ever contact.

This setup is useful for several reasons:

Nginx is an open-source web server released under the 2-clause BSD license, and it handles all of the above. Because it uses an event-driven architecture, it can handle more than 10,000 simultaneous connections with a low memory footprint, which is why it is so widely used as a reverse proxy and load balancer.

Prerequisites

Before you start you need:

• A Linux server (Ubuntu/Debian commands are shown below) running Nginx.
• At least one back-end application listening on a port (for example a Node.js or Python app on http://127.0.0.1:3000). The back end does not need to be publicly reachable — the reverse proxy talks to it on the local network.
• A domain name and, for TLS, a way to get a certificate (Let's Encrypt free certificates work well with Nginx).

Installing Nginx

On Debian or Ubuntu, install with:

sudo apt update
sudo apt install nginx

When it finishes, Nginx starts automatically and serves a default page on port 80. Check it is running with:

sudo systemctl status nginx

On most distributions the config lives in /etc/nginx/nginx.conf, with site configs in /etc/nginx/sites-available/ and symlinked into /etc/nginx/sites-enabled/. You can also put small config snippets in /etc/nginx/conf.d/.

A minimal reverse proxy config

The core directive that makes Nginx a reverse proxy is proxy_pass. It tells Nginx which back-end server to forward a request to. A minimal server block looks like this:

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://127.0.0.1:3000;
    }
}

With this config, a request to http://example.com/ is forwarded to http://127.0.0.1:3000/, where your Node.js or Python app runs. Clients only ever see example.com; they never learn that the app lives on port 3000.

proxy_pass can target any upstream: a local port, another server on your network, or a Unix socket for a service like the PHP-FPM application server.

Forwarding the right HTTP headers

By default Nginx forwards the request but the back-end app does not know the original client's IP or protocol, because to the back end every request looks like it comes from the proxy (usually 127.0.0.1). You fix this by forwarding the standard proxy headers:

location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

What each header does:

• Host $host — tells the back end which hostname the client asked for. Without this, apps that route by domain name (virtual hosts) break.
• X-Real-IP $remote_addr — the client's real IP address.
• X-Forwarded-For — the full chain of client IPs the request passed through. Many apps read this to get the real client IP.
• X-Forwarded-Proto $scheme — whether the original request was HTTP or HTTPS. Apps use this to build correct absolute links and to know if the request arrived over TLS.

Note that these headers are not inherently trustworthy: a client can set a fake X-Forwarded-For. Only trust the values your own proxy adds. $remote_addr in Nginx is the IP Nginx actually saw, so X-Real-IP is usually the one to log and trust.

Load balancing with the upstream directive

To spread requests across several back-end instances, define an upstream block and point proxy_pass at it by name:

upstream app_servers {
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://app_servers;
    }
}

By default Nginx uses round-robin — each new request goes to the next server in the list. You can change the strategy:

• least_conn — sends each request to the server with the fewest active connections; good for uneven request durations.
• ip_hash — always sends the same client IP to the same back-end; useful when an app keeps per-user state in memory.

upstream app_servers {
    least_conn;
    server 127.0.0.1:3000;
    server 127.0.0.1:3001;
}

If one back-end goes down, Nginx notices and stops routing new requests to it (inactive servers are marked down), so a single failing instance does not take the whole site with it.

WebSocket and long-lived connections

Most modern apps use upgrade headers for WebSockets or streaming. You forward them like this:

location / {
    proxy_pass http://127.0.0.1:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
}

proxy_http_version 1.1 is required because HTTP/1.1 is needed for the Upgrade and Connection headers to work correctly with a proxy. Many guides add this block to application routes that stream data (SSE, WebSockets, gRPC-Web).

Adding TLS with a free certificate

HTTPS is not optional for a production site. The easiest path is Let's Encrypt via certbot:

sudo apt install certbot python3-certbot-nginx
sudo certbot --nginx -d example.com -d www.example.com

Certbot edits your Nginx config to add the certificate and redirects HTTP to HTTPS. The result is a server block that terminates TLS and forwards plain HTTP to your back end:

server {
    listen 443 ssl;
    server_name example.com;

    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

One important detail for apps behind TLS: because the connection between Nginx and your back end is now plain HTTP, your app will see X-Forwarded-Proto: https. If your app needs to know the original protocol (to generate secure links or decide whether to set a Secure cookie), it must read X-Forwarded-Proto — and you must make sure the app trusts that header only from your own proxy.

Checking and reloading the config

Nginx config is strict: one syntax error and the service fails to start. Always test before reloading:

sudo nginx -t

If the test passes, reload Nginx without dropping connections:

sudo systemctl reload nginx

nginx -t validates the config; reload applies it gracefully. This is the safe loop for every config change.

Common pitfalls

• The proxy is a security surface. Since every public request hits Nginx, apply rate limits, keep it patched, and don't expose debug endpoints through it.
• Forgotten proxy headers. An app that sees 127.0.0.1 as every client IP and generates HTTP links behind TLS is almost always missing the X-Forwarded-* headers above.
• Trusting client-supplied headers. Letting arbitrary clients set X-Forwarded-For unchecked lets attackers spoof their IP in your logs. Only append headers from the proxy; don't blindly trust the whole chain.
• One server block per site. Put each site in its own server block with its own server_name; the listen 80 default_server block catches requests for domains you did not configure.
• Reload, not restart. systemctl restart nginx drops active connections; reload does not.

When you do not need a reverse proxy

If you have a single static site and no rolling deployments, a plain web server without proxying is enough. Add a reverse proxy when you: run a dynamic app on a local port, run several apps on one machine (each gets its own server_name or path), need load balancing across instances, want to centralize TLS and caching, or want to hide your origin server. For many cloud setups, managed providers (Cloudflare Pages, load balancers, API gateways) already give you reverse-proxy behavior, so you only operate Nginx yourself when you are self-hosting.

FAQ

Is Nginx a web server or a reverse proxy?

Both. Nginx originally served static files, but it is equally capable as a reverse proxy, load balancer, HTTP cache and mail proxy. In practice the same proxy_pass configuration turns Nginx into a reverse proxy in front of one or more back-end servers.

What is the difference between a reverse proxy and a load balancer?

A load balancer distributes traffic across several back-end instances; a reverse proxy forwards traffic from clients to back-end servers and can add TLS, caching, compression and security. Because Nginx does both, the two roles often blur — in Nginx you define upstream blocks for load balancing, which is one feature inside the broader reverse-proxy setup.

Do I need a certificate on both Nginx and my back-end app?

No. With Nginx terminating TLS, the public connection is HTTPS while the connection from Nginx to your back-end is often plain HTTP on a local port. That is normal and the standard setup for a single reverse proxy. Your app must then read X-Forwarded-Proto to know the original request was HTTPS.

How do I make my app see the real client IP?

Forward the proxy headers from Nginx (X-Real-IP, X-Forwarded-For, X-Forwarded-Proto) as shown above, and configure your app to trust a single upstream — your Nginx instance. Never trust an unbounded list of X-Forwarded-For values from the Internet.

Does a reverse proxy slow my site down?

A well-configured reverse proxy adds only a few milliseconds per request and is usually a net win because it offloads TLS, compression and caching from your application servers. The proxy should be stateless and never store session data, so any instance can handle any request and a failed instance can be replaced without downtime.

Verdict

A reverse proxy is the standard way to expose dynamic applications safely and reliably. Nginx makes it straightforward: one server block, a proxy_pass to your back-end port, the four forwarding headers, and optionally an upstream block for load balancing. Start with the minimal config, always run nginx -t before reload, forward the headers your app needs, and terminate TLS once at the edge. Once it is in place, your origin servers stay private, certificates live in one place, and you can scale by adding more back-end instances behind the same proxy.

What a monolithic application is

A monolithic application is software in which all the components that make up a system are combined into a single, unified program. The user interface, business logic, and data-access layer live together in one codebase and are built and deployed as a single artifact.

The traditional model, a monolith is often the simplest way to build software:

• Simple development — one codebase, one build, one place to look. Getting started is fast because there is no distribution overhead.
• Simple deployment — a single artifact ships to a server; there is nothing to coordinate across many moving parts.
• Simple testing — an end-to-end test can exercise the whole application at once because everything lives in one process.
• Shared data is easy — a single database means foreign keys, joins, and transactions all work the way you expect; consistency is immediate.

The trade-off appears as the application and the team grow. Because everything is tightly coupled in one deployable unit, any change — even a small one — can mean rebuilding and redeploying the whole application. Scaling means scaling everything together, and a failure in one part can take down the entire application. As the codebase grows, a monolith can become harder to understand, test, and maintain.

What a microservices architecture is

A microservices architecture, or just microservices, is an architectural pattern that organizes an application into a collection of loosely coupled, fine-grained services that communicate through lightweight protocols such as HTTP or message queues.

Each service:

• Is deployed independently — you can update, release, and roll back a single service without touching the others.
• Owns its own data — each service manages its own database or data store, so teams do not share and fight over a single schema.
• Scales independently — a service under heavy load scales alone instead of forcing the entire application to scale.
• Maps to a business capability — services are often organized around functions such as "orders", "payments", or "search", and small teams can own each one end to end.

Because services are independent processes, they can be developed by separate teams using the tools that suit each problem. Fault isolation is better: if one service fails, the others keep running. And independent deployment lets teams ship changes continuously without coordinating a single big release.

The cost is complexity. Microservices are a distributed system, and distributed systems bring new problems:

• Network calls are slow and can fail — in-process function calls become remote calls with latency and failure modes.
• Consistency becomes eventual — cross-service transactions are hard; data consistency across services often has to tolerate delay (a pattern called eventual consistency).
• Operational burden rises — you need service discovery, load balancing, monitoring, and deployment tooling for many moving parts.
• Testing is harder — a test that spans multiple services must deal with a distributed environment.

Monolith vs microservices: the key differences

When to choose a monolith

Choose a monolith — or the middle-ground modular monolith (a single deployable unit with clearly separated internal modules) — when:

• You are building a new product or MVP, and the domain boundaries are not yet certain. A monolith lets you change structure cheaply as you learn.
• Your team is small, and a single codebase with one deployment is far more productive than paying for distributed-systems infrastructure.
• You do not yet need independent scaling — one service is not so much heavier than the others that the whole app has to scale, or you can scale the monolith vertically on a bigger machine first.

The honest guideline: the large majority of projects should start as a monolith (ideally a modular one). You get speed, simplicity, and cheap changes, and you avoid paying for distributed-systems complexity you do not yet need. Many teams later migrate to microservices only when a real, specific need — uneven scaling, many autonomous teams, or independent release cadence — makes the complexity worth it.

When microservices become worth it

Reach for microservices when several of these are genuinely true:

• Uneven scaling — one part of the application (say search or payments) needs to scale far beyond the rest, and scaling the whole monolith wastes resources.
• Many autonomous teams — you have enough teams that independent ownership, deployment, and release of a service is worth the coordination cost. This is Conway's Law in practice: systems mirror the teams that build them.
• Independent release cadence — different parts of the product ship on their own schedule and should not be blocked by a single whole-app release.
• Fault isolation matters — a failure in one capability should not take down the entire product.
• Technology freedom — you want to pick the best tool per service rather than one stack for everything.

Migrating a working monolith to microservices is a significant project with real risk — mostly around distributed data: joins become API calls, transactions become eventual, and multi-step operations need patterns like sagas to keep things consistent. A common low-risk path is the strangler pattern: gradually carve out one service at a time from the monolith while the rest keeps working, rather than doing a big-bang rewrite.

FAQ

Are microservices always better than a monolith?

No. Microservices and a monolith trade different costs. Microservices give you independent scaling, deployment, and team ownership, but add the complexity of running a distributed system: network failures, eventual consistency, and heavier operational tooling. A monolith is simpler, cheaper to build and operate at first, and easier to test end to end. The better choice depends on team size, scale, and product needs, not on a blanket rule.

Should I start with a monolith or microservices?

Start with a monolith, ideally a modular one — a single deployable unit with clearly separated modules. It is fastest to build, easiest to change while your domain boundaries are still uncertain, and avoids paying for distributed-systems complexity before you need it. Move toward microservices only when a real need (uneven scaling, many autonomous teams, or independent release cadence) makes the added complexity worth it.

What is the difference between a monolith and microservices in data handling?

A monolith typically uses one shared database, so foreign keys, joins, and transactions work directly and consistency is immediate. In a microservices architecture each service owns its own database, which gives clean data ownership but means joins become API calls or data duplication and cross-service consistency becomes eventual rather than immediate. Distributed data is often the hardest part of microservices and the part teams most underestimate.

What is a modular monolith?

A modular monolith is a single deployable unit whose internal modules are clearly separated along business boundaries, but which still ships and scales as one application. It gives you many of the maintainability benefits of good boundaries without the complexity of running a distributed system. It is a common recommended middle ground: start with a modular monolith and carve out real microservices later if a particular module needs to scale or release independently.

Verdict

Do not default to microservices because they sound modern. Default to a monolith — preferably a modular one — for new products, MVPs, and small-to-mid teams, and adopt microservices only when a concrete need appears: one part of the application scales unevenly, you have multiple teams that need independent ownership and release, or fault isolation is critical. When you do migrate, do it incrementally with the strangler pattern, and budget for the hardest problem of all: distributed data. In almost every case, the simpler architecture that meets today's real needs is the right one — you can always split later.

What a load balancer does

At its core, a load balancer is a reverse-proxy style component that intercepts traffic destined for a service and forwards each request to one member of a backend pool. In doing so it solves several problems at once:

• Availability. If one server goes down or stops responding, the load balancer stops sending it traffic and routes new requests to the healthy servers, keeping the service up. This is called failover.
• Scalability. Because traffic is spread across many servers, you can handle more requests by adding servers to the pool instead of making one machine more powerful.
• Performance. Distributing load avoids unevenly overloading some compute nodes while other nodes sit idle, which improves response time and utilization.
• Single entry point. Clients and DNS see one address; the complexity of many servers is hidden behind it.

The load balancer itself is typically aware of the health of the pool: it monitors servers, for example by fetching a known page or checking a health endpoint, and only routes traffic to members that respond. When a server comes back online, the balancer starts routing to it again.

Layer 4 vs Layer 7 load balancing

Load balancers are commonly described by the OSI layer they operate on. The two most common types are Layer 4 and Layer 7.

Layer 4 (transport) load balancing works at the transport layer and makes routing decisions using only header information such as IP address, port, and protocol. It does not look inside the payload. Because it does less inspection, Layer 4 balancing is very fast and has low overhead, which makes it a good fit for TCP and UDP traffic, database connections, VoIP, and other long-lived or high-throughput workloads.

Layer 7 (application) load balancing works at the application layer and can inspect the HTTP request itself — the URL path, headers, cookies, and body. This lets it route requests intelligently, for example sending /api traffic to one backend pool and /images to another, or using cookies for session affinity. It is more feature-rich but carries more processing overhead per request, so it is best suited to HTTP workloads where that intelligence pays off.

In practice many real-world deployments use both: a Layer 4 balancer in front for speed and a Layer 7 balancer for application-aware routing.

Load balancing algorithms

A load balancer needs a rule for choosing which server gets each request. These rules are called algorithms or scheduling methods, and they fall into two broad families: static algorithms, which ignore the current state of the servers, and dynamic algorithms, which take the servers' live load into account.

Round robin

Round robin is the simplest static method. The first request goes to the first server, the second to the second server, and so on down to the last, then it cycles back to the first. It is easy to set up and works well when all servers have roughly equal capacity and requests take similar time to process. Its weakness is that it ignores how loaded each server actually is.

Weighted round robin

Weighted round robin is round robin with a weight assigned to each server, so that more powerful servers receive proportionally more requests — and, depending on the implementation, receive them first. It is a good choice when servers in the pool differ in capacity.

Least connections

Least connections is a dynamic method that routes each new request to the server with the fewest active connections at that moment. It suits workloads where requests vary a lot in processing time or are long-lived, such as WebSocket connections, because a server with a few heavy connections is not confused for a lightly loaded one based on connection count alone. It can also be weighted.

Hashing

Hash-based methods map each request to a server using a hash of some value, most commonly the client IP address (IP hash). Because the same client always hashes to the same server, this provides session persistence (affinity) without reading cookies — useful for stateful applications. This is why IP-hash balancing is a popular choice for certain types of workloads.

Randomized and power-of-two choices

Randomized static load balancing simply assigns tasks to servers at random. Power of two choices picks two servers at random and sends the request to whichever of the two is less loaded. The latter is a surprisingly effective dynamic method: with only two random samples it balances load far better than pure random assignment and is used in distributed systems where a central scheduler is undesirable.

Health checks and failover

Load balancing is often used to implement failover — the continuation of service after the failure of one or more components. The servers are monitored continually; when one becomes unresponsive, the balancer is informed and stops sending it traffic, and when it comes back online the balancer restarts routing to it. For failover to keep capacity when a server drops, the pool needs at least one server in excess of the service's capacity (often called N+1 redundancy). Note that this technique can make the load balancer itself a single point of failure, so production setups usually balance at more than one level.

Choosing an algorithm

There is no universal best algorithm — the right choice depends on your traffic and your servers:

• Equal servers, similar requests: start with round robin. It is simple and predictable.
• Servers of different capacity: weighted round robin (or weighted least connections).
• Long-lived or variable-length connections (WebSockets, media): least connections.
• Stateful sessions without cookies: IP hashing for session affinity.
• HTTP with distinct resource types (paths, headers): a Layer 7 balancer so routing can be content-aware.

The practical pattern is to start with the simplest method that meets your needs, add health checks, and move to a more dynamic algorithm only when your workload shows it needs one.

FAQ

What is the difference between Layer 4 and Layer 7 load balancing?

Layer 4 load balancing makes routing decisions from transport-layer headers only (IP address, port, protocol) and is fast with low overhead, making it good for TCP/UDP and many long-lived workloads. Layer 7 load balancing inspects the HTTP request — path, headers, cookies, body — which allows content-aware routing and session affinity but costs more processing per request. Many systems use a Layer 4 balancer in front of a Layer 7 balancer to get both speed and intelligence.

What is the best load balancing algorithm?

There is no single best algorithm. Round robin is simplest and works for equal-capacity servers with similar requests; weighted round robin suits servers of different capacity; least connections suits long-lived or variable-length connections; and IP hashing gives session affinity for stateful apps. Pick the simplest method that meets your needs, then move to a more dynamic algorithm only if your workload requires it.

What is session affinity in load balancing?

Session affinity (also called sticky sessions or persistence) ensures that requests from a given client are consistently sent to the same server, which matters for stateful applications that keep session data in memory on one server. It can be achieved with cookies at Layer 7 or with IP hashing so that the same client always routes to the same server.

Does load balancing make the load balancer a single point of failure?

It can. If the load balancer itself fails, all the traffic through it stops. Production setups mitigate this by running load balancers in redundant pairs, balancing at multiple levels, or using DNS-level failover, so the balancing layer itself does not become the bottleneck.

What JWT gives you (and what it does not)

A JWT is a compact, URL-safe string with three base64url-encoded parts — header.payload.signature — separated by dots. The server signs the header.payload with a secret (or private key) and the receiver verifies the signature. Because the signature covers the payload, a client cannot tamper with the claims without the server noticing.

JWT gives you:

• Stateless authentication. The server does not keep a session table; the token is the proof. A cluster of API nodes can verify any token as long as they share the signing key.
• Self-contained claims. The payload can carry the user id, role and expiry, so the API does not need a database round-trip on every request to know who the caller is.
• Cross-service trust. The same token can be verified by multiple services that share the public key (with RS256/ES256).

JWT does not give you:

• Revocation for free. A signed token is valid until it expires, even if the user logs out or is banned. You need a denylist, short expirations, or refresh-token rotation to mitigate that.
• Encrypted payload. Base64url is encoding, not encryption. Anyone who can read the token can read the payload. Never put passwords, tokens or PII in JWT claims unless you use a JWE.
• Password storage. JWT is for sessions. Storing user passwords is a separate job that needs a slow, salted hash (bcrypt, scrypt or Argon2).

For a deeper look at the structure and the claims, see the JWT explained guide; for the surrounding HTTP layer, see REST API with Node.js and Express.

Prerequisites

• Node.js 20+ installed.
• A working Express app. If you do not have one yet, the REST API with Node.js and Express guide walks you through the basics.
• A database or in-memory store for users (a users array is enough for the demo; the same code works with Postgres, MongoDB, etc.).
• The packages jsonwebtoken, bcrypt and express.

npm install express jsonwebtoken bcrypt

All three are open source. jsonwebtoken is the most widely used JWT library for Node.js, bcrypt is the reference password-hashing function (designed by Provos and Mazières in 1999 and still considered safe with a sufficient cost factor).

The data model

A user record needs at least an id, an email and a password hash. The plain password is never stored. A minimal shape looks like this:

// users store — in production this is a database table
const users = [];
let nextId = 1;

In a real app you would also store a unique index on email, a createdAt timestamp, and either a passwordUpdatedAt field (to invalidate tokens after a password change) or a tokenVersion counter (to bump every time the user logs out everywhere).

Hashing passwords with bcrypt

Never store passwords in plain text or with a fast hash like SHA-256. bcrypt is the standard choice: it is slow on purpose, salts automatically, and the cost factor can be raised over time to keep pace with hardware.

import bcrypt from "bcrypt";

const COST = 12; // 2^12 = 4096 rounds — balance between security and login latency

export async function hashPassword(plain) {
  return bcrypt.hash(plain, COST);
}

export async function verifyPassword(plain, hash) {
  return bcrypt.compare(plain, hash);
}

The COST value is the log2 of the number of internal rounds. 12 is the common production default: on a modern server it takes roughly 200–300 ms, which slows a brute-force attack on a stolen database to a crawl while still being acceptable for a real login. Drop to 10 for test environments, raise to 13 if your hardware can take it.

Two important properties of bcrypt, straight from the spec:

• Salting is automatic. Every call to bcrypt.hash generates a fresh 16-byte salt and embeds it in the output. You do not store the salt separately.
• The hash carries its own parameters. The output looks like $2a$12$R9h/cIPz0gi.URNNX3kh2O.PST9/PgBkqquzi.Ss7KIUgO2t0jWMUW — the $2a$ part identifies the algorithm, 12 the cost, and the next 22 characters the salt. Verification extracts all three from the stored hash.

bcrypt.compare is constant-time for a given hash, which avoids timing attacks on the comparison.

Signing a JWT

The jsonwebtoken package exposes jwt.sign(payload, secret, options):

import jwt from "jsonwebtoken";

const ACCESS_SECRET = process.env.JWT_ACCESS_SECRET;
const REFRESH_SECRET = process.env.JWT_REFRESH_SECRET;

export function signAccessToken(user) {
  return jwt.sign(
    { sub: user.id, email: user.email },
    ACCESS_SECRET,
    { algorithm: "HS256", expiresIn: "15m" }
  );
}

export function signRefreshToken(user) {
  return jwt.sign(
    { sub: user.id, typ: "refresh" },
    REFRESH_SECRET,
    { algorithm: "HS256", expiresIn: "7d" }
  );
}

A few details that matter:

• algorithm: "HS256" is pinned explicitly. Without it, some libraries honor the alg field of an incoming token, which is the well-known “algorithm confusion” attack. Pin it.
• sub (subject) is the user id. The JWT spec reserves a small set of claim names — iss, sub, aud, exp, nbf, iat, jti — and using them keeps your tokens interoperable.
• expiresIn controls lifetime. Short access tokens (15 minutes is a common default) limit the damage if one leaks. Refresh tokens (7 days) are the way the client gets a new access token without re-entering the password.
• Two different secrets. Access and refresh tokens are signed with different keys so a leaked refresh token cannot be used to forge access tokens, and vice versa.
• Secrets come from the environment. process.env.JWT_ACCESS_SECRET is loaded from your deployment environment, never hardcoded. A 32-byte random string is enough; openssl rand -hex 32 gives you one.

The login endpoint

The POST /auth/login route takes an email and password, verifies the hash, and returns both tokens:

app.post("/auth/login", async (req, res) => {
  const { email, password } = req.body ?? {};
  if (typeof email !== "string" || typeof password !== "string") {
    return res.status(400).json({ error: "email and password required" });
  }

  const user = users.find((u) => u.email === email);
  if (!user || !(await verifyPassword(password, user.passwordHash))) {
    // same response for unknown email and wrong password
    return res.status(401).json({ error: "invalid credentials" });
  }

  return res.json({
    accessToken: signAccessToken(user),
    refreshToken: signRefreshToken(user),
  });
});

Two security details worth pointing out:

• Same error for unknown email and wrong password. Telling the client “user not found” leaks which emails are registered; “invalid credentials” is the only safe response. This is the standard mitigation against user enumeration.
• Rate limiting belongs here. Even with a slow hash, an attacker can try millions of passwords per second if you let them. A simple per-IP and per-email rate limiter (e.g. express-rate-limit) on the login and refresh routes is not optional in production.

The matching POST /auth/register endpoint is symmetrical: validate input, hash the password, save the user, return tokens. Same error-handling rules.

Verifying tokens in middleware

Every protected route runs through a middleware that reads the Authorization header, verifies the token, and attaches the user to the request:

function requireAuth(req, res, next) {
  const header = req.get("authorization") || "";
  const [scheme, token] = header.split(" ");
  if (scheme !== "Bearer" || !token) {
    return res.status(401).json({ error: "missing bearer token" });
  }
  try {
    const payload = jwt.verify(token, ACCESS_SECRET, { algorithms: ["HS256"] });
    req.user = { id: payload.sub, email: payload.email };
    return next();
  } catch {
    return res.status(401).json({ error: "invalid or expired token" });
  }
}

Then mount it on the routes that need it:

app.get("/me", requireAuth, (req, res) => {
  res.json({ id: req.user.id, email: req.user.email });
});

jwt.verify does three things for you, all of which you want:

• Checks the signature. A tampered or unsigned token fails.
• Checks exp and nbf. An expired token or one whose nbf is in the future is rejected. No clock skew by default.
• Pins the algorithm. The algorithms: ["HS256"] option disables the “alg=none” and “RS256 verified with the public key as an HMAC secret” attacks. Pin it the same way on the signing side.

The middleware never re-throws the underlying JWT error to the client — a generic invalid or expired token is enough; details like “signature mismatch” vs “expired at …” help an attacker tune their approach and are best left out.

Refresh tokens

When the access token expires, the client sends the refresh token to POST /auth/refresh and gets a new pair. The refresh token is rotated: the old one is invalidated and a new one is returned.

app.post("/auth/refresh", (req, res) => {
  const { refreshToken } = req.body ?? {};
  if (typeof refreshToken !== "string") {
    return res.status(400).json({ error: "refreshToken required" });
  }
  try {
    const payload = jwt.verify(refreshToken, REFRESH_SECRET, { algorithms: ["HS256"] });
    if (payload.typ !== "refresh") {
      return res.status(401).json({ error: "wrong token type" });
    }
    const user = users.find((u) => u.id === payload.sub);
    if (!user) return res.status(401).json({ error: "user not found" });
    return res.json({
      accessToken: signAccessToken(user),
      refreshToken: signRefreshToken(user),
    });
  } catch {
    return res.status(401).json({ error: "invalid or expired refresh token" });
  }
});

Why rotation matters: if a refresh token leaks, the legitimate client will use it next and the server will see the same jti twice. That is the signal to invalidate the user's whole token family. In a real system, store the jti of the last issued refresh token per user; if the same jti shows up again, revoke everything and force a re-login.

Logging out

Stateless JWT makes logout tricky. The token is still mathematically valid after the user leaves — there is nothing on the server to delete. Three options, in order of complexity:

1. Client-side discard. The client deletes the tokens. Simple, but does not protect a leaked token.
2. Short access tokens. With a 15-minute access token, a stolen token is useful for at most 15 minutes even without server-side action.
3. Server-side denylist. Keep a Redis set of revoked jtis and check it in requireAuth. For most apps, option 2 alone is enough; add the denylist if you have a real reason to invalidate tokens immediately (password change, ban, suspected leak).

app.post("/auth/logout", requireAuth, (req, res) => {
  // client should also delete the tokens
  res.status(204).end();
});

Common pitfalls

• Storing the secret in source control. If JWT_ACCESS_SECRET lives in the repo, rotate it the moment the repo leaks. Use environment variables and a secret manager.
• Putting the token in localStorage. localStorage is readable by any script on the page; a single XSS bug drains it. Store tokens in an HttpOnly; Secure; SameSite=Strict cookie when the client is a browser. For native apps or server-to-server callers, the Authorization header is fine.
• Trusting the alg field. The classic JWT pitfall: a server that does not pin the algorithm can be tricked into accepting a token with alg: none or alg: HS256 where the public RSA key is the “secret”. Always pass algorithms: ["HS256"] to jwt.verify.
• Long-lived access tokens. A token that never expires is a permanent backdoor. 15 minutes is a sensible default for human-driven APIs; service-to-service tokens can be longer but should still expire.
• No rate limiting on auth routes. Without a limiter, an attacker can grind the password field. express-rate-limit keyed on (ip, email) is the standard fix.
• Sensitive data in the payload. JWT payload is readable by anyone who has the token. Put only the user id, role and similar non-sensitive claims in there.
• Forgetting HTTPS. Tokens travel in the Authorization header on every request. Without TLS, anyone on the network can read them. Terminate TLS at the edge (Nginx, a managed load balancer, a CDN) and never run an authenticated API over plain HTTP.

End-to-end flow

Putting it all together, a client does this:

1. POST /auth/login with { email, password }. Receives { accessToken, refreshToken }.
2. Calls any protected route with Authorization: Bearer <accessToken>. The server returns the user data.
3. After 15 minutes, accessToken is expired. The client calls POST /auth/refresh with { refreshToken } and gets a new pair.
4. On logout, the client deletes both tokens and calls POST /auth/logout to tell the server.

That is the full loop. The server is stateless except for the user store and (optionally) a refresh-token version counter or denylist.

FAQ

How long should an access token live?

For human-driven APIs, 5 to 30 minutes is the common range. Shorter is safer (a leaked token is useful for less time) but worse for UX (the client has to refresh more often). 15 minutes is a balanced default. For service-to-service tokens, an hour to a day is common; rotate them with the same refresh flow.

Where should I store JWTs on the client?

For a browser app, the safest option is an HttpOnly; Secure; SameSite=Strict cookie set by the server. JavaScript on the page cannot read it, so an XSS bug cannot exfiltrate it. For a mobile or server-to-server client, store the token in the OS keychain or a secret manager, and send it as Authorization: Bearer … on every request. localStorage is convenient but exposes the token to any script that runs on the page.

Should I use HS256 or RS256?

HS256 uses a single shared secret: the same key signs and verifies. It is fast and simple, and it is the right choice when the same service issues and verifies the token. RS256 uses a private key to sign and a public key to verify; the public key can be shared with other services so they can verify tokens without being able to mint new ones. Pick RS256 when you have multiple services that need to verify tokens but only one service that issues them.

What is the difference between an access token and a refresh token?

The access token is the short-lived credential the client sends on every API call. The refresh token is a longer-lived credential whose only job is to mint new access tokens at the refresh endpoint. Keeping them separate means a leaked access token expires in minutes, and a leaked refresh token can be rotated and revoked. The signing secrets are different so a compromise of one does not give the attacker the other.

Do I need a database for users?

Yes — at least a place to store { id, email, passwordHash }. The demo in this guide uses an in-memory array so the example is self-contained, but in production the user store is a real database table. The same JWT code works with any store because the auth layer only needs the user record after the token is verified.

Is JWT the right choice for sessions?

JWT is a great fit for stateless APIs and for cross-service auth. It is the wrong choice when you need instant server-side revocation (no denylist), or when your sessions carry large amounts of state (the token is sent on every request, so the smaller the better). For traditional server-rendered web apps with Set-Cookie sessions, a server-side session table is often simpler.

Verdict

JWT authentication on a Node.js API is a 100-line job once the parts are clear: bcrypt for passwords, jsonwebtoken for signing, two routes (/auth/login and /auth/refresh) and one middleware (requireAuth). The hard part is not the code, it is the discipline — short access tokens, pinned algorithms, rate limiting, no secrets in the repo, and HTTPS in front of everything. Once those are in place, the API is stateless, horizontally scalable, and the same token works across every node behind the load balancer.

What makes JSON so widely used?

JSON's design goals, set out in RFC 8259, are that it be minimal, portable, textual, and a subset of JavaScript. Those four properties explain its dominance:

• It is tiny and readable. There is no tag markup, no namespace syntax, no schema language required to read a small document — a human can scan a JSON object and understand it at a glance.
• It is language-independent. Although it looks like JavaScript, standard libraries in almost every programming language can parse and generate JSON. You send JSON from a JavaScript client, a Python server, or a Go service, and any of them can read it.
• It transmits cleanly over the network. Because JSON is just text encoded in UTF-8, it fits naturally into HTTP request and response bodies, and it is trivial to log, cache, and test.
• It is strict enough to be unambiguous. The small, well-defined grammar means two different implementations agree on the same document — the property that made it possible to build interoperability across the web.

The JSON data types

A JSON document can be any one of the following values:

• Object — an unordered collection of zero or more name/value pairs, written as { "name": value, ... }. The names are strings, and the values can be any JSON type. Curly braces delimit the object and a colon separates each name from its value.
• Array — an ordered sequence of zero or more values, written as [ value, value, ... ] with square brackets. The values can be any JSON type, and an array's elements do not have to be the same type.
• String — a sequence of zero or more Unicode characters enclosed in double quotes, with a backslash escaping syntax for special characters.
• Number — a signed decimal number that may contain a fractional part and may use exponential (E) notation. JSON makes no distinction between integer and floating point, and it cannot represent NaN or Infinity.
• Boolean — the literal value true or false.
• null — the literal word null, representing an empty value.

A simple example of a complete JSON object:

{
  "name": "Debod",
  "isActive": true,
  "posts": 54,
  "tags": ["json", "api"],
  "meta": { "editor": "Hermes", "reviewed": null }
}

JSON's strict syntax rules

JSON is a subset of JavaScript object literal syntax, but it is stricter in ways that matter:

• Strings must use double quotes ("name"), never single quotes.
• Every key must be quoted. In JavaScript you can write {name: "x"}; in JSON you must write {"name": "x"}.
• No trailing commas. A comma after the last element in an object or array is invalid in JSON, though it is common in JavaScript.
• No comments. JSON deliberately has no comment syntax. Douglas Crockford removed comments because people used them to hold parsing directives, which would have destroyed interoperability.
• Only serializable types are allowed. JSON cannot represent functions, Date, Set, Map, undefined, NaN, or Infinity.
• Numbers use decimal notation. There is no support for hex literals or leading zeros.

Because the grammar is so confined, a single misplaced comma or missing quote makes a JSON document invalid. That strictness is a feature: it keeps every conforming parser in agreement.

JSON and JavaScript: parse and stringify

Even though JSON is not JavaScript, the JSON object in JavaScript provides the two operations you need almost every day:

• JSON.stringify(value) — converts a JavaScript object or array into a JSON string so it can be stored or transmitted.
• JSON.parse(string) — converts a JSON string back into a JavaScript object so you can read its values.

These map to the general concepts of serialization (object → text) and deserialization (text → object). On the web, you also meet JSON through the Fetch API: response.json() reads a response body and returns a JavaScript object directly.

JSON vs XML vs YAML

JSON is not the only text data format, but it holds a particular niche:

• JSON vs XML. JSON needs far less markup — no closing tags, no namespaces — so the same data is shorter and easier to read. XML predates JSON and is still used where complex document structure, namespaces, or strict schema validation (XSD) are required. For typical API payloads and config, JSON is the lighter, more common choice.
• JSON vs YAML. YAML 1.2 is a superset of JSON, meaning any valid JSON is valid YAML. YAML adds comments, multi-line strings, and a more readable indentation-based syntax, which makes it popular for configuration files where humans edit by hand. But YAML's syntax is richer and its parsers more varied, so JSON remains the safer choice for program-to-program data exchange.

In short, use JSON when data crosses a network or is read by programs; use YAML when humans hand-edit configuration; use XML when you need namespaces, attributes, or formal document schemas.

When and where JSON appears

JSON's stamp is everywhere in modern development:

• Web APIs. REST and GraphQL APIs return JSON bodies; Content-Type: application/json is the norm for request and response payloads. Tools like Postman, Insomnia, and Swagger are built around it.
• Configuration and metadata. package.json, TypeScript config, VS Code settings, and countless tool manifests are JSON (or JSONC, which adds comments and trailing commas). JSON-LD, used for structured data on the web, is JSON that carries linked-data semantics.
• Databases. Document databases like MongoDB store records as BSON (a binary superset of JSON). PostgreSQL, MySQL, and SQL Server all support a native JSON type and query operators.
• Client/server communication. Sending structured data from a browser to a server, or between microservices, is almost always done with JSON today.

FAQ

What does JSON stand for?

JSON stands for JavaScript Object Notation. It is a data interchange format derived from JavaScript object syntax, standardized by RFC 8259 (an IETF Internet Standard, STD 90) and by ECMA-404.

Is JSON a programming language?

No. JSON is a data interchange format, not a language. It contains only data — no functions, variables, or logic — and it can be parsed and generated by standard libraries in almost any programming language.

What are the valid data types in JSON?

JSON supports six types: objects (name/value pairs in curly braces), arrays (ordered lists in square brackets), strings (double-quoted Unicode text), numbers (decimal values, optionally with fractions or exponents), booleans (true/false), and null. It cannot represent functions, dates, sets, maps, undefined, NaN, or Infinity.

What is the difference between JSON and XML?

JSON is a compact, minimal format derived from JavaScript, using objects, arrays, and quoted keys with no closing tags or namespaces. XML is an older markup format with opening and closing tags, attributes, and namespaces. For API payloads and configuration, JSON is shorter and more common; XML is still used where complex document structure, namespaces, or XSD schema validation are required.

Declarative vs Imperative: the two ways to describe infrastructure

There are two broad approaches to writing IaC definitions, and the Wikipedia IaC article frames the distinction clearly: "The declarative approach focuses on what the eventual target configuration should be; the imperative focuses on how the infrastructure is to be changed to meet this."

Declarative is the dominant approach in modern IaC. You write a definition that describes the desired end state — "I want three servers, each with 4 GB of RAM, running Ubuntu 22.04, behind a load balancer" — and the IaC tool figures out the exact sequence of API calls to get there. Cloudflare's Terraform documentation describes Terraform's model: "You define the final state of the configuration rather than the step-by-step procedure." If you run the same declarative definition twice, you get the same result — a property called idempotency that declarative IaC delivers by design.

Imperative defines the exact sequence of steps to reach a target state. You write a script that says "create server A, then install Nginx, then configure the firewall, then start the service." This gives you more control over the individual steps but requires you to manage the procedure yourself. Tools like Ansible can work in both modes.

Why teams adopt IaC

The Wikipedia IaC history section traces the practice back to 1993 and CFEngine, but cloud computing is what made it mainstream. As organisations moved from physical servers to virtual machines they could provision through an API, the manual console workflow became a bottleneck. Cloudflare's Workers IaC documentation frames the value proposition: "Define and store configuration in source code repositories like GitHub, track and version changes over time, and roll back when needed."

The concrete benefits teams report:

Reproducibility. Every environment — development, staging, production — is described by the same files, instantiated by the same tool. The staging environment is not "mostly like production"; it is a precise replica built from the same definitions.

Version control. Infrastructure changes go through the same review process as application code. A pull request shows exactly what will change, who approved it, and the diff is auditable. If a change causes an outage, you revert the commit.

Speed. Provisioning a new environment that once took a week takes minutes. The definition file describes the target; the tool does the work. This makes experiments cheap — if an idea does not work, you destroy the environment and try another.

Consistency at scale. Running ten servers and running ten thousand is the same command. Manual configuration accumulates small differences — the one server where someone set max_connections to a different value — that become production incidents. IaC applies the definition uniformly.

Security as code. Security policies — encryption at rest, least-privilege IAM roles, firewall rules — are expressed in the definition file. They are not a checklist someone runs manually; they are enforced every time the infrastructure is built. Cloudflare's Terraform provider documentation shows this pattern: you describe the desired state of your firewall rules and the tool enforces them, with every change tracked in the repository.

Popular IaC tools

The Wikipedia tool table lists a wide range. The most widely used today:

Terraform (HashiCorp, 2014) is the most popular IaC tool in the industry. It uses its own HashiCorp Configuration Language (HCL), a declarative language designed to be readable. Terraform is provider-agnostic — it can manage resources across AWS, Azure, Google Cloud, Cloudflare, and hundreds of other providers from a single definition file. Cloudflare's Terraform documentation shows exactly this: the same Terraform configuration that manages Cloudflare DNS also manages your cloud VMs and your CI/CD pipeline.

Ansible (Red Hat, 2012) started as a configuration management tool and grew into a general-purpose automation platform. It communicates over SSH, requires no agent on target machines, and can run in both declarative and imperative modes. Ansible playbooks describe the target state for servers, networks, and applications.

AWS CloudFormation is AWS's native IaC service. It is declarative and JSON- or YAML-based. If you live entirely inside AWS, CloudFormation integrates tightly with every AWS service. If you use multiple clouds, Terraform's provider model is more flexible.

Pulumi (2018) takes a different approach: you write infrastructure definitions in a general-purpose programming language — Python, TypeScript, Go, C# — instead of a domain-specific language like HCL. This makes IaC more accessible to developers who already know those languages and allows you to use loops, conditionals, and libraries in your infrastructure code.

OpenTofu (Linux Foundation, 2023) is an open-source fork of Terraform, created after HashiCorp switched to a non-open-source license. It is functionally compatible with Terraform and aims to be the community-driven continuation of the project.

FAQ

Is IaC the same as configuration management?

No. IaC focuses on provisioning — creating and deleting the underlying infrastructure (servers, networks, load balancers). Configuration management focuses on the software and settings on systems that already exist. In practice, teams use both: Terraform to provision the infrastructure, Ansible or a similar tool to configure the operating system and deploy applications on top of it.

What is configuration drift and why does it matter?

Configuration drift is the gradual divergence between your infrastructure's actual state and the state described in your code. It happens when someone makes a manual change — adjusting a setting through the cloud console, tweaking a configuration file on a running server — that never gets written back to the definition files. Over time, environments that started identical become different in small, invisible ways. Those differences are where incidents hide. IaC tools detect and correct drift by re-applying the definition.

Does IaC slow things down initially?

Setting up the first IaC definitions takes longer than clicking through a console. That is the real cost. The return on that investment comes when you need to make a second environment, recover from an incident, or audit what changed last quarter. For a one-time project with a server you will never touch again, IaC may be overkill. For anything you will maintain, update, or scale, it pays back quickly.

Which IaC tool should I start with?

Terraform is the most widely applicable starting point. It works across every major cloud provider, has extensive documentation, a large community, and the HCL language is purpose-built for infrastructure definitions — readable even by people who do not write code every day. If you are an AWS-only shop and want tight integration with every AWS service, CloudFormation is a natural fit. If your team is more comfortable with Python or TypeScript, Pulumi lowers the learning curve by letting you use languages you already know.

Verdict

Infrastructure as Code is not optional once you have more than one environment or more than one person touching infrastructure. The manual console workflow that works for a single server becomes a liability at scale — inconsistent, unauditable, unrecoverable. IaC turns infrastructure into software: testable before you deploy it, reviewable through pull requests, versionable through git, and reproducible on demand.

The Wikipedia definition captures the essence: "the process of managing and provisioning computer data center resources through machine-readable definition files." That process — defining infrastructure as files, not clicks — is what makes modern cloud infrastructure manageable, auditable, and safe to change. Pick Terraform as a starting point, version your definitions from day one, and treat infrastructure changes the same way you treat code changes: with a review, a test run, and a rollback plan.

Elements and tags: the anatomy of HTML

HTML documents are built from elements. An element is made of three parts: an opening tag, the content, and a closing tag. The tags — written inside angle brackets < and > — tell the browser how to interpret the content between them.

<p>This is a paragraph element.</p>

<p> is the opening tag. This is a paragraph element. is the content. </p> is the closing tag. The forward slash before the element name signals the closing tag.

Some elements have attributes — additional values that configure the element or adjust its behavior. Attributes are written inside the opening tag:

<a href="https://example.com">Visit Example</a>
<img src="photo.jpg" alt="A landscape photograph" />

Here, href is an attribute of the <a> (anchor) element, and src and alt are attributes of the <img> element.

Element tag names are case-insensitive — <TITLE>, <Title>, and <title> all work — but the MDN recommendation is to write tags in lowercase. That is the universal convention.

Not all elements have content and closing tags. Void elements (also called empty elements) have no content and no closing tag. The <img> element is a common example: it embeds an image but wraps no text. These elements may optionally use a trailing slash in the opening tag (<img src="photo.jpg" alt="..." />), which is permitted but not required.

A complete HTML document

The MDN first-HTML tutorial shows a minimal complete HTML page:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>My test page</title>
  </head>
  <body>
    <img src="" alt="My test image" />
  </body>
</html>

Here is what each part does:

• <!doctype html> is the doctype — a required preamble that tells the browser this is an HTML5 document. Modern browsers use it to switch to standards mode.
• <html lang="en"> is the root element. Everything on the page lives inside it. The lang attribute sets the document's primary language.
• <head> is the head — a container for content that is not displayed on the page. It holds the page title, character encoding, links to CSS stylesheets, and metadata.
• <meta charset="utf-8"> sets the character set to UTF-8, which covers characters from most written languages. There is no reason to omit this.
• <meta name="viewport" content="width=device-width"> tells the browser to match the page width to the device screen, preventing mobile browsers from shrinking the page.
• <title> sets the text shown in the browser tab and used when bookmarking the page.
• <body> contains all visible content — text, images, forms, links, and more.

Browsers do not display the HTML tags themselves. They use the tags to interpret the content and render the page. A paragraph <p>...</p> becomes a block of text; an <a href="..."> becomes a clickable link; an <img src="..."> becomes a displayed image.

Semantic HTML: meaning over appearance

One of the most important concepts in HTML is semantics — writing markup that describes the meaning of content, not just its appearance.

The MDN Semantics glossary explains: "In HTML, the <h1> element is a semantic element that gives the text it wraps around the role of 'a top level heading on your page.'" You can make any element look like an <h1> heading with CSS, but it will not carry the meaning of a heading for search engines, screen readers, or browser features.

The benefits of semantic HTML, according to MDN:

• Search engines treat semantic content as important keywords that influence page rankings.
• Screen readers use semantic elements as signposts to help visually impaired users navigate a page.
• Developers find meaningful markup easier to read and maintain than a wall of <div> tags.
• Semantic naming mirrors proper component naming in modern frontend frameworks.

There are roughly one hundred semantic elements in HTML5. The most important:

Compare a semantic page to one built only from <div> elements with classes like class="sidebar" or class="red-text". The semantic version carries machine-readable meaning; the <div> version is just a styled box.

The Wikipedia HTML article notes that the World Wide Web Consortium (W3C) "encouraged the use of CSS over explicit presentational HTML since 1997" — keeping HTML for structure and semantics, and CSS for appearance.

HTML, CSS, and JavaScript: the three layers

HTML handles content and structure. Two other technologies work alongside it:

CSS (Cascading Style Sheets) describes how HTML elements look — colors, fonts, spacing, layout. The Wikipedia HTML article describes the relationship: "It is often assisted by technologies such as Cascading Style Sheets (CSS) and scripting languages such as JavaScript." Separating CSS from HTML keeps both files easier to maintain.

JavaScript adds behavior and interactivity — handling clicks, validating forms, updating content without a page reload. JavaScript can read and modify the HTML document through the DOM (Document Object Model).

These three layers stack: HTML provides the structure, CSS makes it look right, and JavaScript makes it work. A well-built web page keeps them separate: HTML for content, CSS for presentation, JavaScript for behavior.

FAQ

What does HTML do?

HTML structures and annotates web content so browsers can render it. It marks up text as headings, paragraphs, lists, or links; embeds images, videos, and forms; and provides the semantic meaning that search engines and assistive technologies rely on. HTML does not style content (that is CSS) or define behavior (that is JavaScript).

Is HTML a programming language?

HTML is a markup language, not a programming language. A programming language can execute logic, loops, and conditions. A markup language annotates content to describe its structure and meaning. HTML describes what content is — a heading, a paragraph, a list — not what the computer should do. HTML, CSS, and SQL are the most common markup/data-query languages in web development, but they are distinct from Turing-complete programming languages.

Why does semantic HTML matter?

Semantic elements tell browsers, search engines, and screen readers what each part of a page means — not just how it looks. A <nav> element marks a navigation region; a <article> marks self-contained content; an <h1> marks the page title. This structure lets assistive technologies skip to the main content, lets search engines understand the page hierarchy, and makes the code readable by developers. MDN's Semantics article lists these as the primary benefits: better SEO, accessible navigation, maintainable code, and proper browser features.

How does a browser read HTML?

When you request a URL, the web server sends an HTML document. The browser reads the document from top to bottom, parsing each tag and building a tree structure called the DOM (Document Object Model). The DOM represents every element — headings, paragraphs, images, links — as objects the browser can manipulate. The browser then renders the DOM visually on screen. Tags are not displayed; they are instructions the browser follows to interpret the content.

What is the difference between HTML and HTML5?

HTML (the original) was developed by Tim Berners-Lee in 1993. The Wikipedia HTML article tracks its history: "HTML 4 was published in 1997; HTML5 was a major evolution, adding native support for video, audio, canvas graphics, and web applications." HTML5 also introduced semantic elements (<header>, <nav>, <article>, <section>, <footer>) that were absent in earlier versions. Today, "HTML" and "HTML5" are often used interchangeably — the WHATWG maintains a "living standard" that evolves continuously, and all modern browsers implement HTML5 features.

Verdict

HTML is the foundation everything else on the Web is built on. Understanding it means understanding the difference between structure (HTML), presentation (CSS), and behavior (JavaScript) — and why keeping those three layers separate makes sites faster, more accessible, and easier to maintain.

The key practical takeaways: write semantic HTML (use <article> when you mean an article, <nav> when you mean navigation), keep content separate from styling, and write <!doctype html> and <meta charset="utf-8"> in every document. A page built with meaningful HTML elements is automatically more accessible to screen readers, more readable to search engines, and easier for developers to work with — all without writing a single line of CSS or JavaScript.

The problem HSTS solves

The web uses HTTP over TLS (HTTPS) to protect communication between browsers and servers. HTTPS encrypts data in transit, authenticates the server, and prevents tampering. But HTTP — the unencrypted predecessor — still works on most servers, and users still type http:// into their address bars.

This creates an attack window. A man-in-the-middle (MITM) attacker on a shared Wi-Fi network can intercept the first unencrypted HTTP request a user makes to a website. The attacker forwards nothing, or forwards a tampered version. The user sees the page, but the attacker has read or modified everything. This class of attack — where the attacker downgrades a secure connection to an insecure one — is called a protocol downgrade attack or SSL stripping.

The Wikipedia article on HSTS describes the mechanism: when a server sends the Strict-Transport-Security header over HTTPS, it instructs user agents to "automatically interact with it using only HTTPS connections." Browsers that obey this policy refuse to make plain HTTP requests to that domain.

How the Strict-Transport-Security header works

The MDN TLS documentation explains that browsers only respect the HSTS header when it arrives over a secure (HTTPS) connection. An HTTP response carrying this header is simply ignored — a server cannot force browsers to use HTTPS over an insecure channel.

A minimal HSTS header looks like this:

Strict-Transport-Security: max-age=31536000

This tells the browser to only use HTTPS when connecting to this domain for the next 315,360 seconds — one year. After the browser has received this header once, it will refuse HTTP requests to the domain until the max-age period expires.

The three HSTS directives

The HSTS specification (RFC 6797) defines three directives:

max-age is the only mandatory directive. A value of 0 disables HSTS for that domain. Security-sensitive sites typically use max-age=63072000 (two years) or higher with includeSubDomains and preload.

What HSTS protects against

The Wikipedia TLS article describes TLS as providing three properties: confidentiality (encryption, so eavesdroppers cannot read data), integrity (tampering is detected), and authenticity (client and server prove their identities). HSTS extends these guarantees by preventing the protocol downgrade that would break all three.

Specifically, HSTS prevents:

• SSL stripping attacks — where an attacker intercepts the first HTTP request and leaves the user on an unencrypted connection.
• Mixed content warnings — browsers automatically upgrade subresource requests from http:// to https:// on HSTS-enabled sites.
• Cookie hijacking — session cookies sent over HTTP can be stolen; HSTS ensures all traffic (including cookie transmission) goes over HTTPS.

As the Wikipedia HSTS article notes, HSTS also addresses "cookie hijacking," where an attacker steals a session cookie over an unencrypted connection.

The trust-on-first-use limitation

The main weakness of HSTS is described in the Wikipedia HSTS article: the protection "normally only applies after a user has visited the site at least once." The first visit to a domain — whether typed manually, clicked from an email, or followed from a bookmark — is still made over HTTP unless the browser already has a cached HSTS policy.

This is called trust on first use (TOFU). An attacker who can intercept that first connection can still perform an SSL stripping attack, because the browser has not yet received the HSTS header.

The HSTS preload list

To eliminate this first-visit window, major browsers maintain HSTS preload lists — hard-coded lists of domains that must always be accessed over HTTPS, baked into the browser binary. The Wikipedia HSTS article explains: "If a domain is on this list, the browser skips the initial request and encrypts all communication immediately."

The Chrome HSTS preload list (also used by Safari, Firefox, and other Chromium-based browsers) is managed at hstspreload.org. Any domain can submit itself, provided it meets these requirements:

• A valid certificate for the domain and all subdomains.
• Serving a redirect from HTTP to HTTPS, or refusing HTTP connections entirely.
• Serving the HSTS header on the HTTPS response with at least max-age=31536000; includeSubDomains; preload.

Once on the list, removing a domain is difficult — it requires browser updates, which can take months. This makes the preload commitment serious. Sites should only request preload after they are certain HTTPS will be permanent.

Mixed content and HSTS

The MDN TLS documentation explains that an HTTPS page loading subresources over HTTP is called mixed content — the main page is encrypted, but the scripts, images, or stylesheets loaded over HTTP are not. An attacker who intercepts the page can replace those subresources with malicious code.

HSTS helps here too. When a browser has a cached HSTS policy for a domain, it automatically upgrades any http:// subresource requests to https:// before sending them. The MDN TLS documentation describes how browsers "either upgrade the load request to use HTTPS, or block the request entirely" for mixed content on secure pages.

Deploying HSTS: practical steps

1. Enable HTTPS first — HSTS only works over HTTPS. A site must fully migrate to HTTPS before setting the header.
2. Start with a short max-age — begin with max-age=300 (5 minutes) and verify nothing breaks.
3. Increase gradually — once verified, increase to max-age=86400 (1 day), then a week, then a month.
4. Add includeSubDomains — once stable, add the directive to cover all subdomains.
5. Request preload — after running at max-age=31536000; includeSubDomains for a few weeks with no issues, submit to hstspreload.org.

The MDN TLS documentation also recommends sending the Strict-Transport-Security header on every HTTPS response, not just the first one, to refresh the browser's policy and extend its expiry.

HSTS and CDN edge deployments

When a site uses a CDN, the HSTS header must be set at the CDN edge, not just the origin server. Cloudflare, for example, adds HSTS headers through its dashboard (Speed → Optimization → HSTS) or via a Strict-Transport-Security Page Rule. If the CDN terminates TLS, the HSTS header should be set on the CDN's response — it is the response the browser receives.

Sites using Cloudflare Pages can set HSTS via the public/_headers file:

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

FAQ

What does HSTS protect against?

HSTS protects against protocol downgrade attacks (SSL stripping) and cookie hijacking. When a browser has a cached HSTS policy for a domain, it refuses to make unencrypted HTTP requests to that domain — even if the user types http:// directly. This prevents attackers on shared networks from intercepting the first request or stealing session cookies.

Does HSTS make a site completely secure?

No. HSTS protects the transport layer — it ensures all connections use HTTPS. It does not protect against vulnerabilities in the application itself, such as XSS (cross-site scripting), CSRF, SQL injection, or weak passwords. It also cannot protect the very first visit to a domain, unless the domain is on the browser's HSTS preload list. HSTS is one layer of a defense-in-depth strategy.

Why is a preload list needed if max-age is set?

The first visit to a domain is made before the browser has received the HSTS header. An attacker who intercepts this first unencrypted request can strip HTTPS and leave the user on HTTP. The preload list bakes the HTTPS requirement into the browser itself, so the upgrade happens on the very first request — before any network traffic leaves the browser. The Wikipedia HSTS article describes this as addressing "the problem of the first request."

What is the difference between HSTS and HTTPS redirect?

An HTTPS redirect (a server responding to http:// requests with a 301 Moved Permanently or 302 Found redirect to the HTTPS URL) tells the browser where to go. HSTS tells the browser never to ask for the HTTP URL in the first place. Redirects can be intercepted and stripped before they take effect; HSTS eliminates the unencrypted request. The two work together: redirects handle existing links and bookmarks, while HSTS prevents the initial unencrypted connection.

How does HSTS interact with subdomains?

The includeSubDomains directive in the HSTS header extends the HTTPS-only policy to every subdomain of the domain. If example.com sets Strict-Transport-Security: max-age=31536000; includeSubDomains, then api.example.com, cdn.example.com, and any other subdomain are also covered. Without includeSubDomains, only the exact domain receives HSTS protection — a subdomain served over HTTP is unaffected.

Verdict

HSTS is a simple, high-impact security header. A single Strict-Transport-Security response header eliminates the SSL stripping attack class on every subsequent visit — for free, with no performance cost, and minimal deployment risk when introduced with a short max-age. The combination of a long max-age, includeSubDomains, and preload registration makes it a near-complete defense against protocol downgrade for the vast majority of users.

The practical takeaway: set max-age=31536000; includeSubDomains on your HTTPS responses today, submit to hstspreload.org once stable, and never serve any part of your site over HTTP again.

What problem does GraphQL solve?

Traditional REST and endpoint-based APIs have two common problems: over-fetching and under-fetching. Over-fetching happens when an endpoint returns a large fixed response and a client needs only a few fields — it downloads data it will never use. Under-fetching happens when a client needs data spread across several resources, so it has to make multiple round trips, one per endpoint, and then assemble the pieces.

GraphQL addresses both with a single endpoint and a declarative query. The client states the exact fields it needs, and the server resolves them from however many underlying data sources are required. The client gets one predictable response, shaped exactly like the query, with no more and no less than what it asked for.

A brief history

Facebook started developing GraphQL in 2012 and released a draft specification with a reference implementation as open source in 2015. In 2018 the project moved to the newly established GraphQL Foundation, hosted by the non-profit Linux Foundation. Since then many public APIs have adopted GraphQL, including those of GitHub, Shopify and Yelp. The specification is open, and the reference implementation GraphQL.js is MIT-licensed.

The schema and type system

A GraphQL service is built by defining a schema: the collection of types and fields that completely describes what the API can do. The schema uses a small type system that includes scalars (such as String, Int, Float, Boolean, and ID), object types, and a root Query type that lists everything that can be queried.

type Query {
  currentUser: User
}

type User {
  id: ID!
  name: String!
  email: String
  posts: [Post]
}

type Post {
  id: ID!
  title: String!
  published: Boolean!
}

A field is non-nullable when it ends with an exclamation mark, and a field can be defined as a list by wrapping its type in square brackets. The schema is the contract between client and server: the client can explore it and build queries that are guaranteed to match, and the server validates every request against it before execution.

Queries

A query defines the exact shape of the data the client needs. The server validates the query against the schema, executes it using functions called resolvers (one per field), and returns a result that mirrors the shape of the original query, typically as JSON.

query {
  currentUser {
    name
    email
  }
}

{
  "data": {
    "currentUser": {
      "name": "John Doe",
      "email": "john@example.com"
    }
  }
}

Because the client selects the fields, adding a field to the query changes only the response, not the endpoint. There is no need to version the API or spin up a new endpoint when a client needs more data — it simply asks for it.

Mutations and subscriptions

GraphQL has three kinds of operations. Queries read data. Mutations modify data — creating, updating or deleting it — and they accept input variables and can return data matching the mutation's shape so the client gets the result of the change in one round trip.

mutation CreatePost($title: String!) {
  createPost(title: $title) {
    id
    title
    published
  }
}

Subscriptions push live updates from the server to the client. A client subscribes once and receives updates whenever the relevant data changes, commonly transported over WebSockets. Subscriptions are how GraphQL supports real-time features such as notifications and live dashboards.

How GraphQL works over HTTP

GraphQL is typically served over HTTP: a client sends a query, typically as POST to a single endpoint such as /graphql, and receives the result. The GraphQL specification is agnostic about transport, so the same query language can work over other transports such as WebSockets for subscriptions.

One key difference from REST is the single endpoint. A REST API usually exposes many URL endpoints, one per resource, and uses HTTP methods such as GET and POST. A GraphQL API exposes one endpoint and lets the body of the request describe the operation and the fields wanted. HTTP remains the underlying protocol, so the caching, status and header semantics defined in RFC 9110 still apply to the transport.

GraphQL vs REST

The two are not a strict either/or. GraphQL is often considered an alternative to REST, but it does not replace it — a GraphQL server can even sit in front of REST APIs, aggregating them behind a single schema, which is a common pattern for a microservice architecture acting as an API gateway.

REST shines when APIs are simple, stable and well-understood, and it benefits from mature caching across the web. GraphQL shines when clients have varied and evolving data needs — especially mobile clients and front-end applications with many views — and when you want one strongly typed contract that clients can explore.

When to use GraphQL

Use GraphQL when different clients need different fields of the same data, when a client-side view composes data from several sources in one round trip, or when you want a self-documenting, strongly typed API. It is also a strong fit as an API gateway in front of multiple services.

• Mobile + web parity: a small screen and a large screen rarely want the same fields; GraphQL lets each ask for exactly what it renders.
• Aggregation: one schema that combines data from several back-end services without the client making many calls.
• Rapid front-end evolution: adding fields usually means editing the query, not the server.

Skip GraphQL when your API is a small set of stable endpoints, when you rely heavily on HTTP caching of raw responses, or when a straightforward REST surface is enough — GraphQL adds conceptual and operational overhead (resolvers, caching strategy, tooling) that is only worth paying when the flexibility pays off.

Security considerations

GraphQL introduces a few concerns specific to its design. Because a single request can ask for nested data, you should protect the server against expensive or abusive queries:

• Query depth limits: cap how deeply a query can nest to stop recursive queries from exhausting resources.
• Query cost / complexity limits: estimate and cap the total work a query represents.
• Timeouts: bound how long the server spends executing a single operation.
• Introspection control: introspection, which lets clients explore the schema, can be disabled in production if you do not want to expose the full schema.
• Variables and batching: as with any API, validate input and protect against attacks such as injection and denial of service, and consider batching attacks where many queries are sent in one request.

Many of the security concerns with GraphQL are the same ones that apply to any API — authentication, authorization, input validation — handled in your business logic layer with common patterns such as JWTs or OAuth.

Conclusion

GraphQL is an open, transport-agnostic query language that gives clients precise control over the data they receive. Its explicit schema, strong typing and three operation types (queries, mutations, subscriptions) solve over-fetching and under-fetching, and its single-endpoint design exposes a self-documenting contract. Choose it when your clients have varied and evolving data needs, and keep REST when a simple, stable set of endpoints and mature HTTP caching are the better fit. For the request/response side of API design, see the REST API with Node.js and Express guide, and for how these endpoints connect across services, read what an API gateway is.

FAQ

What is GraphQL vs REST?

GraphQL is a query language where the client asks for exactly the fields it needs from a single endpoint. REST is an architectural style where each resource has its own URL endpoint and a fixed response shape. GraphQL avoids over-fetching and under-fetching by letting the client shape the response, while REST benefits from simpler URLs and mature HTTP caching. They can coexist — a GraphQL server can aggregate multiple REST APIs behind one schema.

Is GraphQL a database language like SQL?

No. GraphQL is a specification for remote client-server communication and is agnostic to the data source. Unlike SQL, it does not query a database directly. A GraphQL server coordinates resolver functions, which delegate to a business-logic layer that talks to whatever data sources you use — databases, other APIs or in-memory data.

Do I need a GraphQL client?

No, you can start by issuing GraphQL requests with a regular HTTP client. GraphQL clients such as Apollo and Relay add batching, caching, build-time schema validation and UI updates, and become valuable as your application grows, but they are optional.

How do I secure a GraphQL API?

Apply the same protections as any API — authentication, authorization, input validation — and add GraphQL-specific guards such as query depth limits, query cost limits, timeouts, and controlling introspection. Resolve authorization in your business-logic layer, and authenticate with standard patterns such as JWTs or OAuth.

Is GraphQL only for JavaScript or React?

No. GraphQL is a specification that can be implemented in any language, and many runtimes exist across the ecosystem. It was introduced at a React conference and the reference implementation is in JavaScript, which is why it is associated with that stack, but it is transport- and language-agnostic.

Verdict

GraphQL is a powerful, open query language that earns its place when clients have diverse and evolving data requirements and you want a single, strongly typed, self-documenting API contract. It solves real problems — over-fetching, under-fetching and multiple round trips — that grow more painful as front-ends and clients multiply. The trade-off is added complexity in resolvers and caching, so match the tool to the surface: reach for GraphQL when flexibility pays, and reach for REST when a simple, stable, cache-friendly API is all you need.

What CI/CD and GitHub Actions mean

CI/CD is the combined practice of continuous integration (CI) and continuous delivery (CD). Continuous integration means merging small changes into your main branch frequently and verifying each change with automated builds and tests. Continuous delivery means releasing software in short cycles so reliable software can be deployed at any time with a simple, repeatable process. When CI and CD run together they form a pipeline.

GitHub Actions provides the platform to build that pipeline. Its four core components are:

• Workflow — a configurable automated process defined in a YAML file in the .github/workflows directory.
• Event — what triggers the workflow: a push, a pull request, a schedule, or a manual dispatch.
• Job — a set of steps executed on the same runner.
• Runner — a machine that runs a job; GitHub provides Linux, Windows, and macOS virtual machines (GitHub-hosted runners), or you can host your own.

A repository can have multiple workflows, each doing a different job: build and test every pull request, deploy on release, or add a label when an issue opens.

Prerequisites

• A GitHub repository (public or private) with your code committed.
• A project with defined build/test commands (npm test for Node.js, python -m pytest for Python, etc.).
• Basic familiarity with YAML.

GitHub Actions is free for public repositories, and free minutes are included for private repositories on the free plan. You do not need a separate CI server.

Step 1 — Create your first workflow file

Workflows live in a .github/workflows directory at the root of your repository. Create the file .github/workflows/ci.yml with the minimum valid workflow:

name: CI

on: push

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - run: echo "Hello from GitHub Actions"

Commit and push this file. Every push to the repository now triggers a run: GitHub looks in .github/workflows, finds a workflow whose on value matches the event, and starts the job. You can watch the run under the Actions tab of your repository.

Why actions/checkout? Each job starts on a fresh virtual machine with an empty working directory. The actions/checkout action clones your repository into the runner so later steps can access your code.

Step 2 — Configure triggers with on

The on key defines which events start the workflow. Common triggers:

on: push            # any push to any branch

on:
  push:
    branches: [ main ]      # only the main branch
    paths: [ "src/**" ]     # only when files under src change

on:
  pull_request:             # run on pull requests
    branches: [ main ]

on:                         # run on a schedule, cron syntax
  schedule:
    - cron: "0 8 * * *"

on: workflow_dispatch       # manual trigger from the Actions tab

You can combine events. For example, on: [push, pull_request]. Branch and path filters use glob patterns; quoted '*' and '**' match any number of characters.

Step 3 — Build and test your code

The most common CI goal is to verify your code builds and passes tests on every change. Here is a practical workflow for a Node.js project:

name: CI

on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"
      - run: npm ci
      - run: npm run build --if-present
      - run: npm test

Key points:

• actions/setup-node installs a specific Node.js version and adds it to PATH. It also caches dependencies (the cache: "npm" input) to make future runs faster.
• npm ci installs dependencies from the lock file — it is generally faster and more reliable than npm install.
• The --if-present flag makes the build step optional, so the same workflow works for projects without a build script.
• The matrix keyword lets you test multiple versions or operating systems in parallel without duplicating the job:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: ["18", "20"]
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm test

Each value in the node-version array creates a separate job running the same steps, using the matrix context to pick its version.

Step 4 — Store secrets and use them in workflows

Never hard-code credentials in a workflow file. Instead, store them as secrets on the repository (Settings → Secrets and variables → Actions) and reference them with the secrets context:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - run: npm test
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

GitHub redacts secrets in logs. Use the principle of least privilege: give each secret the minimum scope it needs, and rotate them if a workflow or repository is compromised.

Step 5 — Deploy automatically (continuous delivery)

The final step of a pipeline ships your verified code. A common pattern is to run tests first, then deploy only when tests pass. One workflow can do both by making the deploy job depend on the test job with needs:

name: CI/CD

on:
  push:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"
      - run: npm ci
      - run: npm test

  deploy:
    needs: test          # only runs if 'test' succeeds
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - name: Deploy
        run: ./deploy.sh
        env:
          DEPLOY_TOKEN: ${{ secrets.DEPLOY_TOKEN }}

The needs: test line makes the deploy job wait for the test job and run only if it succeeds — so broken code never deploys. In place of ./deploy.sh you would call the deploy action for your platform (for example, a Cloudflare Pages or static hosting deployment action) or run the CLI with your secret token.

Step 6 — Check, debug, and extend your workflow

• Open the Actions tab to see run status, logs, and failures.
• Each step's output is shown in the run log; failed steps are highlighted.
• Use continue-on-error: true on a step when a known failure should not block the job.
• Limit how long a job may run with timeout-minutes so a hanging job does not consume minutes forever.
• Store build/test artifacts (test reports, screenshots, logs) with the actions/upload-artifact action so you can inspect them after a run.

Common mistakes to avoid

• Missing .github/workflows path — workflows must live in that exact directory at the repository root.
• Hard-coded credentials — always use secrets, never commit tokens or keys in YAML.
• Forgetting actions/checkout — later steps cannot see your code without it.
• Unsafe shell interpolation — treat any value coming from a PR title or issue body as untrusted input when passing it to run:.
• Wrong action versions — pin the major version (e.g. actions/checkout@v6) and review new majors before upgrading.
• Ignoring the event context — a pull_request event runs in the context of the PR, so make sure your test commands are valid there too.

FAQ

Is GitHub Actions free?

Yes for public repositories, where workflows run at no cost. Private repositories include a monthly allowance of free action minutes on GitHub's hosted runners (the exact amount depends on your plan), after which you pay for additional minutes. Self-hosted runners use your own machines and have no per-minute fee.

What is the difference between CI and CD?

Continuous integration (CI) automatically builds and tests every change you merge, catching integration problems early. Continuous delivery (CD) automates deploying the verified build so reliable software can be released at any time with a repeatable process. CI/CD refers to the two working together in one pipeline.

How do I run a workflow only on the main branch?

Use the on.push.branches filter:

on:
  push:
    branches: [ main ]

The workflow then runs only when the push targets main. You can also add paths filters or use pull_request events with branch filters.

What is a GitHub Actions runner?

A runner is a machine that executes the steps of a job. GitHub provides GitHub-hosted runners (fresh Linux, Windows, or macOS virtual machines) with a set of preinstalled tools, or you can set up self-hosted runners on your own infrastructure.

How do I use secrets without exposing them in logs?

Reference secrets through the secrets context (${{ secrets.NAME }}) and set them as environment variables or action inputs rather than printing them. GitHub automatically redacts secret values in logs. Keep secrets scoped to the minimum permission needed and rotate them regularly.

Summary

GitHub Actions turns a GitHub repository into a CI/CD platform. You define workflows as YAML files in .github/workflows, trigger them with events like push and pull_request, build and test code on hosted runners, secure credentials with secrets, and deploy automatically once tests pass. Start with one small workflow that builds and tests your project, then extend it with a deploy job — a working pipeline for free on public repositories.

Why a branching strategy matters

A branching strategy gives a team a shared, predictable way to work with Git. Instead of each developer inventing their own branch names and merge rules, a workflow standardizes three things: where new work starts, where finished work goes, and how a release is produced. The result is fewer accidental conflicts, clearer code review, and a main branch whose state you can trust.

Branches themselves are a mechanism of version control: a branch is an independent line of development that forks off the mainline. Git makes branching cheap because its commits are content-addressed snapshots, so creating and merging branches is fast even in large repositories. Because branches are cheap, teams can afford to isolate work — the strategic question is how much isolation you want and for how long.

There are two broad philosophies. Feature-branch workflows isolate each piece of work on its own branch and merge it back when complete. Trunk-based development keeps a single mainline and integrates changes almost continuously. Most teams use something between these two extremes.

Trunk-based development

Trunk-based development (TBD) is a version control management practice where developers integrate small, frequent changes into a single main branch — the trunk — rather than working on long-lived feature branches. It is closely tied to continuous integration and continuous delivery, because the trunk is expected to remain in a deployable state at all times.

How it works

In a pure trunk-based workflow, developers commit directly to the main branch, or use very short-lived branches that are merged within hours. The key constraints are:

• One long-lived branch (main, sometimes called master or trunk).
• Changes are small and integrated frequently — often several times per day.
• Strong automated testing and builds guard the trunk, so integration is safe.

# Get the latest main and start work
git checkout main
git pull origin main

# Small change, committed directly to main
git add .
git commit -m "fix: handle empty response in API client"

# Or use a very short-lived branch that is merged the same day
git checkout -b fix/empty-response
git add .
git commit -m "fix: handle empty response in API client"
git checkout main
git merge fix/empty-response

Requirements and trade-offs

Trunk-based development works well for teams with fast release cycles, strong CI/CD, and a culture of small frequent commits. Because everything lands on one branch, it minimizes merge conflicts and keeps every developer working with the latest code. The trade-offs are real: it requires disciplined automated testing, and long-running or partially finished features need other mechanisms to stay safe — most commonly feature flags, which let you merge incomplete work behind a runtime switch that is off by default.

Trunk-based development is often associated with high-performing DevOps teams and is considered a foundation of continuous integration: if work sits on isolated branches for weeks before integration, you cannot call the process continuous integration, no matter how automated your builds are.

GitHub Flow

GitHub Flow is a simplified branching model built around a single main branch and short-lived feature branches, with pull requests (PRs) as the review and merge gate. It was designed for teams that deploy continuously and want a workflow that is easy to learn.

Branch structure

GitHub Flow uses just two kinds of branches:

• main — always deployable; everything is merged here and deployed from here.
• feature branches — created from main, live for hours or days, and deleted after merge.

There is no separate develop branch and no release branches; every feature branch is treated the same way.

The workflow

# 1. Create a feature branch from the latest main
git checkout main
git pull origin main
git checkout -b feature/add-user-dashboard

# 2. Work and commit
git add .
git commit -m "feat: add user dashboard widget"

# 3. Push and open a pull request
git push -u origin feature/add-user-dashboard
# Open a PR on the host, request review, discuss changes

# 4. After approval, merge on the host, then clean up locally
git checkout main
git pull origin main
git branch -d feature/add-user-dashboard

The defining rules of GitHub Flow are that main is always in a deployable state, and that any merging into main happens through a reviewed pull request. Because there is no dedicated release branch, the pipeline that deploys from main must be trustworthy — the model works because every merged change is immediately deployable.

When GitHub Flow fits

GitHub Flow is a good fit for small to medium teams deploying web applications and SaaS continuously, and for open-source projects where random contributors open PRs. It is simple, keeps every merge reviewed, and ships fast. It is less well suited to projects that must maintain multiple simultaneously supported versions, because there is no built-in mechanism for release branches.

GitFlow

GitFlow is a branching model introduced by Vincent Driessen in 2010 that adds a permanent develop branch and a set of supporting branch types to handle releases and hotfixes explicitly. It is stricter than GitHub Flow and is designed for projects with scheduled releases and a formal release process.

Branch structure

GitFlow defines two long-lived branches plus three types of short-lived supporting branches:

• main — always reflects the state of production, tagged for each release.
• develop — the integration branch where features are merged; the most recent "in-development" code.
• feature/ (from develop)* — new functionality.
• release/ (from develop, merged to main and develop)* — preparing a release: only bug fixes, version bumps, and documentation.
• hotfix/ (from main, merged to main and develop)* — urgent fixes to production.

The workflow

# Start a feature from develop
git checkout develop
git checkout -b feature/user-auth

# ... work on the feature, commit ...

# Merge the finished feature back into develop
git checkout develop
git merge --no-ff feature/user-auth
git branch -d feature/user-auth

# Prepare a release: branch off develop
git checkout develop
git checkout -b release/2.0.0
# bump version, fix last-minute bugs
git checkout main
git merge --no-ff release/2.0.0
git tag -a v2.0.0 -m "Release 2.0.0"
git checkout develop
git merge --no-ff release/2.0.0
git branch -d release/2.0.0

# Hotfix straight to production
git checkout main
git checkout -b hotfix/emergency-fix
# fix the bug, commit
git checkout main
git merge --no-ff hotfix/emergency-fix
git tag -a v2.0.1 -m "Hotfix 2.0.1"
git checkout develop
git merge --no-ff hotfix/emergency-fix

Strengths and weaknesses

GitFlow's strength is structure: it separates stable production code from integration work, supports scheduled releases cleanly, and gives hotfixes a clear, low-risk path to production. It is a reasonable choice for larger teams and software that ships on a defined cadence and must maintain multiple versions.

The cost is complexity. For teams deploying continuously, GitFlow adds overhead — two permanent branches to keep in sync, release-branch bookkeeping, and merges in two directions for every release and hotfix. Many commentators, and Driessen himself in later reflections, have noted GitFlow is best suited to teams that do not deploy continuously and that the simpler continuous-delivery workflows fit most modern products better.

Comparing the three strategies

How to choose the right strategy

There is no single best workflow — the choice depends on your release cadence, team size, and deployment process.

• Choose GitFlow if you ship on a defined schedule, need a formal release and QA phase, or must maintain multiple supported versions of the same product.
• Choose GitHub Flow if you are a small or medium team deploying continuously from main and want a simple, PR-driven workflow with little ceremony.
• Choose trunk-based development if you ship very frequently, have strong automated testing and CI/CD, and want to minimize merge conflicts — and you can pair it with feature flags to keep incomplete work out of the production path.

Whichever you pick, keep the workflow consistent, keep branches short-lived, and make the deploy pipeline trustworthy. The best strategy is the one your team can follow every day without friction.

FAQ

What is the difference between a branch and a fork?

A branch lives inside a repository and shares its history with the mainline. A fork is a copy of a whole repository on a hosting service, typically used when a contributor does not have write access to the original — changes travel back through pull requests. Branches are for isolating work within a project; forks are for collaborating across repositories.

Should I use GitFlow or GitHub Flow?

Use GitHub Flow if you deploy continuously, ship often, and want simplicity. Use GitFlow if you release on a schedule, need a formal release and QA phase, or must support multiple releases in parallel. For most modern web products, GitHub Flow or trunk-based development is the simpler, more appropriate choice.

What is a feature flag and why does trunk-based development need it?

A feature flag (or feature toggle) is a runtime switch that controls whether a piece of code is active. It lets you merge incomplete or risky work into the main branch behind a flag that is off by default, so the trunk stays deployable even while a feature is being built. Trunk-based development relies on flags because it has no long-lived feature branches to hide unfinished work.

How long should a feature branch live?

As a rule of thumb, a feature branch should live hours, not days, and rarely more than a couple of days. The longer a branch lives, the more it diverges from the mainline and the more merge conflicts and integration delays you accumulate. Trunk-based development keeps branches under a day; GitHub Flow keeps them short enough that a PR review can happen promptly.

What Flexbox is

Flexbox is the layout system you use for a single line of items in a row or a column. You turn any element into a flex container by setting display: flex (or display: inline-flex), and its direct children become flex items that the browser arranges along a main axis you control with the flex-direction property.

The four flex-direction values give you the four possible flows:

• row — items line up horizontally, left to right in a left-to-right language.
• row-reverse — same as row but the start and end lines are swapped.
• column — items stack vertically, top to bottom.
• column-reverse — same as column but the start and end lines are swapped.

The key insight from the MDN flexbox guide is that flexbox has two axes: the main axis (set by flex-direction) and a cross axis that runs perpendicular to it. Every flex property you write either moves things along one of these axes, or grows and shrinks items along them. The two most useful flex-item properties are the flex shorthand and the alignment properties.

A typical flexbox row with three items that share the available space is two lines of CSS:

.nav {
  display: flex;
}
.nav > a {
  flex: 1; /* flex: 1 1 0 — grow, shrink, zero base */
}

The flex shorthand expands to flex-grow, flex-shrink, and flex-basis in that order. Writing flex: 1 is equivalent to flex: 1 1 0, which means the items start at zero size and grow equally to fill the available space. Writing flex: 2 1 0 would give that item twice the share of the space.

Flexbox is also the right tool for centering an item inside its parent, which is famously awkward with earlier CSS. The web.dev flexbox guide walks through this exact pattern: justify-content aligns along the main axis, align-items aligns along the cross axis, and combining center on both gives you a centered item in any flex container.

The reason flexbox stays useful even when grid exists is that most layout work on a page is one-dimensional. A navigation bar, a toolbar, a row of action buttons, a list of cards that wraps to the next line, a footer with three columns of links — all of these are single-line layouts, and flexbox is the simplest, most predictable tool for each one.

What Grid is

CSS Grid is the layout system for two-dimensional layouts: when items need to line up in rows and columns at the same time. You turn any element into a grid container with display: grid (or display: inline-grid), and its direct children become grid items that the browser places into a grid you define.

The grid is defined with two properties: grid-template-rows and grid-template-columns. Each one lists the sizes of the rows and columns. The fr unit — short for "fraction" — is a grid-only length that represents a share of the available space in the container:

.page {
  display: grid;
  grid-template-columns: 1fr 2fr 1fr; /* three columns: 25% / 50% / 25% */
  gap: 16px;
}

This produces a page with a left sidebar, a main column twice as wide, and a right sidebar, with a 16-pixel gap between every track. The two outer columns are equal because they both have 1fr; the middle column is twice as wide because it has 2fr. If you resize the browser, all three columns resize proportionally without any media query.

The repeat() function and minmax() function take this further. A common pattern is a fluid card grid that always keeps the cards above a minimum readable width:

.cards {
  display: grid;
  grid-template-columns: repeat(auto-fit, minmax(250px, 1fr));
  gap: 16px;
}

The auto-fit keyword asks the browser to fit as many 250-pixel-minimum column tracks as it can into the container, and minmax(250px, 1fr) says each track can grow beyond 250 pixels to share leftover space. On a wide screen you might see four cards per row; on a tablet, three; on a phone, one. All from those two functions, with no media query.

The other defining feature of grid is that you can place items precisely. Grid lines are numbered from the start edge, and you can tell an item which lines to start and end at:

.header  { grid-column: 1 / 3; grid-row: 1; } /* spans columns 1 to 2, row 1 */
.sidebar { grid-column: 1;     grid-row: 2; }
.main    { grid-column: 2;     grid-row: 2; }

The MDN grid basics guide shows the full longhand (grid-column-start, grid-column-end, grid-row-start, grid-row-end) as well as the shorthand. Either form works; the shorthand is what you will see in real code.

Grid is the right tool for whole-page layouts, dashboard layouts, image galleries, form layouts where labels and inputs need to line up across rows, and any other case where the relationship between items in both dimensions matters.

The key differences

When Flexbox is the better choice

Reach for flexbox when the layout is genuinely one-dimensional, or when the items themselves should drive the sizing:

• Navigation bars and toolbars — a row of links or buttons that should share the available width, or right-align a single action button. Flexbox is the simplest, most readable way to express this.
• Evenly spaced groups of items — tags, chips, action menus, where you want them distributed with justify-content: space-between or space-around. The MDN justify-content reference lists every value (flex-start, flex-end, center, space-between, space-around, space-evenly, and a few writing-mode-aware ones).
• Centering a single element in its parent — combining justify-content: center and align-items: center on a flex container is the cleanest one-liner.
• Lists that wrap to a new line gracefully — setting flex-wrap: wrap and giving each item a flex-basis produces a row that reflows on narrow screens without any media query.
• Component internals — the inside of a card, the inside of a button, the inside of a form field. Flexbox rarely surprises you when the scope is one component.

If you are writing flexbox and find yourself setting width on a flex item to make it line up with the items above it, you are probably fighting the one-dimensional model. That is a strong signal to switch to grid.

When Grid is the better choice

Reach for grid when the layout needs both axes, or when you are designing a layout in advance and want items to fall into it:

• Whole-page layouts — header, sidebar, main, footer. The MDN relationship guide shows this exact pattern using grid-template-areas with named regions. It is the most readable CSS you can write for a page-level layout.
• Card grids and image galleries — the repeat(auto-fit, minmax(250px, 1fr)) pattern from earlier handles a fluid grid with no media queries and no JavaScript.
• Dashboards and admin panels — multiple panels that need to align both horizontally and vertically. Grid's align-items and justify-items keep every cell of the dashboard aligned without you having to track heights.
• Forms with aligned labels and inputs — grid is the only way to make a column of labels line up with the inputs across every row, because that is a two-axis problem.
• Overlapping content — cards with badges that overlay the top-right corner, image captions that sit on top of a hero image. Grid's grid-column / grid-row placement and z-index make overlapping items clean.

The MDN relationship guide also describes grid as "layout-in" — you create the layout first (the rows, the columns, the gaps), and then items slot into it. This is the opposite of flexbox's "content-out" model, where items ask for space and the container distributes it. Picking the right mental model is half the battle.

They work well together

You do not have to pick one and use it everywhere. The MDN relationship guide is explicit that grid is designed to be part of a complete system that includes flexbox. A typical real page uses grid for the page-level skeleton and flexbox for the navigation bar, the toolbar, the inside of every card, and every other one-dimensional component inside the grid cells. Treat them as two complementary layout systems, not two competitors.

A common pattern is a grid for the page skeleton, and a flexbox for the page header:

.page {
  display: grid;
  grid-template-rows: auto 1fr auto; /* header / main / footer */
  min-height: 100vh;
}
.page > header {
  display: flex;
  justify-content: space-between;
  align-items: center;
}

The outer grid makes the footer stick to the bottom of the viewport on short pages. The inner flexbox lays out the logo on the left and the navigation links on the right. Neither layout system was used for something the other does better.

FAQ

Can I use Flexbox and Grid on the same page?

Yes, and you usually should. Grid is the right tool for the page-level skeleton (the rows and columns that define where the major regions sit) and flexbox is the right tool for one-dimensional components inside those regions (the navigation bar, the toolbar, the contents of a card, the row of action buttons). The MDN relationship guide describes them as complementary parts of the same layout system, not alternatives. Most real-world designs use both.

Which is better for responsive design?

Neither is "better" — they solve different problems responsively. A flex container with flex-wrap: wrap reflows items to new lines as the container narrows, which is great for navbars and tag lists. A grid container with repeat(auto-fit, minmax(250px, 1fr)) keeps cards in a strict grid while still reflowing the number of columns. For page-level layouts that need to align in both dimensions, grid is the only realistic choice. For one-dimensional rows, flexbox usually needs fewer properties to express the same behavior.

When should I use Grid over Flexbox?

The MDN rule is the cleanest test: if you only need to control the layout by row or column, use flexbox; if you need to control it by row and column, use grid. Concrete signals to switch from flexbox to grid: you find yourself setting width on flex items to make them line up across rows, you need items to overlap, or you are building any kind of dashboard, card grid, or form layout.

Does Grid replace Flexbox?

No. Grid was added to CSS later, and for a moment the narrative was that grid was the modern replacement for flexbox. That turned out to be wrong. Flexbox is still the right tool for one-dimensional layouts, and the two systems now share most of their alignment properties through the CSS Box Alignment module. The web.dev flexbox guide and the MDN relationship guide both describe them as partners, not competitors.

Verdict

Pick flexbox for one-dimensional layouts (a row, a column, a wrapping list) and pick grid for two-dimensional layouts (a page, a card grid, a dashboard). If you find yourself disabling the flexibility of flexbox to force items into a grid shape, you want grid. If you find yourself creating a one-row grid to lay out a navigation bar, you want flexbox. Use them together: grid for the page skeleton, flexbox for the components inside.

The pieces

The runtime that runs your JavaScript is built from a few small, cooperating parts:

• Call stack — a LIFO (last-in, first-out) stack of execution contexts. The function you are inside is on top; the function that called it is below. When a function returns, its frame pops.
• Heap — where objects and closures live. Mostly out of the picture for the loop itself.
• Web APIs / Node APIs — the host's async primitives: setTimeout, fetch, addEventListener, file I/O, network sockets. These run outside the JS engine; when they finish, they place a callback into a queue.
• Task queue (a.k.a. macrotask queue) — a FIFO queue of tasks. Tasks include the initial script, event callbacks, setTimeout/setInterval callbacks, I/O completion callbacks.
• Microtask queue — a FIFO queue of microtasks. Microtasks include Promise.then/catch/finally callbacks, queueMicrotask() callbacks, and MutationObserver callbacks.
• Event loop — a simple loop that, on each iteration, runs one task, then drains the microtask queue, then optionally renders, then repeats.

The "host" here matters: the exact list of what counts as a task vs a microtask is defined by HTML in browsers and by the libuv model in Node.js. The two are not identical, but the shape is.

How one iteration works

In HTML terms, each iteration of the event loop does roughly this:

1. Pick the oldest task from the task queue. Run it to completion — that is, the call stack is never re-entered until the current task's top-level synchronous code finishes.
2. After the task ends, drain the microtask queue: keep popping microtasks and running them until the queue is empty. New microtasks scheduled while draining also run in the same pass.
3. If the document is visible, perform rendering and paint.
4. Go back to step 1 and wait for the next task.

Two guarantees follow from that shape:

• Run-to-completion — a task cannot be interrupted by another task. Once it starts, it finishes synchronously.
• Microtasks drain after every task — a microtask queued at any point will run before the next task starts, even if it was scheduled during rendering work or during the task itself.

That second guarantee is the one that explains most "why did the promise run first?" questions.

A worked example

Consider this snippet, which is the canonical event-loop interview question:

console.log("1");

setTimeout(() => console.log("2"), 0);

Promise.resolve().then(() => console.log("3"));

console.log("4");

Walk through it line by line:

1. console.log("1") runs synchronously on the call stack. Output: 1.
2. setTimeout(..., 0) schedules a task. The host's timer service remembers the callback and the (zero) delay; when the timer expires, the host places the callback into the task queue. Nothing runs yet.
3. Promise.resolve().then(...) schedules a microtask via the promise machinery. The callback is placed into the microtask queue.
4. console.log("4") runs synchronously. Output: 4.
5. The current task (the initial script) is now complete. The event loop drains the microtask queue: the promise callback runs. Output: 3.
6. The event loop picks the next task — the setTimeout callback — and runs it. Output: 2.

Final output: 1 4 3 2. The key idea: microtasks (promises) beat tasks (timers) by design.

Why microtasks first?

The HTML spec made microtasks higher priority than tasks for a practical reason: a promise resolution is supposed to feel synchronous to user code. If you write:

const result = await fetch(url);
use(result);

you want the use(result) callback to run as soon as fetch finishes — not to wait behind an unrelated setTimeout from another piece of code. Draining the microtask queue after every task makes that promise nearly always true, while still letting tasks carry the bulkier work (I/O, rendering, events).

The same rule is what makes queueMicrotask(fn) a useful primitive: it lets you schedule a callback to run after the current synchronous code, but before the next task — a guaranteed "very soon" without blocking.

How this looks in Node.js

Node.js implements the same model on top of libuv, with a few extra phases to handle I/O more fairly. The user-facing story is the same: microtasks (promise callbacks) run after the current task and before the next I/O or timer phase. Node also exposes process.nextTick(fn), which is a separate, even more eager queue that runs after the current operation and before any microtasks — a detail worth knowing because mixing it with promise code can produce surprising ordering.

If you debug "the server responded first but the promise logged after the timer", the loop is usually where the answer lives: the timer had a 0 ms minimum, the I/O callback was scheduled into a later phase, and the microtask drained in between.

Common pitfalls

• Long synchronous work blocks the loop. A while loop that takes 500 ms freezes input, timers, and rendering. The fix is the usual: chunk the work, or move it off the main thread with a Web Worker.
• Stack overflow on deeply nested calls. The call stack has a finite size; recursive code without a base case can throw RangeError: Maximum call stack size exceeded. Microtasks scheduled recursively from inside their own drain pass can also starve the loop — each new microtask runs before the queue is "empty", so rendering and tasks never get a turn.
• setTimeout(fn, 0) is "after tasks and microtasks", not "immediately". A 0 ms delay still yields to the loop. Use queueMicrotask(fn) if you really want the soonest possible callback.
• Order across I/O is not guaranteed by delay alone. Two fetch calls that complete at the same time will resolve in the order their responses arrived at the network, not the order you called them. Don't rely on the literal source order.

FAQ

What is the difference between a task and a microtask?

A task is the unit of work the event loop pulls on each iteration: the initial script, an event callback, a setTimeout callback, or an I/O completion. A microtask is a smaller, higher-priority unit that always runs after the current task and before the next task. Microtasks include promise callbacks (then/catch/finally), queueMicrotask() callbacks, and MutationObserver callbacks. The event loop drains the entire microtask queue after every task, so a queued microtask is guaranteed to run before the next task starts.

Why does Promise.then run before setTimeout(fn, 0)?

Promise callbacks are microtasks; setTimeout callbacks are tasks. The event loop drains the microtask queue after every task and before pulling the next task from the task queue. So in a single iteration, a microtask scheduled during the current task — like the one a resolved promise places — always runs before any newly-pending task, including one whose timer has already expired. This is a deliberate HTML spec choice: it makes promise resolution feel synchronous to user code.

Is the event loop part of JavaScript?

No. The event loop is provided by the host environment — the browser (per the HTML specification) or Node.js (per its libuv-backed model). The JavaScript language itself defines the execution model — a single thread of execution, a stack of execution contexts, a job queue for promise reactions — but the loop that ties it to timers, I/O, and rendering lives in the host. Different hosts can have different scheduler details while keeping the same user-facing shape.

What starves the event loop?

A microtask that schedules another microtask, recursively, will keep the microtask queue non-empty and starve tasks and rendering. Recursive Promise.resolve().then(...) chains are the classic example; a while loop that synchronously schedules microtasks is another. On the task side, a long synchronous computation — parsing a huge JSON, sorting a large array, a tight loop — blocks the call stack, which is the same thing as blocking the loop because nothing else gets a chance to run until the stack unwinds.

Summary

The event loop is the small, repeating rule that turns a single-threaded language into a responsive runtime: run one task, drain all microtasks, render if it's time, repeat. Once you internalize the stack/queue split and the microtask-first rule, the seemingly magical ordering of setTimeout, fetch, and Promise.then becomes obvious — and bugs caused by treating JavaScript as "multi-threaded under the hood" become much easier to spot.

What process.env is

In Node.js, process.env is a property that returns an object containing the user environment — the set of key-value variables the node process inherits from the shell or the environment you started it in. You read a value simply by accessing the property name:

console.log(process.env.NODE_ENV);
// "development" if you set NODE_ENV=development before starting the process

You can also set a value at runtime by assigning a property:

process.env.MY_VAR = "value";

In older Node.js versions assigning a non-string value was implicitly converted to a string; that behavior is deprecated and future versions may throw when the value is not a string, number, or boolean. Use delete process.env.KEY to remove a property.

Environment variables are not namespaced per app — they are inherited from the process that launched Node. That is exactly why they are good for configuration that differs between your laptop and production: the same code reads process.env.DB_URL, and each environment supplies its own value.

Setting variables in the shell

Before the process starts, you set variables in the environment. In a POSIX shell:

NODE_ENV=production node app.js

or export it for the current session:

export DB_URL=postgres://localhost:5432
node app.js

Values set this way are visible to the process through process.env. For local development, however, exporting dozens of values by hand is tedious, which is why .env files exist.

Reading .env files

A .env file (also called a dotenv file) is a plain-text list of KEY=VALUE pairs that define environment variables an application can load. The format was popularized by the dotenv package, and modern Node.js can read these files natively without any dependency.

Option 1: Node's built-in --env-file (Node.js 20.6+)

Since Node.js 20.6 you can load a .env file directly from the command line with no package install. If you have a .env file:

DB_URL=postgres://localhost:5432
PORT=8080

start the process with:

node --env-file=.env app.js

The variables in the file populate process.env, so your code reads them the same way as any other environment variable. If the same variable is defined both in the shell environment and in the file, the value from the environment takes precedence. You can pass multiple --env-file arguments, and later files override variables set by earlier ones; --env-file-if-exists is the variant that silently skips a missing file instead of failing.

Node.js also exposes two programmatic helpers: process.loadEnvFile(path) loads a file and populates process.env, and util.parseEnv() parses raw .env content into an object.

Option 2: the dotenv package

The dotenv package loads a .env file at runtime. Install it, call config() early in your entry file, and process.env is populated:

npm install dotenv

require("dotenv").config();
console.log(process.env.DB_URL);

You can point it at a custom path with dotenv.config({ path: "/custom/path/to/.env" }). dotenv is a fine choice when you want the loader inside your code or need to support older Node.js versions, but with a modern Node.js release the built-in --env-file flag removes the dependency entirely.

.env file format rules

The .env format follows a few simple rules, and both Node's implementation and dotenv honor them:

• Variable names must match ^[a-zA-Z_]+[a-zA-Z0-9_]*$ — start with a letter or underscore, then letters, digits, or underscores. MY_VAR, my_var, and My_Var123 are valid; 1_VAR, my-var, my var, and trailing VAR_ are not.
• Values are arbitrary text and may be wrapped in single or double quotes. Quoted values can span multiple lines; unquoted ones are single-line. Values are always parsed as text — 0, true, and {"hello":"world"} all become the literal strings "0", "true", and '{"hello":"world"}', not a number, boolean, or object.
• Spacing around keys and values is ignored unless enclosed in quotes.
• Comments start with # and run to the end of the line; a # inside quotes is a normal character.
• export prefixes are allowed and ignored — useful when you also want to source the same file in a shell.

Example:

# application config
NODE_ENV=development
DB_URL="postgres://localhost:5432"
# a value with a literal hashtag must be quoted
API_TOKEN='abc#123'

Why secrets must not go in code

Environment variables solve a real security problem: configuration and credentials should not be committed to source control. Hardcoded values in code end up in your repository history, where anyone with access — and anyone who consumes a leaked snapshot — can read them. A .env file that contains real secrets should be listed in .gitignore so it is never committed, while a committed .env.example file documents which variables the app needs, with placeholder values.

Separating secrets from code also makes rotation possible without a redeploy. When a key leaks or expires, you update the variable in the deployment platform and restart, instead of editing source and shipping a new build. Managed secret stores — a secrets manager such as 1Password or a purpose-built store such as HashiCorp Vault — take this further by injecting values into the runtime environment or into deployment pipelines, which is the approach most mature teams move toward.

Environment variables in deployment and CI

The same code that reads process.env in development reads it in production. Hosting platforms let you set environment variables in their dashboard or config rather than in code, and CI/CD systems expose them as encrypted secrets. In a CI pipeline such as GitHub Actions, you define secrets in the repository settings and reference them in the workflow, so keys never appear in the workflow file or the log. Containerized apps follow the same principle — tools like Docker pass environment variables into containers, keeping runtime configuration out of the image.

FAQ

Should I use --env-file or the dotenv package?

Use Node.js's built-in --env-file flag (Node.js 20.6+) when you can pick the Node.js version — it needs no dependency and is the most direct option. Use the dotenv package when you need to load the file from inside your code, call it at a specific point at runtime, or must support an older Node.js version. Both read the same .env format. The native flag also has the advantage that the loader runs before your application code, so configuration is ready from the first line.

Are environment variables secure?

Environment variables keep secrets out of your source code and out of version control, which is a meaningful improvement — but they are not a complete security boundary. Anyone who can run code or read memory on the host can read them, and they can leak through logging or client-side exposure if you ship them to the browser. Never hardcode or log secrets, keep real ones out of committed .env files (.env should be in .gitignore), and use a dedicated secrets manager for the most sensitive credentials.

What is the difference between process.env and a .env file?

process.env is the live object Node exposes, pre-populated from the environment the process was launched in. A .env file is just a convenient on-disk format for declaring those variables so they can be loaded (via --env-file or dotenv) instead of being typed by hand. Loading a .env file populates process.env; it does not create a separate store.

How do I set environment variables in production?

You do not read or set them in your application code. Instead, configure them in your hosting platform's dashboard, config file, or deployment manifest, and for CI define them as encrypted repository secrets. The running process receives them just like local variables, so your code reads process.env unchanged across every environment.

Verdict

Use process.env to read configuration, and prefer Node.js's native --env-file flag (Node.js 20.6+) for loading local .env files so you need no extra dependency — or reach for dotenv if you must load from inside your code on an older runtime. Keep every real secret out of source control: put .env in .gitignore, commit an .env.example of placeholders, and let your deployment platform or a secrets manager like 1Password or HashiCorp Vault inject production values into the environment. The rule that makes all of it safe is simple: configuration belongs in the environment, never in the code.

What Docker is: the container platform

Docker is an open platform for developing, shipping, and running applications. Its core idea is the container: a lightweight, isolated environment that packages your application code together with its runtime, system libraries, and settings. Containers are self-contained and run the same way on a developer's laptop, a test server, or a production cloud VM — because they carry everything they need and do not depend on what is already installed on the host.

A container is not a virtual machine. A VM runs a full guest operating system on top of a hypervisor; a container shares the host operating system's kernel and runs as an isolated process. Because containers share the kernel and are not booting a whole OS, they are much lighter: container images are typically tens of megabytes, and a container can start in seconds or less.

Two key Docker concepts to hold onto:

• An image is a read-only template with instructions for creating a container — often built from another image, plus your application and its configuration. You define how to build it in a Dockerfile.
• A container is a runnable instance of an image. You can start, stop, and restart it, and anything you write inside it that is not stored in a persistent volume disappears when the container is removed.

Docker was launched in 2013 as an open-source project. It built on Linux primitives called cgroups and namespaces: cgroups limit and account for CPU, memory, and I/O, while namespaces isolate processes, networking, filesystems, and users so each container behaves as if it owns the machine. Docker also standardized how containers are built and shared through registries such as Docker Hub.

What Kubernetes is: the orchestration platform

If Docker answers "how do I run one app in a container?", Kubernetes answers "how do I run and manage many containers across many machines, reliably?" Kubernetes (from the Greek for helmsman or pilot, abbreviated k8s) is the open-source system for automating deployment, scaling, and management of containerized applications.

Google open-sourced Kubernetes in June 2014, drawing on more than fifteen years of experience running production workloads at scale — an internal cluster-management system called Borg, and its successor Omega, pioneered the ideas. Version 1.0 shipped on July 21, 2015, and the same year Google donated the project to the newly formed Cloud Native Computing Foundation (CNCF), a Linux Foundation body. Kubernetes was the CNCF's first project to graduate from incubation, which happened on March 6, 2018.

A Kubernetes deployment is called a cluster, and it has two parts:

• The control plane makes global decisions about the cluster — scheduling work, detecting and responding to events — and exposes the API you talk to.
• Worker nodes are the machines (physical or virtual) that actually run your applications. Each node runs a kubelet agent that makes sure the containers are running as requested, and a kube-proxy that maintains network rules.

The smallest deployable unit in Kubernetes is the Pod: one or more tightly coupled containers that share a network namespace and volumes. Higher-level objects like Deployments and Services build on pods to give you replicas, rolling updates, stable networking, and self-healing.

What Kubernetes actually gives you, in plain terms:

• Self-healing — if a container or node fails, Kubernetes restarts or reschedules the work to meet the state you declared.
• Scaling — you say how many replicas you want, and Kubernetes adds or removes them, including automatically based on load.
• Rolling updates and rollbacks — you can ship a new version without taking the service down, and roll back if it misbehaves.
• Service discovery and load balancing — a stable address that routes traffic to the healthy pods of a service.

The key difference: one container vs a fleet

Docker is about containers — packaging and running a single unit of software. Kubernetes is about orchestration — managing a fleet of containers distributed across a cluster, keeping them healthy, scaled, and reachable.

You can run Docker without Kubernetes: a single command, docker run my-app, starts one container. That is often enough for a local project or a small service. Kubernetes without Docker does not really apply to this question — Kubernetes runs containers (Docker-provided or via other container runtimes), so it sits on top of the packaging that tools like Docker provide.

Crucially, Kubernetes is not a faster or "better Docker." It is a much larger and more complex piece of infrastructure. Running a production Kubernetes cluster means operating the control plane, handling node upgrades, securing the API, and maintaining the surrounding ecosystem of networking and storage. That overhead only pays off when you have enough moving parts that manual container management becomes the bottleneck.

When you only need Docker

You probably only need Docker when you are running one application, or a handful of them on one machine:

• Developing or demoing a service locally with a consistent runtime.
• Deploying a single containerized app to a single server or VM.
• Using Docker Compose to run a small multi-container app (for example a web server plus a database) on one host — Compose manages a set of containers on a single machine without a cluster.

For one or a few containers on one machine, a cluster manager adds nothing but complexity. You can start with docker run, then move to Compose when you have a few services, and keep using Docker for months or years without ever touching Kubernetes.

When Kubernetes becomes worth it

Consider Kubernetes when you are running many containers across multiple machines, and you care about any of these:

• High availability — your service must survive a machine or container failing.
• Horizontal scaling — you need many replicas that grow and shrink with load, across nodes.
• Rolling deployments — you ship continuously and want zero-downtime updates and easy rollbacks.
• Multi-service consistency — many services share networking, config, and secrets in one declarative system.

The honest guideline: if you can run your workload on one machine, you do not need Kubernetes yet. If your workload spans several machines and one of the above concerns is real, Kubernetes starts earning its complexity. Many teams take the path Docker → Docker Compose → a managed Kubernetes service (like those offered by the major cloud providers, where the control plane is operated for you) rather than running the control plane themselves.

Docker vs Kubernetes compared

FAQ

Are Docker and Kubernetes competitors?

No. Docker standardizes how you build and package a single container; Kubernetes manages many containers running across many machines. They work together: Kubernetes schedules and runs the containers that tools like Docker help you build. The more accurate mental model is "Docker for packaging, Kubernetes for orchestration."

Do I need Kubernetes if I use Docker?

Only if you run many containers across multiple machines and need high availability, scaling, or rolling deployments. For one or a few containers on a single machine, Docker (optionally with Docker Compose) is enough — adding Kubernetes would only add operational complexity.

Can Kubernetes run without Docker?

Kubernetes can use other container runtimes besides Docker, but the question of "Kubernetes vs Docker" is really "do you need orchestration?" Kubernetes still runs containers; it manages the ones you package, however you build them. Orchestration does not replace packaging.

Should I start learning Docker before Kubernetes?

Yes. Kubernetes assumes you understand images, containers, and how to run them. Learn Docker, get comfortable running and shipping a container, add Docker Compose for multi-container apps, and move to Kubernetes only when a real deployment outgrows a single machine.

Verdict

Docker and Kubernetes are partners, not rivals. Start with Docker for reliable, reproducible single-container development and deployment. Reach for Docker Compose when a few services run together on one host. Move to Kubernetes when your workload spans several machines and you need self-healing, scaling, or rolling deployments — and if possible, start with a managed Kubernetes service to hand off the control-plane burden. When in doubt, the simpler option is usually right: you can always adopt orchestration after a real deployment demands it.

What containerization and Docker mean

A container is a runnable instance of an image — a lightweight, standalone, executable package that includes everything the software needs to run: code, runtime, system tools, libraries, and settings. Containers share the host operating system kernel and run as isolated processes, which is why they start in seconds and use far fewer resources than a full virtual machine.

Docker is a platform for building, sharing, and running containers. Its main pieces are:

• Image — a read-only template with instructions for creating a container.
• Container — a runnable instance of an image.
• Dockerfile — a text file of instructions that build an image.
• Docker Engine — the background service that builds and runs images.
• Docker Hub / registries — places to store and share images.

Because the image bundles the exact runtime and dependencies, the same image behaves identically on a laptop, a test server, or a cloud production machine. This removes the classic "it works on my machine" problem.

Prerequisites

• Docker installed and running — Docker Desktop on Windows and macOS, Docker Engine + CLI on Linux. Verify with docker --version.
• A Node.js project — at minimum a package.json with a start script and a source entry point (for example index.js).
• Basic familiarity with the command line.

You do not need Node.js installed locally to build or run the container, because the base image provides the runtime.

Step 1 — Understand the parts of a Dockerfile

A Dockerfile is a build script for your image. Each instruction is a line that Docker executes, and each instruction creates a layer. Common instructions:

• FROM — the base image your image starts from (the runtime and OS it builds on).
• WORKDIR — sets the working directory for the commands that follow.
• COPY — copies files or directories from your build context into the image.
• RUN — executes a command during the build (used to install dependencies).
• ENV — sets an environment variable.
• EXPOSE — documents the port the app listens on (informational; it does not publish the port).
• CMD — the default command run when a container starts from the image.

The simplest possible Node.js Dockerfile looks like this:

FROM node:20
WORKDIR /app
COPY . .
RUN npm install --omit=dev
EXPOSE 3000
CMD ["node", "index.js"]

This is a valid starting point, but it has problems: it copies your whole directory, installs everything including tools you only need for building, and can be large. The next sections improve it.

Step 2 — Choose a good base image and use .dockerignore

The base image in FROM provides the runtime. For Node.js, the official images are published as node:<version> with tags such as node:20, node:20-alpine, and node:20-slim. The plain variant is based on Debian, while the -alpine variant is much smaller because it is built on Alpine Linux. Smaller images are faster to pull and have a smaller attack surface, so a common choice for the final runtime stage is an -alpine or -slim variant.

Create a .dockerignore file in your project root. It works like .gitignore and tells docker build which files to leave out of the build context, keeping the build faster and preventing local folders such as node_modules from being copied in:

node_modules
npm-debug.log
.git
.env
.containerignore
Dockerfile
.dockerignore

If you already have a local node_modules, excluding it here prevents the build from copying the wrong (host-specific) dependencies into the image.

Step 3 — Install only what you need

Inside the container you install dependencies with npm. A key detail is the difference between npm install and npm ci:

• npm ci installs from the lock file (package-lock.json) and is faster and more reproducible — it is the recommended choice when a lock file is present.
• npm install --omit=dev installs only production dependencies, skipping devDependencies such as test frameworks and build tools.

A production-first install looks like:

FROM node:20-alpine
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY . .
EXPOSE 3000
CMD ["node", "index.js"]

Copying package.json and package-lock.json before the rest of the source lets Docker cache the dependency-install layer. Because package.json changes far less often than your source code, Docker reuses the cached layer on rebuilds and only reinstalls when the lock file actually changes — this makes most rebuilds much faster.

Step 4 — Use a multi-stage build to keep the image small

Many Node.js projects need a build step: compiling TypeScript, bundling a frontend, or running a build tool. If you run that build inside the same image that ships to production, the final image carries the build tools and dev dependencies with it — unnecessarily large.

A multi-stage build uses multiple FROM instructions. Each FROM starts a new stage, and you copy only selected artifacts from an earlier stage into the final one with COPY --from. The earlier stages are discarded, so the final image contains only what the app needs at runtime.

# Stage 1 — build
FROM node:20 AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2 — production runtime
FROM node:20-alpine
ENV NODE_ENV=production
WORKDIR /app
COPY --from=build /app/package.json ./
COPY --from=build /app/dist ./dist
RUN npm ci --omit=dev
EXPOSE 3000
CMD ["node", "dist/index.js"]

Here the build stage installs all dependencies (including dev ones) and runs npm run build, and the final stage copies only the built output and the lock file, then installs only production dependencies. The result is a small image that ships exactly the runtime artifacts.

Step 5 — Handle environment variables and non-root users

Configuration that changes between environments — database URLs, API keys, feature flags — should come from environment variables, not be baked into the image. You can set a default in the image with ENV, but you override it at runtime.

FROM node:20-alpine
ENV NODE_ENV=production
ENV PORT=3000
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY . .
EXPOSE 3000
CMD ["node", "index.js"]

Pass real values when you run the container:

docker run -e PORT=8080 -e DATABASE_URL=postgres://user:pass@host/db -p 8080:8080 my-app

Never store secrets in the image — do not put tokens or passwords in the Dockerfile or copy an .env file in. Pass secrets at runtime with -e (or a secrets mechanism) instead.

It is also good practice to avoid running the app as root. The official node images already provide a node user; switch to it before starting the app:

FROM node:20-alpine
WORKDIR /app
COPY . .
RUN npm ci --omit=dev
USER node
ENV PORT=3000
EXPOSE 3000
CMD ["node", "index.js"]

Running as a non-root user reduces the impact if the container is compromised.

Step 6 — Build and run the container

With the Dockerfile in place, build the image from your project root:

docker build -t my-app .

The -t my-app gives the image a name and tag. (The trailing . is the build context directory.) You can view it with docker images.

Run a container from the image:

docker run -p 3000:3000 my-app

The -p 3000:3000 maps port 3000 on your host to port 3000 in the container — this is what makes the app reachable from your browser. (The EXPOSE line in the Dockerfile only documents the port; the -p flag actually publishes it.) Open http://localhost:3000 to see your app.

Run it detached in the background and give it a name:

docker run -d --name my-app -p 3000:3000 my-app

Manage containers with docker ps (running), docker ps -a (all), docker logs my-app (logs), docker stop my-app, and docker rm my-app. To remove an unneeded image, use docker image rm my-app.

The ordinary workflow is: change your code → docker build again → docker run the new image → test. Because the dependency layers are cached, iterative rebuilds after the first are fast.

Common mistakes to avoid

• Missing .dockerignore — your local node_modules gets copied into the image, producing a huge, broken image.
• Installing dev dependencies in production — use npm ci --omit=dev in the final stage.
• Copying all source before installing — copy package.json/package-lock.json first to keep the dependency cache valid.
• Ship the build environment — a single-stage build with a build step ships build tools; use a multi-stage build to copy only the output.
• Hard-coding secrets — never bake tokens or .env files into the image; pass them at runtime.
• Forgetting -p — EXPOSE only documents the port; you must publish it with -p to reach the app from your host.
• Running as root — use the non-root node user unless you have a concrete reason not to.

FAQ

What is the difference between an image and a container?

An image is a read-only template with the instructions (layers) for creating a container. A container is a runnable instance of that image. You can build one image and run many containers from it, each isolated on the host.

Is Docker free to use?

Docker Engine and the Docker CLI are open source, and the Docker platform offers a free tier; Docker Desktop's licensing has a paid tier for larger companies. The container format and the docker build/docker run commands themselves are free to use on your own machines, and many container registries offer free public storage.

Why use a multi-stage build?

A multi-stage build separates building from running. You do the heavy work (installing dev dependencies, compiling, bundling) in a build stage, then copy only the needed artifacts into a slim final stage. The final image is smaller, faster to pull, has a smaller attack surface, and does not leak build tools or secrets.

What does -p 3000:3000 do?

It publishes the container's port 3000 to port 3000 on your host, so the application is reachable at http://localhost:3000. The left number is the host port, the right the container port; they can differ (for example -p 8080:3000).

Do I need Node.js installed to run the container?

No. The base image (node:<version>) contains the Node.js runtime. The whole point of containerizing is that the image bundles everything the app needs, so the machine only needs Docker — not a local Node.js install.

Summary

Containerizing a Node.js app is a small, repeatable process: write a Dockerfile that selects a base image, copies your manifest first for fast cached installs, installs only production dependencies, exposes and maps the port, and starts your app with CMD. Use a .dockerignore to keep the build clean, use a multi-stage build when you have a build step so the production image stays small, keep secrets and environment configuration out of the image, and run as a non-root user. Then docker build -t my-app . and docker run -p 3000:3000 my-app give you a portable containerized version of your app that runs identically anywhere Docker is installed.

What Docker Compose is

Docker Compose is a tool for defining and running multi-container Docker applications. It controls your whole stack — services, networks, and volumes — in a single YAML configuration file. With one command you create and start all the services from that file, instead of running many long docker run commands with dozens of flags.

Compose lets you:

• Start, stop, and rebuild services
• View the status of running services
• Stream the log output of running services
• Run a one-off command on a service

Compose is included with Docker Desktop (Windows and macOS) and with the Docker Engine on Linux, so after installing Docker you usually already have it. Verify with docker compose version.

The Compose file

The default file is compose.yaml in your project directory (older docker-compose.yml is also accepted). It is written in YAML, and its most important top-level elements are services, volumes, and networks:

• services — the containers to run, each built from an image or a Dockerfile.
• volumes — named storage that persists data between container restarts.
• networks — how services communicate (Compose creates a default network for you).

A minimal two-service Compose file for a Node.js app plus a Redis cache looks like this:

services:
  web:
    build: .
    ports:
      - "3000:3000"
    volumes:
      - .:/app
      - /app/node_modules
    depends_on:
      redis:
        condition: service_healthy
    environment:
      - REDIS_HOST=redis
      - REDIS_PORT=6379

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 10s

Here the web service is built from the Dockerfile in the current directory (note build: .), while redis uses the public redis:7-alpine image pulled from Docker Hub. Every Compose project gets a default network, so web can reach redis by its service name.

Starting the stack

The command that creates and starts every service from your file is:

$ docker compose up

This builds the images that need building, pulls any remote images, and starts all containers. Running it in the foreground shows streaming logs from every service. To run in the background and free your terminal, add the detached flag:

$ docker compose up -d

Viewing status and logs

Check the state of all services:

$ docker compose ps

Follow log output from all services, color-coded by service name:

$ docker compose logs -f

To follow logs for one service only, pass its name:

$ docker compose logs -f web

Running commands inside a running container

docker compose exec runs a command inside an already-running container — the primary tool for live debugging. For example, to check that your Node.js app can reach Redis, resolve the service name:

$ docker compose exec web node -e "const r=require('redis');console.log('ok')"

To confirm environment variables propagated correctly:

$ docker compose exec web env | grep REDIS

Environment variables with .env

Keep configuration out of the Compose file where possible. Compose automatically reads a .env file in the project directory and makes its values available for interpolation in your YAML. For example, with .env containing:

APP_PORT=3000
REDIS_HOST=redis
REDIS_PORT=6379

You can reference these in compose.yaml as ${APP_PORT}, ${REDIS_HOST}, and ${REDIS_PORT}. Keeping values in .env makes it easier to change configuration across environments without editing YAML.

Health checks and startup order

Containers start in parallel, so a web app often tries to connect to a database before it is ready. A health check tells Compose when a service is truly ready, and depends_on with a condition makes one service wait for another.

The healthcheck block on redis runs redis-cli ping and expects a PONG reply. The start_period of 10 seconds lets Redis initialize before checks begin, and interval/timeout/retries control how often the check runs and how many failures are tolerated.

depends_on with condition: service_healthy makes web wait until redis is confirmed healthy before starting. This fixes the classic startup race where the app connects before the cache is up.

Persisting data with named volumes

Container filesystems are ephemeral — when you remove a container, everything written inside is lost. A named volume keeps data on the host and reattaches it on the next up. Attach a volume with volume-name:/container/path:

services:
  redis:
    image: redis:7-alpine
    volumes:
      - redis-data:/data

volumes:
  redis-data:

The top-level volumes key registers redis-data so it persists across docker compose down and up cycles. This is how you keep a database or cache between restarts.

Compose Watch for live reload

Compose's develop.watch block gives live updates during development. Running docker compose up --watch, Compose watches files and applies rules you define:

services:
  web:
    build: .
    develop:
      watch:
        - action: sync+restart
          path: .
          target: /app

• sync+restart copies changed files into the running container, then restarts it.
• rebuild triggers a full image rebuild — useful when you add a dependency (for example a new package.json entry) that requires reinstalling packages.

Inspecting config before it runs

docker compose config shows the fully resolved configuration without starting anything. It works purely from your files and is invaluable for debugging variable substitution and file merging — it shows environment variables replaced, port notation expanded into canonical fields, and default network and volume names made explicit.

Tearing down the stack

Stop and remove the containers (named volumes are kept):

$ docker compose down

Remove containers and named volumes too (this permanently deletes stored data — use intentionally):

$ docker compose down -v

FAQ

What is the difference between Docker and Docker Compose?

Docker builds and runs individual containers; Docker Compose orchestrates multiple containers as one application defined in a single YAML file. You use Docker to build and run one image, and Compose to define and run a whole stack of services, networks, and volumes together.

Do I need a separate install for Docker Compose?

No. Docker Compose is included with Docker Desktop on Windows and macOS and with the Docker Engine on Linux. After installing Docker, verify it with docker compose version.

What does docker compose up do?

It reads your compose.yaml, builds any services with a build: directive, pulls any remote images, and starts all containers. Adding -d runs the stack in detached (background) mode; --watch enables live reload during development.

How do I keep data when I restart a container?

Use a named volume. Declare it in the top-level volumes key and mount it at the data path (for example redis-data:/data). Named volumes persist across docker compose down and up. To delete stored data as well, run docker compose down -v.

What DNS records do

DNS (Domain Name System) is the phone book of the internet. When you visit example.com, your browser asks a DNS resolver for the IP address of that domain. The resolver queries authoritative nameservers, which return DNS records that map your domain name to specific destinations: web servers, mail servers, verification tokens, or other domains.

Every DNS record has a type (what it does), a name (usually @ for the root domain or a subdomain), a value (the destination), and a TTL (Time To Live — how long resolvers cache the record before asking again). Most hosting providers give you a DNS management panel where you add and edit these records.

A record — point a domain to an IPv4 address

The A record maps a domain or subdomain to a 32-bit IPv4 address (for example, 192.0.2.1). This is the most common record type — it tells browsers the numeric address of your web server.

Example — root domain to a server:

Type: A
Name: @
Content: 192.0.2.1
TTL: 300

Example — www subdomain redirect:

Type: A
Name: www
Content: 192.0.2.1
TTL: 300

If your hosting provider gives you a dedicated IP address, use an A record. Many modern platforms (Cloudflare Pages, Netlify, Vercel) use CNAME records instead of A records.

AAAA record — point a domain to an IPv6 address

The AAAA record (quad-A) maps a domain to a 128-bit IPv6 address (for example, 2001:db8::1). It works identically to an A record but for the newer IP addressing scheme. Most websites have both A and AAAA records so visitors on both IPv4 and IPv6 networks can reach the site.

Example:

Type: AAAA
Name: @
Content: 2001:db8::1
TTL: 300

If your host does not provide an IPv6 address, you can skip the AAAA record for now. Both records are optional, but having both improves reachability.

CNAME record — alias one domain to another

A CNAME record makes one domain name an alias for another. When a resolver looks up a CNAME, it gets the canonical name and then resolves that instead. CNAMEs are commonly used for subdomains that point to a third-party service.

Example — www pointing to root domain:

Type: CNAME
Name: www
Content: @
TTL: 3600

Example — custom subdomain for a third-party service:

Type: CNAME
Name: crm
Content: salesforce.com
TTL: 3600

A CNAME cannot be set on the root domain (@) in most DNS systems. The root domain must use an A or AAAA record. This is a common mistake — if you try to create a CNAME on @, the DNS provider will reject it.

MX record — direct email to a mail server

The MX (Mail Exchange) record tells mail servers where to deliver email sent to your domain. Unlike other records, MX records point to mail server hostnames, not IP addresses, and include a priority value.

Lower priority = higher preference. If you list multiple MX records, mail servers try the lowest number first and fall back to higher numbers if the first server is unavailable.

Example — Google Workspace mail:

Type: MX
Name: @
Content: ASPMX.L.GOOGLE.COM
Priority: 1
TTL: 3600

Example — multiple mail providers (backup):

Type: MX    Name: @    Content: mail.primary.com    Priority: 10
Type: MX    Name: @    Content: mail.backup.com    Priority: 20

MX records must always point to a hostname — never an IP address. If you are using a managed email service (Google Workspace, Microsoft 365, Proton Mail), your provider gives you specific MX values to enter.

TXT record — verification, SPF, DKIM, and DMARC

TXT records store arbitrary text and serve several purposes: domain ownership verification, email authentication (SPF, DKIM, DMARC), and security records.

Domain verification

When you connect a domain to a service (Google Workspace, Microsoft 365, a CDN, or an SSL provider), the service asks you to add a TXT record to prove you control the domain.

Type: TXT
Name: @
Content: google-site-verification=abc123xyz789
TTL: 3600

The TXT value is unique to your account. Copy it exactly — spaces and quotes matter.

SPF — specify which servers can send email from your domain

SPF (Sender Policy Framework) is a TXT record that lists the mail servers authorized to send email on behalf of your domain. Receiving mail servers check this to detect forged sender addresses.

Type: TXT
Name: @
Content: v=spf1 include:_spf.google.com ~all
TTL: 3600

• v=spf1 — SPF version 1
• include:_spf.google.com — allow Google's servers to send mail
• ~all — softfail (mail from other servers is suspicious but not rejected)

For a simple setup using one email provider, the provider gives you the full SPF string. For complex setups (multiple providers), you add include: entries for each.

DKIM and DMARC

DKIM (DomainKeys Identified Mail) and DMARC (Domain-based Message Authentication, Reporting and Conformance) are more advanced email authentication records. Many email providers set these up automatically or provide the exact records to add.

DNSSEC — signing your DNS data

DNSSEC adds cryptographic signatures to your DNS records so resolvers can verify the records have not been tampered with in transit. When you enable DNSSEC, your domain registrar publishes a DS (Delegation Signer) record in the parent zone.

Note: If your DNS provider supports DNSSEC, enable it in the DNS provider's dashboard — the registrar then needs the DS record information. Cloudflare, Google Cloud DNS, and Route 53 all support DNSSEC.

How to verify your DNS changes

DNS changes are not instant. After you update a record, it can take anywhere from a few seconds to 48 hours to propagate globally, depending on the TTL of the old record and the resolver's cache. The average is 5–30 minutes.

Check propagation with a DNS lookup tool:

• dig A example.com — query A record from your default resolver
• dig @1.1.1.1 A example.com — query Cloudflare's resolver directly
• dig @8.8.8.8 A example.com — query Google's resolver
• Online tools like dnschecker.org show how your records resolve from resolvers worldwide

Check email DNS:

• dig MX example.com — verify MX records are pointing to the right servers
• dig TXT example.com — verify SPF, DKIM, and verification TXT records

Flush your local cache: If your browser or computer is still resolving the old value, flush the local DNS cache:

• macOS: sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder
• Windows: ipconfig /flushdns
• Linux: systemd-resolve --flush-caches or service nscd restart

Common mistakes to avoid

• CNAME on root domain (@) — Most DNS providers block this because RFC 1034 prohibits it. Use an A/AAAA record instead.
• MX records pointing to IP addresses — MX must point to hostnames, not IP addresses.
• Forgetting the trailing dot — In some DNS UIs, values ending in a dot (mail.example.com.) are fully qualified and not relative to the domain. Check your provider's format rules.
• Conflicting records — Having both an A record and a CNAME on the same name causes conflicts. One name, one canonical record.
• Setting TTL too high — When planning a migration, lower your TTL to 300 seconds 24 hours in advance so the change propagates faster.
• TXT records with quotes — Enter the TXT value without surrounding quotes in most DNS UIs. The quotes are part of the zone file format, not the record value.

Summary

Configure these five record types and you can set up virtually any domain for web hosting, email, and third-party services. Always verify changes with dig or an online DNS checker, and plan TTL changes before migrations to minimize downtime.

FAQ

How long does it take for DNS changes to propagate?

Most changes take 5–30 minutes to reach the majority of resolvers globally. The old record's TTL and the resolver's cache behavior determine the actual time. After 48 hours, virtually all resolvers will have the new record.

Can I have both A and CNAME records for the same subdomain?

No. A name cannot be both a canonical record (A/AAAA) and an alias (CNAME) at the same time. If you need a CNAME on a subdomain, do not also create an A record on that same name.

Should I use Cloudflare proxy (orange cloud) with my A record?

For web traffic, yes — Cloudflare proxy adds CDN caching, DDoS protection, and performance optimization. However, MX records should never be proxied. DNS-only MX records are required for email delivery to work correctly.

What is the difference between TTL and propagation?

TTL (Time To Live) is a setting on each DNS record telling resolvers how many seconds to cache it before asking the authoritative nameserver again. Propagation is the informal term for the time it takes all resolvers worldwide to fetch the updated record after you change it.

Do I need to set up DNSSEC?

DNSSEC is recommended for production domains. It prevents DNS spoofing attacks where an attacker redirects your visitors to a malicious server. Most DNS providers make it a one-click toggle once your registrar supports DS records.

Why deploy to Cloudflare Pages

A static site is a collection of fixed HTML, CSS, and JavaScript files — no server, no database, no runtime. Because everything is prebuilt, a static host only needs to store files and serve them fast. Cloudflare Pages does exactly that, serving your files from its global network with no bandwidth charges. The free plan removes the usual cost of hosting, which is why Pages is a common home for blogs, documentation, and marketing sites built with frameworks such as Next.js (static export), Eleventy, Hugo, Astro, or plain HTML.

Pages gives you a clean workflow: commit code, push to Git, and a new version goes live automatically. Every deployment is versioned, so you can roll back to a previous build if something breaks.

Path 1: Deploy from a Git repository

Git integration is the recommended path when you want changes to publish automatically on every push. Pages supports both GitHub and GitLab.

1. Build your site locally and confirm the build command and output directory for your framework. For a Next.js static export the build command is npx next build and the output directory is out. For a Vite React app it is npm run build and dist. A plain HTML site with no build step can leave the build command blank.
2. In the Cloudflare dashboard, open the Workers & Pages page and select Create application → Pages → Connect to Git.
3. Sign in with your Git provider and choose the repository to deploy. Both public and private repositories are supported.
4. In Set up builds and deployments, enter your project name (it becomes <project>.pages.dev) and choose your production branch — usually main or master. The production branch serves your live site; other branches become preview deployments.
5. Set your build command and build output directory from the framework preset, or leave the build command blank for a site with no build step.
6. Optionally add environment variables for the build, then select Save and Deploy.

Pages runs the install and build steps, then uploads the output to its edge network. When a build process returns a non-zero exit code, Pages marks it as failed; an exit code of 0 means the build succeeded and the assets are uploaded. After the first deploy you get a URL like <project>.pages.dev for your live site, and Cloudflare injects system build variables such as CF_PAGES (set to 1) and CF_PAGES_BRANCH so builds can detect they are running on Pages.

Path 2: Direct Upload with prebuilt assets

Choose Direct Upload when you build elsewhere — at a separate CI platform or on your own machine — and only want Cloudflare Pages to host the finished files. Assets are uploaded either through the Wrangler CLI or by dragging and dropping in the dashboard.

With Wrangler:

1. Log in with wrangler login, then create a project: npx wrangler pages project create.
2. Deploy your build output: npx wrangler pages deploy <BUILD_OUTPUT_DIRECTORY>.
3. Your production site is served at <project>.pages.dev. To deploy a preview, pass --branch=<BRANCH_NAME> and the branch alias becomes available at <branch>.<project>.pages.dev.

With drag and drop, open Workers & Pages → Create application → Get started → Drag and drop your files, name the project, and upload a folder (or a zip) of your assets. Drag and drop is limited to 1,000 files, while Wrangler supports up to 20,000 files per project.

Direct Upload projects do not support the functions folder via the dashboard drag and drop — to deploy a functions folder you must use Wrangler.

Adding a custom domain

After your site is live at <project>.pages.dev, attach your own domain from the project's Custom domains settings. Cloudflare can issue an SSL/TLS certificate automatically, and the site continues to serve over HTTPS through the edge. The free plan allows up to 100 custom domains per project.

Free plan limits to plan around

Cloudflare Pages free plan limits (as documented by Cloudflare):

Bandwidth and requests are unlimited on the free plan, which is why Pages suits a high-traffic static site without surprise bills. The practical constraint is the file count (20,000 per site) and the monthly build budget (500 builds). A large documentation site that publishes often can approach those numbers; a typical blog stays far below them.

What about dynamic features?

Pages also supports Pages Functions, which let you run server-side code for dynamic behavior such as handling forms or reading cookies, with requests counting toward your Workers quota. On the free plan this quota is modest, so most users keep dynamic work minimal and lean on the static asset hosting, which is effectively free.

Summary

Cloudflare Pages turns a static site into a fast, globally-cached deployment with no hosting bill. Connect a Git repository for automatic rebuilds, or use Direct Upload for manual builds. Set a custom domain, and keep the free-plan limits in mind — especially the 20,000-file and 500-builds-per-month caps. For purely static output, Pages is hard to beat on price and performance.

FAQ

Is Cloudflare Pages free?

Yes. The free plan provides 20,000 files per site, 500 builds per month, and unlimited bandwidth. Paid plans raise the file limit to 100,000 per site and increase monthly builds.

What is the difference between Cloudflare Pages and Cloudflare Workers?

Pages is static hosting with optional Pages Functions; Workers is a serverless execution platform where you write code that runs at the edge. They are related products, and Pages Functions requests count toward your Workers quota. Many static sites use Pages alone, without Workers.

Can I use Cloudflare Pages without GitHub or GitLab?

Yes. Direct Upload lets you upload prebuilt assets with Wrangler or drag-and-drop without any Git provider. You lose automatic rebuilds on push, but still get the same edge hosting and HTTPS.

Does Cloudflare Pages support Next.js?

Yes. Next.js can be deployed as a static HTML export using npx next build with the out output directory, or via the pages adapter for full-stack deployments. For a purely static site, the static export path needs no server runtime.

What happens without an index

Imagine a users table with one million rows and a query that fetches a single row by email:

SELECT * FROM users WHERE email = 'ada@example.com';

Without an index on email, the database cannot know where the matching row is, so it must read the table from the first row to the last and compare email on every row until it finds a match. This is called a full table scan, and on a large table it means reading (potentially) a million rows. A full scan is correct but slow, and it becomes useless the moment the table grows.

An index changes this: it keeps the email values sorted in their own structure, with a pointer from each value back to the corresponding row. To satisfy the query, the database searches the sorted index, finds ada@example.com, follows the pointer, and reads only that one row. Instead of a million comparisons, the lookup takes roughly the depth of the tree — a few dozen steps even for huge tables.

How a B-tree index works

Most relational databases index with a B-tree — more precisely a B+ tree variant. A B-tree is a multi-way balanced tree: a root node points down to branch nodes, branch nodes point down to leaf nodes, and the leaf level holds the index keys in sorted order together with a pointer to each row. "Balanced" means every leaf sits at the same shallow distance from the root, so any value — common or rare — is found in the same small number of steps regardless of how many rows there are.

Because the leaves are sorted, a B-tree index accelerates more than exact equality lookups:

• Equality: WHERE email = 'x'
• Range: WHERE age BETWEEN 18 AND 25, WHERE price < 100
• Prefix search: WHERE name LIKE 'Ada%' (a leading wildcard, '%da', cannot use the index)
• Ordering: ORDER BY created_at
• Joins: matching rows between two tables on the indexed column

PostgreSQL's documentation notes that any data type that can be sorted into a well-defined linear order can be indexed by a B-tree — which is why it is the default index type and covers the vast majority of use cases.

Clustered vs non-clustered indexes

The single most important distinction in indexing is clustered vs non-clustered, and it comes down to where the data actually lives.

Clustered index. The leaf level of a clustered index is the table data — the rows are physically stored on disk in the order of the index keys. Clustered indexes are great for range queries and ordered reads, because once you find the start of a range, the rest of the matching rows are physically adjacent. But because the rows can be laid out in only one order, a table can have at most one clustered index. In MySQL's InnoDB engine the primary key is always the clustered index; SQL Server clusters the primary key by default.

Non-clustered index. A non-clustered index is a completely separate B-tree that stores the sorted key columns plus a pointer back to the row — either a physical row locator or, in clustered-table engines, the clustered key value. The underlying table rows stay in their own natural order, so a table can have many non-clustered indexes, each tuned to a different query pattern.

In practice a typical table has one clustered index (often the primary key) and several non-clustered indexes for the queries that matter most.

Composite and covering indexes

You are not limited to a single column. A composite index covers several columns in a fixed order, for example (last_name, first_name). Composite indexes obey the leftmost-prefix rule: a query can use the index when it filters on the first column, or the first and second, but not on the second column alone. So an index on (last_name, first_name) helps WHERE last_name = 'Lovelace' and WHERE last_name = 'Lovelace' AND first_name = 'Ada', but not WHERE first_name = 'Ada' on its own.

A covering index (in SQL Server, an index with included columns) holds every column the query needs inside the index itself, so the database can answer the query without ever going back to the table — a "covering" or index-only scan. This is the fastest possible read, at the cost of a wider index.

The trade-offs: writes, storage, and maintenance

Indexes are not free. Every index on a table must be updated on every write to that table, so:

• Write overhead. Each INSERT, UPDATE, and DELETE now has to maintain every index along with the table. A table with many indexes slows down writes and can slow down transactions.
• Storage. Each index is its own structure on disk. A wide or covering index can rival the table itself in size.
• Selectivity. An index only pays off when the query is selective — when a value matches a small fraction of rows. Indexing a column where most rows share the same value (like a boolean that is true 99% of the time) rarely helps, because the optimizer may decide that scanning the table is cheaper.
• Function calls. Wrapping the indexed column in a function, as in WHERE LOWER(email) = 'ada@example.com', usually prevents the index from being used unless a functional index exists for that expression.

The practical guideline is to index the columns that appear in your most important WHERE, JOIN, and ORDER BY clauses, keep the number of indexes modest, and drop indexes that queries never use.

How to verify an index is working

After creating an index, confirm the query planner actually uses it. In PostgreSQL and MySQL you prefix a query with EXPLAIN ANALYZE (or just EXPLAIN) and read the plan:

EXPLAIN ANALYZE SELECT * FROM users WHERE email = 'ada@example.com';

The plan shows whether the engine chose an index scan or a sequential scan (table scan). If a query you expect to be fast is still doing a sequential scan, the index may not be selective enough, or the query may be written in a way the index cannot serve (for example, a leading wildcard or a function call on the column).

FAQ

What is the difference between a clustered and a non-clustered index?

A clustered index determines the physical order in which a table's rows are stored on disk — its leaf level is the data — and a table can have only one. A non-clustered index is a separate sorted structure that stores the key columns plus a pointer back to the row, and a table can have many of them. Clustered indexes suit range and ordered reads; non-clustered indexes support many different query patterns on the same table.

Why do database indexes slow down writes?

Every time a row is inserted, updated, or deleted, the database must also update every index that covers the affected columns. Indexes speed up reads by trading write cost and storage for that speed — the more indexes a table has, the more work each write has to do.

When should I not create an index?

Avoid an index when the column has low selectivity (most rows share the same value, so the query still matches most of the table), when the table is tiny and a full scan is already fast, or when the column is rarely used in WHERE, JOIN, or ORDER BY. Too many unused or low-value indexes add write overhead and consume storage without helping reads.

Why is my index not being used?

Common reasons the optimizer skips an index: the query is not selective enough (matching too large a fraction of rows), the column is wrapped in a function (WHERE LOWER(email) = ...), a LIKE pattern starts with a wildcard ('%ada'), or the query filters on a column that is not the leftmost column of a composite index. Running EXPLAIN shows whether the planner chose an index scan or a sequential scan.

CSS Selectors

A CSS selector is the pattern that tells the browser which HTML elements a rule applies to. The MDN basic selectors lesson describes it as the first part of a CSS rule — a pattern of elements and terms that identifies the subject of the selector. Selectors range from trivially simple to precisely targeted.

Type, Class, and ID Selectors

A type selector (also called tag-name or element selector) targets an HTML tag directly:

p { color: blue; }

A class selector starts with a dot (.) and targets every element with that class. Multiple classes can be chained without a space — p.highlight targets only <p class="highlight"> — and an element can carry several classes, each independently styleable. The MDN lesson gives the example of a .notebox with additional .warning or .danger classes applied to the same element for layered styling.

An ID selector starts with # and targets exactly one element per page, since id must be unique in a document. Using the same ID twice produces invalid HTML. The MDN lesson warns: "Using the same ID multiple times in a document may appear to work for styling purposes, but don't do this. It results in invalid code and will cause strange behavior in many places."

Attribute Selectors

Attribute selectors target elements based on the presence or value of an attribute. The MDN attribute selectors lesson covers two groups.

Presence and value selectors: