Scaling is one of the most important concepts in system design. As traffic increases, applications must handle more requests efficiently without crashing or slowing down.
This article covers vertical scaling, Node.js clustering, autoscaling strategies, capacity estimation, and queue-based architectures for large-scale applications.
Vertical Scaling
What is Vertical Scaling?
Vertical scaling means increasing the resources of a single machine to handle higher traffic.
This includes:- Increasing CPU power- Adding more RAM- Upgrading storage- Better machine configurations
Example:2 CPU + 4GB RAM → 8 CPU + 32GB RAM
Vertical scaling is simple to implement but has hardware limitations.
Limitation in Node.js
Node.js runs on a single-threaded event loop.
This means:- One process typically uses only one CPU core- Multi-core CPUs are not fully utilized automatically
In contrast, languages such as:- Go- Rust- Java
support multithreading and can efficiently utilize multiple CPU cores.
A Naive Approach (Incorrect)
A beginner approach might be manually starting multiple Node.js processes:
node index.jsnode index.jsnode index.jsnode index.jsnode index.jsnode index.jsnode index.jsnode index.js
Problems with This Approach
1. Difficult process management2. Port conflicts between processes3. No built-in load balancing4. No fault tolerance or auto recovery
Solution: Node.js Cluster Module
Node.js provides a built-in cluster module to use multiple CPU cores.
Benefits:- Creates worker processes- Uses multiple CPU cores- Workers share the same port- Requests are distributed automatically
Application Logic (index.js)
import express from 'express'export const app = express()
console.log(`Worker ${process.pid} started`)
app.get('/', (req, res) => { res.send('Hello World!')})
app.get('/api/:n', (req, res) => { let n = parseInt(req.params.n) let count = 0 if (n > 5000000000) { n = 5000000000 } for (let i = 0; i <= n; i++) { count += i } res.send(`Final count is ${count} (handled by PID ${process.pid})`)})
Cluster Entry (bin.js)
import cluster from 'cluster'import os from 'os'import { app } from './index.js'
const totalCPUs = os.cpus().lengthconst port = 3000
if (cluster.isPrimary) { console.log(`Number of CPUs: ${totalCPUs}`) console.log(`Primary process ${process.pid} is running`)
for (let i = 0; i < totalCPUs; i++) { cluster.fork() }
cluster.on('exit', worker => { console.log(`Worker ${worker.process.pid} died`) console.log('Restarting worker...') cluster.fork() })} else { app.listen(port, () => { console.log(`App listening on port ${port}`) })}
Parallel Computation Using Clusters
Heavy CPU tasks can be divided among workers.
Instead of one process calculating everything sequentially, the workload is split across multiple CPU cores.
Cluster-Based Parallel Computation
import cluster from 'cluster'import os from 'os'
const numCPUs = os.cpus().lengthconst n = 1_000_000_000
if (cluster.isPrimary) { const chunkSize = Math.floor(n / numCPUs) let completed = 0 let totalSum = 0 const startTime = Date.now()
for (let i = 0; i < numCPUs; i++) { const start = i * chunkSize + 1 const end = i === numCPUs - 1 ? n : (i + 1) * chunkSize const worker = cluster.fork()
worker.send({ start, end })
worker.on('message', partialSum => { totalSum += partialSum completed++
if (completed === numCPUs) { const endTime = Date.now() console.log('Total Sum:', totalSum) console.log('Execution Time:', endTime - startTime, 'ms')
for (const id in cluster.workers) { cluster.workers[id]?.kill() } } }) }} else { process.on('message', ({ start, end }) => { let sum = 0 for (let i = start; i <= end; i++) { sum += i } process.send?.(sum) })}
Sequential Computation
const n = 1_000_000_000let sum = 0const start = Date.now()for (let i = 1; i <= n; i++) { sum += i}const end = Date.now()console.log('Sum:', sum)console.log('Time:', end - start, 'ms')
Capacity Estimation
Before scaling, estimate how much traffic your system must handle.
Step 1: Estimate Requests Per Second (RPS)
Example:10,000 users ≈ 100 RPS
Step 2: Measure Server Capacity
Benchmark a single machine.
Example:1 server = 500 RPS
Step 3: Estimate Required Machines
Formula:Total RPS / RPS per machine
Example:1000 RPS / 500 = 2 machines
Autoscaling Groups (ASGs)
Autoscaling automatically adjusts infrastructure based on demand.
Scaling decisions can be based on:- CPU usage- Memory usage- Traffic spikes- Queue length
Benefits:- Saves cost- Handles sudden traffic- Improves availability
Scaling Complex Applications
For high-scale systems, simple server scaling is often not enough.
A queue-based architecture is commonly used.
Architecture Flow
Client → Queue → Workers → Database/Storage → Response
Queue-Based Processing
Typical flow:1. Request arrives2. Request enters a queue3. Workers consume tasks4. Processing happens asynchronously5. Results are stored
Dynamic Worker Scaling
As load increases:More Queue Jobs → More Workers
As load decreases:Fewer Jobs → Fewer Workers
This approach enables efficient horizontal scaling.
Benefits of Queue-Based Systems
- Handles high traffic efficiently- Prevents server overload- Enables horizontal scaling- Improves reliability- Supports asynchronous workloads
Conclusion
Simple applications often scale using better machines and additional servers. However, large-scale systems rely on clustering, autoscaling, queues, and worker architectures to handle millions of requests efficiently.
Understanding when to use vertical scaling, horizontal scaling, or distributed processing is essential for designing reliable systems.
