Skip to content
Français
Account: not signed in, sync available My learning
Select theme
Français

Cloud elasticity vs scalability: telling the 2 mixed-up concepts apart

By updated

Improve this page

Elasticity and scalability are very often confused in technical conversations. The two concepts are distinct but related: scalability is the ability to grow, elasticity is the ability to grow and shrink quickly, most often automatically, according to the real load. An application can be scalable without being elastic, like an on-premise server fleet sized for the peak. To become truly elastic, it must meet precise architectural prerequisites: no local state, externalised sessions, observability. This page lays out the definitions, distinguishes vertical from horizontal scaling, and explains why designing a new project so that it can be elastic (no local state, fast startup) is a matter of design, not of optimisation. Automatic autoscaling, for its part, is only justified on certain load profiles (section 5).

What you will learn

  • The precise difference between scalability and elasticity
  • Vertical scaling (scale-up) vs horizontal scaling (scale-out) and their limits
  • The architectural prerequisites of elasticity (stateless, sessions, observability)
  • How to add and lose an instance without breaking the service, and measure the load during the exercise
  • When to aim for automatic elasticity and when to avoid it (elastic architecture is planned from the design stage, the autoscaler only when the load varies)
  • The classic pitfalls of a poorly tuned autoscaling
  • The limits that a sudden spike runs into despite autoscaling (startup delay, database, service ceilings)

Prerequisites: having understood the 5 characteristics of the cloud (Definition and promises of the cloud (in French)) and the design principles of a stateless application (one that keeps no user-specific data on the machine that serves it), presented in Loose coupling and stateless design (in French).

Section 4 is practised on a real account (level B, optional) and creates billed resources: if you want to replay it, first open your practice account with Setting up your practice environment (in French), and, to rehearse the syntax of the scw commands at no cost before the real account, get started with the local emulator through the clients to point at Feint. Sections 1 to 3 and 5 to 7 require no account.

1. The two precise definitions

First, let us fix the vocabulary. The confusion comes from the fact that in everyday language, "scalable" is used for any application that can grow, without distinguishing the modes.

Scalability

Scalability is the ability of a system to handle a growing load by adding resources (vertically or horizontally). You can scale the system manually, on command and at a given frequency: scalability is not implicitly automatic.

A scalable application can grow, but it does not grow automatically. An on-premise cluster of 10 servers can be scaled to 20 servers, but that requires ordering, racking, configuring, which takes a few weeks to several months.

Elasticity

Elasticity is the ability of a system to provision and release resources quickly, both up AND down, to follow the load. This is the NIST wording (the US standards institute, "rapid elasticity" characteristic): NIST specifies that this happens automatically "in some cases". Elasticity is an additional property compared with scalability, and autoscaling (automatic scaling: a mechanism that adds or removes instances according to a measured metric) is the tool that automates it.

An elastic application is necessarily scalable, but a scalable application is not necessarily elastic. That is the crucial difference.

PropertyScalableElastic
Can growYesYes
Can shrinkYes, but slowlyYes, quickly
Adapts without human interventionNoPossible with an autoscaler, not guaranteed by the property alone
Adjustment delayMonths (on-prem) to minutes (cloud)Minutes to seconds depending on the service
Cost at idleFixed (reserved capacity)Reduced, down to almost nothing with scale to zero (bringing the number of instances down to zero when nobody uses the service)

2. The two axes of scaling

When we talk about scaling, two directions are possible. They are not equivalent and each has its own limits.

Vertical scaling (scale-up / scale-down)

Vertical scaling means increasing the power of a single machine: going from 2 vCPU to 8 vCPU (a vCPU is a virtual processor core), from 4 GB RAM to 32 GB RAM, from gp3 to io2 for the disk (two SSD disk types at AWS: gp3 is general purpose, io2 guarantees a number of operations per second).

Vertical scaling has three advantages:

  • It is simple to implement, with no change to the application architecture.
  • It suits monolithic applications that are not distributed.
  • It avoids distributed consistency (locks, transactions).

It also has four limits:

  • Physical ceiling: you eventually reach the maximum capacity (for example 1,920 vCPU and 32 TiB of RAM for u7inh-32tb.480xlarge, one of the largest instances in the EC2 memory optimized instances documentation, read on 13 November 2026).
  • Cost proportional, gain rarely proportional: going from m5.xlarge (4 vCPU) to m5.24xlarge (96 vCPU) costs 24 times more in Paris (0.224 against 5.376 USD per hour on-demand, AWS price list retrieved on 14 November 2026), because the price follows the vCPUs, but the application is rarely 24 times faster.
  • Single point of failure (SPOF: the component whose failure stops everything): a single big machine remains a SPOF.
  • Downtime at resize: the VM must be restarted in most cases.

Horizontal scaling (scale-out / scale-in)

Horizontal scaling means adding more identical machines to spread the load: from 5 servers to 50 servers behind a load balancer (it distributes requests among the servers and removes those whose health check fails).

Horizontal scaling has four advantages:

  • It is almost unlimited: adding the 1001st instance is done like the 100th.
  • It is more resilient: if one server goes down, the others carry on.
  • Its cost is linear: 10 servers cost 10 times the price of one server, no more.
  • It is compatible with automatic elasticity.

In return, it requires three things:

  • A distributed architecture: load balancer, shared sessions, stateless code.
  • Enough observability to detect when to scale.
  • Distributed consistency to manage, with externalised locks, saga transactions (a series of local transactions, each with a compensating action in case of failure), and eventual consistency (the copies of a piece of data are only identical after a short delay, see Synchronous, asynchronous and eventual consistency).

Decision table

This table helps you decide between vertical and horizontal scaling. Look first at the ceiling and the compatibility with elasticity, then at the application complexity: horizontal scaling goes higher but demands more from the application.

CriterionVertical scalingHorizontal scaling
Application complexityLowHigh
CeilingCapacity of one VMAlmost unlimited
Elasticity-compatibleLimited: resizing often needs a restart, except for services that adjust continuously (Aurora Serverless v2, an AWS database whose capacity varies continuously, or VPA, the Kubernetes Vertical Pod Autoscaler, which adjusts the CPU and memory reserved for a pod)Yes
Monolith-compatibleYesPartial (depends on design)
Relative costProportional to vCPUs, gain rarely proportional (see above)Proportional to the number of instances
Fault toleranceSPOFTolerant

3. The architectural prerequisites of elasticity

Automatic elasticity is not just a matter of cloud service, it requires precise application properties. Without these prerequisites, you can configure an autoscaler but it will not work properly.

Prerequisite 1, Stateless

Your application servers must not keep any critical local state (the design is detailed in Loose coupling and stateless design (in French)). User sessions, application caches and important temporary files must live elsewhere, Redis for sessions, Memcached for caches (two services of memory shared between instances), S3 for uploads (AWS object storage).

Without stateless, a scale-in kills the connected users (their session disappears with the instance), and a scale-out does not work properly (the new user lands on an instance that does not have their context).

Prerequisite 2, Idempotent operations

When a request times out (timeout), it is the client that replays it, and the load balancer only if it is configured to retry: it then lands on another instance, and the application must be able to reprocess it without side effects. An order placed twice must not create two orders; this is idempotence, the subject of Idempotence in the cloud (in French).

Prerequisite 3, Realistic health checks

The load balancer must know quickly when an instance is failing. A /health that always returns 200 is not enough: an instance whose application is stuck still answers it. You need a check on the instance itself (process alive, configuration loaded, ability to handle a local request). Be careful not to put the database connectivity or shared services in it: if the database goes down, all instances fail at the same time, the load balancer removes them all, and a partial outage becomes a total stop (cascading failure). Shared dependencies are monitored separately (alerts, circuit breaker, degraded mode).

There are two kinds of check. The liveness check says when to restart an instance: if it no longer answers, it is relaunched. The readiness check says when to send it traffic: an instance that is starting or overloaded stays alive, but no longer receives requests until it is ready.

Prerequisite 4, Actionable metrics

The autoscaler scales on metrics. If your only metrics are CPU and RAM, you will scale poorly. The relevant metrics for scaling are application-level: p95 latency (the response time under which 95% of requests fall), queue depth, requests per second per instance. Subject of the monitoring and observability (in French) guide.

Prerequisite 5, Fast startup

If an instance takes 5 minutes to become available (OS boot + runtime install + code loading), your autoscaler always reacts 5 minutes after the spike. To gain reactivity: pre-built images (with Packer, a tool that builds machine images), managed containers (CaaS), or serverless (FaaS: functions run on demand, with no server to manage).

4. How do you make a workload elastic without an autoscaler?

A workload is elastic when you can add an instance to it without touching the code, and remove one without an error for a client that retries. These two gestures are practised before any autoscaler: the autoscaler only triggers them on its own, and if they do not work by hand, they will not work automatically. This is the gesture to master before automating.

Add an instance without changing the code

An added instance must become useful without modifying the application or the configuration of the others. Three conditions make this possible: it starts from the same image as the others, it receives its configuration at bootstrap rather than through a manual edit, and the load balancer registers it as soon as its health check answers. An add script does these three things, with no infrastructure as code at this stage. This gesture is practised on a real account, at the optional level B of the course (see Setting up your practice environment (in French)): it creates a load balancer, servers and IP addresses that are billed as long as they exist, and the cleanup at the end of the section is mandatory.

The local Feint emulator cannot play it: no machine starts in it and no real health check answers there. You therefore first need two web servers, web-1 and web-2, created with the scw instance server create command shown below for web-3: same image, and same cloud-init.yaml bootstrap script. This script is a cloud-init file (see the lesson cloud-init (in French)); the minimal version this section needs installs nginx and publishes a page:

#cloud-config
package_update: true
packages:
- nginx
write_files:
- path: /var/www/html/index.html
content: "amorce par cloud-init\n"
runcmd:
- systemctl enable --now nginx

With Scaleway's scw CLI, for example, you then create web-3 from the same image with the same bootstrap script, then add it to the load balancer. The --wait option waits for the running state, and cloud-init=@cloud-init.yaml loads the script from a file:

Terminal window
scw instance server create type=DEV1-S image=ubuntu_noble name=web-3 \
cloud-init=@cloud-init.yaml zone=fr-par-1 --wait -o json
scw lb backend add-servers backend-id=<backend-id> \
server-ip.0=<web-3-address> zone=fr-par-1

The address is the one the load balancer will use to reach the machine: without a private network, it is its public address (public_ip.address field of the JSON response), because the machine then has no private address. The command adds the address to the backend's server list (pool) without touching the others.

The backend and the load balancer must exist before this gesture; for a trial, an LB-S load balancer is created with scw lb lb create type=LB-S, then scw lb backend create with forward-protocol=http, forward-port=80 and a health check that requires all of health-check.port=80, health-check.http-config.uri=/ and health-check.http-config.method=GET (the command refuses otherwise), then scw lb frontend create with inbound-port=80.

The verification is the load balancer's health check: the instance must reach the healthy state before receiving traffic. If adding it requires editing a file on another instance, the workload is not elastic, whatever the scaling tool.

Lose an instance without a client-side error

Losing an instance must be invisible to a client that retries. To verify it, you generate a steady load, abruptly delete an instance, and count the errors received by the client: the load balancer must remove the lost instance after a few health checks, and the following requests must succeed on the others. To tolerate the loss of a zone and not only of an instance, the instances and the entry point must be located in at least two zones: a load balancer confined to a zone that goes down is the single point of failure, whatever the number of instances behind it (see Resilience, HA and DR (in French)). The gesture comes down to two commands run in two terminals: a loop that calls the service without retrying and counts the response codes (the LOAD_BALANCER_ADDRESS variable holds the IP address of the load balancer), then the deletion of an instance while it runs.

Terminal window
for i in $(seq 1 200); do curl -s -m 5 -o /dev/null -w '%{http_code}\n' "http://${LOAD_BALANCER_ADDRESS}/"; sleep 0.5; done | sort | uniq -c
# meanwhile, in a second terminal:
scw instance server terminate <instance-id> with-ip=true with-block=true zone=fr-par-1

Without with-ip=true or with-block=true, terminate leaves behind the machine's IP address and volume, which stay billed: this is exactly the leftover that the lesson Rebuild then destroy an environment from code (in French) teaches you to look for. The terminate command shuts down the instance, then deletes it and waits for the operation to finish: its duration varies from a few seconds to more than ten minutes depending on how the system shuts down. Run the loop long enough, or bound it by the return of terminate, to be sure you have observed the disappearance.

The count must show a large majority of 200, with only a few error lines: for example a 000 code (the client gave up after 5 seconds on a request sent to the instance that was disappearing), then a 503, before the return to 200. The exact number of error lines depends on the length of the detection window, record it on your own trial.

This 000 comes from the loop's -m 5 option, which bounds the wait to 5 seconds: without it, curl would wait longer. A few 502 or 503 right after the deletion are normal: this is the detection window, during which the load balancer has not yet detected the failure, and its duration depends on how the checks are tuned, which ties back to prerequisite 3 above. The loop above does not retry, so it exposes this window; it is a client that replays its request after an error or a timeout that masks it, not the load balancer. Errors that last point to a health check that is too slow or an instance that was keeping state.

Measure the load during the exercise

The same experiment serves for measuring. Record the CPU and memory usage of each instance while the load runs, before and after the loss of an instance. These readings give a quantified basis to justify the size of an instance: if a single instance would saturate at twice the nominal load, the size or the number of instances is badly chosen. For the calculation, the capacity left after the loss of an instance is (minimum − 1) × safe capacity per instance, and you compare it with the peak load. For example, with 4 instances at the minimum and 500 requests per second of safe capacity each, there remain (4 − 1) × 500 = 1,500 requests per second: this plan holds a peak of 1,400, with 100 of margin, but not a peak of 1,700. A measurement made on your application is worth more than sizing by habit.

Clean up the billed resources of the exercise

The exercise leaves a load balancer, servers and IP addresses that keep being billed: delete them as soon as the measurements are taken. First delete the load balancer, releasing its IP address (release-ip=true, otherwise the address stays reserved in the account), then each server still present (scw instance server list zone=fr-par-1 gives their identifiers), with its address and volume:

Terminal window
scw lb lb delete <load-balancer-id> release-ip=true zone=fr-par-1
scw instance server terminate <server-id> with-ip=true with-block=true zone=fr-par-1

Then take the inventory of each resource type through the API, in all zones (zone=all), load balancers and load balancer addresses included: only empty lists prove that nothing is left to bill.

Terminal window
scw instance server list zone=all
scw instance volume list zone=all
scw block volume list zone=all
scw instance ip list zone=all
scw block snapshot list zone=all
scw instance snapshot list zone=all
scw lb lb list zone=all
scw lb ip list zone=all

5. When to aim for automatic elasticity

Elasticity has a significant setup cost (distributed architecture, observability, resilience tests). It is not justified on all load profiles.

Profiles where elasticity pays off

  • Seasonal variable load: e-commerce with a Black Friday peak, event sites, ticketing.
  • Daily variable load: consumer applications with a 6 pm to 10 pm peak, B2C platforms.
  • Unpredictable bursty load: public APIs, event-driven functions.
  • One-off compute workloads: ML training, video encoding, night batch.

On these profiles, the gap between the peak and the baseline load is significant, and elasticity can save a significant share of the compute bill, all the more so as this gap is large. The break-even point depends on your price grid and your setup cost: calculate it rather than retaining a ready-made ratio (method in CAPEX vs OPEX (in French)).

Profiles where elasticity is superfluous

  • Stable 24/7 load with little variation (an author's benchmark: less than a third gap between the peak and the baseline): internal B2B applications, ERP, transactional databases.
  • Workloads with strict latency constraints: autoscaling cold starts (the startup time of a new instance) can degrade p99 (the response time under which 99% of requests fall, hence the slowest ones).
  • Complex stateful applications where the cost of going stateless exceeds the benefit of elasticity.

On these profiles, fixed sizing with Reserved Instances (a one- or three-year commitment with the provider, in exchange for a reduced rate) is simpler and often cheaper in TCO (total cost of ownership).

6. The classic pitfalls of autoscaling

Five pitfalls come up regularly when autoscaling is badly tuned. Each card describes the mechanism, then the mitigation.

Scaling too slow

The thresholds are at 80% CPU over 5 minutes and the instance takes 3 minutes to start: that makes 8 minutes of degradation for the user before the scale-out relieves the service.

To react faster, lower the thresholds (60%), speed up startup (containers) and keep a pool of pre-warmed instances (pre-warming).

Overly aggressive scale-in

The load drops at 2 pm and the autoscaler removes 80% of the instances; at 2:05 pm, a new peak arrives and everything has to be restarted.

Plan a long scale-in cooldown (the minimum delay imposed after a scaling action before another is allowed, 15 to 30 min), a minimum number of instances and, if possible, a load forecast based on history. Before removing an instance, let it drain its connections: the load balancer stops sending it new requests and you wait for a set draining delay (in seconds) so that in-flight requests finish, then you remove it.

Irrelevant metrics

You scale on CPU while latency climbs because of a slow database: the CPU stays at 30%, the autoscaler triggers nothing and users suffer the slowness.

Scale instead on application metrics (p95 latency, queue depth), with an extension for custom triggers such as KEDA (Kubernetes Event-driven Autoscaling, a Kubernetes extension that triggers scaling on events or external metrics).

Exploding cost

Without an instance cap, a DDoS attack or a bug can make it scale up to 500 instances and multiply the bill sharply.

Set max_size in the Auto Scaling Group (AWS's autoscaling group), set cost alarms, and put a WAF (web application firewall) and rate limiting upstream of the load balancer.

Flapping

The load oscillates around the threshold and the scaler adds then removes instances in a loop: this is flapping, which wears out the platform and inflates the bill for no reason.

Use different thresholds for scale-out (60%) and scale-in (40%), a long cooldown and a metric smoothed over 5 minutes.

Why does a sudden spike exceed autoscaling?

The cloud scales, but neither infinitely nor instantly, and a sudden spike runs into three limits that the autoscaler does not compensate. Take a traffic spike multiplied by 100 in 30 seconds (product launch, viral message): the autoscaler will not keep up.

  • The startup delay. A new instance generally takes several minutes to become useful (machine startup, then health check). During that time, the existing instances absorb the load alone, as described by the "Scaling too slow" pitfall.
  • The database. Vertical scaling of a database has a ceiling (see section 2), and horizontal scaling of a database requires partitioning it (sharding), which changes the application. A database does not widen in five minutes.
  • Managed service ceilings. A service can limit its own throughput, independently of your instances. As an example, the Amazon API Gateway quotas documentation indicates, as of 12 October 2026, 10,000 requests per second by default, per account and per region, and 2,500 in a list of recent regions (Cape Town, Milan, Zurich, Spain, among others), a quota that can be raised on request. An additional burst is added to it, but AWS sets it itself.

Two remedies exist. Pre-warming keeps instances on standby, at the price of capacity paid for nothing; serverless scales up very fast, but remains subject to concurrency quotas and to a limited ramp-up rate. In both cases, check your account's ceilings before the spike: the lesson Operating a cloud environment details the quota families and the reflex of raising them in advance.

7. When do you aim for elasticity from the design stage?

Section 5 laid out the load profiles that make elasticity pay off. What remains is the question of timing: should it be built in from the design stage, even if the current load does not require it? Two criteria avoid both over-engineering and the technical debt of a stateful start that is hard to fix afterwards.

The cost of a retroactive switch. A stateful application with local sessions, files stored on the local disk and in-memory caches can stay that way for years. When the need for elasticity finally arrives, the transition requires reworking sessions, caches, uploads and periodic tasks. This retroactive switch costs more than an initial design that anticipates elasticity, even if the application does not immediately need it.

The operational benefit, not only the financial one. Beyond cost, an elastic system gives room to manoeuvre against unexpected spikes, reduces the risk of degradation at 3 a.m., and simplifies the on-call work. These operational benefits count as much as the bill, but can only be measured after several months of operation.

Put it into practice

The lab gives the measurements and constraints of an application tier, and you write an elasticity plan in elasticity-plan.json. The plan says where sessions, uploads and orders live, how a retry avoids creating a second order, and what the two health checks are, liveness and readiness. It also sets the minimum and maximum replicas, calculates the capacity left after the loss of an instance, and describes the ramp-up and ramp-down of the load separately, with connection draining before any removal. The validation recomputes the capacity and the margin from the measurements, re-reads each threshold, each window and each delay, and checks that the plan explains how to add then lose an instance. Nothing is deployed: the exercise is about design.

Key points

  • Scalability = ability to grow (manual or automatic). Elasticity = ability to grow AND shrink quickly, generally through an autoscaler.
  • Scale-up (vertical) increases the power of one machine, capped. Scale-out (horizontal) adds machines, almost unlimited.
  • Elasticity requires 5 prerequisites: stateless, idempotence, instance health checks, actionable metrics, fast startup.
  • A workload is elastic when you can add an instance to it without changing the code and lose one without an error for a client that retries; both gestures are verified by hand, with a load reading, before any autoscaler.
  • Variable profiles (large gap between peak and baseline load) make elasticity pay off; stable 24/7 profiles prefer Reserved Instances.
  • Five classic autoscaling pitfalls: slow scaling, aggressive scale-in, irrelevant metrics, exploding cost, flapping.
  • A sudden spike exceeds autoscaling: a startup delay of several minutes, a database that is hard to widen, throughput ceilings of managed services.
  • Elasticity brings a clear benefit on variable loads and unexpected spikes; on stable 24/7 loads, fixed instances with commitments often remain simpler.

Knowledge check

Check that the essentials of this guide are in place. The questions only cover what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions
6 min
70% to pass

How it works

  • The timer starts when you click Start
  • Multiple choice, true/false and short answer questions
  • You can move between questions
  • Detailed results are shown at the end

Starts the quiz and the timer