Skip to content
Français
Account: not signed in, sync available My learning
Select theme
Français

Managed data: databases, caches and queues in the cloud

By updated

Improve this page

A managed database does not excuse you from knowing how to administer a database, it only relieves you of certain precise tasks. Confusing the two leads to a classic outage: a team discovers, the day it needs it, that nobody ever tested the restore. This lesson explains what the provider takes on, what stays with you, and how to choose between the families of data services. It is aimed at people who already know a database and must decide where and how to run it; protecting the database (replication, backup, failover) and connecting it privately each have their own lesson, cited below. The module has four lessons: this one, replication and backup, private access, then PostgreSQL backup (mandatory), which comes last. A lab, which consists of choosing a managed data service from a brief, is at the end of the third lesson, on private access.

What you will learn

  • Decide between a managed service and a database you install yourself.
  • Choose the right family: relational, NoSQL, cache, queue, stream.
  • Avoid the trap of the restore nobody ever tested.
  • Anticipate the two difficulties of a cache: invalidation and its outage.

Managed or self-managed: what the line moves

A managed service takes over the operation of the engine, not the responsibility for your data. Here is the real split, the one that matters when something goes wrong.

TaskManaged databaseDatabase you install
Install and configure the engineprovideryou
Apply security patchesprovider, during a maintenance windowyou, whenever you want
Back up automaticallyprovideryou
Check that the restore worksyouyou
Design the schema and the indexesyouyou
Tune the engine parametersyou, within the allowed limitsyou, without limit
Choose the replication strategyyouyou

The bold line is the one people forget. The provider guarantees that a backup was taken, not that it is usable for your case: it is up to you to restore periodically into a test environment and check that the data is there.

Managed brings a real gain when your team has no database administrator, and it imposes two constraints you must accept with your eyes open: the maintenance windows where the provider applies its patches (with Amazon RDS, you choose the time slot, not whether it happens), and the limited access to the engine, with no superuser account and no arbitrary extension.

Five families, five questions

Each family of data service answers a different question, and mixing them produces architectures that work badly without anyone knowing why.

FamilyThe questionExample engines
RelationalDoes my data have relationships and rules to guarantee?PostgreSQL, MySQL
Document NoSQLDo I have many loosely structured objects, read by identifier?MongoDB, DynamoDB
CacheCan I avoid recomputing or rereading the same thing?Redis, Valkey, Memcached
QueueDo I need to decouple a producer from a consumer?RabbitMQ, SQS
Event streamDo I need to replay the history, or serve several consumers?Kafka, Kinesis

Relational remains the right default choice, and this advice is not conservative. A relational database handles transactions, guarantees consistency, and supports volumes far above what most projects will ever reach. Choose something else when you know precisely why, not in anticipation of a hypothetical scale.

The difference between a queue and a stream matters, because they look alike. In a queue, a consumed message disappears: a single recipient processes it. In a stream, messages stay readable for a retention period, and several independent consumers each advance at their own pace. If you must replay the history after a bug fix, you need a stream.

The cache, and the two difficulties it brings

A cache solves a performance problem and creates two others, which are better known before adding one.

Invalidation. A cached piece of data becomes wrong as soon as the source changes. Deciding when to throw it away is the hard part, and there is no general answer: a short lifetime simplifies everything at the price of smaller gains, a long one forces you to invalidate explicitly on every write.

The cache outage. If your application stops working when the cache is empty, it is no longer a cache but a database, and it has neither the guarantees nor the backups of a database. Test what happens when the cache starts empty: after a cache restart, an application that uses it for recomputable data should slow down, not fall over. User sessions are an exception, as the next paragraph shows.

A particularly worthwhile use, and often overlooked: storing user sessions in a shared cache rather than in the server's memory. It is what makes the application stateless, hence genuinely load-balanceable, as the lesson on load balancing (in French) explains. This choice has a price: the cache then becomes the only place where sessions live, and an emptied cache logs every user out, without the application "falling over". It is a deliberate trade-off, to be handled with full knowledge of the facts: a lost session forces a new login, which is acceptable for many applications, and is not for a payment in progress. If the loss is unacceptable, the session goes in a database, not in a cache.

Where are replication, backup and private access covered?

Protecting a database and connecting it to the network are covered in two other lessons of the module. The lesson Replicate, back up and fail over a database explains why a replica is not a backup, what high availability costs, which replica to choose and what a failover does. The lesson Limit connections and reach a database privately covers the connection limit of a managed database and how to reach it without going through the Internet. For the gesture on a self-managed database, the lesson Back up and restore PostgreSQL (in French) teaches pg_dump, pg_restore and how to check a restore.

Key points

  • Managed takes over operation, not the responsibility for your data. Checking the restore stays with you, in both cases.
  • Check the list of allowed extensions before designing: it is a common blocking point when migrating to managed.
  • Relational is the right default choice. Change when you know why, not in anticipation.
  • A queue delivers a message to one consumer, a stream keeps it for several and lets you replay history.
  • If your application falls over when the cache is empty, it is no longer a cache but a database without guarantees; only sessions are an exception, at the price of users logging in again.
  • A cache solves a performance problem and creates two: invalidation and outage.

Knowledge check

Check that you have grasped the essentials of this guide. The questions cover only what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions
6 min
70% to pass

How it works

  • The timer starts when you click Start
  • Multiple choice, true/false and short answer questions
  • You can move between questions
  • Detailed results are shown at the end

Starts the quiz and the timer