3.1 Object storage

What this is and why it exists

Object storage is where almost everything that is not a database lives: uploads, images, backups, exports, logs, static sites. It is cheap, effectively unlimited, and reachable over the network rather than attached to a machine — and it is also the single most common source of accidental public data exposure in the whole industry. Both halves of that sentence are this lesson.

The vocabulary

• Object — one stored item: the bytes, plus a name and some metadata. Amazon calls the service S3, Azure calls it Blob Storage, Google calls it Cloud Storage.
• Bucket (or container) — the named space objects live in; the unit that carries permissions and configuration.
• Key — the object's full name inside the bucket, which looks like a path but is a single flat string.
• Storage class (or tier) — a price-and-behaviour setting per object: instant access, or cheaper storage with slower and charged retrieval.
• Lifecycle rule — an automatic policy that moves or deletes objects as they age.
• Presigned URL — a temporary link, signed by you, granting one specific operation on one object for a limited time.
• Versioning — keeping previous copies of an object when it is overwritten or deleted.

The mental model

There are no folders. The key 2026/august/invoice.pdf contains slashes, and the console draws them as a tree, but the store holds one flat map from key to bytes — there is no directory object anywhere. Two consequences follow. Renaming a "folder" is not a rename; it is copying every object to a new key and deleting the old ones, which for a million objects is a real job with a real cost. And listing is a prefix scan, so a well-chosen key prefix is what makes listing cheap, exactly as an index makes a query cheap.

Storage classes are the price of patience. The frequently-accessed class costs the most to store and nothing extra to read; colder classes cost dramatically less to store and charge you to retrieve, sometimes with a wait measured in hours, and often with a minimum storage duration you are billed for even if you delete early. That last clause is what catches people: moving objects to a cold class and then deleting them a week later can cost more than leaving them alone. Lifecycle rules automate the movement — after thirty days go colder, after a year go colder still, after seven years delete — and are the honest way to hold a retention policy, because a rule written once is applied to objects nobody remembers.

Then the leak, which deserves its own paragraph because it has embarrassed very large organisations repeatedly. A bucket can be made readable by anyone on the internet, and the setting is one click, has legitimate uses, and does not look dangerous at the moment it is set. The failures are almost always the same shape: a bucket opened for one quick share and never closed; a permission set at bucket level to make one file public; a backup written into a bucket that was already public. Two habits prevent nearly all of it. Turn on the account-level setting that refuses public access and leave it on, so making something public becomes a deliberate exception rather than a default anyone can reach. And for anything that genuinely needs sharing, do not make the bucket public at all.

Which is what presigned URLs are for. You keep the bucket private and generate a link that carries a signature, permitting exactly one operation — read this object, or upload to this key — for a stated number of minutes. The link works without any account or credentials, expires by itself, and cannot be widened into access to anything else. That covers the two cases people wrongly open a bucket for: letting a user download their own file, and letting a browser upload directly without the bytes passing through your server. Treat the link itself as the secret, keep the lifetime short, and remember that expiry is the only revocation you get.

What you should now be able to explain or do

Explain why there are no folders and what that means for renaming and listing. Choose a storage class for three different kinds of data, including one where the cheaper class would cost more. Write a lifecycle rule in plain words for logs you must keep for a year. Describe the two habits that prevent public-bucket leaks. Say when to use a presigned URL instead of making something public, and name the two things you control about one.

Check yourself