AWS Deep Dive Part 2: Storage & Databases — S3, EBS, EFS, RDS, DynamoDB, ElastiCache

☁️ AWS DEEP DIVE SERIES (4 PARTS · 20 SERVICES)

• Part 1: Compute & Networking — EC2, Lambda, ECS, EKS, Fargate, VPC, Route 53 →
• ▸ Part 2: Storage & Databases — S3, EBS, EFS, RDS, DynamoDB, ElastiCache YOU ARE HERE
• Part 3: Application Integration — API Gateway, SQS, SNS, EventBridge, Step Functions →
• Part 4: Security, Observability & DevOps — IAM, CloudWatch, CloudFormation, Secrets Manager →

If Part 1 of this series was about where your code runs, Part 2 is about where your data lives. AWS offers a deep catalog of storage and database services, but in practice six of them carry the bulk of real-world workloads: S3 for object storage, EBS for block storage attached to EC2, EFS for shared file storage, RDS for managed relational databases, DynamoDB for a managed key-value/document store, and ElastiCache for in-memory caching. This post walks through what each one does, what tradeoffs it forces, and how to choose between them.

Three Storage Modalities, Three Tools

The first thing to internalize is that AWS storage breaks into three modalities, each with a single dominant service. Object storage (S3) is the right answer when you have files identified by a key, accessed via an API, and durability matters more than low-latency random access. Block storage (EBS) is what an operating system mounts as a disk; it gives you raw blocks and full POSIX semantics, but attaches to one (or few) instances at a time. File storage (EFS) sits between the two: it speaks NFS, is mountable by many clients at once, and grows elastically.

1. Amazon S3 — The Object Store

Amazon S3 (Simple Storage Service) is arguably the most important service on AWS. It stores arbitrary blobs as objects inside buckets, accessed via REST API by key. Each object is replicated across multiple Availability Zones, giving 99.999999999% (eleven nines) of durability and at least 99.99% availability on the standard storage class.

The thing that makes S3 a platform rather than just a disk is its storage classes. The same API stores hot data in S3 Standard, infrequently accessed data in Standard-IA or One Zone-IA, automatically tiered data in Intelligent-Tiering, and archival data in Glacier Instant Retrieval, Glacier Flexible Retrieval, and Glacier Deep Archive. Lifecycle rules move objects between classes automatically based on age, dramatically cutting cost for write-once-read-rarely datasets like logs and backups.

S3 also offers features that quietly do a lot of heavy lifting. Versioning keeps every overwrite as a separate version, protecting against accidental deletion. Object Lock enforces write-once-read-many compliance retention. S3 Replication copies objects to another bucket (potentially in another region) for DR or geo-locality. S3 Select lets you push SQL into individual CSV / JSON / Parquet objects so you scan only the rows you need. Presigned URLs let your backend hand a time-limited upload or download link to clients without proxying the bytes.

For security, every bucket is private by default. Block Public Access at both the account and bucket level catches the most common mistake (an accidentally public bucket). Encryption is on by default — either SSE-S3 (AWS-managed keys) or SSE-KMS (your KMS key); SSE-KMS adds an audit trail and per-key access control. Pair this with VPC Gateway Endpoints so workloads in private subnets reach S3 without traversing the public internet or a NAT Gateway.

2. Amazon EBS — Block Storage for EC2

Amazon EBS (Elastic Block Store) provides virtual disks that attach to EC2 instances. The volume lives in a single Availability Zone (an instance in another AZ cannot mount it), and you can pick from several volume types tuned for different workloads.

gp3 is the default general-purpose SSD; you provision capacity up to 16 TiB and independently dial IOPS up to 16,000 and throughput up to 1,000 MB/s. io2 Block Express is the high-end SSD for the most demanding databases — up to 256,000 IOPS, sub-millisecond latency, and 99.999% durability. st1 and sc1 are HDD volumes optimized for sequential throughput (big data, cold logs) at very low cost per GB.

Two operational features matter every day. Snapshots are incremental block-level backups stored in S3; they form the basis of AMIs, cross-region copies, and point-in-time recovery. Multi-Attach on io1/io2 lets a single volume attach to up to 16 Nitro-based instances simultaneously, enabling clustered databases that coordinate access at the application layer.

3. Amazon EFS — Shared File System

Amazon EFS is a fully managed NFSv4 file system that grows and shrinks automatically and is accessible by thousands of clients across all AZs in a region. It is the right choice when multiple instances need read/write access to the same files — think shared content directories, build caches across CI workers, or container workloads needing persistent shared state.

EFS comes in two performance modes (General Purpose, Max I/O) and two throughput modes (Bursting, Provisioned or Elastic). Lifecycle Management automatically moves files that have not been touched to EFS IA or EFS Archive tiers, cutting cost by up to 90% for cold data. EFS is significantly more expensive per GB than S3, so it pays to use it only where multi-writer POSIX semantics are actually required.

4. Amazon RDS — Managed Relational Databases

Amazon RDS provides managed instances of six engines: PostgreSQL, MySQL, MariaDB, Oracle, SQL Server, and Amazon’s own Aurora (a PostgreSQL- and MySQL-compatible engine with a custom storage layer). AWS handles provisioning, patching, backups, minor-version upgrades, and failover — you keep ownership of schema design, query tuning, and connection management.

The two operational features you will use most are Multi-AZ and read replicas. Multi-AZ keeps a synchronous standby in a different Availability Zone, with automatic failover in roughly 60–120 seconds when the primary fails (or in 35 seconds with the newer Multi-AZ Cluster deployment that adds two readable standbys). Read replicas are asynchronous copies, usable for read scaling or as a target for online schema changes; for Aurora, replicas share the same storage layer and lag is typically under 20 milliseconds.

For most new workloads, Aurora is the right starting point on AWS. Its storage tier is a distributed, log-structured system that replicates 6 ways across 3 AZs, automatically scales from 10 GB up to 128 TiB, and survives the loss of an AZ without data loss. Aurora Serverless v2 scales capacity (measured in ACUs) up and down in seconds, which is excellent for spiky or development workloads. The tradeoff is that Aurora is locked to AWS, so portability is reduced compared to vanilla RDS for PostgreSQL or MySQL.

5. Amazon DynamoDB — Serverless NoSQL at Scale

Amazon DynamoDB is the serverless key-value and document database that powers many of the largest workloads on AWS, including parts of Amazon.com itself. You define a table with a partition key (and optional sort key), pick a capacity mode, and Dynamo handles partitioning, replication, and indexing for you. Reads are single-digit-millisecond at any scale, and you never run a server.

Two capacity modes are available. Provisioned mode reserves a specific number of Read Capacity Units and Write Capacity Units, optionally with Auto Scaling; it is cheaper at steady-state. On-Demand mode charges per request and absorbs any spike automatically; it shines for unpredictable or seasonal workloads. Switching between the two is allowed (with one switch per 24 hours).

The biggest mental shift for engineers coming from relational databases is that DynamoDB is access-pattern-first. You enumerate the queries your application will make, then design the partition key, sort key, and Global Secondary Indexes to serve those queries directly. Joins do not exist; instead you denormalize, often into a single-table design that holds multiple entity types differentiated by the sort key prefix.

Beyond raw key-value access, Dynamo offers Streams (a change feed consumed by Lambda for event-driven workflows), Global Tables (active-active multi-region replication with last-writer-wins conflict resolution), Transactions (ACID across up to 100 items), and DAX (an in-memory write-through cache for microsecond reads). Used well, these eliminate most of the cases where a team thinks they need a separate cache or message bus.

6. Amazon ElastiCache — Managed Redis and Memcached

Amazon ElastiCache is the managed in-memory data store, available in two engines: Redis (now branded as ElastiCache for Valkey in newer regions) and Memcached. It runs on EC2 nodes inside your VPC, with the engine and OS patched by AWS and clustering handled for you.

The two engines solve different problems. Memcached is a simple distributed cache: pure key-value, no persistence, no replication, multi-threaded. It is the right fit when you want to scale a transient cache horizontally and you can lose the whole thing without consequence. Redis is a feature-rich in-memory database: strings, lists, hashes, sorted sets, streams, pub/sub, geospatial, Lua scripting, server-side TTL, optional persistence (RDB snapshots and AOF), and replication with automatic failover. Most teams pick Redis.

Most workloads use ElastiCache as a cache-aside read accelerator in front of RDS or DynamoDB: the application reads from Redis first, falls back to the database on miss, and writes the result back with a TTL. Other common patterns include using Redis as a session store, a rate limiter, a leaderboard backend, a pub/sub bus for small-scale fanout, and a stream queue (Redis Streams) for lightweight workflows.

The two pitfalls to watch for are memory pressure (Redis eviction policies matter when the working set exceeds memory) and the thundering herd when a popular key expires and many clients hit the database at once. Both are solved with capacity planning, a good eviction policy (typically allkeys-lru), and request coalescing patterns in the application.

Choosing the Right Data Service

The decision framework that works in practice: start with the access pattern and consistency requirement, then pick the service that natively gives you both. Need ACID transactions and ad-hoc analytical queries across normalized tables? Use RDS or Aurora. Need predictable single-digit-ms latency at any scale with known access patterns? Use DynamoDB. Need to store huge blobs cheaply with high durability? Use S3. Need a disk for an EC2 database? Use EBS. Need many machines to share files? Use EFS. Need to soak up read traffic or store transient state? Use ElastiCache in front of the primary store.

Cost Heuristics

Storage and database costs add up quietly because they keep accruing whether traffic is high or low. A few habits help. Turn on S3 Intelligent-Tiering for any bucket whose access pattern is mixed or unknown — it pays back fast. Set lifecycle rules to expire incomplete multipart uploads and to transition old logs to Glacier. Right-size EBS gp3 volumes: most older gp2 volumes are over-provisioned for IOPS and saving 20% is a one-command change. On RDS, use Graviton (db.r7g, db.m7g) instances where supported and consider Aurora I/O-Optimized when I/O is more than ~25% of the bill. On DynamoDB, profile read patterns before paying for On-Demand on a workload that is actually steady — Provisioned with Auto Scaling is often 30–60% cheaper.

What is Next

Compute, networking, and now storage and databases give you the building blocks for a complete application. The next post in this series covers application integration — how services talk to each other asynchronously. We will look at API Gateway as the front door, SQS and SNS for messaging, EventBridge for event routing, and Step Functions for orchestrating multi-step workflows. Part 4 will close the series with security, observability, and infrastructure as code.

☁️ CONTINUE THE AWS DEEP DIVE SERIES

• Part 1: Compute & Networking →
• ▸ Part 2: Storage & Databases (you are reading this)
• Part 3: Application Integration →
• Part 4: Security, Observability & DevOps →