Cloud Computing, the Bezos Mandate, AWS and Azure

43.0 What this chapter gives you#

1. You will be able to say what “the cloud” actually is in one plain sentence, without any mystery, and say exactly what it changed and what it did not.
2. You will be able to describe a real datacentre from the outside wall to the individual server, including power, cooling and what a rack unit is.
3. You will be able to explain regions, availability zones and edge locations precisely, and name the real regions the big three run inside India.
4. You will be able to explain what a hypervisor does, why virtual machines became cheap, and how special hardware removed the last of the overhead.
5. You will be able to state clearly that a container is a process, not a small virtual machine, and name the exact Linux features that make it work.
6. You will be able to read a Dockerfile, a Kubernetes manifest and a Terraform file line by line, and say what each line causes to happen.
7. You will be able to quote the 2002 Bezos API mandate, explain why it forced Amazon to build a platform, and say honestly what it cost.
8. You will be able to tell the true story of how AWS started, and correct the popular myth about spare retail capacity with evidence.
9. You will be able to name the eighteen AWS services you meet in real work, say what problem each solves, and map each to Azure and Google Cloud.
10. You will be able to read a cloud bill, predict where it will explode, and argue both sides of the buy-versus-rent question with real numbers.

43.1 What the cloud actually is#

PLAIN43.1.1 in simple words#

1. The cloud is somebody else’s computers, sitting in somebody else’s building.
2. You rent them by the hour or by the second, instead of buying them.
3. You reach them over the internet, and you never touch them physically.
4. That is the whole idea. There is nothing floating in the sky.
5. The word “cloud” came from old network drawings, where people drew a fuzzy cloud shape to mean “the internet, and we are not going to draw the details”.
6. So “in the cloud” originally just meant “over there, past our own wires”.
7. Four things genuinely changed when this became a business you could buy.
8. First, elasticity: you can ask for 200 machines at 9am and give them back at 11am, and pay only for those two hours.
9. Second, pay per use: no big purchase up front, no machine sitting idle.
10. Third, self-service: you get the machines by making a request from a program, in seconds, with nobody’s approval and no phone call.
11. Fourth, somebody else deals with the building, the power, the cooling, the broken disks and the security guard at the door.
12. Here is what did not change. These are still real physical computers.
13. They still overheat, lose power, lose disks and lose network cables.
14. Your program can still be slow, still be wrong, and still fall over.
15. The cloud moved the pain. It did not delete the pain.

PLAIN43.1.2 a picture in your head#

1. Think about how you get electricity in your home.
2. You do not own a generator. You do not buy fuel or service a machine.
3. You plug in, you use what you need, and once a month a meter reading turns into a bill.
4. If you buy an air conditioner, you do not have to also buy a bigger generator. You just use more, and the bill goes up.
5. If you go away for a month, you use almost nothing, and you pay almost nothing.
6. Somebody else runs enormous power stations so that you can avoid running a small one badly.
7. Renting computers works the same way. Amazon, Microsoft and Google run the enormous machine rooms so that you can avoid running a small one badly.
8. This comparison is not new. The writer Nicholas Carr made it the centre of his 2008 book The Big Switch, comparing cloud computing with the way factories stopped running their own power plants a century earlier.

Where this comparison breaks: electricity is one product. One volt is like any other volt, and you can change supplier without rewiring your house. Cloud services are all different from each other, and moving from one supplier to another means rewriting real code. Also, electricity is metered on one dimension. A cloud bill meters compute time, storage, requests, data movement, addresses and dozens of other things at once, and the surprise is almost always in a dimension you were not watching.

PLAIN43.1.3 a worked example#

1. You are two students launching a website for a college festival.
2. Old way: you buy one server. Say it costs 90,000 rupees, plus a rack space rental, plus a person to look after it.
3. You must guess the busiest moment months in advance and buy for that.
4. On the festival night 40,000 people arrive at once. Your one server dies.
5. For the other 360 days of the year, the server is 3 per cent busy, and you have paid for all of it anyway.
6. Cloud way: you rent two small machines for the quiet months. On festival night you rent forty machines for six hours, then give them back.

7. Real prices, from the AWS public price list for the North Virginia region in August 2026: a small t3.medium machine costs 0.0416 US dollars per hour.
8. Forty of them for six hours is 40 x 6 x 0.0416 = 9.98 US dollars.
9. That is the actual thing that changed. Ten dollars, decided at 7pm, with no purchase order and no meeting.

PLAIN43.1.4 what is really happening inside#

1. You send a small message over the internet to the provider’s control system. It says, in effect, “give me one machine of this size, in this place”.
2. That message is a normal web request, signed with your secret key so the provider knows it is really you.
3. A piece of software called the control plane takes the request. Its job is to decide and to record, not to run your program.
4. It finds a physical server in the right building that has room, and picks it.
5. On that physical server, a program called a hypervisor carves out a slice: some processor time, some memory, some disk, one network address.
6. It boots an operating system into that slice from a saved disk image.
7. Within about 30 to 60 seconds, that slice answers to a network address, and you can log in as if it were a computer on your desk.
8. Everything after that is ordinary computing. The slice does not know or care that it is a slice.
9. A separate meter records that this slice existed, second by second, and that record eventually becomes a line on your bill.
10. When you say “delete it”, the hypervisor throws the slice away and the physical server is offered to the next customer.

TECHNICAL43.1.5 the engineer’s version#

1. The working definition of cloud computing is not marketing. It is a standard document: NIST Special Publication 800-145, published September 2011 by the United States National Institute of Standards and Technology.
2. It names five essential characteristics. A service is only cloud computing if it has all five.
3. On-demand self-service: the consumer provisions capacity unilaterally, with no human interaction with the provider.
4. Broad network access: capabilities are available over the network through standard mechanisms.
5. Resource pooling: physical and virtual resources are pooled and assigned to many consumers, with location independence at some level of abstraction.
6. Rapid elasticity: capabilities can be scaled outward and inward, in some cases automatically, and appear unlimited to the consumer.
7. Measured service: use is metered and reported, giving transparency to both sides.
8. The same document names three service models, IaaS, PaaS and SaaS, and four deployment models: private, community, public and hybrid.
9. That test is genuinely useful. A rented server with a two-week ticket queue fails point three and point six, and is therefore hosting, not cloud.
10. Scale of the industry, from published company filings. Amazon reported AWS net sales of 128,725 million US dollars for the full year 2025, with segment operating income of 45,606 million.
11. In the quarter ending June 2026 AWS reported 42.2 billion US dollars, up 37 per cent year on year, an annualized run rate near 169 billion.
12. Microsoft first disclosed Azure’s own revenue for its fiscal year 2025, at 75 billion US dollars, and reported that Azure passed 100 billion for fiscal 2026, growing 41 per cent.
13. Alphabet reported Google Cloud revenue of 17.664 billion US dollars for the quarter ending December 2025, up 48 per cent, and said the business ended 2025 at an annual run rate above 70 billion.

14. Commands that make the abstraction visible: aws ec2 describe-instances, az vm list, gcloud compute instances list. Each is a signed web request to a control plane, nothing more.

WORDS43.1.6 remember these#

1. Cloud — renting computers you reach over the internet — on-demand, self-service, metered, pooled and elastic computing per NIST SP 800-145.
2. Elasticity — getting more and giving it back quickly — automatic or manual scaling of provisioned capacity in both directions.
3. Control plane — the part that decides and records — the API and orchestration layer that provisions and tracks resources.
4. Data plane — the part that does the actual work — the running instances, packets and stored bytes themselves.
5. Provisioning — asking for and receiving a resource — the allocation of capacity to a tenant in response to an API call.
6. Tenant — one paying customer sharing the hardware — an isolated account whose workloads share pooled physical resources.

43.2 Inside a real datacentre#

PLAIN43.2.1 in simple words#

1. A datacentre is a warehouse full of computers, built so they never stop.
2. Inside, computers are not on desks. They are bolted into tall metal frames called racks, like books on a shelf but lying flat.
3. Each computer is a flat metal tray about as wide as a shelf and as thick as a few textbooks. Rows of them fill each rack from floor to head height.
4. Every rack has a small switch at the top that all its computers plug into.
5. Those switches plug into bigger switches, which plug into the internet.
6. Power comes in from the electricity grid, on two separate paths, so that one failure does not stop the room.
7. Between the grid and the computers sit big batteries, which cover the few seconds of a cut, and diesel generators, which cover the hours.
8. All that electricity turns into heat, so a large part of the building is machinery for taking heat away.
9. Cold air is pushed up through the floor into one aisle, sucked through the computers, and comes out hot into the next aisle.
10. Getting in is hard on purpose: fences, cameras, guards, card and fingerprint doors, and a locked cage around your own rack.
11. These buildings are enormous. A big one holds tens of thousands of computers and uses as much electricity as a small city.

PLAIN43.2.2 a picture in your head#

1. Think of a very large library.
2. The racks are the shelves, standing in long parallel rows.
3. Each server is one book lying flat on the shelf, and the shelf spacing is fixed so books of standard thickness always fit.
4. Every row of shelves has a librarian at the end of it, the top-of-rack switch, who handles all messages in and out of that row.
5. The librarians all report to a small group of head librarians, and any librarian can reach any other through any head librarian.
6. The reading rooms alternate: one cold room, one warm room, one cold, one warm, because readers give off heat and you keep the cold air away from the warm exhaust.
7. The building has its own power station in the yard, in case the city cuts supply, and a battery room that keeps the lights on while it starts.

Where this comparison breaks: a library’s books are the point of the library. Here, the servers are interchangeable and expected to die. Roughly two to five per cent of servers fail in a year, and staff replace them without anybody noticing. Also, a library is quiet. A datacentre hall is loud enough that long-stay staff wear ear protection, because thousands of fans are running.

PLAIN43.2.3 a worked example#

1. Take one standard rack. The width of the mounting rails is 19 inches, which is 482.6 millimetres. This is the EIA-310 standard, and it dates from telephone equipment in the 1920s.
2. Height is measured in rack units, written U. One U is 1.75 inches, which is 44.45 millimetres, exactly.
3. A common rack is 42U tall, about 1.87 metres of usable height.
4. A basic web server is 1U thick. So 42 of them fit, minus space for switches and cable management. In practice you plan for about 38 servers plus 2 switches.
5. Now the power. Suppose each server draws 350 watts on average.
6. 38 x 350 W = 13,300 W, which is 13.3 kilowatts for one rack.
7. An older datacentre was designed for 5 to 10 kilowatts per rack, so that rack would not fit its power budget. You would fill the rack only half full.
8. A modern rack of graphics processors for artificial intelligence work is a different animal. An NVIDIA GB200 NVL72 rack draws up to about 120 kilowatts, which is roughly what forty Indian homes use at peak.
9. Air cannot carry that much heat away. Those racks are water-cooled, with cold liquid piped directly onto the chips.

PLAIN43.2.4 what is really happening inside#

1. Grid electricity arrives at high voltage and is stepped down by transformers outside the building.
2. It passes through switchgear that can move the whole load from one source to another in a fraction of a second.
3. It then reaches the uninterruptible power supply, the UPS. This is a bank of batteries or a spinning flywheel.
4. The UPS is not there to run the building for hours. It is there to hold the load for the 10 to 30 seconds a diesel generator needs to start and take over.
5. Generators keep enough fuel on site for a day or more, with contracts for refuelling after that.
6. From the UPS, power goes to rack-level power strips, called PDUs. Servers have two power supplies, plugged into two independent strips, fed from two independent paths.
7. Cooling runs in a loop. Computer room air handlers blow cold air under a raised floor or through overhead ducts into the cold aisle.
8. Server fans pull that air front to back, over the hot parts, and blow it into the hot aisle behind.
9. Physical barriers, doors and roofs, seal the aisles so cold and hot air never
   1009. Mixing is the single biggest waste in a machine room.
10. The hot air returns to the coolers, gives its heat to a water loop, and the water carries the heat to chillers or to outdoor cooling towers.
11. For very hot racks, air is skipped entirely. A cold plate sits directly on the chip and water takes the heat out of the rack.
12. Meanwhile the network: each server has one or two cables to the switch at the top of its own rack, and those switches connect upwards to a layer of larger switches, so that any server can reach any other in a small, fixed number of hops.

TECHNICAL43.2.5 the engineer’s version#

1. Rack mechanics follow EIA-310-D. Rail spacing 19 inches, 1U equals 44.45 mm, common heights 42U, 45U, 48U and 52U, common depths 1000 to 1200 mm.
2. Rackmount servers are 1U, 2U or 4U. A 1U dual-socket server today typically holds 2 CPUs of 32 to 128 cores each, 12 to 32 memory slots and a handful of NVMe drives.
3. Blade servers put many thinner server modules into a shared chassis, sharing power supplies, fans and network fabric. They win on density and cabling and lose on vendor lock-in. Hyperscale operators mostly use custom rackmount designs instead, following the Open Compute Project, which Facebook founded in 2011.
4. Network topology is spine-leaf, also called a folded Clos network after Charles Clos’s 1953 telephone switching paper. Every leaf connects to every spine, and no leaf connects to another leaf.

        [spine 1]   [spine 2]   [spine 3]   [spine 4]
           |  \   /   |   \  /    |   \  /     |
           |   \ /    |    \/     |    \/      |
           |   / \    |    /\     |    /\      |
        [leaf A]   [leaf B]   [leaf C]   [leaf D]
          | | |      | | |      | | |      | | |
        servers    servers    servers    servers

  Any server to any server: leaf -> spine -> leaf. Two hops.
  Add a spine to add bandwidth. Add a leaf to add racks.

5. Leaf switches are the top-of-rack switches, typically 48 ports of 25 or 100 gigabits per second down to servers, and 4 to 8 uplinks of 100 to 400 gigabits per second up to the spine.
6. The ratio of downlink to uplink bandwidth is the oversubscription ratio. 3:1 is common in general fleets. Storage and AI clusters are built at 1:1, meaning non-blocking.
7. Power redundancy is described as N, N+1 or 2N. The Uptime Institute Tier classification runs Tier I to Tier IV, where Tier III allows concurrent maintenance and Tier IV is fault tolerant.
8. ASHRAE Thermal Guidelines class A1 recommends an inlet temperature of 18 to 27 degrees Celsius. Operators run near the top of that band deliberately, because warmer air needs less chilling.
9. Power usage effectiveness, PUE, is total facility power divided by IT equipment power. A PUE of 2.0 means one watt of overhead for every watt of computing. The theoretical floor is 1.0.

10. Google’s published history shows fleet PUE falling from about 1.23 in 2008 to 1.09 today. The global average has been stuck near 1.54 since 2019, which is an honest and unflattering industry fact.
11. Physical security is audited, not asserted. The relevant reports are SOC 2 Type II, ISO/IEC 27001 and, for cardholder data, PCI DSS. Controls include perimeter fencing, vehicle barriers, two-factor mantrap entry, biometric readers, per-cage access lists and asset destruction on decommission.
12. Scale and cost, from 2026 construction cost surveys: roughly 11.3 million US dollars per megawatt of IT capacity for a standard facility, and 20 million or more per megawatt for an AI-optimized build.
13. On that basis a 500 megawatt campus costs about 5.5 to 6 billion US dollars standard, or over 10 billion for a dense AI build. Electrical systems alone are 40 to 50 per cent of construction cost.
14. Microsoft stated that it deployed about 2 gigawatts of new datacentre capacity during its fiscal year 2025. That single year of one company is comparable to the entire generating capacity of a large power station.

WORDS43.2.6 remember these#

1. Rack unit (U) — one shelf slot — 44.45 mm of vertical mounting height under EIA-310-D.
2. Top-of-rack switch — the small switch every server in a rack plugs into — the leaf layer of a Clos fabric, typically 48 x 25G with 100G uplinks.
3. Spine-leaf — a wiring plan where any machine is two hops from any other — a folded Clos topology with full leaf-to-spine meshing.
4. UPS — the battery that covers a power cut — an uninterruptible power supply sized to bridge to generator start, typically 10 to 30 seconds.
5. Hot and cold aisle — keeping cold air and hot exhaust apart — containment of supply and return airflow to prevent recirculation.
6. PUE — how much extra power the building wastes — total facility power divided by IT load, global average 1.54, hyperscale 1.10 to 1.15.
7. CRAC and CRAH — the room’s air conditioners — computer room air conditioner with its own compressor, versus air handler fed by chilled water.

43.3 Regions, availability zones and edge locations#

PLAIN43.3.1 in simple words#

1. A region is a place on the map where a provider has built capacity. Mumbai is a region. Frankfurt is a region. Ohio is a region.
2. Regions are far apart, and they are run as almost separate businesses. Prices differ, and some services exist in one region and not another.
3. Inside a region there are availability zones. A zone is one or more buildings with its own power, its own cooling and its own network links.
4. Two zones in one region are close enough that talking between them is fast, but far enough apart that one flood, fire or substation failure cannot take out both.
5. That is the whole point. A zone is a failure domain: a boundary that damage is not supposed to cross.
6. So you never put your only copy of anything in one zone. You run at least two copies, in two zones, and something in front that sends traffic to whichever is alive.
7. An edge location is a third thing and much smaller. It is a small set of machines in many cities that hold copies of files and terminate connections close to users.
8. Edge locations make things feel fast. They do not run your application. Do not confuse them with regions.
9. Rule of thumb: zones protect you from a building. Regions protect you from a city or a country. Edge locations protect you from distance.

PLAIN43.3.2 a picture in your head#

1. Think of a bank with branches.
2. A region is a city where the bank operates.
3. Availability zones are three separate branch buildings in that city, on different streets, on different electrical supplies.
4. If one branch burns down, your money is still in the other two, because the bank writes every transaction to all three before confirming it.
5. An edge location is an ATM. There are hundreds of them, in many towns, and they are fast to reach.
6. But an ATM cannot open an account or approve a loan. It holds cash and handles simple requests. For anything real it talks back to a branch.
7. Countries also have rules: some money must stay inside the country. That is data residency.

Where this comparison breaks: bank branches in a city are genuinely independent businesses that can run alone. Cloud availability zones share a regional control plane, and some regional services have components that live in only one zone. In the AWS event of 20 October 2025, one subsystem in one region failed and the effect was felt across the whole region and by customers worldwide, even though the individual zones had power and cooling the entire time.

PLAIN43.3.3 a worked example#

1. You run a shop website for Indian customers. You pick the AWS Mumbai region, whose code name is ap-south-1.
2. That region has three availability zones, named ap-south-1a, ap-south-1b and ap-south-1c.
3. You place two web servers in zone a and two in zone b, with a load balancer in front that spreads traffic across both.
4. You place your database as a primary in zone a and a standby copy in zone b, with automatic promotion if the primary is lost.
5. Now zone a loses power. The load balancer stops sending traffic to the two dead servers within about 30 seconds. The database standby is promoted in one to two minutes. Customers see a short blip.
6. Cost of that safety: you run four servers where two would do, and you pay for traffic that crosses between zones, at 0.01 US dollars per gigabyte in each direction.
7. Now suppose the whole Mumbai region fails. Nothing you have deployed helps. Surviving that needs a second region, which needs a second copy of your data kept up to date, which is a much larger project.

PLAIN43.3.4 what is really happening inside#

1. Zone names are shuffled per account on purpose. Your ap-south-1a and my ap-south-1a are usually different physical zones, so that customers do not all crowd into the alphabetically first one.
2. The stable identifier is a zone ID such as aps1-az1, which is the same for everyone. That is what you compare when you need to know two accounts are truly in the same building.
3. Between zones, the provider runs its own dedicated fibre, usually many redundant paths. Round trip time is typically well under two milliseconds.
4. Between regions, traffic crosses the provider’s private long-distance backbone. Mumbai to Singapore is roughly 60 to 70 milliseconds round trip. Mumbai to North Virginia is roughly 190 to 220 milliseconds.
5. Those numbers come from the speed of light in glass and cannot be improved by any amount of money. Light travels about 200,000 kilometres per second in fibre, so 1,000 km of fibre costs about 5 milliseconds one way, before any switching delay.
6. A region failure, in practice, is rarely the building burning. It is far more often a software or configuration fault in a service that everything else in the region depends on, such as identity, DNS or the metadata store.
7. When that happens, machines that are already running usually keep running, but you cannot start new ones, cannot log in, and cannot change anything.

TECHNICAL43.3.5 the engineer’s version#

1. Current published footprints, checked in August 2026. AWS states 123 availability zones across 39 geographic regions, with plans announced for the Kingdom of Saudi Arabia and Chile. Microsoft states over 70 Azure regions. Google states 43 regions and 130 zones.
2. AWS defines an availability zone as one or more discrete datacentres with redundant power, networking and connectivity, physically separated by a meaningful distance, up to 100 km apart, with single-digit millisecond latency between them.
3. Azure calls the equivalent an availability zone too, and pairs most regions into region pairs for platform-managed geo-replication. You cannot choose your pair.
4. Google Cloud calls them zones, and a zone is a deployment area within a region, isolated for failure purposes.
5. The India footprint, with names as the providers publish them:

6. Azure also offers Jio India West and Jio India Central, operated in partnership with Reliance Jio. Azure’s three 2015 regions are in Pune, Chennai and Mumbai; the 2026 India South Central region is in Hyderabad and opened with three availability zones.
7. AWS announced an investment of 4.4 billion US dollars in India by 2030 at the Hyderabad launch. Microsoft’s stated India commitment reached 20.5 billion US dollars by 2026, combining announcements from January 2025 and December 2025.
8. Data residency is a legal question, not a technical one. In India the relevant instruments are the Digital Personal Data Protection Act, 2023, and the Reserve Bank of India’s April 2018 directive requiring payment system data to be stored in India.
9. In the European Union it is the General Data Protection Regulation of 2016, applied from May 2018, plus sector rules and sovereign cloud offerings.
10. Choosing a region therefore has three inputs: latency to your users, price, and law. They frequently disagree.
11. The honest note on multi-region. Marketing implies you tick a box. The reality is that you must solve write conflicts, replication lag, failover orchestration, DNS behaviour under partial failure, cost of duplicated capacity, and testing.
12. Almost nobody tests region failover under load. A failover path that has never been exercised is a hypothesis, not a capability.
13. Real example to anchor this. On 20 October 2025 AWS suffered a major disruption in us-east-1, caused by a latent race condition in the automated DNS management of DynamoDB. The trigger occurred around 06:49 UTC, DNS was restored by 09:25 UTC, and full recovery of dependent systems such as EC2 instance launches continued until about 20:50 UTC the same day, with some Redshift work finishing the next morning.
14. Many companies that believed they were multi-region discovered that their identity provider, their build system or their DNS was single-region.
15. Commands to inspect this: aws ec2 describe-availability-zones --region ap-south-1 shows both the name and the stable zone ID.

WORDS43.3.6 remember these#

1. Region — a city-sized place where the provider has capacity — an independent deployment area with its own service catalogue, pricing and endpoints.
2. Availability zone — one building group with its own power — an isolated failure domain within a region, single-digit millisecond latency to peers.
3. Failure domain — the boundary damage should not cross — the blast radius of a single correlated fault.
4. Edge location — a small cache near users — a point of presence for content delivery and connection termination, not general compute.
5. Data residency — the rule that data must stay in a country — a legal obligation on storage location, distinct from encryption or access control.
6. Region pair — Azure’s fixed partner region — a Microsoft-assigned pairing used for platform geo-replication and staged updates.

43.4 Virtualization, the technology that made cloud possible#

PLAIN43.4.1 in simple words#

1. One physical computer used to run one operating system, and that was that.
2. Most of the time it was mostly idle, waiting for something to do.
3. Virtualization is a trick that lets one physical computer pretend to be many separate computers at once.
4. The program that does the pretending is called a hypervisor.
5. It sits between the real hardware and the pretend computers, and it hands out slices of processor, memory, disk and network.
6. Each pretend computer is called a virtual machine. It boots its own operating system and believes it is alone.
7. It is not alone. Ten or forty of its neighbours are on the same metal, completely unaware of each other.
8. This is what made renting computers by the hour possible. Without it, giving a customer a machine would mean physically plugging one in.
9. With it, giving a customer a machine means writing a few numbers into a table and starting a process.
10. It also lets the provider move a running customer off a server that needs repair, without switching that customer off. That is called live migration.

PLAIN43.4.2 a picture in your head#

1. Think of a large old house converted into flats.
2. The house is the physical server. The flats are the virtual machines.
3. Each family has its own front door, own kitchen, own furniture, and never sees the others.
4. There is one water main, one electricity supply and one roof, shared. A caretaker, the hypervisor, splits them fairly and stops one family from flooding another.
5. If one family plays loud music, the others may hear it a little. That is resource contention, the noisy neighbour problem.
6. If the caretaker needs to fix the roof over one flat, a very good caretaker can move the whole family, mid-dinner, to an identical flat next door, so fast they barely notice. That is live migration.

Where this comparison breaks: a family in a flat knows it is in a flat. A well built virtual machine genuinely cannot tell, by any ordinary means, that it is virtual. That indistinguishability is a formal design goal, stated by Gerald Popek and Robert Goldberg in their 1974 paper, and it is stronger than any housing analogy suggests. Also, the caretaker in this story has absolute power: the hypervisor can read every byte of every flat, which is why the trust model of the provider matters so much.

PLAIN43.4.3 a worked example#

1. Take a physical server with 2 processors of 64 cores each, so 128 physical cores, plus 1,024 gigabytes of memory.
2. With hyper-threading on, the operating system sees 256 logical processors.
3. You want to sell virtual machines with 4 virtual processors and 16 gigabytes of memory each.
4. On memory alone, 1,024 divided by 16 gives 64 virtual machines, minus about 5 per cent kept for the hypervisor itself. Call it 60.
5. On processors, 60 virtual machines x 4 virtual processors = 240 virtual processors against 256 logical ones. That is roughly 1:1, no oversubscription.
6. If instead you sell to customers whose machines are usually idle, you might place 120 virtual machines, giving 480 virtual processors on 256 logical ones, an oversubscription ratio of nearly 2:1.
7. That is a business decision, not a technical one, and it is exactly why cheap virtual servers feel inconsistent and expensive ones do not.

8. You can measure this from inside a Linux guest. Run top and look at the st column, short for steal time. It is the percentage of time your virtual processor was ready to run but the hypervisor gave the physical core to somebody else. Anything consistently above 5 is a crowded host.

PLAIN43.4.4 what is really happening inside#

1. A processor has at least two privilege levels: a privileged one for the operating system and an unprivileged one for applications.
2. Normally the operating system sits in the privileged level and controls the hardware directly.
3. Under virtualization, the hypervisor takes the privileged level, and the guest operating system is pushed down a level, even though it still thinks it is privileged.
4. When the guest executes an instruction that only a truly privileged program may run, the processor stops and hands control to the hypervisor. This is called a trap.
5. The hypervisor looks at what the guest was trying to do, does something safe and equivalent, and resumes the guest.
6. Memory is handled with two levels of translation. The guest maps its own pages as usual; the hardware then maps those again to real physical pages, using a second table the hypervisor controls.
7. Disks are usually just large files on the host, or slices of a network storage system. A write inside the guest becomes a write to that file.
8. Network cards are emulated or, better, partly real: modern cards can present many independent copies of themselves, and the hypervisor gives one copy directly to each guest.
9. Live migration works by copying memory pages to the destination host while the guest keeps running, repeatedly re-copying pages that changed, until the set of changed pages is small enough to copy in a brief pause.
10. That final pause is typically well under a second, and the guest’s network connections survive because the address moves with it.

TECHNICAL43.4.5 the engineer’s version#

1. Gerald Popek and Robert Goldberg, in Formal Requirements for Virtualizable Third Generation Architectures, Communications of the ACM, July 1974, gave three requirements: equivalence, resource control and efficiency.
2. Their theorem: an architecture is classically virtualizable if the set of sensitive instructions is a subset of the privileged instructions.
3. x86 failed that test. John Robin and Cynthia Irvine identified 17 problematic instructions in a 2000 USENIX Security paper. Instructions such as POPF silently did the wrong thing instead of trapping.
4. Two workarounds followed. VMware, founded in 1998, shipped binary translation: rewrite the guest’s privileged instruction stream on the fly.
5. Xen, from the University of Cambridge, presented at SOSP in 2003, shipped paravirtualization: modify the guest kernel so it calls the hypervisor deliberately, through hypercalls, instead of executing offending instructions. Faster, but needs a modified guest.
6. Hardware assistance ended the argument. Intel VT-x shipped in November 2005; AMD-V, codenamed Pacifica, shipped in May 2006. Both add a new guest mode, so an unmodified guest kernel can run at its expected privilege level and trap cleanly to the hypervisor.
7. Memory virtualization was still expensive until second level address translation arrived: AMD Nested Page Tables in Barcelona in 2007, Intel Extended Page Tables in Nehalem in 2008. Before that, hypervisors maintained shadow page tables in software.
8. Device passthrough needs an IOMMU: Intel VT-d and AMD-Vi. PCI-SIG’s Single Root I/O Virtualization specification, published 2007, lets one physical card present many virtual functions, each assignable to one guest.

9. The type 1 versus type 2 split is a useful teaching device and a slightly leaky one. KVM, merged into Linux 2.6.20 in February 2007, turns the Linux kernel itself into a type 1 hypervisor while Linux remains a normal operating system. Experts differ on which box it belongs in.
10. From the host’s point of view a KVM virtual machine is an ordinary process. You can see it in ps, one thread per virtual processor, its guest memory is an anonymous mapping, and you can kill it with kill.
11. Live migration in QEMU/KVM is iterative pre-copy: transfer all pages, then repeatedly transfer the pages dirtied during the previous pass, then stop the guest, transfer the remainder and the device state, and resume on the target. Typical stop time is tens to a few hundred milliseconds.
12. The hypervisor tax is the fraction of a machine consumed by virtualization rather than by customers. Werner Vogels has written that on older AWS designs this was around 30 per cent of the resources.
13. AWS attacked it with hardware. The path: C3 in 2013 offloaded network processing, C4 in 2014 offloaded storage, Amazon acquired the Israeli chip company Annapurna Labs in early 2015, and the full Nitro System shipped with C5 instances in 2017.
14. Nitro has three parts: Nitro Cards, which are dedicated hardware running VPC networking, EBS storage and instance storage; the Nitro Security Chip, which verifies firmware; and the Nitro Hypervisor, a thin KVM-based component that now mainly assigns memory and CPU.
15. Because almost nothing is left in software, AWS can also sell .metal instances, where a customer gets the whole physical server with no hypervisor at all, yet still gets normal EBS volumes and VPC networking.
16. Microsoft’s equivalent story runs through programmable hardware. The Catapult project put FPGAs in Azure servers from 2015, Accelerated Networking used those FPGA-based SmartNICs to move virtual switching off the CPU, and Azure Boost, announced in 2023, offloads storage and networking to a dedicated card in the same spirit as Nitro.
17. Observation commands: lscpu shows the hypervisor vendor line inside a guest; systemd-detect-virt names the hypervisor; dmidecode -s system-manufacturer often reveals it; virsh list and virsh domstats inspect KVM guests from the host.

WORDS43.4.6 remember these#

1. Hypervisor — the program that splits one computer into many — a virtual machine monitor providing equivalence, resource control and efficiency.
2. Type 1 versus type 2 — running on bare metal versus on top of an existing operating system — native versus hosted VMM.
3. Paravirtualization — the guest knows it is a guest and cooperates — replacing sensitive instructions with explicit hypercalls, as in Xen.
4. VT-x and AMD-V — processor features that make the trick cheap — hardware virtualization extensions adding a distinct guest execution mode, 2005 and
5. Oversubscription — selling more slices than you physically have — allocating virtual resources beyond physical capacity, relying on idle time.
6. Steal time — time your machine was ready but not scheduled — the st counter in Linux CPU accounting, the direct measure of host contention.
7. Live migration — moving a running machine without switching it off — iterative pre-copy of memory and device state to another host.
8. Nitro — AWS’s offload hardware — dedicated cards and a minimal hypervisor that remove virtualization overhead and enable bare-metal instances, from 2017.

43.5 Containers#

PLAIN43.5.1 in simple words#

1. A virtual machine is a whole pretend computer, with its own operating system inside. That costs memory and takes tens of seconds to start.
2. Most of the time you did not want a computer. You wanted to run one program, with the exact files and libraries it expects.
3. A container gives you that. It is one ordinary program, running on the normal operating system, but wrapped so that it can only see its own files, its own processes and its own network.
4. Say this clearly, because almost every beginner gets it wrong: a container is not a small virtual machine. There is no second operating system inside it.
5. From the host’s point of view it is just a process, visible in the process list, killable, schedulable, like any other.
6. The wrapping is done by features built into the Linux kernel. The container asks for a private view of the world, and the kernel gives it one.
7. Because there is nothing to boot, a container starts in tens of milliseconds instead of tens of seconds.
8. Because there is no second operating system, it uses megabytes of extra memory instead of hundreds of megabytes.
9. The other half of the idea is the image: a frozen, shareable copy of all the files the program needs, built once and run anywhere.
10. That is why containers solved the oldest complaint in software: “it works on my machine”. Now the machine travels with the program.
11. The trade is isolation. Virtual machines are separated by hardware enforcement. Containers are separated by kernel bookkeeping, which is thinner, and a kernel bug can cross it.

PLAIN43.5.2 a picture in your head#

1. Imagine a big office building with one security system, one power supply and one set of stairs. That is the operating system kernel.
2. A virtual machine is like building a whole separate house in the car park: its own foundations, its own wiring, its own everything.
3. A container is like giving one team a locked room inside the existing building, plus a rule at reception that they may not ask about any other team, plus a cap on how much electricity and how many desks they may use.
4. The team genuinely cannot see the other teams. Ask them who else is in the building and they will answer, honestly, “nobody”.
5. Setting up a room takes minutes. Building a house in the car park takes months.
6. But the room is inside a building that somebody else controls. If the building’s fire alarm has a flaw, everybody in it is affected.

Where this comparison breaks: the locked room suggests a physical wall. There is no wall. There is only the receptionist consistently refusing to answer certain questions. If you find a way to make the receptionist answer, you are out. That is exactly what a container escape is, and it is why you do not run untrusted code in a plain container without further sandboxing.

PLAIN43.5.3 a worked example#

1. Here is a real Dockerfile for a small Python web service, with every line explained afterwards.

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
RUN useradd -m appuser && chown -R appuser /app
USER appuser
EXPOSE 8000
ENV PORT=8000
CMD ["gunicorn", "-b", "0.0.0.0:8000", "app:application"]

2. FROM python:3.12-slim starts from an existing image that already contains a minimal Linux userland and Python 3.12. You are not building from nothing.
3. WORKDIR /app creates and enters a directory. Later commands run there.
4. COPY requirements.txt . copies just the list of dependencies in first.
5. RUN pip install ... installs those dependencies. Because this is a separate layer, and because only the requirements file was copied so far, this expensive step is reused from cache whenever your source changes but your dependency list does not. This ordering is the single most valuable trick in the file.
6. COPY . . now copies your actual source code.
7. RUN useradd ... creates a non-root user and gives it ownership.
8. USER appuser means every later instruction, and the running container, operate as that user. Running as root inside a container is a common and bad default.
9. EXPOSE 8000 is documentation plus metadata. It does not open a port by itself.
10. ENV PORT=8000 sets an environment variable inside the image.
11. CMD [...] is the command run when the container starts. The list form avoids a shell, so signals reach your process directly.
12. The commands to build and run it:

docker build -t festival-site:1.4 .
docker run -d --name web -p 8080:8000 \
  --memory 512m --cpus 1.5 festival-site:1.4
docker ps
docker logs -f web
docker exec -it web /bin/sh
docker stop web && docker rm web

13. -p 8080:8000 maps port 8080 on the host to 8000 in the container.
14. --memory 512m --cpus 1.5 are the limits. Without them, one container can consume the whole machine.

PLAIN43.5.4 what is really happening inside#

1. When you run a container, the runtime asks the Linux kernel to start a process with a private view of six or seven kinds of thing. Each private view is called a namespace.
2. The PID namespace gives the process its own process numbering. The first process inside sees itself as PID 1 and cannot see the host’s processes.
3. The network namespace gives it its own network interfaces, its own routing table and its own firewall rules. Usually a virtual cable is created, one end inside, one end on a bridge in the host.
4. The mount namespace gives it its own filesystem tree, so / inside is not / outside.
5. The user namespace lets an identity that is root inside map to an ordinary unprivileged identity outside.
6. The IPC namespace separates shared memory and message queues. The UTS namespace lets the container have its own hostname.
7. Separately, cgroups put ceilings on what the process may consume: so much memory, so much processor time, so much disk bandwidth. If it exceeds the memory ceiling, the kernel kills it.
8. The filesystem is assembled from stacked read-only layers plus one writable layer on top. Reads fall through the stack to the first layer that has the file. Writes go to the top layer only, copying the file up first.
9. Because the read-only layers are shared, ten containers from the same image store one copy of the common files, not ten.
10. The image itself is just those layers plus a small JSON document describing the command, environment, user and layer order.
11. A registry is a web service that stores and serves those layers and documents, addressed by content hash.

TECHNICAL43.5.5 the engineer’s version#

1. The idea is old. chroot appeared in Version 7 Unix in 1979: change the apparent root directory of a process. It isolates the filesystem view and nothing else, and it was never a security boundary.
2. FreeBSD jails, by Poul-Henning Kamp, arrived in FreeBSD 4.0 in March 2000, adding process, network and user isolation to the chroot idea.
3. Solaris Zones shipped in Solaris 10 in 2005, with resource controls and a full virtualized environment per zone.
4. On Linux the pieces arrived separately. The mount namespace landed in kernel 2.4.19 in 2002. Process containers, developed at Google by Paul Menage and Rohit Seth, were renamed control groups and merged in kernel 2.6.24 in January
   2008. The user namespace was completed in kernel 3.8 in 2013.
5. LXC, released in 2008, was the first tool to assemble those pieces into something usable. It required real Linux skill.
6. Docker changed the packaging, not the kernel. Solomon Hykes demonstrated it at PyCon in Santa Clara and it was open sourced in March 2013, out of a company then called dotCloud, renamed Docker Inc. later that year.
7. Docker’s contribution was the image format, the layer cache, the Dockerfile and a public registry: a developer workflow, not an isolation mechanism. Early versions used LXC underneath, replaced by its own libcontainer in version 0.9 in 2014.
8. Standardization followed. The Open Container Initiative was founded in June 2015 under the Linux Foundation, by Docker, CoreOS and others. Its runtime and image specifications reached version 1.0 in July 2017, and the distribution specification followed in 2021.
9. runc is the OCI reference runtime. containerd and CRI-O are the higher level runtimes that Kubernetes actually calls. Docker Engine today is a developer-facing layer over containerd.
10. Namespaces in the kernel, with their clone flags:

11. There are also cgroup and time namespaces, added later. cgroup v2, the unified hierarchy, is the current standard on modern distributions.
12. Overlay filesystems: overlayfs was merged in kernel 3.18 in 2014 and is the default storage driver. It presents a merged view of lower read-only directories and one upper writable directory, with copy-up on first write.
13. Image layers are content-addressed by SHA-256 digest. The image manifest lists layer digests; the config blob holds the command, environment and history. Pulling an image you already partly have transfers only the missing digests.
14. Additional hardening beyond namespaces and cgroups: seccomp-bpf to restrict system calls, AppArmor or SELinux for mandatory access control, dropping Linux capabilities, and read-only root filesystems.
15. For untrusted workloads, use a stronger boundary: gVisor, which intercepts system calls in userspace, or Kata Containers and AWS Firecracker, which put a genuine lightweight virtual machine around each container. Firecracker, open sourced by AWS in 2018, boots a micro virtual machine in about 125 milliseconds and is what runs AWS Lambda and Fargate.
16. Inspection commands: lsns lists namespaces, nsenter enters one, systemd-cgls and systemd-cgtop show cgroup trees, docker image history shows the layers and their sizes, crictl ps lists containers on a Kubernetes node.

WORDS43.5.6 remember these#

1. Container — an isolated process with its own files and limits — a process group running under restricting namespaces and cgroups on a shared kernel.
2. Namespace — a private view of one kind of system resource — a kernel mechanism partitioning PID, network, mount, user, IPC, UTS and cgroup views.
3. cgroup — a ceiling on how much a process may consume — control group hierarchy enforcing CPU, memory and I/O limits, cgroup v2 unified.
4. Image — a frozen copy of everything the program needs — an ordered set of content-addressed layers plus a config manifest, per the OCI image spec.
5. Layer — one saved change on top of the previous ones — a tar archive of filesystem differences, shared and cached by digest.
6. Registry — the shop where images live — an OCI distribution-spec service storing manifests and blobs, such as Docker Hub or ECR.
7. OCI — the agreed rules so tools interoperate — the Open Container Initiative runtime, image and distribution specifications, founded June 2015.
8. Container escape — breaking out into the host — exploitation of a kernel or configuration flaw to cross the namespace boundary.

43.6 Kubernetes and orchestration#

PLAIN43.6.1 in simple words#

1. Containers are easy to run one at a time on your laptop.
2. Now imagine 400 of them, across 30 machines, each needing to find the others, restart when they die, and move when a machine is taken away.
3. Doing that by hand is impossible. You need a program that does it for you. That program is called an orchestrator.
4. Kubernetes is the one that won. The name is Greek for helmsman, and it is often shortened to K8s, because there are eight letters between K and s.
5. The idea at its heart is simple and worth memorizing: you describe what you want, and the system continuously works to make reality match.
6. You do not say “start three copies”. You say “there should be three copies”.
7. If one dies at 3am, nobody is paged, because nothing has been violated except temporarily. The system notices the gap and starts another.
8. If you edit the number to five, the system notices the gap and starts two more. The same machinery handles both cases.
9. This is called declarative reconciliation: state the desired end, let a control loop close the difference, forever.
10. Everything else in Kubernetes is detail on top of that one idea.

PLAIN43.6.2 a picture in your head#

1. Think of a thermostat on a wall.
2. You do not tell the heater to switch on for eleven minutes. You set 22 degrees.
3. The thermostat measures, compares with your setting, and acts. Then it does it again, forever.
4. Open a window and the temperature drops; the thermostat responds without being told about the window.
5. Kubernetes is a building full of thermostats, one for each kind of thing.
6. One watches “how many copies of this program are running” against “how many should be”. Another watches “which machines are alive”. Another watches “what traffic goes where”.
7. Each loop is small, dull and independent. The intelligence is in the fact that they never stop running.

Where this comparison breaks: a thermostat has one input and one output. Kubernetes controllers act on each other’s outputs, so an error can propagate. A wrong setting does not sit still and look wrong; it is enthusiastically and repeatedly enforced across your entire cluster in seconds. That is the same property that makes it powerful and makes mistakes fast.

PLAIN43.6.3 a worked example#

1. Here is a minimal but real pair of Kubernetes objects.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: festival-web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: festival-web
  template:
    metadata:
      labels:
        app: festival-web
    spec:
      containers:
      - name: web
        image: registry.example.com/festival-site:1.4
        ports:
        - containerPort: 8000
        resources:
          requests:
            cpu: "250m"
            memory: "256Mi"
          limits:
            memory: "512Mi"
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8000
---
apiVersion: v1
kind: Service
metadata:
  name: festival-web
spec:
  selector:
    app: festival-web
  ports:
  - port: 80
    targetPort: 8000

2. kind: Deployment says this is a managed set of identical copies.
3. replicas: 3 is the desired state. This one number is the whole contract.
4. selector.matchLabels says which pods this deployment considers its own. The labels in the template must match, or nothing works.
5. image: names the exact image, including a version tag. Using latest here is the classic mistake, because two nodes can then run different code.
6. requests is what the scheduler uses to decide where the pod fits. limits is what the kernel enforces at runtime.
7. Note there is a memory limit but no CPU limit. That is a deliberate and widely recommended pattern: CPU throttling hurts latency, while unlimited memory risks killing the node.
8. readinessProbe tells Kubernetes when this copy may receive traffic. Without it, traffic arrives before the program is ready and users see errors during every deployment.
9. The Service gives the three copies one stable internal name and one stable internal address, and spreads connections across whichever copies are ready.
10. Apply and observe:

kubectl apply -f festival.yaml
kubectl get deploy,rs,pods -l app=festival-web
kubectl describe pod festival-web-6c9f7d8b4-x2ktr
kubectl logs -f deploy/festival-web
kubectl rollout status deploy/festival-web
kubectl rollout undo deploy/festival-web

PLAIN43.6.4 what is really happening inside#

1. kubectl apply sends your document to one program, the API server, which validates it and writes it into a database.
2. Nothing has run yet. All that exists is a written wish.
3. The deployment controller notices a Deployment with no matching ReplicaSet and creates one.
4. The ReplicaSet controller notices it should have three pods and has zero, so it creates three pod records. Still nothing is running.
5. The scheduler notices three pods with no node assigned. For each, it filters nodes that could fit, scores the survivors, picks the best, and writes the node name into the pod record.
6. On that node, an agent called the kubelet is watching for pods assigned to it. It sees one, pulls the image, and asks the container runtime to start it.
7. The kubelet then reports status back: starting, running, ready, or failed.
8. The endpoints controller notices a ready pod matching a Service’s label selector, and adds its address to that Service’s endpoint list.
9. On every node, a component programs the local packet rules so that traffic to the Service address is redirected to one of the current endpoint addresses.
10. If a pod dies, the kubelet reports it, the ReplicaSet controller sees two where three should be, and creates one more. The loop closes.
11. Nobody issued a command. Every step was a controller observing a difference and reducing it.

TECHNICAL43.6.5 the engineer’s version#

1. Kubernetes descends from Borg, Google’s internal cluster manager, described publicly in the 2015 EuroSys paper Large-scale cluster management at Google with Borg, and from its successor experiment Omega.
2. The first Kubernetes commit was pushed to GitHub on 6 June 2014: 250 files and 47,501 lines of Go, shell and Markdown. Joe Beda, Brendan Burns and Craig McLuckie are the usually named originators, with Borg engineers including Brian Grant and Tim Hockin joining early.
3. Version 1.0 was released in July 2015, and Google donated the project to the newly formed Cloud Native Computing Foundation under the Linux Foundation at the same time. That donation is the reason competitors adopted it.
4. Core objects you must know:

5. A pod is the smallest schedulable unit. Containers in a pod share a network namespace, so they reach each other on localhost, and can share volumes.

6. Control plane components: the API server, the only component that talks to the datastore; the scheduler, which assigns pods to nodes; the controller manager, which runs the built-in control loops; and etcd, a distributed key-value store using the Raft consensus algorithm.

7. On each worker: the kubelet, which owns pod lifecycle on that node, a network proxy component, and a container runtime speaking the Container Runtime Interface, usually containerd or CRI-O.

8. Be honest about Secrets. By default they are base64-encoded, not encrypted, in etcd. Encryption at rest must be enabled explicitly, and anyone with read access to the namespace can read them.

9. Ingress is being superseded. The Gateway API reached general availability for its core resources in October 2023 and is the direction of travel; Ingress remains extremely common and is not going away soon.

10. Scale limits, from the official documentation: no more than 5,000 nodes, 150,000 pods and 300,000 containers in a single cluster, with at most 110 pods per node.

11. Managed offerings remove the control plane work but not the concepts: EKS on AWS from June 2018, AKS on Azure from June 2018, and GKE on Google Cloud from 2015, which was the first.

12. Now the honest section, which matters more than the object list.

13. Most teams do not need Kubernetes. This is not a fashionable opinion; it is what the cost structure says.

14. Kubernetes solves problems that appear at a certain size: many services, many teams, high change rate, bin-packing pressure across many machines, and a genuine need for self-healing across node failures.

15. Below that size it adds a permanent, non-trivial tax: cluster upgrades every few months, a networking plugin to understand, ingress controllers, certificate rotation, RBAC, admission controllers, storage classes, and a new class of failure modes nobody on the team has seen before.

16. A useful test with three questions. Do you run more than about ten distinct services? Do you have someone whose job includes the platform? Would a single machine restarting cause a real outage today?

17. If the answer to all three is no, a managed container service such as ECS, Cloud Run or App Service, or even two virtual machines behind a load balancer, will serve you better and cost less in time.

18. Where experts disagree: some argue the standard interface is worth the tax even for small teams, because hiring and tooling are common. Others argue that complexity you do not need is complexity that will page you at 3am. Both are stated in good faith and depend on team size and turnover.

WORDS43.6.6 remember these#

1. Orchestrator — the program that runs your containers for you — a cluster scheduler and lifecycle manager across many hosts.
2. Pod — the smallest thing Kubernetes runs — one or more containers sharing a network namespace and storage volumes, scheduled as a unit.
3. Deployment — the object that keeps N copies of your app alive — a controller managing ReplicaSets to provide declarative rolling updates.
4. Reconciliation — repeatedly closing the gap between wish and reality — a level-triggered control loop comparing desired and observed state.
5. etcd — the cluster’s memory — a strongly consistent distributed key-value store using Raft, the only stateful control plane component.
6. kubelet — the agent on each machine — the node-level daemon that starts pods assigned to it and reports their status.
7. Service — a stable name in front of moving copies — a virtual IP and DNS name load-balancing to the current ready endpoints.

43.7 The Bezos API mandate#

PLAIN43.7.1 in simple words#

1. Around 2002, Jeff Bezos sent an internal message to everyone building software at Amazon.
2. It said, in effect: every team must publish its work as a service that other teams call over the network, and no team may reach into another team’s data any other way.
3. That sounds like a boring technical rule. It was one of the most consequential management decisions in the history of the industry.
4. Before it, teams shared databases and linked directly to each other’s code. That is fast to write and impossible to change later.
5. After it, the only way to use another team’s work was to call its published interface, over the network, like an outsider.
6. And the last rule was the sharp one: build every interface as if a stranger outside the company would use it.
7. Nobody was selling anything to outsiders yet. But once every internal service was already built for outsiders, selling it later was a small step rather than a rewrite.
8. That is the link between the memo and AWS. AWS is what you get when a company spends four years turning itself into a set of public-quality services and then notices it can sell them.
9. Amazon never published the memo. What everyone quotes is one former employee’s recollection, written down in 2011.

PLAIN43.7.2 a picture in your head#

1. Think of a large hospital where every department shares one giant paper filing cabinet.
2. Radiology reaches into the pharmacy’s drawer to check a prescription. Pharmacy reaches into billing’s drawer. Everyone rummages everywhere.
3. It works, and it is fast, right up until one department reorganizes its drawer. Then four other departments break, and nobody knows who or why.
4. Now the director issues a rule: no department may open another department’s drawer. If you need something, you fill in that department’s request form and they answer it.
5. At first this is slower and everybody hates it. Forms take longer than rummaging.
6. But now pharmacy can completely redesign its drawer over a weekend, and nobody else notices, because the form did not change.
7. And one day the hospital realizes that since every department already answers formal requests, it can answer requests from other hospitals too, for a fee.

Where this comparison breaks: a paper form is slow but reliable. A network call can be slow, can fail halfway, can succeed while appearing to fail, and can arrive twice. The hospital analogy makes the mandate look like pure gain. The real version bought flexibility with a permanent bill in latency and in failure handling, and that bill never stops being paid.

PLAIN43.7.3 a worked example#

1. Here is the mandate as it is publicly reported. The source is a long post by Steve Yegge, an engineer who worked at Amazon for about six years and then at Google. He wrote it in October 2011 as an internal Google memo and posted it publicly by accident.
2. His introduction is “His Big Mandate went something along these lines”. The rules, quoted:
3. One. “All teams will henceforth expose their data and functionality through service interfaces.”
4. Two. “Teams must communicate with each other through these interfaces.”
5. Three. “There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.”
6. Four. “It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols – doesn’t matter.”
7. Five. “All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.”
8. Then the closing line: “Anyone who doesn’t do this will be fired.”
9. The honest version: in Yegge’s original post this is a list of seven items, not five plus a closing line. Item six is the firing line, and item seven is a one-line pleasantry, which he immediately identifies as his own joke. The five substantive rules plus the firing line is the accurate summary.
10. Yegge also wrote that item six “was quite real, so people went to work”. He was recalling, not quoting a document. Amazon has never released the memo, so the wording is a recollection and should always be described that way.
11. Note what rule four is not doing. It does not pick a technology. That is deliberate and correct: the mandate is about boundaries, not protocols.

PLAIN43.7.4 what is really happening inside#

1. Follow what the rule forces, step by step. This is the part that matters.
2. Once teams may only call each other over the network, they must first be able to find each other. So somebody has to build a service registry, and the registry is itself a service.
3. Once hundreds of services call each other, any team can accidentally flood another. Yegge’s words: “every single one of your peer teams suddenly becomes a potential DOS attacker”. So quotas and throttling become mandatory everywhere, not optional.
4. Once a request crosses twenty services, an alarm has to reach the right team. Yegge again: a ticket “might bounce through 20 service calls before the real owner is identified”, and at fifteen minutes per bounce that is hours. So you must build ownership metadata and automated pager routing.
5. Once you cannot see inside another team’s code, “monitoring and QA are the same thing”. A service that answers “I am fine” may have nothing working except the part that says “I am fine”. So health checks must exercise real behaviour, and at that point monitoring has become automated testing.
6. Once debugging spans teams, it “is basically impossible unless there is a universal standard way to run every service in a debuggable sandbox”. So you must build sandboxes and standardized local environments.
7. Look at that list again: discovery, quotas, paging, deep health checking, sandboxes. Every one of those is a component of a public cloud platform.
8. Amazon did not set out to build platform tooling. The mandate made it unavoidable, and four years later they had it.

TECHNICAL43.7.5 the engineer’s version#

1. Organizational consequence one: the two-pizza team. Bezos’s stated rule was that a team should be small enough to be fed by two pizzas, roughly six to ten people. The number is not the point; the bound on communication paths is.
2. Organizational consequence two: you build it, you run it. Werner Vogels, Amazon’s Chief Technology Officer, stated it in an ACM Queue interview with Jim Gray in 2006. The team that writes a service also operates it and carries its pager.
3. That closes a loop. If the author of the 3am page is also the person woken by it, quality arguments stop needing management.
4. Organizational consequence three: Conway’s law. Melvin Conway wrote in How Do Committees Invent?, published in Datamation in April 1968, that organizations produce designs which copy their own communication structures.
5. The mandate is Conway’s law used deliberately. Change the communication structure, and the architecture follows. This is sometimes called the inverse Conway manoeuvre.
6. Now the costs, stated plainly, because the reader is owed them.
7. Latency. An in-process function call is on the order of nanoseconds. A call to another service in the same datacentre is on the order of 0.5 to 2 milliseconds. That is a factor of roughly a million.

8. Failure semantics. A local call either returns or throws. A network call can also time out with the work completed, be retried and applied twice, or succeed with the response lost. Every service boundary therefore needs idempotency keys, retries with backoff, timeouts and circuit breakers.
9. Data integrity. You can no longer wrap two teams’ updates in one database transaction. You are pushed into eventual consistency and compensating actions.
10. Operational cost. Each service needs its own build, deployment, monitoring, on-call rota, dependency updates and security patching.
11. The most important honest point in this chapter: most companies that copied the microservice structure did not copy the mandate’s discipline.
12. The mandate had four properties that copies usually lack. It was absolute, with no exceptions. It was enforced from the top. It required interfaces to be externalizable, which forces documentation, versioning and stability. And it was accompanied by real investment in the platform tooling that the consequences demanded.
13. Split a system into thirty services without discovery, quotas, tracing, ownership routing and sandboxes, and you have not built a service-oriented architecture. You have built a distributed monolith, which has all the latency and failure modes of distribution and none of the independence.
14. Where experts disagree: some argue the mandate is a timeless engineering principle, others that it was a response to a specific scale and coupling crisis at Amazon in 2001 and 2002, and that applying it to a twelve-person company is cargo cult. The second view is the majority among practitioners today.

WORDS43.7.6 remember these#

1. API mandate — the 2002 rule that teams talk only through published interfaces — an organization-wide prohibition on any interprocess communication other than service calls.
2. Externalizable — built as if a stranger will use it — designed with the documentation, versioning and stability required for third-party exposure.
3. Two-pizza team — a team small enough to feed with two pizzas — an organizational unit sized to bound internal communication paths.
4. You build it, you run it — the authors carry the pager — development teams own production operation of their own services.
5. Conway’s law — you ship your org chart — systems mirror the communication structure of the organizations that design them, Conway 1968.
6. Distributed monolith — services that must all deploy together — a system with the costs of distribution and the coupling of a monolith.

43.8 How AWS actually started#

PLAIN43.8.1 in simple words#

1. By 2000 Amazon was a large online shop with a serious internal problem.
2. Every new project spent months on the same groundwork: getting machines, setting up databases, storage, networking and monitoring.
3. Teams were reinventing the same plumbing over and over, and waiting on each other to do it.
4. The API mandate had already forced them to expose everything as a service. The next thought was obvious: make the infrastructure itself a service.
5. In 2003, engineers wrote a proposal describing computing capacity offered as a service, so that any team could ask for a server through an interface.
6. In parallel, a senior manager wrote a proposal arguing that this could be a business, sold to the outside world.
7. Amazon approved both. Development of the compute service was set up in Cape Town, South Africa, of all places, because the engineer leading it wanted to move home.
8. The public services arrived in 2006: storage in March, and rentable computers in August.
9. It was not a side project selling leftovers. It was a deliberate new business with its own staffing plan.

PLAIN43.8.2 a picture in your head#

1. Imagine a large restaurant chain that keeps building its own kitchens.
2. Every new branch designs plumbing, gas, extraction and cold storage from scratch, badly, and slowly.
3. Head office finally standardizes: one kitchen design, ordered as a kit, ready in a week.
4. Branches stop thinking about plumbing and start thinking about food. Opening a branch goes from a year to a month.
5. Then somebody points out that other restaurant chains have the same problem, and that the kit could be sold.
6. The kit business becomes larger and far more profitable than the original restaurants.

Where this comparison breaks: a kitchen kit is a product you build once and ship. AWS had to be operated forever, at growing scale, for customers who could leave. Also, the myth this analogy tempts you into is that head office sold its spare kitchens. It did not. It designed and built a product from scratch, then sold it, which is a completely different thing.

PLAIN43.8.3 a worked example#

1. The dates, in order, from public records.
2. July 2002: Amazon launches Amazon.com Web Services, a free interface letting affiliates pull product data. This is where the name comes from. It is not cloud computing.
3. 2003: Chris Pinkham and Benjamin Black write a short internal paper proposing standardized, automated infrastructure delivered as a service, and noting that virtual servers could be sold.
4. 2003: Andy Jassy, who had joined Amazon in 1997 and become Bezos’s technical assistant in 2001, writes a separate proposal for the business, including a plan for the first 57 hires. It is approved by the senior team.
5. 2004: Pinkham moves to Cape Town and opens a development centre there to build the compute service.
6. 3 November 2004: Simple Queue Service launches in beta. Chronologically this is the first thing recognizably cloud.
7. 14 March 2006: Simple Storage Service, S3, launches generally available, at 15 US cents per gigabyte-month.
8. 25 August 2006: Elastic Compute Cloud, EC2, launches in limited beta. One instance size only, 1.7 GHz, 1.75 GB of memory, 160 GB of local disk, at 10 US cents per hour. It reached general availability in October 2008.
9. From 10 cents an hour in 2006 to 128.7 billion US dollars of revenue in 2025 is the whole arc.

PLAIN43.8.4 what is really happening inside#

1. Why Amazon and not IBM, Microsoft, Sun or a telephone company.
2. First, Amazon already had to run huge, spiky, low-margin infrastructure, and was already good at it out of necessity.
3. Second, the API mandate had already forced internal services to be built for external use. The hard organizational work was done before the product was conceived.
4. Third, retail is a low-margin business, so Amazon’s culture was to treat infrastructure cost as an enemy. A company used to 3 per cent margins prices very differently from one used to 60 per cent margins.
5. Fourth, nobody else wanted the customers. In 2006 the buyers were startups with no money. Established vendors sold to enterprises with purchase orders and salespeople, and a self-service credit-card product looked like a toy.
6. Fifth, Amazon was willing to be unprofitable for years and did not have to protect an existing licence business. Microsoft and others did.
7. Now correct the myth. The popular story is that AWS was built to rent out servers left idle after the Christmas shopping peak.
8. It is false. Werner Vogels, Amazon’s Chief Technology Officer, publicly rejected it in 2011, and Andy Jassy has repeated the correction many times.
9. The mechanics also do not work. Retail’s peak is Q4, exactly when external customers would need capacity most, so you would have to evict them at the worst moment. And AWS ran on different hardware, in different facilities, with different software, from the start.
10. The kernel of truth is smaller and duller: Amazon’s own experience of running infrastructure badly and expensively is what taught it what the product should be.

TECHNICAL43.8.5 the engineer’s version#

1. The service launch timeline, with dates as published:

2. Read the order. Storage, then compute, then block storage, then content delivery, then load balancing and monitoring, then networking, then managed databases. Each launch is a response to what the previous one made painful.
3. VPC arriving three years after EC2 is the most instructive gap. The original EC2 put every instance on a flat shared network with a public address, which enterprises would not accept.
4. Financially, AWS was reported separately for the first time in Amazon’s first quarter 2015 results, revealing a business with roughly 5 billion US dollars of annualized revenue that was already profitable. That disclosure repriced Amazon’s stock and forced competitors to respond.
5. Current figures, from company filings. Full year 2025 AWS net sales were 128,725 million US dollars, with segment operating income of 45,606 million, an operating margin near 35 per cent, against total Amazon net sales of 716,924 million.
6. In the quarter ending June 2026, AWS reported 42.2 billion US dollars, growth of about 37 per cent, and operating income of 16.6 billion, which Amazon’s chief executive described as the fastest growth in eighteen quarters.
7. Reference customer for scale: Netflix began moving to AWS in 2008 after a database corruption incident halted DVD shipping for three days, and completed the shutdown of its last own-datacentre systems in January 2016. It took seven years, and Netflix is a well-resourced engineering organization.
8. Where sources disagree. Some accounts credit the 2003 Pinkham and Black paper as the origin of AWS; others credit Jassy’s business proposal; Black himself has written that theirs was one of several parallel efforts. The honest summary is that both existed in 2003 and both mattered.

WORDS43.8.6 remember these#

1. AWS — Amazon’s cloud business — Amazon Web Services, launched publicly 2006, 128.7 billion US dollars of net sales in 2025.
2. EC2 — rentable computers by the hour — Elastic Compute Cloud, virtual machines billed per second with a 60 second minimum, beta August 2006.
3. S3 — a place to put files, reached over the web — Simple Storage Service, object storage launched 14 March 2006, the first generally available service.
4. SQS — a queue you do not have to run — Simple Queue Service, beta November 2004, chronologically the first AWS service.
5. Two-pizza origin — small teams owning services — the organizational precondition that let Amazon staff many independent services at once.
6. Spare capacity myth — the false story that AWS resold leftover retail servers — publicly denied by Amazon’s CTO in 2011 and contradicted by the timeline.

43.9 The AWS services you actually need to know#

PLAIN43.9.1 in simple words#

1. AWS advertises more than 200 services. You will use about eighteen.
2. Here is each one, in a sentence, as the problem it solves.
3. EC2: I need a computer. Rent a virtual machine by the second.
4. S3: I need somewhere to put files that never fills up. Store objects by name, reached over the web.
5. EBS: my rented computer needs a hard disk that survives a reboot. Attach a network-backed disk to one machine.
6. VPC: I need my machines on a private network of my own design, not on the open internet.
7. Route 53: I need names to turn into addresses, and I need to steer traffic. This is DNS, run for you.
8. RDS: I need a normal database, but I do not want to install, patch, back up or fail it over myself.
9. DynamoDB: I need a store that never slows down as it grows, and I can live with looking things up only by key.
10. Lambda: I need to run a small piece of code when something happens, and I do not want a server sitting there waiting.
11. IAM: I need to say exactly who may do what. This is the permission system, and it applies to every other service.
12. CloudFront: my users are far away and my files are large. Keep copies near them.
13. ELB and ALB: I have several machines and one address. Spread the traffic and skip the dead ones.
14. SQS: I need to hand work to something else without waiting for it, and without losing it if the other side is down.
15. SNS: I need to tell many interested parties that something happened.
16. ECS: I have containers and I want them run for me, simply.
17. EKS: I have containers and I want Kubernetes, without operating the control plane myself.
18. CloudWatch: I need to see what is happening. Logs, numbers and alarms.
19. CloudFormation: I want my whole setup written down as a file, so it can be rebuilt exactly.
20. Secrets Manager: I need to keep passwords somewhere that is not the source code.

PLAIN43.9.2 a picture in your head#

1. Think of building a shop in a market town.
2. EC2 is renting the shop unit. EBS is the storeroom attached to your unit. S3 is the town warehouse where anyone with a key can drop off boxes.
3. VPC is the private courtyard you rent, with your own gate and your own path layout. Route 53 is the street sign that tells people where your shop is.
4. ELB is the doorman who splits an arriving crowd between your three counters and stops sending people to the one that has closed.
5. RDS is a bookkeeper the market provides, who keeps your ledger, backs it up nightly and covers for himself when ill.
6. SQS is the in-tray by the back door. Deliveries go in whether you are ready or not, and you take them at your pace.
7. IAM is the master key system: who may open which door, when, from where.
8. CloudWatch is the set of meters and the alarm bell. CloudFormation is the architect’s drawing from which the whole shop could be rebuilt.

Where this comparison breaks: in a market town you rent one unit and that is the bill. Here, almost every one of these has three or four separate meters running at once, and the doorman charges by the crowd as well as by the hour.

PLAIN43.9.3 a worked example#

1. A three-tier web application, in AWS parts, in the order you would create them.
2. VPC with six subnets across two availability zones: two public, two private for applications, two private for the database.
3. An Application Load Balancer in the public subnets, holding the TLS certificate.
4. An Auto Scaling group of EC2 instances in the private application subnets, each with a gp3 EBS root volume.
5. RDS for PostgreSQL in the database subnets, with a standby in the second availability zone.
6. S3 for user uploads, with CloudFront in front of it for delivery.
7. Route 53 holding the hosted zone and an alias record pointing at the load balancer.
8. IAM roles attached to the instances so that no access key is ever written to disk.
9. Secrets Manager holding the database password, fetched at startup.
10. CloudWatch collecting logs and metrics, with alarms on error rate and database CPU.
11. CloudFormation or Terraform describing all of the above in one repository.

PLAIN43.9.4 what is really happening inside#

1. Every one of these is reached the same way: an HTTPS request to a region-specific endpoint, signed with a key derived from your credentials.
2. That signature scheme is called Signature Version 4. It hashes the request, the date, the region and the service name, so a signature stolen for one region and service is useless elsewhere.
3. The AWS console, the aws command line tool and every language SDK all do exactly this. There is no private channel.
4. Services are separated into control plane operations, which create and modify resources, and data plane operations, which use them.
5. That split matters during outages. A control plane failure stops you creating new things while existing things keep serving.
6. Almost every service integrates with two others by default: IAM decides whether the call is allowed, and CloudTrail records that it happened.

TECHNICAL43.9.5 the engineer’s version#

1. Equivalents across the three large providers. Names are current as of August 2026; providers rename services often, so check before quoting.

2. The mapping is approximate on purpose. DynamoDB and Cosmos DB have different consistency models. SQS is a queue with one consumer per message; Pub/Sub is a topic with fan-out, so it maps to both SQS and SNS.
3. Durability and availability commitments, as published in the service level agreements: S3 Standard is designed for 99.999999999 per cent durability, eleven nines, with a 99.9 per cent availability service level agreement.
4. Eleven nines means that for ten million objects you would statistically expect to lose one object every ten thousand years. It is a design target derived from redundancy, not a measured result.
5. EC2 offers a 99.99 per cent regional service level agreement when instances are spread across two or more availability zones, and 99.5 per cent for a single instance with certain storage configurations.
6. Storage duplication: S3 Standard stores objects redundantly across at least three availability zones. S3 One Zone-IA deliberately does not, which is why it is cheaper and why you must not use it for anything irreplaceable.
7. Service quotas are real and will surprise you: default limits such as 5 VPCs per region, 5 Elastic IP addresses per region, and per-instance-family vCPU limits are common causes of failed deployments. Check aws service-quotas list-service-quotas.
8. Useful commands: aws sts get-caller-identity answers “who am I”, which is the first thing to run when something is denied; aws configure list shows which credentials and region are in effect.

WORDS43.9.6 remember these#

1. EC2 — a rented computer — Elastic Compute Cloud instances, billed per second with a 60 second minimum, in families tuned for compute, memory or storage.
2. S3 — files stored by name on the web — object storage with eleven nines design durability and a flat key namespace per bucket.
3. EBS — a disk for one machine — network-attached block storage, gp3 by default, snapshottable, tied to one availability zone.
4. VPC — your own private network in the cloud — a logically isolated virtual network with your own address ranges, subnets and route tables.
5. IAM — the rules about who may do what — Identity and Access Management, evaluating identity and resource policies on every API call.
6. Lambda — run code without a server — a function-as-a-service runtime billed per millisecond of execution and per request.
7. Signature Version 4 — how a request proves it is really you — the AWS request signing scheme scoped to date, region and service.

43.10 Azure, Google Cloud and the rest#

PLAIN43.10.1 in simple words#

1. Microsoft announced its cloud, then called Windows Azure, in October 2008, and opened it for paying customers on 1 February 2010.
2. Its first version made a bet that turned out to be wrong. It offered a platform: you gave it your application, it ran it, and you never saw a machine.
3. That was elegant, but customers did not want elegant. They wanted to move the servers they already had, unchanged.
4. So Microsoft turned around and added plain virtual machines, and then Linux, and then everything else. It renamed itself Microsoft Azure in 2014 to signal that it was no longer a Windows product.
5. That late start cost Microsoft the developer-first market, which AWS held.
6. But Microsoft had something AWS did not: it was already inside almost every large company on earth, selling Windows, Office and, crucially, the identity system that decides who employees are.
7. So Azure won the enterprise from the inside. If your staff already log in with a Microsoft account, the cloud that understands that account has an enormous advantage.
8. Google Cloud started earlier in one narrow sense and later in every other. It is respected for containers, networking and data analysis, and is a clear third by revenue.
9. Below the big three there is a real second tier, and for many small projects it is cheaper and simpler.

PLAIN43.10.2 a picture in your head#

1. Imagine three shops selling building materials.
2. The first, AWS, opened years earlier, stocks absolutely everything, and its staff assume you are a professional builder. The catalogue is enormous and the labels are terse.
3. The second, Azure, is owned by the company that already supplied the doors, locks and keys for every office in town. It says: your existing keys work here, and we will send someone to your site.
4. The third, Google Cloud, is run by engineers who built the largest structures in the world for themselves, and now sell the tools. The tools are excellent. The shop assistants are fewer.
5. Around the corner are small shops selling four or five items at half the price, with a printed price list you can understand in a minute.

Where this comparison breaks: building materials are interchangeable between shops. Cloud services are not. Once your identity, your networking and your managed databases are with one provider, moving is a project measured in quarters, not an afternoon of carrying bricks.

PLAIN43.10.3 a worked example#

1. Azure’s naming conventions are worth learning, because they are consistent and they differ from AWS.
2. Everything lives in a resource group, which is a folder for related resources with a shared lifecycle. AWS has no exact equivalent; the closest habits are tags and stacks.
3. Above resource groups sit subscriptions, which are the billing boundary, and above those, management groups, which apply policy.
4. Identity is not per subscription. It comes from a tenant in Entra ID, which can span many subscriptions.
5. So the Azure hierarchy reads: tenant, management group, subscription, resource group, resource.
6. In AWS the hierarchy reads: organization, organizational unit, account, resource. There is no resource group layer, which is why AWS practitioners use separate accounts for separation far more aggressively.
7. Creating a machine, side by side:

# Azure
az group create -n festival-rg -l centralindia
az vm create -g festival-rg -n web1 \
  --image Ubuntu2404 --size Standard_B2s \
  --admin-username azureuser --generate-ssh-keys

# AWS
aws ec2 run-instances --image-id ami-0abcd1234 \
  --instance-type t3.medium --key-name mykey \
  --subnet-id subnet-0123456789abcdef0

PLAIN43.10.4 what is really happening inside#

1. The reason Entra ID is described as Microsoft’s real moat is worth understanding mechanically.
2. Most large organizations authenticate employees against Active Directory, a system Microsoft shipped in Windows 2000. It holds users, groups, machines and policies.
3. Entra ID, which was called Azure Active Directory until Microsoft renamed it on 11 July 2023, is the cloud version. It issues tokens for web applications using OAuth 2.0 and OpenID Connect.
4. Once employees sign in to Microsoft 365 with Entra ID, the same identity can authorize Azure resources, third-party software and internal applications.
5. That means the decision “which cloud” is partly already made by a decision taken years earlier about email and file sharing.
6. Switching cost is therefore not mainly about servers. It is about identity, group membership, conditional access rules and hundreds of application integrations.
7. Hybrid follows from the same fact. Azure Arc, Azure Stack and ExpressRoute exist so that machines in a company’s own building appear in the same management and identity plane as machines in Azure.

TECHNICAL43.10.5 the engineer’s version#

1. Azure timeline. Announced at the Professional Developers Conference on 27 October 2008 by Ray Ozzie, under the internal codename Red Dog, led by Dave Cutler, the architect of Windows NT.
2. Commercially available 1 February 2010, initially as a platform service with web roles and worker roles, Blob storage, SQL Azure and Service Bus.
3. Infrastructure services, meaning ordinary virtual machines, entered preview in June 2012 and reached general availability in April 2013. Renamed from Windows Azure to Microsoft Azure on 25 March 2014.
4. Position today: Azure passed 100 billion US dollars of annual revenue in Microsoft’s fiscal year 2026, growing 41 per cent from 75 billion in fiscal
   2025. Microsoft states over 70 Azure regions.
5. Google Cloud timeline. App Engine, a platform service, previewed 7 April
   2008. Compute Engine was announced at Google I/O on 28 June 2012 in limited preview, opened broadly in May 2013 and reached general availability in December 2013. Google Kubernetes Engine launched in 2015, the first managed Kubernetes service.
6. Google Cloud’s strengths are concentrated: Kubernetes, since it wrote it; networking, since it owns an unusually large private backbone; and data analytics, where BigQuery separates storage from compute and is genuinely ahead. Revenue was 17.664 billion US dollars in the December 2025 quarter, up 48 per cent.

7. The rest of the field, briefly and honestly. Oracle Cloud Infrastructure competes on database licensing terms and, since 2024, on large AI compute contracts. IBM Cloud is strongest in regulated industries and mainframe adjacency. Alibaba Cloud leads in China and much of South East Asia.
8. DigitalOcean, Hetzner and OVHcloud sell simple virtual servers at a fraction of hyperscaler list prices, with flat pricing and generous or unmetered traffic. For a small production workload the difference can be five to ten times.
9. Cloudflare is a different shape: a network-first provider whose Workers run code at edge locations and whose R2 object storage charges no egress fee, which is a direct attack on the industry’s most disliked charge.
10. The trade is real. The second tier gives you fewer managed services, fewer compliance certifications, smaller regional footprints and less depth of support. For a database-plus-web-server application, that often does not matter. For a regulated enterprise, it does.

WORDS43.10.6 remember these#

1. Azure — Microsoft’s cloud — announced October 2008, commercially available 1 February 2010, over 100 billion US dollars in fiscal 2026.
2. Entra ID — the Microsoft login your company already uses — the cloud identity service formerly named Azure Active Directory, renamed 11 July 2023.
3. Resource group — an Azure folder for related things — a management and lifecycle boundary containing resources within one subscription.
4. Hybrid cloud — some machines yours, some rented, managed together — a deployment model joining on-premises infrastructure with public cloud under one control and identity plane.
5. Hyperscaler — one of the very large providers — an operator running global fleets at a scale where custom hardware and custom networks are economic.
6. Egress-free storage — object storage with no charge to read data out — Cloudflare R2’s pricing model, contrasted with per-gigabyte egress charges.

43.11 The service models: IaaS, PaaS, SaaS and FaaS#

PLAIN43.11.1 in simple words#

1. The models differ in one thing only: where the line is drawn between what you look after and what the provider looks after.
2. IaaS, infrastructure as a service. You rent machines, disks and networks. You install and patch the operating system, the runtime and your code. You have the most control and the most work.
3. PaaS, platform as a service. You give the provider your code. It provides the operating system, the runtime, scaling and deployment. You lose some control and most of the work.
4. SaaS, software as a service. You do not write anything. You use finished software over the web and configure it. Gmail, Salesforce, Zoom.
5. FaaS, function as a service, also called serverless. You give the provider one function. It runs it only when something triggers it, and charges only for the milliseconds it runs.
6. These are not four products. They are four points on one line, and the line is “how much of the stack do I own”.
7. Nothing forces you to pick one. Real systems mix all four, usually without anybody deciding to.

PLAIN43.11.2 a picture in your head#

1. The standard analogy is pizza as a service, and it is genuinely useful.
2. Made at home: you buy flour, cheese, an oven, gas and a table. That is owning your own servers.
3. Take and bake: you buy a ready pizza and use your own oven, gas and table. That is infrastructure as a service.
4. Delivered: someone else makes and cooks it; you supply the table, the drinks and the dining room. That is platform as a service.
5. Dining out: you supply nothing but yourself and money. That is software as a service.

Where this comparison breaks, and it breaks in three specific places. First, it suggests you own less as you move along, when in fact you still own all the important things: your data, your access rules and your legal obligations. Second, pizza has one dimension of choice. Cloud stacks let you take the oven from one supplier and the dough from another, which the analogy cannot express. Third, and worst, the analogy implies dining out costs the most. In cloud pricing the fully managed option is often the cheapest at small scale and the most expensive at large scale, and the crossover is the whole argument.

PLAIN43.11.3 a worked example#

1. The same task, a small web application, at each level.
2. IaaS: launch an EC2 instance, install Ubuntu updates, install Python, install a web server, write a service unit file, configure log rotation, set up certificate renewal, and repeat all of it when you scale out.
3. PaaS: push your code to Azure App Service or Google Cloud Run. It builds a container, runs it, terminates TLS, scales copies with traffic, and rolls back if health checks fail.
4. FaaS: write one function that handles one request, register it against an HTTP trigger, and let the provider create and destroy execution environments as demand moves.
5. SaaS: realize you were building a form-and-spreadsheet tool and buy one.

PLAIN43.11.4 what is really happening inside#

1. The important consequence of the split is not convenience. It is who is accountable when something goes wrong.
2. This is formalized as the shared responsibility model, and every major provider publishes a version of it.
3. The provider is responsible for security of the cloud: the buildings, the hardware, the hypervisor, the physical network, and the managed service software itself.
4. The customer is responsible for security in the cloud: operating system patching where applicable, network rules, identity and permissions, data encryption choices, and application code.
5. The line moves with the model. On IaaS you patch the guest operating system. On PaaS the provider does. On SaaS almost everything technical is theirs.
6. One thing never moves. Access control and data classification are always yours, in every model, without exception.
7. This is why almost every publicized cloud breach is a customer-side misconfiguration rather than a provider failure: a storage bucket made public, a permission granted too widely, a key committed to a repository.

TECHNICAL43.11.5 the engineer’s version#

1. The definitions are standardized in NIST SP 800-145, September 2011, which defines IaaS, PaaS and SaaS. FaaS post-dates it and is generally treated as a specialization of PaaS.
2. Responsibility split, stated as the providers state it:

3. Encryption responsibilities are subtler than the table suggests. Providers supply encryption at rest by default in most storage services, but key custody is a choice: provider-managed keys, customer-managed keys in a key management service, or customer-supplied keys held outside entirely.
4. Compliance is inherited, not transferred. A provider’s ISO/IEC 27001 or SOC 2 report covers their controls. Your auditor still tests yours. The document that connects the two is the provider’s customer responsibility matrix.
5. Service level agreements are financial instruments, not promises of uptime. A 99.99 per cent monthly agreement permits about 4 minutes 23 seconds of downtime per month, and the remedy for breach is typically a service credit of 10 to 100 per cent of that service’s fee. It does not compensate your business loss.
6. Availability arithmetic that engineers get wrong: dependencies multiply. A system depending on four independent components each at 99.9 per cent has a theoretical availability of 0.999 to the fourth power, which is 99.6 per cent, or about three hours per month.

WORDS43.11.6 remember these#

1. IaaS — renting raw machines — infrastructure as a service, where the tenant manages the guest operating system and everything above it.
2. PaaS — hand over code, get a running service — platform as a service, provider-managed runtime, scaling and deployment.
3. SaaS — finished software you just use — software as a service, delivered multi-tenant over the network, configured not coded.
4. FaaS — one function, run on demand — function as a service, event-triggered, billed per invocation and per millisecond.
5. Shared responsibility — who is to blame for what — the provider secures the cloud, the customer secures what they put in it.
6. Service level agreement — the promised uptime and the refund if missed — a contractual availability target with service credits as the sole remedy.

43.12 Serverless, properly#

PLAIN43.12.1 in simple words#

1. Serverless does not mean there are no servers. There are servers. You do not see them, name them, patch them or pay for them when idle.
2. A better name would have been “someone else’s servers, allocated per request”. That name lost.
3. You upload a function: a piece of code with one entry point.
4. You attach a trigger: an HTTP request, a file appearing in storage, a message in a queue, a database change, a timer.
5. When the trigger fires, the provider finds or creates an execution environment, runs your function, and returns the result.
6. You are billed for the milliseconds it ran and the memory it was given. Between events the cost is zero.
7. If a thousand events arrive at once, a thousand copies run at once. You did not configure that.
8. The catch is that each run starts with no memory of any previous run. Nothing you leave in a variable can be relied on next time.
9. The other catch is that the very first run after a pause is slower, because the environment has to be created. That is the cold start.

PLAIN43.12.2 a picture in your head#

1. Think of a hotel meeting room you can book by the minute.
2. Ordinary servers are like renting an office all year: it is yours, your things are on the desk, and you pay in August when nobody is there.
3. Serverless is like walking to reception, being handed an empty room in seconds, using it for ninety seconds, and leaving.
4. The room is always empty when you arrive. Anything you left last time is gone.
5. Most of the time a cleaned room is already waiting and you walk straight in. That is a warm start.
6. Occasionally none is ready, and staff must prepare one while you wait. That is a cold start.
7. If two hundred of your colleagues arrive at once, reception opens two hundred rooms, and you get one bill for two hundred short bookings.

Where this comparison breaks: a hotel has a fixed number of rooms and will eventually say no. Serverless platforms feel unlimited but have concurrency quotas, and hitting one produces throttling errors rather than a polite refusal. Also, in a hotel you pay per room. Here you pay per room multiplied by its size, so a function given four times the memory costs four times as much per second, even when it does not use the memory.

PLAIN43.12.3 a worked example#

1. A real AWS Lambda function in Python, triggered when an image is uploaded to S3, which records the upload in DynamoDB.

import os, json, boto3, urllib.parse

ddb = boto3.resource("dynamodb")
table = ddb.Table(os.environ["TABLE_NAME"])

def handler(event, context):
    for record in event["Records"]:
        bucket = record["s3"]["bucket"]["name"]
        key = urllib.parse.unquote_plus(
            record["s3"]["object"]["key"])
        size = record["s3"]["object"]["size"]
        table.put_item(Item={
            "pk": f"{bucket}/{key}",
            "size": size,
            "eventTime": record["eventTime"],
        })
    return {"statusCode": 200,
            "body": json.dumps({"ok": len(event["Records"])})}

2. The two lines creating ddb and table are outside the handler on purpose. Code outside the handler runs once per execution environment, not once per event, so the connection is reused across warm invocations.
3. event describes what happened. context carries the request id and the remaining time in milliseconds.
4. unquote_plus is not decoration. S3 event keys are URL-encoded, and spaces arrive as plus signs. Forgetting this is one of the most common real bugs.
5. The function returns; the environment is kept warm for a while in case another event arrives; eventually it is destroyed.
6. Now the cost. Suppose 2 million invocations a month, 200 milliseconds each, at 512 MB.
7. Compute: 2,000,000 x 0.2 s x 0.5 GB = 200,000 GB-seconds. At 0.0000166667 US dollars per GB-second that is 3.33 US dollars.
8. Requests: 2,000,000 at 0.20 US dollars per million is 0.40 US dollars.
9. Total about 3.73 US dollars per month, before the free allowance of one million requests and 400,000 GB-seconds, which would cover most of it.
10. A single small EC2 instance running all month costs about 30 US dollars. For this workload serverless is roughly ten times cheaper.

PLAIN43.12.4 what is really happening inside#

1. A cold start has four phases. The platform first downloads your code package.
2. It then creates an isolated execution environment. On AWS this is a Firecracker micro virtual machine, which boots in roughly 125 milliseconds.
3. It then starts the language runtime: interpreter, virtual machine or process.
4. Finally it runs your initialization code, the part outside the handler, and only then calls your handler.
5. After the handler returns, the environment is frozen rather than destroyed. A later event thaws it, and only the handler runs. That is a warm start, typically a few milliseconds of overhead.
6. Environments are frozen for some minutes and then reclaimed. The exact time is not documented and has changed over the years.
7. This is why state is unreliable. A warm environment keeps its variables, so caching sometimes appears to work, which is more dangerous than if it never worked.
8. Concurrency is not threads. Each concurrent event gets its own environment, so your function does not need to be thread-safe, but it also cannot share memory with its siblings.

TECHNICAL43.12.5 the engineer’s version#

1. AWS Lambda launched 13 November 2014, the first mainstream function service. Google Cloud Functions followed in 2016, Azure Functions in 2016.
2. Current published limits for Lambda: memory from 128 MB to 10,240 MB in 1 MB steps; maximum execution time 15 minutes; deployment package 50 MB zipped direct upload, 250 MB unzipped, or up to 10 GB as a container image; request and response payload 6 MB synchronous, 256 KB asynchronous; /tmp from 512 MB to 10 GB.
3. CPU is not configured separately. It is allocated in proportion to memory, with roughly one full virtual CPU at about 1,769 MB. Raising memory to make a function faster is a legitimate and frequently profitable tactic.
4. Pricing, x86, at the time of writing: 0.20 US dollars per million requests and 0.0000166667 US dollars per GB-second, with an always-free tier of one million requests and 400,000 GB-seconds per month. Provisioned concurrency costs 0.0000041667 US dollars per GB-second on top.
5. Measured cold start durations at the median, from public benchmarking:

6. Cold start frequency depends entirely on traffic shape. At sustained high traffic it is well under 1 per cent of invocations. At a handful of requests per hour, 30 to 50 per cent of invocations are cold.
7. Two mitigations exist. Provisioned concurrency keeps environments warm at a price. SnapStart, introduced for Java in 2022 and later extended, takes a snapshot of the initialized environment and restores from it, cutting multi-second Java starts to a few hundred milliseconds.
8. The VPC penalty is historical. Before 2019, attaching a Lambda to a VPC added 10 to 15 seconds to a cold start because an elastic network interface was created per environment. AWS re-engineered this with shared interfaces, and the penalty is now typically under 50 milliseconds. Beware of old advice.
9. When serverless is right: spiky or unpredictable traffic; event processing glue between services; scheduled jobs; low steady volume where a server would sit idle; teams with no operations capacity.
10. When serverless is a trap: sustained high throughput, where per-request pricing exceeds a reserved instance by a wide margin; workloads needing more than 15 minutes; anything needing large in-memory state or a warm cache; latency-critical paths where a cold start is unacceptable; heavy database use, because a thousand concurrent functions open a thousand connections and exhaust the database, which is what RDS Proxy exists to fix.
11. Note also the debugging cost. Local reproduction is imperfect, stack traces cross service boundaries, and understanding a failure requires distributed tracing rather than a debugger.
12. Vendor lock-in is strongest here. The function body is portable. The triggers, permissions, event shapes and surrounding services are not.

WORDS43.12.6 remember these#

1. Serverless — you never see or pay for an idle machine — an execution model with per-invocation billing, automatic scaling and no capacity management.
2. Cold start — the slow first run — the latency of creating an execution environment, starting the runtime and running initialization code.
3. Warm start — reusing a paused environment — invocation of a thawed environment, adding only milliseconds of overhead.
4. Trigger — the thing that causes your code to run — an event source binding such as HTTP, object creation, queue message or schedule.
5. Concurrency limit — how many copies may run at once — an account or function quota beyond which invocations are throttled.
6. Provisioned concurrency — paying to keep some copies warm — pre-initialized environments billed continuously to eliminate cold starts.
7. Firecracker — the tiny virtual machine underneath — AWS’s open-source micro-VM monitor, booting in about 125 milliseconds, used by Lambda and Fargate.

43.13 Storage in the cloud#

PLAIN43.13.1 in simple words#

1. There are three kinds of cloud storage, and choosing the wrong one is a common and expensive mistake.
2. Block storage is a virtual hard disk. You attach it to one machine, and the machine puts a filesystem on it. It is fast and it belongs to one owner.
3. File storage is a shared drive. Many machines mount it at once and see the same folders. Slower, but shared.
4. Object storage is different in kind. There are no folders and no disk. You store a whole item under a name, and you get the whole item back by name, over the web.
5. Object storage cannot be partly edited. To change one byte you upload the whole object again.
6. In exchange it is cheap, effectively unlimited, and enormously durable.
7. Choose block for databases and operating system disks. Choose file when several machines genuinely need the same files. Choose object for uploads, backups, logs, images, video and anything a browser will download.
8. There is one more thing everyone learns painfully. Putting data in is usually free. Taking data out to the internet is not.

PLAIN43.13.2 a picture in your head#

1. Block storage is a blank notebook on your desk. You can rub out one word on page 40 and write another. Only you have it.
2. File storage is a shared filing cabinet in the office. Several people open drawers at once, and there are rules about who may write.
3. Object storage is a left-luggage counter. You hand over a sealed bag and get a ticket. Later you show the ticket and get the whole bag back.
4. You cannot ask the counter to change one shirt inside the bag. You take the bag, change the shirt, and hand back a new bag.
5. The counter is astonishingly cheap and never runs out of shelves.
6. But it charges you a small fee every time you carry a bag out of the building.

Where this comparison breaks: a left-luggage counter has one shelf per bag. Object storage keeps several copies of your bag in several buildings, which is why it survives a fire and a notebook does not. Also, the counter here is reachable by anyone in the world with the right ticket, which is a security property the analogy hides and which has caused a great many public data leaks.

PLAIN43.13.3 a worked example#

1. Real S3 list prices for the North Virginia region, August 2026, per gigabyte per month.

2. Deep Archive is about 23 times cheaper than Standard. That is why lifecycle rules exist.
3. Requests cost separately: 0.005 US dollars per 1,000 PUT requests and 0.0004 per 1,000 GET requests in Standard. In Glacier Deep Archive a PUT costs 0.05 per 1,000, ten times more, because the class is designed for few, large objects.
4. Store 10 terabytes of logs. In Standard: 10,240 GB x 0.023 = 235.52 US dollars per month. In Deep Archive: 10,240 x 0.00099 = 10.14 US dollars.
5. Now the trap. You want to download those 10 terabytes to your own datacentre.
6. Data transfer out to the internet is billed at 0.09 US dollars per gigabyte for the first 10 TB each month, after 100 GB free.
7. 10,240 GB x 0.09 = 921.60 US dollars, minus the free 100 GB, so about 912 US dollars.
8. Read that again. Storing the data for a month cost 10 dollars. Reading it once cost 912. The exit costs 90 times the rent.
9. That asymmetry is the single most important commercial fact about cloud storage, and it is not an accident.

PLAIN43.13.4 what is really happening inside#

1. When you PUT an object, the service splits it, computes checksums, and writes copies to storage nodes in at least three separate availability zones before it answers “done”.
2. That is why the write is slower than a local disk write and why the durability claim is so high.
3. Object names look like paths, but there are no directories. 2026/08/log.txt is one flat key that happens to contain slashes, and the console draws fake folders from them.
4. A lifecycle policy is a background process the service runs on your bucket daily. It looks at object age and moves or deletes objects by rule.
5. Transitions are not free: each object moved costs a lifecycle transition request. Moving ten million tiny objects to Glacier can cost more than leaving them where they were.
6. Egress is metered at the edge of the provider’s network. Traffic to the internet is charged; traffic to another service inside the same region is usually free; traffic between availability zones or regions has its own smaller rate.

TECHNICAL43.13.5 the engineer’s version#

1. Consistency. S3 offered eventual consistency for overwrites and deletes until 1 December 2020, when AWS made it strongly read-after-write consistent for all operations, at no extra cost and with no performance penalty. A great deal of published advice about “eventual consistency workarounds” predates that date and is now wrong.
2. Strong consistency here means a successful PUT is immediately visible to a subsequent GET or LIST from any client. It does not provide transactions across objects, and there is still no locking beyond conditional writes.
3. Performance figures: at least 3,500 PUT, COPY, POST or DELETE and 5,500 GET or HEAD requests per second per partitioned prefix, scaling by adding prefixes. Maximum object size 5 TB; maximum single PUT 5 GB; multipart upload required above that, with up to 10,000 parts.
4. Block storage on AWS is EBS: gp3 at 0.08 US dollars per GB-month with 3,000 IOPS and 125 MB/s included, extra IOPS at 0.005 each per month; io2 at 0.125 per GB-month plus 0.065 per provisioned IOPS. Snapshots cost 0.05 per GB-month standard, 0.0125 archived, and are incremental.
5. gp2, the older class, costs 0.10 per GB-month and ties performance to size at 3 IOPS per GB. gp3 is cheaper and decouples the two. Migrating gp2 volumes to gp3 is one of the easiest genuine savings in AWS.
6. File storage is EFS on AWS, Azure Files, or Filestore on Google Cloud. EFS speaks NFSv4.1 and costs several times more per gigabyte than EBS, so use it only when sharing is a real requirement.
7. Lifecycle policy, expressed in the JSON that the API actually takes:

{"Rules": [{
  "ID": "logs-tiering",
  "Filter": {"Prefix": "logs/"},
  "Status": "Enabled",
  "Transitions": [
    {"Days": 30, "StorageClass": "STANDARD_IA"},
    {"Days": 90, "StorageClass": "GLACIER"},
    {"Days": 365, "StorageClass": "DEEP_ARCHIVE"}],
  "Expiration": {"Days": 2555}
}]}

8. Minimum storage durations are billed even if you delete early: 30 days for Standard-IA and One Zone-IA, 90 days for Glacier Flexible, 180 days for Deep Archive. Standard-IA and One Zone-IA also bill a minimum object size of 128 KB, so tiering many small files can increase the bill.
9. Retrieval from Deep Archive takes up to 12 hours in standard mode and costs per gigabyte on top. It is an archive, not cold storage you can browse.
10. Egress pricing tiers for data out to the internet from US and EU regions: first 100 GB per month free across all services, then 0.09 US dollars per GB to 10 TB, 0.085 to 50 TB, 0.07 to 150 TB, and 0.05 above that. Cross- availability-zone traffic is 0.01 per GB in each direction, and NAT gateway processing adds 0.045 per GB on top of everything else.
11. Two changes have eroded the egress lock. From 5 March 2024 AWS offers free data transfer out for customers who are leaving AWS entirely, with 90 days to complete the move and a support request required; other major providers made similar announcements. And the European Union’s Data Act, Regulation 2023/2854, requires switching charges including egress fees to be fully withdrawn from 12 January 2027.
12. Cloudflare R2 and several second-tier providers charge no egress at all, which is a deliberate competitive attack on this pricing model.

WORDS43.13.6 remember these#

1. Object storage — whole items stored and fetched by name — a flat key-value store over HTTP, immutable per object, replicated across zones.
2. Block storage — a virtual hard disk for one machine — network-attached block device presented as raw sectors, formatted by the guest.
3. File storage — a shared network drive — a managed NFS or SMB filesystem mountable by many clients at once.
4. Storage class — how cheap and how slow — a tier trading price per gigabyte against retrieval time, minimum duration and request cost.
5. Lifecycle policy — automatic tiering and deletion by age — bucket rules that transition or expire objects on a daily background pass.
6. Egress — the charge for data leaving — per-gigabyte data transfer out to the internet, typically 0.09 US dollars per GB at the first tier.

43.14 Cloud networking#

PLAIN43.14.1 in simple words#

1. A VPC, virtual private cloud, is your own private network inside the provider’s network. You choose the address range and the layout.
2. Inside it you create subnets, which are slices of that address range, each pinned to one availability zone.
3. A subnet is called public if its route table sends unknown traffic to an internet gateway, and private if it does not. That is the only difference.
4. An internet gateway lets traffic in and out. A NAT gateway lets private machines reach out to the internet while nothing can reach in.
5. Security groups are firewalls attached to individual machines. They allow traffic; they cannot deny it, and they remember connections.
6. Network ACLs are firewalls attached to whole subnets. They can allow and deny, and they remember nothing.
7. Load balancers sit in front of your machines and spread traffic. Layer 4 forwards connections without reading them. Layer 7 reads the request and can route by hostname or path.
8. Private endpoints let you reach a provider service without your traffic going over the internet at all.
9. Peering joins two networks. A transit gateway joins many, in a hub pattern, instead of one link per pair.

PLAIN43.14.2 a picture in your head#

1. Think of a gated business park that you have rented and laid out.
2. The park boundary is the VPC. The address range is the numbering scheme for all the plots.
3. Subnets are the blocks of plots. Two blocks are in one part of the park and two in another, on separate power and water.
4. The main gate is the internet gateway.
5. A NAT gateway is a courier desk. Staff in the inner blocks can send parcels out through it and get replies, but nobody outside can address a parcel to them directly.
6. Security groups are the locks on each individual building’s doors.
7. Network ACLs are the barriers at the entrance to each block, checking every vehicle in both directions with no memory of who came in.
8. A load balancer is the receptionist at the front, directing arrivals to whichever office is open.

Where this comparison breaks: in a real park a delivery van drives itself. Here, every packet’s path is decided by a route table you wrote, and if no route matches, the packet is silently dropped. There is no lost-property office and no error message. Silence is the normal failure mode, which is why the first thing to check in any cloud networking problem is the route table.

PLAIN43.14.3 a worked example#

1. A standard three-tier VPC, spread over two availability zones.

 VPC 10.0.0.0/16   Region: ap-south-1
 +------------------------------------------------------+
 |   AZ ap-south-1a          |   AZ ap-south-1b         |
 |                           |                          |
 | PUBLIC 10.0.0.0/24        | PUBLIC 10.0.1.0/24       |
 |  [ALB node]  [NAT gw]     |  [ALB node]  [NAT gw]    |
 |      |            ^       |      |           ^       |
 |      v            |       |      v           |       |
 | APP 10.0.10.0/24          | APP 10.0.11.0/24         |
 |  [web ec2] [web ec2] -----+  [web ec2] [web ec2]     |
 |      |                    |      |                   |
 |      v                    |      v                   |
 | DATA 10.0.20.0/24         | DATA 10.0.21.0/24        |
 |  [RDS primary]            |  [RDS standby]           |
 +------------------------------------------------------+
        ^
        |  Internet gateway  <-> the internet

2. Address maths. 10.0.0.0/16 gives 65,536 addresses. Each /24 gives 256, of which AWS reserves 5, leaving 251 usable.
3. Route table for the public subnets: 10.0.0.0/16 local, 0.0.0.0/0 to the internet gateway.
4. Route table for the application subnets: 10.0.0.0/16 local, 0.0.0.0/0 to the NAT gateway in the same availability zone.
5. Route table for the data subnets: 10.0.0.0/16 local, and nothing else. The database cannot reach the internet at all, which is deliberate.
6. Security group chain: the load balancer group allows 443 from anywhere; the web group allows 8000 only from the load balancer’s group; the database group allows 5432 only from the web group.
7. Referencing a group rather than an address range is the important habit. It keeps working when instances are replaced.
8. One NAT gateway per availability zone, not one shared, because a shared NAT gateway makes the other zone depend on the first, defeating the whole design.

PLAIN43.14.4 what is really happening inside#

1. None of this is physical. There is no cable and no real router.
2. Every packet leaving an instance is intercepted by the host’s virtualization hardware, which looks up your VPC’s rules and rewrites the packet into an encapsulated form.
3. That encapsulated packet crosses the provider’s physical network as ordinary traffic, is decapsulated at the destination host, and delivered.
4. Your route tables and security groups are entries in a distributed database that is pushed to every host that needs them.
5. On AWS this mapping work happens on the Nitro card, not on the CPU, which is why a security group costs almost nothing in performance.
6. Because rules are enforced at the source host, a denied packet never reaches the network. There is no device to log it, which is why denied traffic is invisible unless you enable flow logs.

TECHNICAL43.14.5 the engineer’s version#

1. Address planning uses RFC 1918 private ranges: 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. AWS VPC CIDR blocks may be /16 to /28. Overlapping ranges between VPCs make peering impossible later, so allocate deliberately from day one.
2. Security groups versus network ACLs, the exam question and the real one:

3. Stateful means the return traffic of an allowed outbound connection is automatically permitted. Stateless means you must write both directions, including the ephemeral port range 1024 to 65535 for replies. This is the same distinction drawn in Chapter 34 for ordinary firewalls, and it is the most common source of network ACL mistakes.
4. Load balancers by layer. A network load balancer operates at layer 4, forwards TCP or UDP flows, preserves the client address, handles millions of connections per second and adds tens of microseconds. An application load balancer operates at layer 7, terminates TLS, understands HTTP, and can route by host header, path, method or header value, at the cost of about one millisecond and a more complex failure surface.
5. Private connectivity to services: gateway endpoints, which are route table entries for S3 and DynamoDB and are free; and interface endpoints, which are PrivateLink elastic network interfaces in your subnets, billed per hour and per gigabyte. Using a gateway endpoint for S3 removes both NAT gateway processing charges and the internet path.
6. VPC peering is one-to-one, non-transitive, and requires non-overlapping CIDR blocks. If A peers with B and B peers with C, A cannot reach C. That limitation is exactly why AWS Transit Gateway exists: a hub that all VPCs attach to, giving transitive routing, at a per-attachment hourly charge plus per-gigabyte processing.
7. Hybrid connectivity: site-to-site VPN over the internet using IPsec, typically 1.25 Gbps per tunnel and subject to internet variability; or Direct Connect, a dedicated physical circuit from 50 Mbps to 100 Gbps with predictable latency and lower egress rates. Azure calls the equivalent ExpressRoute; Google calls it Cloud Interconnect.
8. Observation: VPC Flow Logs record accepted and rejected flows to CloudWatch or S3. Reachability Analyzer traces a path through your configuration and tells you which rule blocked it, without sending a packet. aws ec2 describe-route-tables and aws ec2 describe-security-groups are the two commands most often needed at 3am.

WORDS43.14.6 remember these#

1. VPC — your own private network in the provider — a logically isolated virtual network with tenant-defined CIDR, subnets and routing.
2. Subnet — a slice of that network in one zone — a CIDR range bound to a single availability zone with one associated route table.
3. Internet gateway — the way in and out — a horizontally scaled VPC component providing a route to and from the public internet.
4. NAT gateway — outbound only, for private machines — a managed network address translation service, billed hourly and per gigabyte.
5. Security group — a stateful allow-list on one machine — instance-level rules permitting traffic, with automatic return-path allowance.
6. Network ACL — a stateless filter on a whole subnet — numbered allow and deny rules evaluated in order, requiring explicit return rules.
7. Transit gateway — one hub instead of many pairwise links — a regional router providing transitive connectivity between many VPCs and on-premises networks.

43.15 Identity and access#

PLAIN43.15.1 in simple words#

1. Every action in a cloud account is an API call, and every API call is checked against rules before it runs.
2. The system holding those rules is identity and access management, IAM.
3. There are two kinds of actor. A user is a person with a long-lived password or key. A role is a set of permissions that anything can borrow temporarily.
4. A policy is a document listing which actions are allowed on which resources, and under which conditions.
5. The single most important habit is least privilege: grant exactly what is needed and nothing more, then review it.
6. The second most important habit is to stop using long-lived keys.
7. A long-lived key is a password that never expires, is often copied into a config file, and frequently ends up in a code repository.
8. A role gives out temporary credentials that expire in an hour. If they leak, the window is small.
9. Machines should never hold permanent keys. A server gets a role attached to it, and the credentials appear automatically and rotate themselves.

PLAIN43.15.2 a picture in your head#

1. Think of a hospital again.
2. A user is a staff member with a permanent badge that opens certain doors.
3. A role is a coat hanging by the door of the operating theatre. Anyone authorized may put it on, and while wearing it they can do surgery.
4. The coat is returned. It cannot be taken home, and it stops working at the end of the shift.
5. A policy is the written list of which doors that coat opens.
6. Least privilege means the surgery coat does not also open the pharmacy safe, even though it would occasionally be convenient.
7. The commonest real-world failure is a coat that opens everything, made once during an emergency, and never taken out of circulation.

Where this comparison breaks: a coat is physical, so only one person wears it at a time. Temporary credentials can be copied and used from anywhere in the world until they expire. That is why conditions such as source address, multi-factor authentication and session duration matter as much as the permission list.

PLAIN43.15.3 a worked example#

1. A policy allowing an application to read one bucket and nothing else.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "ReadUploadsOnly",
    "Effect": "Allow",
    "Action": ["s3:GetObject", "s3:ListBucket"],
    "Resource": [
      "arn:aws:s3:::festival-uploads",
      "arn:aws:s3:::festival-uploads/*"],
    "Condition": {
      "Bool": {"aws:SecureTransport": "true"}
    }
  }]
}

2. Two resource lines are needed because ListBucket acts on the bucket and GetObject acts on objects inside it. Forgetting this is a daily support question.
3. The condition rejects any request that did not arrive over TLS.
4. Version is a fixed policy language date, not your version number. It must be 2012-10-17.
5. Attach this policy to a role, attach the role to the instance, and the application never sees a key.
6. Compare with the wildcard policy that people write when they are tired: "Action": "s3:*", "Resource": "*". That grants delete on every bucket in the account, including the backups.

PLAIN43.15.4 what is really happening inside#

1. Policy evaluation follows a fixed order, and knowing it saves hours.
2. Start from an implicit deny. Nothing is allowed by default.
3. Evaluate all applicable policies: identity policies on the caller, resource policies on the target, permission boundaries, service control policies from the organization, and session policies.
4. If any of them contains an explicit deny that matches, the answer is deny. Nothing can override an explicit deny.
5. Otherwise, if at least one allow matches and no boundary excludes it, the answer is allow.
6. Otherwise the answer is deny by default.
7. In one line: explicit deny beats allow, and allow beats the default deny.
8. Temporary credentials work through a token service. An entity calls “assume this role”, proving its own identity; the service checks the role’s trust policy; if satisfied it returns an access key, a secret key and a session token with an expiry.
9. On an EC2 instance a metadata service delivers those credentials to software on the machine and refreshes them before expiry. On Kubernetes, a service account token is exchanged for the same thing.

TECHNICAL43.15.5 the engineer’s version#

1. Policy types and their interaction: identity-based policies attach to users, groups and roles; resource-based policies attach to buckets, queues and keys and can grant cross-account access without a role; permission boundaries cap the maximum a principal can be granted; service control policies cap an entire account within an organization; session policies narrow one session.
2. A principal’s effective permission is the intersection of all caps and the union of all grants, minus every explicit deny.
3. Instance metadata: IMDSv1 was a plain unauthenticated HTTP GET to 169.254.169.254. IMDSv2 requires a PUT to obtain a session token first and sets a low IP hop limit, which defeats the classic server-side request forgery path. AWS began defaulting new instances to IMDSv2 in 2024. Enforce it explicitly on older infrastructure.
4. That specific weakness produced the best-known cloud breach. In March 2019 an attacker exploited a server-side request forgery flaw in a misconfigured web application firewall at Capital One, reached the metadata service, obtained the credentials of an over-permissive role, and used them to list and copy S3 buckets containing roughly 106 million customer records. Regulators fined the bank 80 million US dollars in 2020 and a class action settled at 190 million.
5. Note that no AWS component failed. Every step used a documented feature. The failure was a firewall misconfiguration plus a role with far more S3 access than the application needed.
6. The two dominant causes of cloud compromise remain over-permissive policies and leaked long-lived keys. Keys leak through public repositories, container images, CI logs, browser bundles and screenshots. Automated scanners find a newly committed AWS key in public GitHub within minutes.
7. Mitigations in order of value: use roles and short-lived credentials everywhere; federate human access through single sign-on so no human has an access key at all; enable multi-factor authentication on every account with console access; turn on GitHub push protection and secret scanning; set an organization-wide service control policy denying dangerous actions outright.
8. Right-sizing tools: IAM Access Analyzer generates a least-privilege policy from CloudTrail history; last-accessed data shows which services a principal has actually used; Access Advisor exposes the same for roles.
9. Naming, for cross-provider fluency. AWS uses roles and JSON policies. Azure uses Entra ID principals with role-based access control, where a role assignment binds a role definition to a scope. Google Cloud binds members to predefined or custom roles at a resource in the hierarchy, and its default Compute Engine service account is famously over-permissive and should be replaced.
10. Diagnostic commands: aws sts get-caller-identity to learn who you are; aws iam simulate-principal-policy to test a decision before deploying it; CloudTrail event history to find the exact denied call and its reason.

WORDS43.15.6 remember these#

1. IAM — the rules about who may do what — identity and access management, evaluated on every API call against identity and resource policies.
2. Role — a borrowable set of permissions — an identity with no long-term credentials, assumed to obtain time-limited session credentials.
3. Policy — the document listing allowed actions — a JSON statement set of effect, action, resource and condition.
4. Least privilege — grant only what is needed — the principle of minimizing granted permissions to those actually exercised.
5. Explicit deny — a refusal nothing can override — the highest-precedence outcome in policy evaluation, above any allow.
6. Instance metadata service — where a machine gets its own credentials — the link-local endpoint at 169.254.169.254, hardened as IMDSv2.
7. Service control policy — an account-wide ceiling — an organization policy limiting the maximum permissions available in member accounts.

43.16 What cloud costs and how bills explode#

PLAIN43.16.1 in simple words#

1. A cloud bill is not one price. It is dozens of meters running at once.
2. The usual dimensions are: time a machine existed, storage held per month, number of requests, gigabytes moved, and per-hour charges for things that feel free, such as addresses and gateways.
3. Three ways to pay for compute. On demand is the list price, no commitment. Reserved, or savings plans, means promising one or three years of spending for a discount of roughly 30 to 72 per cent. Spot means using spare capacity at up to 90 per cent off, on the condition it can be taken away with two minutes’ notice.
4. Bills explode for boring reasons, almost never exciting ones.
5. Something was left running. Somebody chose a machine ten times bigger than needed. Logs were kept forever. Data crossed a boundary that charges.
6. The free tier is genuinely useful and genuinely a trap: it hides which resources cost money until the day it expires.
7. The honest headline: at steady, predictable load, owning hardware is usually cheaper than renting it. Cloud wins on speed, elasticity and not needing a team, not on unit price.

PLAIN43.16.2 a picture in your head#

1. Think of a mobile phone bill from before unlimited plans.
2. The handset rental is the machine. The data pack is the storage. Calls are requests. Roaming is egress.
3. Everyone understood the handset charge. Everyone was shocked by roaming.
4. Roaming had three properties that made it dangerous: it was per unit, it was invisible while it happened, and it was triggered by crossing a boundary nobody was thinking about.
5. Cloud egress and cross-zone traffic are exactly that. The boundary is invisible in your code and expensive in your bill.
6. And there is one extra hazard the phone bill did not have: your program can generate charges by itself, all night, with nobody watching.

Where this comparison breaks: a phone bill has a ceiling because a human can only talk so much. A misconfigured job can generate a five-figure bill in hours, and there is no hard spending limit you can set on the major clouds. Budgets alert you; they do not stop you.

PLAIN43.16.3 a worked example#

1. A real monthly bill for a modest production application in North Virginia, using the list prices quoted earlier in this chapter. Two web servers across two zones, a managed database with a standby, a load balancer, storage and 500 GB of traffic to users.

2. Now read the bill rather than the total. The two web servers, the thing everybody thinks of as “the cost”, are 18 per cent of it.
3. The database costs more than the servers. The load balancer costs as much as the servers. The NAT gateway costs 42 dollars to forward packets.
4. Egress of half a terabyte, which is a small website, costs more than the object storage holding everything.
5. Savings available without changing the architecture: a one-year compute savings plan takes about 30 per cent off the EC2 and RDS lines, roughly 47 dollars. A gateway endpoint for S3 removes much of the NAT data charge. Log retention set to 14 days cuts CloudWatch.
6. For comparison, a dedicated server of similar capability from a low-cost European host costs about 40 to 70 US dollars per month including generous traffic, but with no managed database, no standby, no load balancer and no support.

PLAIN43.16.4 what is really happening inside#

1. Every service writes usage records continuously, usually per hour, sometimes per second, into a billing pipeline.
2. Those records land in a detailed usage report with one row per resource, per usage type, per hour. The monthly invoice is a summary of tens of millions of rows.
3. This is why bills are hard to read: the invoice groups by service, but costs are caused by resources and by teams.
4. Tags are the only mechanism that connects the two. An untagged resource is effectively anonymous spending.
5. Charges accrue whether or not anything useful happens. A stopped EC2 instance still bills its EBS volume, its snapshots and its allocated public address.
6. Since February 2024 AWS charges for every public IPv4 address, in use or not, at 0.005 US dollars per hour, about 3.60 per month each. Many organizations discovered hundreds of forgotten addresses.

TECHNICAL43.16.5 the engineer’s version#

1. Commitment mechanisms: Reserved Instances lock instance family and region; Savings Plans commit to a dollar per hour of spend and are more flexible. Discounts run roughly 30 to 40 per cent for one year with no upfront payment, and up to 72 per cent for three years paid fully in advance.
2. Spot instances draw on spare capacity at typically 70 to 90 per cent off, with a two-minute interruption notice. Correct uses are stateless web tiers behind a load balancer, batch processing, continuous integration runners and fault-tolerant analytics. Incorrect uses are databases and anything holding state a customer cares about.
3. Free tier changed materially on 15 July 2025. Accounts created from that date receive 100 US dollars of credit at sign-up and up to 100 more for completing activities, on a plan that ends after six months or when credits run out. The old twelve-month free tier applies only to accounts created before that date. Always-free allowances, such as one million Lambda requests per month, continue under both schemes.
4. The classic explosion patterns, in the order they actually occur: a development environment left running over a holiday; log retention set to never expire; a NAT gateway processing data that should have used an endpoint; cross-availability-zone chatter between services that could have been zone-aware; oversized instances chosen from a template; unattached EBS volumes and old snapshots; and a recursive trigger, where a function writes to the bucket that triggers it.
5. FinOps is the discipline that grew up around this. The FinOps Foundation was formed in 2019 and moved under the Linux Foundation in 2020. Its model is three iterative phases: inform, meaning visibility and allocation; optimize, meaning rightsizing and commitments; and operate, meaning continuous governance.
6. Practical controls, in order of return on effort: mandatory cost allocation tags enforced by policy; AWS Budgets with alerts at 50, 80 and 100 per cent; automatic shutdown schedules for non-production; Compute Optimizer or equivalent for rightsizing; S3 lifecycle rules and log retention limits; gateway endpoints for S3 and DynamoDB; and one named person who reviews the top ten line items every month.
7. Now the honest comparison, which the industry avoided for a decade.
8. A single dedicated server bought outright, amortized over five years, is typically two to five times cheaper per unit of compute than the equivalent on-demand cloud instance, before staff costs.
9. The best-documented case is 37signals, the company behind Basecamp. It spent 3,201,564 US dollars on cloud in 2022, moved seven applications off AWS during 2023 onto about 700,000 US dollars of Dell hardware, and reported savings of nearly 2 million US dollars in the first full year, with total projected savings above 10 million over five years. It then moved roughly 10 petabytes off S3 as well.
10. Dropbox is the other well-known case. It moved the bulk of its storage off AWS onto its own infrastructure, a project called Magic Pocket, and its 2018 filing reported 74.6 million US dollars of infrastructure savings across 2016 and 2017.
11. Both are the same shape: large, steady, predictable workloads, with strong in-house engineering, and a bill dominated by storage and bandwidth.
12. When cloud is genuinely the better economic choice: unpredictable or seasonal load; early-stage products where the requirement is unknown; small teams with nobody to run hardware; workloads needing global reach quickly; and anything where time to market dominates unit cost.
13. When owning is better: steady load at scale; heavy egress; large storage volumes; regulatory requirements that push to dedicated hardware anyway; and organizations that already employ infrastructure staff.
14. The reasonable middle position, and the one most large companies have reached by 2026, is a split: steady baseline on owned or long-committed capacity, burst and experimentation in the cloud.

WORDS43.16.6 remember these#

1. On-demand pricing — full price, no promises — per-second or per-hour billing with no commitment and no discount.
2. Reserved and savings plans — a discount for a promise — one or three year commitments giving roughly 30 to 72 per cent off.
3. Spot — very cheap, can be taken away — interruptible spare capacity at 70 to 90 per cent off with a two-minute notice.
4. Egress — the charge for data leaving — per-gigabyte data transfer out, the most common source of bill surprise.
5. Idle spend — paying for what nobody uses — unattached volumes, stopped instances still holding storage, unused addresses and forgotten environments.
6. FinOps — treating cloud spend as an engineering problem — the inform, optimize and operate practice defined by the FinOps Foundation from 2019.
7. Repatriation — moving workloads back off cloud — relocating steady, high volume workloads to owned hardware for cost reasons.

43.17 DevOps, infrastructure as code and the modern workflow#

PLAIN43.17.1 in simple words#

1. For decades, two groups fought. Developers were paid to ship changes. Operators were paid to keep things stable. Change is the enemy of stability, so they were structurally opposed.
2. DevOps is the idea of removing that opposition: one team, shared goals, shared responsibility for both shipping and running.
3. It is a way of organizing people. It is not a tool, not a job title and not a department, although the industry turned it into all three.
4. The technical half is infrastructure as code: instead of clicking in a web console, you write a file describing what should exist, and a tool makes reality match it.
5. That file goes into version control, gets reviewed, and can rebuild the entire environment from nothing.
6. Immutable infrastructure takes it further: never patch a running machine. Build a new one from a new image and throw the old one away.
7. GitOps takes it further still: the repository is the only source of truth, and an agent inside the cluster continuously pulls from it and corrects drift.
8. Observability is being able to answer questions about a running system that you did not think to ask in advance. It needs logs, metrics and traces together, because each answers a different question.
9. Site reliability engineering is the discipline of deciding, numerically, how reliable a service must be, and spending the remaining unreliability deliberately on shipping features.

PLAIN43.17.2 a picture in your head#

1. Think of two ways to furnish a flat.
2. Way one: over three years you buy things, move them, and drill holes. It looks fine. Now furnish an identical flat next door exactly the same way. You cannot. Nobody wrote anything down.
3. Way two: you keep a plan on one sheet. Every change is drawn on the plan first, then done. Furnishing the identical flat takes an afternoon.
4. Infrastructure as code is the second way.
5. Immutable infrastructure is the rule that you never repaint a wall. You build a new wall to the new plan and remove the old one.
6. GitOps adds a caretaker who compares the flat to the plan every few minutes and undoes anything that does not match, including your own quick fix.

Where this comparison breaks: a flat has no state. A database has state, and you cannot throw it away and build a new one. The immutable rule applies cleanly to servers that hold no data, and needs careful exceptions for anything that does. That distinction, stateless versus stateful, is the whole reason people speak of cattle and pets.

PLAIN43.17.3 a worked example#

1. The same load balancer and instance, written in Terraform.

resource "aws_security_group" "web" {
  name   = "festival-web"
  vpc_id = aws_vpc.main.id

  ingress {
    from_port       = 8000
    to_port         = 8000
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]
  }
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_instance" "web" {
  count                  = 2
  ami                    = var.app_ami_id
  instance_type          = "t3.medium"
  subnet_id              = aws_subnet.app[count.index].id
  vpc_security_group_ids = [aws_security_group.web.id]
  iam_instance_profile   = aws_iam_instance_profile.web.name

  tags = {
    Name = "festival-web-${count.index}"
    Env  = "prod"
    Cost = "festival"
  }
}

2. aws_security_group.alb.id is a reference, not a copied value. Terraform builds a dependency graph from those references and orders creation itself.
3. count = 2 creates two instances. Changing it to four is a one-character edit and a reviewable change.
4. var.app_ami_id points at a machine image built in advance. Changing that variable replaces both instances, which is immutable infrastructure in practice.
5. The tags are not decoration. They are how the monthly bill gets attributed.
6. The workflow:

terraform init
terraform plan -out=tf.plan
terraform apply tf.plan
terraform state list
terraform destroy

7. plan is the important command. It prints exactly what will be created, changed or destroyed, before anything happens. Reviewing a plan is the equivalent of reviewing a diff.

PLAIN43.17.4 what is really happening inside#

1. Terraform keeps a state file: a record of which real resources correspond to which blocks in your code.
2. On plan it does three reads: your configuration, the state file, and the actual cloud API. It then computes the difference.
3. That is why the state file matters so much. Lose it and Terraform no longer knows it owns anything, and will try to create everything again.
4. In any team the state lives remotely, in S3 or an equivalent, with locking so two engineers cannot apply at once.
5. GitOps inverts the direction of control. Instead of a pipeline pushing changes into the cluster, an agent inside the cluster pulls the repository and applies it continuously.
6. That means the cluster needs no inbound credentials from the pipeline, and any manual change made by hand is reverted within minutes.
7. Observability data flows separately. Applications emit structured logs, numeric metrics and trace spans; agents collect them; backends store and index them; dashboards and alerts read them.

TECHNICAL43.17.5 the engineer’s version#

1. Origins. The word DevOps comes from DevOpsDays, organized by Patrick Debois in Ghent, Belgium, in October 2009. The cultural model is often summarized as CALMS: culture, automation, lean, measurement, sharing.
2. Tooling lineage for infrastructure as code: CFEngine 1993, Puppet 2005, Chef 2009, SaltStack 2011, Ansible 2012. Those are configuration management, which converges the inside of an existing machine.
3. Provisioning tools create the machines and the surrounding cloud resources: AWS CloudFormation from February 2011, Terraform from HashiCorp in 2014, Azure Resource Manager templates and the friendlier Bicep, and Pulumi from 2018, which uses ordinary programming languages instead of a domain-specific one.
4. A licensing event worth knowing: HashiCorp changed Terraform’s licence from Mozilla Public License to the Business Source License in August 2023. The community forked it as OpenTofu, adopted by the Linux Foundation in September 2023, with version 1.0 in January 2024. If you are choosing today, this is a live consideration.

5. Immutable infrastructure practice: bake an image with Packer or an equivalent, version it, deploy by replacing instances, and forbid interactive login to production. The phrase “cattle, not pets” comes from a 2012 talk by Bill Baker of Microsoft, popularized by Randy Bias.
6. GitOps was named by Alexis Richardson of Weaveworks in 2017. The CNCF OpenGitOps project states four principles: the desired state is declarative; it is versioned and immutable; agents pull it automatically; and agents continuously reconcile actual to desired. Argo CD and Flux are the two main implementations.
7. Observability, the three signals, and why you need all three. Metrics are cheap numbers over time and tell you that something is wrong. Logs are detailed records of individual events and tell you what happened in one case. Traces follow a single request across service boundaries and tell you where the time went and which service failed.
8. Metrics alone cannot tell you which user. Logs alone cannot show a system-wide pattern or a cross-service path. Traces alone are usually sampled and cannot be the record of truth. The three are complementary, not alternatives.
9. OpenTelemetry, formed in 2019 from the merger of OpenTracing and OpenCensus under the CNCF, is now the vendor-neutral standard for emitting all three. The wire format is OTLP.
10. Site reliability engineering began at Google in 2003 under Ben Treynor Sloss and was published as the book Site Reliability Engineering in 2016. Its core vocabulary:
11. An SLI, service level indicator, is a measured number, such as the proportion of requests served successfully in under 300 milliseconds.
12. An SLO, service level objective, is your internal target for that indicator, such as 99.9 per cent over 28 days.
13. An SLA is the external contract, always weaker than the objective, with financial consequences.
14. An error budget is the allowed failure: 100 per cent minus the objective. At 99.9 per cent over 30 days it is about 43 minutes of failure.
15. The budget is the point. If it is unspent, you are being too cautious and should ship faster. If it is exhausted, feature work stops until reliability is restored. It converts an argument about risk into arithmetic.
16. Blameless postmortems, formalized at Etsy by John Allspaw and colleagues from around 2012, assume that people act reasonably given the information they had, and look for the systemic conditions that made the error possible. Naming an individual ends learning, because the next person hides the near miss.
17. On-call practice that is defensible: a rota no thinner than one week in six, documented runbooks for every alert, alerts that are actionable or deleted, compensation or time off for nights worked, and a rule that a page nobody acted on gets removed rather than ignored.

WORDS43.17.6 remember these#

1. DevOps — one team owning both shipping and running — a cultural model removing the structural conflict between change and stability, from 2009.
2. Infrastructure as code — the environment written down as a file — declarative definitions applied by a tool, versioned and reviewed like source code.
3. State file — the tool’s record of what it owns — Terraform’s mapping between configuration blocks and real resource identifiers, stored remotely with locking.
4. Immutable infrastructure — replace, never patch — deploying new pre-built images instead of modifying running machines.
5. GitOps — the repository is the truth and an agent enforces it — declarative state pulled and continuously reconciled from version control.
6. SLI, SLO, SLA — the measurement, the internal target, the external contract — the three levels of reliability commitment in site reliability engineering.
7. Error budget — the failure you are allowed to spend — 100 per cent minus the objective, used to arbitrate between shipping and stabilizing.
8. Observability — being able to ask new questions of a live system — the combination of metrics, logs and traces, standardized by OpenTelemetry.

43.18 Microservices versus monolith, decided honestly#

PLAIN43.18.1 in simple words#

1. A monolith is one program containing all your features, built and deployed as one unit.
2. Microservices means splitting those features into many small programs that talk over the network and deploy independently.
3. The genuine benefit is independence. Teams ship without coordinating, and one service can be rewritten or scaled without touching the rest.
4. The genuine cost is that every internal function call you replaced with a network call can now be slow, fail, arrive twice or vanish.
5. A method call cannot half-succeed. A network call can.
6. So the question is never “which is better”. It is “do I have enough teams for the independence to be worth the failure handling”.
7. For a team of five, the answer is almost always no.
8. The recommendation, which is now mainstream rather than contrarian: start with a modular monolith. One deployable unit, with strict internal boundaries and no reaching across them.
9. Then, if a module genuinely needs to scale differently, be written in another language, or be owned by another team, extract that one module.
10. You get the mandate’s discipline about boundaries without paying the network bill until you must.

PLAIN43.18.2 a picture in your head#

1. Picture a restaurant kitchen with one large team in one room.
2. Anyone can shout to anyone. Coordination is instant and free. The room does not scale past a certain number of cooks.
3. Now picture five separate kitchens in five buildings, each producing one part of the meal, connected by couriers.
4. Each kitchen can be rebuilt, restaffed or expanded without closing the others.
5. But every request is now a courier trip. Couriers get stuck in traffic. Sometimes a courier arrives and the dish has already been sent again.
6. A modular monolith is one building with five clearly separated stations, fixed hatches between them, and a rule that nobody walks into another station’s area.
7. If the dessert station later needs its own building, it already has a defined hatch to replace with a courier route.

Where this comparison breaks: kitchens are physically fixed, so the analogy suggests the split is permanent and expensive. In software the split is reversible in principle, and several well-known organizations have merged services back. It also understates the failure modes. A courier who never arrives is obvious. A network call that succeeded but whose reply was lost looks exactly like one that failed, and that ambiguity is the hardest problem in the whole subject.

PLAIN43.18.3 a worked example#

1. One user action, “place order”, in both designs.
2. Monolith: one process calls reserve_stock(), charge_card() and create_shipment() inside a single database transaction. If the card fails, the transaction rolls back and no stock was reserved. Total time about 40 milliseconds. The failure model is one exception.
3. Microservices: the order service calls the inventory service, then the payment service, then the shipping service, each over the network.
4. There is no shared transaction. If payment fails after stock was reserved, the reservation must be actively undone by a compensating action.
5. That pattern is called a saga: a sequence of local transactions where each step has a defined undo.
6. Now the hard case. The order service calls payment and the reply is lost. Was the card charged? Unknown.
7. If it retries blindly, it may charge twice. So payment must accept an idempotency key and treat a repeat of the same key as the same charge.
8. Timing: three network calls at roughly 2 milliseconds each, plus serialization at both ends, plus retries, gives something like 15 to 40 milliseconds instead of 40 microseconds of function calls.

PLAIN43.18.4 what is really happening inside#

1. Once a request crosses services, no single log file contains the story.
2. Distributed tracing fixes this. The first service creates a trace identifier and passes it to everything it calls, in an HTTP header.
3. Each service records a span: its own start time, duration, name, and the identifier of the span that caused it.
4. A backend collects the spans and reassembles them into one tree by trace identifier, showing exactly where the time went.
5. Without tracing, debugging a slow request across eight services means guessing and correlating timestamps by hand.
6. Eventual consistency means different services briefly hold different answers. The order exists but the shipment does not yet. Your user interface must be designed for that, not surprised by it.

TECHNICAL43.18.5 the engineer’s version#

1. The fallacies of distributed computing are the checklist. L. Peter Deutsch wrote seven at Sun Microsystems around 1994; James Gosling added the eighth. Each is something engineers assume without noticing.
2. The network is reliable. Latency is zero. Bandwidth is infinite. The network is secure. Topology does not change. There is one administrator. Transport cost is zero. The network is homogeneous.
3. Every one is false, and every microservice architecture that ignores one will fail in exactly the way that fallacy predicts.
4. Distributed tracing derives from Google’s Dapper paper, published in 2010. The modern interoperability standard is W3C Trace Context, a Recommendation since 2021, which defines the traceparent and tracestate HTTP headers.
5. Sagas come from a 1987 paper by Hector Garcia-Molina and Kenneth Salem at Princeton, originally about long-lived database transactions. Two coordination styles exist: choreography, where services react to each other’s events, and orchestration, where one coordinator drives the sequence and knows the undo steps. Choreography is looser and harder to debug; orchestration is clearer and centralizes the logic.
6. The consistency background is the CAP theorem: conjectured by Eric Brewer in 2000 and proved by Seth Gilbert and Nancy Lynch in 2002. Under a network partition you must choose between consistency and availability. Most business systems choose availability and reconcile afterwards.
7. Required infrastructure before splitting, and this list is the real cost: service discovery, centralized configuration, distributed tracing, correlated structured logging, per-service alerting and ownership routing, contract testing between services, an API gateway, retries with exponential backoff and jitter, circuit breakers, idempotency keys, and a deployment pipeline per service.
8. If your organization cannot build or buy that list, splitting will make reliability worse, not better. This is the Bezos mandate lesson restated: Amazon built all of it, and that is why the structure worked.
9. Published reversals are instructive. In March 2023 an Amazon Prime Video team wrote publicly that moving their audio and video monitoring service from a distributed serverless design to a single scaled process reduced infrastructure cost by over 90 per cent. Shopify and Stack Overflow are long-standing examples of very large systems run as well-structured monoliths.
10. Martin Fowler’s 2015 article MonolithFirst argues that almost every successful microservice system he had seen started as a monolith that was later split, and almost every system built as microservices from scratch ran into serious trouble. That remains the best short statement of the position.
11. Where experts disagree: proponents argue that organizational independence is worth the cost from a surprisingly small size, especially with many teams and high turnover. Critics argue the boundaries chosen before you understand the domain are almost always wrong, and wrong boundaries are far more expensive across a network than inside one process. The second view has gained ground since about 2022.
12. The defensible default for a small team: one deployable unit; modules with explicit interfaces; no module reading another module’s tables; a single database with clear schema ownership; and extraction only when a specific, named pressure demands it.

WORDS43.18.6 remember these#

1. Monolith — one program with everything in it — a single deployable unit containing all business capabilities.
2. Modular monolith — one deployable unit with strict internal walls — enforced module boundaries and no cross-module data access, without network calls.
3. Microservices — many small independently deployed programs — services owned by separate teams communicating only over the network.
4. Distributed monolith — the worst of both — services that must be deployed together, paying network costs without gaining independence.
5. Saga — a sequence of steps with defined undos — a long-running transaction composed of local transactions plus compensating actions, Garcia-Molina and Salem 1987.
6. Idempotency key — a token making a repeat safe — a client-supplied identifier letting a server recognize and collapse duplicate requests.
7. Distributed tracing — following one request across services — correlated spans under one trace identifier, propagated via W3C Trace Context headers.
8. Fallacies of distributed computing — the eight false assumptions — Deutsch’s seven from about 1994 plus Gosling’s eighth.

43.98 Common wrong ideas#

1. Wrong: the cloud is just someone else’s computer, so it is the same thing as renting a server. Right: the rented computer is the smallest part. What you are buying is self-service provisioning in seconds, elasticity, a metered bill, and roughly 200 managed services you did not have to build. A rented server with a two-week ticket queue fails the NIST definition on two of its five required characteristics.
2. Wrong: serverless means there are no servers. Right: there are many servers. You do not choose, name, patch or pay for them while idle. You still hit memory limits, execution timeouts, concurrency quotas and cold starts, all of which are properties of real machines.
3. Wrong: cloud is always cheaper. Right: cloud is cheaper for spiky, unpredictable or small workloads, and usually more expensive per unit for steady large ones. 37signals reported saving nearly 2 million US dollars a year by leaving, and Dropbox reported 74.6 million over two years. Both had large, steady, predictable workloads and their own engineers.
4. Wrong: multi-cloud avoids lock-in. Right: it usually multiplies it. You now depend on the intersection of features from two providers, need staff expert in both, pay egress between them, and lose the deep managed services that made the cloud worth using. Genuine portability comes from boundaries in your own code, not from a second account.
5. Wrong: a container is a lightweight virtual machine. Right: a container is a process on the host kernel, isolated by namespaces and limited by cgroups. There is no second operating system and no hardware boundary. That is why it starts in milliseconds and why a kernel flaw crosses it.
6. Wrong: deploying across availability zones protects me from an outage. Right: it protects you from one building. It does not protect you from a regional control plane failure, which is what actually happens most often. The AWS event of 20 October 2025 affected an entire region for more than fifteen hours while the individual zones kept their power.
7. Wrong: Kubernetes is the modern way to run applications, so we should use it. Right: Kubernetes solves problems that appear with many services, many teams and real bin-packing pressure. Below that, it is a permanent tax in upgrades, networking, certificates and new failure modes. Most teams are better served by a managed container service.
8. Wrong: the provider is responsible for security. Right: the provider secures the cloud; you secure what you put in it. Access control and data classification never transfer, in any service model. Nearly every publicized cloud breach has been a customer misconfiguration, not a provider failure.
9. Wrong: AWS was built to sell Amazon’s spare Christmas capacity. Right: AWS was a deliberate new business, proposed in writing in 2003 with its own staffing plan, built on separate hardware. Amazon’s Chief Technology Officer publicly denied the spare capacity story in 2011, and retail’s peak coincides with customers’ peak, so the arrangement could never have worked.
10. Wrong: microservices make systems more reliable. Right: they make failure partial rather than total, which is only an improvement if you have built tracing, retries, idempotency, circuit breakers and ownership routing. Split without those and you get a distributed monolith: all the network failure modes with none of the independence.

43.99 Chapter summary in 20 lines#

1. The cloud is other people’s computers in other people’s buildings, rented by the second and reached over the internet; the mystique is marketing.
2. What genuinely changed is self-service in seconds, elasticity, metered billing and somebody else owning the hardware; what did not change is that physical machines still fail.
3. NIST SP 800-145, from September 2011, gives the five-part test: self-service, network access, pooling, elasticity and measurement.
4. A datacentre is racks measured in 44.45 millimetre units, top-of-rack switches in a spine-leaf fabric, dual power through UPS and generators, and hot and cold aisle cooling; efficiency is PUE, globally stuck near 1.54 while hyperscalers reach 1.09 to 1.15.
5. A region is a place, an availability zone is an independent failure domain inside it, and an edge location is a cache; deploy across zones always, and treat multi-region as a large project rather than a checkbox.
6. India has AWS Mumbai from 2016 and Hyderabad from 2022, Azure’s three regions from 2015 plus India South Central from August 2026, and Google Mumbai from 2017 and Delhi NCR from 2021.
7. Virtualization made renting possible: a hypervisor slices one machine into many, made cheap by Intel VT-x in 2005 and AMD-V in 2006, and made nearly free by offload hardware such as AWS Nitro from 2017.
8. A container is a process with namespaces and cgroups on a shared kernel, not a small virtual machine; the idea runs back through LXC, Solaris Zones, FreeBSD jails and chroot in 1979, and Docker in 2013 supplied the packaging.
9. Kubernetes, first committed on 6 June 2014 and descended from Google’s Borg, works by declarative reconciliation: state the desired end and let control loops close the gap forever.
10. Most teams do not need Kubernetes, and saying so is not contrarian; it is what the cost of upgrades, networking and new failure modes implies.
11. Around 2002 Jeff Bezos mandated that all teams expose data and functionality only through service interfaces, with no back doors, and that every interface be designed to be externalizable, on pain of dismissal.
12. That mandate forced Amazon to build service discovery, quotas and throttling, pager routing, monitoring merged with quality assurance and standard sandboxes, which is precisely the toolkit of a cloud platform.
13. It also bought latency, partial failure and operational cost; most companies copied the structure without the discipline and got a distributed monolith.
14. AWS itself came from a 2003 paper by Chris Pinkham and Benjamin Black and a parallel business proposal by Andy Jassy; SQS launched in beta in November 2004, S3 on 14 March 2006 and EC2 on 25 August 2006 at ten cents an hour.
15. It was never built from spare retail capacity, and AWS reported 128.7 billion US dollars of net sales for 2025 against Azure’s 100 billion for fiscal 2026.
16. The service models differ only in where the line falls between what you manage and what the provider manages, and the security version of that line is the shared responsibility model, in which access control is always yours.
17. Serverless means per-invocation billing and no idle cost, at the price of cold starts of 200 milliseconds to two seconds, statelessness, a fifteen minute ceiling and heavy lock-in.
18. Storage splits into object, block and file; the decisive commercial fact is that storing ten terabytes costs about ten dollars a month in deep archive while reading it out once costs about nine hundred.
19. Cloud networking is a VPC of subnets and route tables, with stateful security groups on machines and stateless network ACLs on subnets, and identity is governed by policies where an explicit deny always wins.
20. Bills explode from idle resources, oversized instances, egress and cross-zone chatter rather than from compute; own the steady baseline, rent the burst, and start a small system as a modular monolith.