Rajesh Medampudi's avatar
Rajesh Medampudi
rajesh@medampudi.com
npub1hv28...kqdn
Platform Engineer · 17+ years building distributed systems. Observability platforms processing 6TB/day. Saved $120K/month on AWS at gaming scale. Self-hosting everything. Verify my keys: https://rajesh.medampudi.com/verify
Rajesh Medampudi's avatar
rajesh 1 month ago
showback before chargeback. the order matters more than people think. showback shows each team its slice of the bill with no money attached. chargeback moves the cost onto its budget. lead with chargeback and you get a turf war — teams litigate the allocation instead of fixing the waste. most over-spend isn't malice, it's invisibility. show a team its number and a real fraction self-corrects before anyone enforces anything. order of operations: inform, then showback, then chargeback for the stubborn remainder. start at the bottom. image
Rajesh Medampudi's avatar
rajesh 1 month ago
the whole "RIs or savings plans?" decision routes in two questions. is it redshift or opensearch? → reserved node / reserved instance. no savings plan exists. done. is it another managed database — rds, aurora, elasticache, dynamodb and the rest? → database savings plan, 1-year no upfront, for steady-state. otherwise it's ec2/fargate/lambda: compute SP for flexibility, ec2 instance SP for the deepest rate, a reserved instance only if you need a capacity guarantee in an AZ or a resale exit. everything else → compute SP, the safe default. image
Rajesh Medampudi's avatar
rajesh 1 month ago
before you fix anything, measure. fifteen minutes in cost explorer tells you whether this is a $30 problem or a $3,000 one. group by usage type. NatGateway-Hours is your fixed cost, number of gateways times hours — not much to chase there. NatGateway-Bytes is the one worth chasing. then turn on vpc flow logs and see where those bytes go. if a large share is heading to s3, ecr, or dynamodb, that's spend you can zero out with a route-table edit. don't optimise on a hunch. #aws #cloudcost image
Rajesh Medampudi's avatar
rajesh 1 month ago
free s3 win almost everyone leaves on the table. if your EC2/ECS/lambda in a VPC talk to s3, by default that traffic leaves through a NAT gateway — which bills an hourly charge plus per-GB data-processing on every byte. pulling terabytes from s3 that way is a tax you're paying for nothing. an s3 gateway VPC endpoint routes the same traffic privately for $0. no hourly, no per-GB, no transfer charge for same-region s3. one route-table edit. caveat: it only covers same-region VPC-to-s3 — on-prem, cross-region or transit-gateway paths need the paid interface endpoint. for the common case you should've created it on day one. image
Rajesh Medampudi's avatar
rajesh 1 month ago
the most expensive AWS mistake isn't paying on-demand. it's locking in a multi-year commitment on a fleet you're about to halve. the gate is simple. fleet rightsized? if no, resize first — don't commit. stable baseline? if yes, buy the compute savings plan, up to 66% off, sized to that baseline. a smaller baseline buys a cheaper, more accurate commitment. that's why the order matters. image
Rajesh Medampudi's avatar
rajesh 1 month ago
a metric that lives in a monthly finance review is not a metric anyone who moves it ever sees. the cost of a code path is a property of that code path, same as its latency. so put cost per request on the same grafana board as latency. same screen, same on-call engineer. an engineer who sees a new endpoint costs 4x per call fixes it in the PR, context hot, for the price of a code review. the same regression caught six weeks later in a finance reconciliation is a forensic investigation. same bug. timing decides the fix cost. image
Rajesh Medampudi's avatar
rajesh 1 month ago
worth noticing: aws now recommends savings plans over reserved instances for compute. the vendor selling you the RI is telling you to buy the other thing. the logic is clean. the convertible RI exists for flexibility — the compute SP does that better, automatically, no manual exchanges. the standard RI exists for the deepest rate — the ec2 instance SP matches the 72% and stays flexible inside the family. both reasons the RI existed for ec2 are now done better by a savings plan. that's why the recommendation flipped. image
Rajesh Medampudi's avatar
rajesh 1 month ago
cutting a nat bill, the order matters more than any single fix, because the cheap no-downside moves capture most of the money. s3 + dynamodb gateway endpoints — free, one route-table edit. start here. interface endpoints for ecr and logs — $0.01/GB, under a quarter of nat's rate. cheap, but do the per-service math. cross-AZ cleanup, and the honest question of whether the subnet needs nat at all. fck-nat — real savings, but no default failover and a patching burden. last resort. most teams never get past step two. #aws #infrastructure image
Rajesh Medampudi's avatar
rajesh 1 month ago
the order isn't decoration. each decision makes the next possible. cut the waste first — free, mostly attention not architecture. then own past your break-even (datadog past my line → built it ~8x cheaper on LGTM). then observe cheaply, because you can't decide the first two without numbers. then staff light. i run all four — k8s, self-hosted observability, the services on top — alone, from a modest house in hyderabad. the load is far lower than the staffing story implies. image
Rajesh Medampudi's avatar
rajesh 1 month ago
there's storage on your s3 bill you can't see in the console and you're paying for it right now. upload a big object in parts, the upload fails partway — dropped connection, crashed job, sdk that didn't clean up — and the parts that landed stay in the bucket. you get billed for them. they don't show in the object listing. they pile up for years. the fix is one lifecycle rule on every bucket: AbortIncompleteMultipartUpload, 7 days. applies to existing + future uploads, and per aws it doesn't trigger early-delete charges. no downside. set it today before you touch anything else — cheapest win in the whole thing. image
Rajesh Medampudi's avatar
rajesh 1 month ago
before you buy a single savings plan, sweep the free money. none of it needs a commitment and all of it is just sitting there. unattached EBS volumes still billing. old snapshots nobody deletes. idle public IPv4 — $0.005/hr each since feb 2024, attached or not. and untagged spend, because you can't cut what you can't see. clean the dead weight first. then size the commitment to a real baseline. image
Rajesh Medampudi's avatar
rajesh 1 month ago
total dollars on a bill lie, and here's exactly how. the signal arrives weeks after the decision that caused it. it lands in front of finance, who can read it but can't fix it. the engineer who can fix it never sees it. and total $ can't tell a bill that doubled on growth from one that doubled on waste. four failures, all structural. divide into a per-unit number and all four fix at once — timely, actionable, owned by the right person, and it tells growth from waste at a glance. image
Rajesh Medampudi's avatar
rajesh 1 month ago
on 2 december 2025 aws finally launched database savings plans — the gap that kept most database spend stuck on reserved instances for years. covers aurora, rds, dynamodb, elasticache, documentdb, neptune, keyspaces, timestream and dms. up to 35% on serverless, 20% on provisioned, and it follows the workload across engine, family, size and region. two catches worth saying out loud: it's 1-year no-upfront only at launch, and it does NOT cover redshift or opensearch — i've already seen that stated wrong. check the official service list. image
Rajesh Medampudi's avatar
rajesh 1 month ago
four signs your nat gateway is doing a job a free endpoint should be doing: s3 reads from private subnets, ecr image pulls on every deploy, a nat sitting in a different AZ than your workloads, and one nat carrying multiple terabytes a month. you don't have to guess. turn on vpc flow logs for the nat'd subnets and look at where the bytes go. a big share heading to s3, ecr, or dynamodb is the smell — and the cheapest fix you'll find this quarter. #aws #devops image
Rajesh Medampudi's avatar
rajesh 1 month ago
people hear "lean" and think "cheap." it's the opposite. cheap is cutting the thing that matters to save a small number — you win a line item and lose the business. lean is spending deliberately on what actually buys you something, so you have room to do the work that moves the business. the point of cutting a $120K/mo observability bill was never the $120K. it was no longer being held hostage by it. image
Rajesh Medampudi's avatar
rajesh 1 month ago
how a $120K/mo observability bill quietly happens: three systems billing at once. datadog the official tool. cloudwatch still alive underneath that nobody turned off. and engineers SSHing into boxes to grep logs during incidents because paid search was too slow to trust. none of them complete, all of them on the meter, spend climbing 15%/mo. the fix isn't clever — one stack you own: grafana LGTM backed by s3, on your own kubernetes, data stays yours. costs become compute + storage, which scale slow. no vendor between you and the 3am fix. one bill you can actually read. #infrastructure #cloudcost #selfhosting image
Rajesh Medampudi's avatar
rajesh 1 month ago
the trap that quietly reverses your s3 savings: standard-IA, one zone-IA and glacier instant all bill every object as if it were at least 128KB. so you take a bucket of 10KB thumbnails, move it to a "cheaper" class to save money, and now each object bills at 128KB. you're paying for ~12x the bytes you actually store. the per-GB label looked cheaper, the bill came back higher. small objects stay in standard. check your average object size before tiering anything down — under 128KB and IA is just wrong, no matter how cold the data is. image
Rajesh Medampudi's avatar
rajesh 1 month ago
give me read access to a $50k/mo AWS bill and i'll find the first 20-30% in a day. not clever — it's always the same four places, same order. data transfer → rightsize → commit → storage cleanup. the order is the whole thing. each step makes the next one cheaper. don't commit before you rightsize, don't rightsize a fleet still leaking free traffic through a paid NAT. run it bottom to top. image
Rajesh Medampudi's avatar
rajesh 1 month ago
the whole unlock is dividing. total infra cost over requests = cost per request. over tenants = cost per tenant. the finops foundation calls it unit economics. the arithmetic is trivial. the shift it forces is not. two buckets worth keeping separate: resource-efficiency (cost per GB, per vCPU, per token) tells you how the machinery is wasting. business metrics (cost per tenant, cost to serve) tell you whether it matters. once cost is per-unit, a rising bill stops being scary by default. image
Rajesh Medampudi's avatar
rajesh 2 months ago
savings plans are the default now, but not the answer to everything. there are exactly three corners where the reserved model is still the only lever you have. redshift uses reserved nodes, opensearch uses reserved instances — no savings plan covers either, even after the dec 2025 launch. a zonal RI reserves capacity in a specific AZ; a savings plan reserves none. and a standard RI can be sold on the marketplace; a savings plan can't be cancelled mid-term. everything outside those three → savings plan. image