Rajesh Medampudi's avatar
Rajesh Medampudi
rajesh@medampudi.com
npub1hv28...kqdn
Platform Engineer · 17+ years building distributed systems. Observability platforms processing 6TB/day. Saved $120K/month on AWS at gaming scale. Self-hosting everything. Verify my keys: https://rajesh.medampudi.com/verify
Rajesh Medampudi's avatar
rajesh 2 weeks ago
vcluster earns its weight in exactly one place: tenants that need their own crds, operators, or cluster-admin. each gets its own api server as a pod on the host cluster, a syncer schedules their pods onto the shared nodes. if your tenants trust each other and just want their own apps — namespaces + the four native controls. don't run a control plane per tenant for isolation you didn't need. #kubernetes #selfhosting #infrastructure #devops image
Rajesh Medampudi's avatar
rajesh 2 weeks ago
networkpolicy without a cni that implements it does nothing. the api server accepts your default-deny, enforcement is zero, traffic flows anyway. the silent failure. test it: two pods, two namespaces, confirm the connection is actually refused. if it isn't refused, your policy is decorative. #kubernetes #security #selfhosting #devops image
Rajesh Medampudi's avatar
rajesh 2 weeks ago
i'll argue for self-hosting observability. but conviction cuts both ways, so: when NOT to. if you fit grafana cloud's free tier — 10k series, 50GB logs, 50GB traces, 14-day retention, 3 users, $0 — self-hosting buys you nothing. it's a hobby with a pager. use the free tier. also skip it if nobody will own the upgrades, or if your real problem is compliance retention not cost. the strongest argument names where it loses. #selfhosting #observability #grafanacloud #devops image
Rajesh Medampudi's avatar
rajesh 2 weeks ago
the k8s quota gotcha that bites everyone once: set a cpu/memory quota on a namespace and every new pod must declare requests or limits, or the api server rejects it at admission. tenants suddenly can't deploy and don't know why. fix: ship a limitrange in the same namespace so pods without requests get sane defaults. quota sets the ceiling, limitrange fills the floor. together or not at all. #kubernetes #infrastructure #devops #selfhosting image
Rajesh Medampudi's avatar
rajesh 2 weeks ago
a k8s namespace is not a security boundary. it gives you two things — naming isolation and policy scope. that's it. the isolation comes from what you wrap around it: namespace-scoped rbac (least privilege, no clusterrolebindings per tenant), resourcequota, default-deny networkpolicy. the namespace is the shelf, the controls are the lock. #kubernetes #security #selfhosting #devops image
Rajesh Medampudi's avatar
rajesh 2 weeks ago
most teams building multi-tenant k8s skip the only decision that matters: do the tenants trust each other? yes → namespaces + rbac + quota + default-deny netpol. native, cheap, enough. no → no amount of yaml fixes a shared api server and a shared kernel. vcluster or separate clusters. walk it in order, stop at the first "no". #kubernetes #selfhosting #infrastructure #devops image
Rajesh Medampudi's avatar
rajesh 2 weeks ago
ran the 3-year math on one personal box (4 vcpu / 16 gb): ~$195 owned (box + power), ~$972 on hetzner, ~$4,400 on aws. a $150 n100 pays its hardware back vs the cheapest serious cloud in under 6 months — under 6 weeks vs aws. after that it's ~$15/yr of electricity against $324+/yr of cloud you stopped paying. the box doesn't just pay for itself. it deletes a recurring bill you were treating as a law of nature. #selfhosting #homelab #infrastructure image
Rajesh Medampudi's avatar
rajesh 2 weeks ago
the used enterprise server is a trap for a personal lab. a dell r720 idles ~100w, an r730 ~85w. an n100 mini-pc: ~10w, running the same personal stack. that's ~10x the running cost, 24/7, for cores a personal workload never touches. cheap to buy, expensive to own — you're buying someone else's discarded power bill. lifetime cost is the number, not sticker price. #selfhosting #homelab #hardware image
Rajesh Medampudi's avatar
rajesh 3 weeks ago
the $5 vps anchor is why nobody runs the home lab math. a lab doesn't replace one tiny box — it replaces pi-hole + nextcloud + jellyfin + git remote + nostr relay + side projects, plus storage, plus the egress you stop paying for. price the real spec (4 vcpu / 16 gb) honestly: ~$27/mo cheapest (hetzner, after their apr 2026 hike), ~$122/mo on aws. every month. going up. against that, a $150 mini-pc breaks even in under 6 months. #selfhosting #homelab #cloudcost image
Rajesh Medampudi's avatar
rajesh 3 weeks ago
the part the scaling docs bury, and it's the whole thing: loki, tempo, mimir each run as a single binary — monolithic mode, every component in one process. not a hack, the supported default. grafana's own docs say monolithic handles ~20GB logs/day. then point all three at one S3 bucket. compute stays small, data lives in object storage at a couple cents/GB-month. do it day one — retrofitting storage later is the migration you don't want. #selfhosting #observability #s3 #grafana image
Rajesh Medampudi's avatar
rajesh 3 weeks ago
most observability advice is written for someone who isn't you. datadog's quote assumes someone else pays. the 40-service enterprise build assumes you have a platform team. small team needs neither. loki + grafana + tempo + mimir, simplest mode each, one alloy agent, one S3 bucket. one box. that's the whole thing. pick the build that matches the team you have, not the one in the tutorial. #selfhosting #observability #grafana #devops image
Rajesh Medampudi's avatar
rajesh 1 month ago
egress is the moat. ship 20 TB out of aws in a month → ~$1,800. the same 20 TB off a hetzner dedicated box → $0. you can't reserve your way out of egress the way you can with compute. it just scales with how many people use your product. if bytes-to-users is your biggest line item, that alone can justify leaving. #infrastructure #awscost #selfhosting #egress image
Rajesh Medampudi's avatar
rajesh 1 month ago
same class of machine: a hetzner box (16c / 128GB / NVMe) is €119/mo. the nearest aws instance, half the RAM, is ~$677. that's ~5x, on compute alone. the number is real — i run six of these boxes because of it. but "5x cheaper" is not "you should move." the part that actually decides it never shows up on a pricing page. #infrastructure #kubernetes #selfhosting #cloud image
Rajesh Medampudi's avatar
rajesh 1 month ago
intelligent-tiering is the right default when you don't know the access pattern — and the wrong one for billions of tiny objects. the difference is a fee most people never read. on unknown access it's excellent: monitors each object, drops it to IA after 30 days idle, archive instant after 90, no retrieval fee when it gets read again. turn it on, stop thinking about it. the catch is the monitoring charge is per object, not per GB. a bucket of a billion small objects pays a monitoring fee that dwarfs the savings — and objects under 128KB are never auto-tiered anyway, so you pay the overhead and get none of the benefit. unknown + decent size → intelligent-tiering. known or tiny → plain lifecycle, or just standard. image
Rajesh Medampudi's avatar
rajesh 1 month ago
storage is rarely the biggest line. it's the easiest free money on the bill. two moves, different in kind. tier it down — S3 lifecycle or intelligent-tiering ages cold data to cheaper classes. standard $0.023 vs deep archive ~$0.001 is 20x on the same bytes. that's recurring savings. then delete the graveyard: unattached volumes, old snapshots, idle IPv4. tiering recurs. the graveyard is pure waste — delete it and it never comes back. image
Rajesh Medampudi's avatar
rajesh 1 month ago
archive is cheap to store and never free to read. everyone watches the storage line; the retrieval and minimum-duration lines are what ambush you. colder class = higher per-GB read fee. glacier flexible and deep archive add a per-request fee on top, and deep archive's slowest tier is measured in hours, not seconds. put frequently-read data in glacier to save on storage and then pay retrieval on every read — that's the most expensive mistake in the whole subject. the retrieval bill can beat what standard would've cost outright. and minimum duration: delete a standard-IA object before 30 days you still pay 30, glacier bills 90, deep archive 180. before you tier anything down, ask the one question that governs all of it — how often is this actually read? image
Rajesh Medampudi's avatar
rajesh 1 month ago
compute optimizer points the direction. but the wins that actually move a bill are three moves, not a CPU spreadsheet. gp2 → gp3: same volume, ~20% cheaper, and faster. x86 → graviton: ARM, ~20% less for the same work. right requests: real usage, not guesses. i moved an EKS fleet (grafana LGTM, 6 TB/day) to ARM graviton2 under karpenter — 40% off compute, no perf loss. do this before you commit. image
Rajesh Medampudi's avatar
rajesh 1 month ago
two questions decide whether you write an s3 lifecycle policy or hand it to intelligent-tiering, and most people skip both. do you know the access pattern? logs cold after 30 days, backups after 90 — if you know, write the policy and s3 moves the data on a schedule. if you don't know — user uploads, a data lake, hot and cold mixed unpredictably — that's exactly what intelligent-tiering is for. second question: are the objects big enough and not billions of them? if they're tiny or short-lived, neither pays off, stay in standard. and if you do write a policy, set Expiration — storage you delete is storage you stop paying for forever, and it's the most underused line in s3. image
Rajesh Medampudi's avatar
rajesh 1 month ago
give me read access to a $50k/mo AWS bill and i'll show you where the first fifth is hiding. usually the same four places. i've run this enough that the pattern is boring. data transfer nobody instruments, boxes sized for a load test two years ago, on-demand on a baseline that never moves, storage rotting in the priciest class. the audit isn't a pitch before the work. the audit is the work. image
Rajesh Medampudi's avatar
rajesh 1 month ago
showback before chargeback. the order matters more than people think. showback shows each team its slice of the bill with no money attached. chargeback moves the cost onto its budget. lead with chargeback and you get a turf war — teams litigate the allocation instead of fixing the waste. most over-spend isn't malice, it's invisibility. show a team its number and a real fraction self-corrects before anyone enforces anything. order of operations: inform, then showback, then chargeback for the stubborn remainder. start at the bottom. image