Cloud modernization in turbulent times
Today’s Best Band Ever™ is Ciśnienie.
Imagine Godspeed you, Black Emperor having a baby with 65daysofstatic while smuggling baritone saxophones in Poland.
Fun fact. The Polish word “ciśnienie” means “pressure”, but unlike the Latin origin of “pressing”, the Slavic word comes from “tightness” or “squeezing”. Pressure can be applied one-sided, making the subject move, while squeezing requires actions from opposite sides. This is an example of why I believe Large Language Models should have been better called Large Culture Models — they don't encode just the grammar, they also encode … a worldview, if you prefer.
Drippling
Difficulty: Hurt me plenty.
The world.
It's been barely a week, but the world is nothing but turbulent.
UN voted on switching from Mercator to a projection that represents continents more fairly.
UK announced sanctions on illegal Israeli settlements. That, after formally recognizing Palestinian state a year ago.
Germany's officially fascist party, wins 43.8% support in the eastern state of Saxony-Anhalt.
Swiss state is piloting a switch that “aims to reduce its dependency on Microsoft”, while Swiss military is already performing a complete replacement. Digital sovereignty was a hot topic several years ago. The argument seems to be settling on the division that governments and international organizations should seek digital independence. Individual companies, on the other hand, should treat this from the perspective of calculated risk.
Russians have reportedly lost 1'500'000 soldiers as dead and wounded. This is more than 1% of their overall population, but accounting that they need relatively healthy adult men for the military, the number is approximately 8% of the “prime-age” men. The same people who were likely productive members of society before the war and now need to be replaced. Russia is often romanticized as the cradle of modern humanism with Tolstoy, Dostoevsky, Chekhov etc. But despite all the great ideas, if it weren't this terrible, the last 400 years of their history would make for a fascinating anthropological and psychological study. The tsar empire put zero value on human life, the soviet system that was supposedly built on humanist ideas also gave zero value to humans (and even negative if they were not ethnically russian), and the modern fascist russia inherits exactly the same disregard for its people.
And of course, finally math has become a political topic after OpenAI solved a Millennium-prize math problem. And the main issue isn't even whether they cheated. It's whether the goal of math (and by extension art) is creating more proofs (pictures, songs) faster, or it's something that betters humanity thanks to the process of humans doing it and gives us better understanding of the world. Many things can be delegated, but for some it makes no sense to do so — a holiday or a family dinner is something that has to be experienced first hand.
Let's get back to the clouds.
In the previous episode, this blog was set up to run on a single virtual machine. Lacking (but good enough) performance, terrible reliability and a price tag of $18 per month. It all started on 17th of August and has been costing about $0.5 every day. Given I have a monthly budget alert of $5, it went off just after ten days. It's not much, but going from $4 (except the domain registrar spikes) to $18 a month is almost a five-fold increase. A five-fold increase for what I know is the most inefficient way of doing things in the cloud.
That's why several days ago I initiated the second phase — refactoring — an optimization without changing the architecture.
The plan is simple — improve reliability without increasing the cost. Ghost already can run as containers, all is left is to find a suitable (managed) container platform.
After all there are only 12 (if you include deprecated App Runner) ways of running containers in AWS. Here's the list in order of fitness for our needs (managed and cheaper on top)
Managed
- Lambda (container image)
- Lightsail Containers
- ECS Express Mode (using Fargate underneath)
- ECS on Fargate
- EKS on Fargate
- Batch with Fargate
EC2-based
- Elastic Beanstalk (Docker)
- ECS/EKS Anywhere
- ECS on EC2
- EKS on EC2
- EC2 + docker/podman direct
Conveniently, the bottom 5 explicitly use virtual machines, so easy to dismiss — there's no reason for me to manage the OS. From the top 6, only the top two don't use Fargate.
The "problem" with Fargate is that in order to integrate it into CloudFront, I would need a load balancer - a go-to approach for a "serious" setup, but the load balancer alone would cost $18/month which is an immediate blocker for me - I have to stay in the budget.
That leaves two options — Lambda or Lightsail. Lambda is an excellent starting point for all things cloud native (event driven, microservices, etc.). Ghost in its current state is a monolith. One could still package it as a lambda function, but it will be similar to attaching a modern engine to a 19th century post carriage. It's just not right, not a good fit for the refactoring.
So Lightsail it is. What an odd service. No persistent storage apart from custom S3. No integration with other standard services. Custom DB, limited logs and metrics.
But it's cheap. Right now this instance of Ghost is running on the second-lowest tier at $10/month. And that already includes a load balancer.
It's safe to assume that Lightsail is just ECS, ALB, Route53 and CloudFront. For an individual solution using all that, the lowest one could get is $25-30/month (ALB is not covered by any savings options). But since all of those aspects are delegated to AWS, and they run millions of Lightsail instances, they can apply massive optimizations - share load balancers, colocate multiple workloads on same virtual machines, use spot instances and so on.
All those possibilities are what allow AWS to cut the price by two thirds, and I'm sure they still make a profit.
For the database, Lightsail offers a custom version of RDS but, again, it's out of my budget, so we need a low-cost workaround. Lightsail doesn't offer shared persistent storage like EFS. At the beginning, I'm planning to run a single node, but it still should survive restarts, deployments and be potentially scalable.
Good news about Lightsail containers is that they are still workloads running in AWS, and that means they have IAM identities. Another lucky strike is that Ghost supports SQLite and, after some search, I discovered that some people already run multi-writer SQLite databases stored on S3. SQLite over S3 is not the most performant, but conveniently enough, high performance isn't required for this humble blog.
I didn't want to taint the project with a Python dependency, but we live in the times when open source software serves two functions — first is the explicit software function (you can run it), second is the contract (if it walks like a duck . . .). Redpanda is famously a Kafka-compatible product; POSIX is a set of standards that defines what is UNIX no matter who wrote it, what it's called and so on. In the times of agentic coding, creating tailored/-like custom drop-ins has become fairly trivial.
There's nothing wrong with running a general-purpose software, especially if one needs enterprise support (e.g. delegate some knowledge and responsibility) for it.
But now agentic coding has given us an option of creating a tailored custom version that will come with all advantages of being hyper-specific to one's needs as well as all disadvantages of being 100% their responsibility. It carries its risks, but being hyper-specific is a strong enabler for cost optimizations.
A traditional general-purpose software (like Kafka, Kubernetes, etc.) is like a tank - it can serve many needs, but none of them very efficiently.
A tailored/-like software is like a shape-shifting car that changes its characteristics depending on the needs and environment.
After some deliberation with Claude, the multi-writer SQLite “engine” is ready (it glitches once an hour, but this glitch only affects me, the author, and is merely an annoyance).
All in all, the new architecture substitutes an EC2+EBS with a Lightsail container, having the database on S3 (and offloading image serving to S3 as well).
Here's the cost breakdown of the refactored setup
| Item | Monthly estimate | Notes |
|---|---|---|
| Lightsail Container Service (Micro) | ~$10.00 | Bundled load balancing + HTTPS included, no separate LB charge |
| S3 (data bucket: SQLite store + images) | ~$1-2 | Storage + requests at ~2GB content; confirmed live content is 15MB/76 files, well under this |
| ECR (private image registry) | ~$0.10-0.20 | One image kept at a time; storage-only cost |
| CloudWatch (3 custom metrics + 3 alarms) | ~$1.20 | 3 metrics × $0.30/mo + 3 alarms × $0.10/mo |
| SNS (alarm email topic) | ~$0 | Well under the 1,000 free email notifications/month |
| CloudFront (existing distribution, extended) | (pre-existing, not incremental) | Same distribution already serving Phase 1; /blog* and image behaviors added, no new distribution |
| Route53 hosted zone | (pre-existing, not incremental) | Already existed before Phase 1 |
| Total (new, incremental) | ~$12-13/month |
To my pleasant surprise what started as “let's modernize without increasing costs” ended up being 30% cheaper. With an added bonus of running containers fully-managed (no patching) and structurally more resilient — stateless and restartable, with the database on a redundant and versioned storage.
What's also interesting, is that the pure compute cost actually went a little up — t3.micro is $7.60/month while Lightsail micro is $10/month but dismissing little extras like VPC, public IP, EBS (I personally consider VPC an architectural “smell”) have more than compensated the slight increase.
Very unlikely. My solution is perfectly adequate for a low-traffic personal blog. When running hundreds or thousands of instances like these, the economy of scale comes into play and Fargate and MySQL on RDS likely become financially (TCO-wise) justified.
There's no and never will be One True Blueprint of Cloud Architecture - every architecture should reflect specific needs of a specific case of a specific organization. Even for similar functional needs, two companies might end up with very different solutions based on factors like team composition or internal knowledge.
In the previous post I proposed following definition of cloud native:
A cloud native solution
- runs only parts that are needed at the moment (on demand)
- runs them only for as long as they are needed (just in time)
- runs in the most cost-efficient way (cost centric)
Where are we on the cloud-nativeness? For simplicity let's assume each point covers 33%.
Run only parts that are needed at the moment
Image serving is now S3 ⇾ CloudFront. Let's say it's 5%.
Run parts only for as long as they're needed
0% improvement. Lightsail is 100% up.
Run in the most cost-efficient way
31%. The Lightsail instance is actually underutilized (CPU at 20% max, memory at 30% max) so I can cut another $3/month. But for the current architecture it's good enough.
All in all, the current setup is 36% cloud native. The VM-setup was 0% — 0% — 10%, so it's a 26% increase. An improvement (30% cheaper with smaller management overhead), but still far from cloud native.
The phase 1 took about 4 hours to implement.
The second phase took approximately 16 hours. So 4x complexity for a significant but not radical improvement.
Stay tuned, stop aggressors and fight Nazis no matter what they pretend to be or the country they come from.
PS. My code is AI-assisted, but this blog is still stubbornly 100% GPT-free, including 100% hand-crafted em-dashes.
Comments ()