Counter-take: I never understood the appeal of virtualizing the hardware. Is that complexity really necessary?
Of course there's tradeoffs, but I think it's a specific perspective that says that Kubernetes is any more complex than virtualizing the hardware and scheduling multiple VMs across real hardware.
I personally am of the opinion that you only really need one of these. Containers don't really need VMs and vice versa to get all of the benefits of abstracting away the fact that your hardware might break. It's just a choice of which abstraction you prefer to operate at (of course, clouds will put your containers in VMs anyway because they need that abstraction). It sort of sounds like the parent prefers the VM abstraction, and you prefer the container abstraction.
You didn't respond with any benefits of k8s though. What's the true value-add? Surely, coding an entire infrastructure in YAML is not it because that's horrific for anyone who wrote actually working software (one that didn't need 20+ commits of "try again" for a single feature to start working anyway).
I find it hard to believe that you've considered this for more than than two seconds and can't think of a single reason why k8s might be a good fit for someone's requirements. But here's one:
It's a curated, extensible API that provides a decent abstraction over a heterogeneous collection of hardware. Nobody's done that before, and it's extraordinarily useful for being able to define intent.
No-one forces you to use YAML, it's just a serialisation format.
Once you have a bunch of components that implement this API, it becomes trivial to deploy pretty much any level of complexity of containerised application, without having to care too much about the actual exact details of how scheduling, networking, storage etc. is implemented. Even better, I can hit two different clusters configured in two completely different ways with the same manifest and get roughly the same result.
> I find it hard to believe that you've considered this for more than than two seconds and can't think of a single reason why k8s might be a good fit for someone's requirements.
That could also tell you that you're in a bubble of converts and have forgotten what it's like to live without k8s. ;) There are always at least two sides of the coin.
> No-one forces you to use YAML, it's just a serialisation format.
Really? So when do I get to describe my singular app that needs Postgres and Kafka in 7-10 lines in a .txt file and not {checks our company's ArgoCD backend repo} ... not 8 YAML files? I ain't got all day or a week, where is that? Nice declarative MINIMAL syntax with all the BS inferred for me (like names and stuff, why should I think of 5-10 names of stuff? Figure it out, automated tool!) that only concentrates on what is being deployed. It can generate everything else just fine, or at least it should, in a more sane world anyway.
Excuse the slightly combative tone, I ain't trolling you here but you also come across as a bit blind about things outside the k8s holy land.
> Once you have a bunch of components that implement this API, it becomes trivial to deploy pretty much any level of complexity
Nope, nothing ever becomes trivial with k8s. I was on call with two senior platform engineers and we needed 3-4 hours to describe a single small app that needs Postgres and Kafka and to listen on a single port while needing a single env var and a few super small config files. These guys provisioned an entire network of 500+ pods working perfectly for years. They made 10+ mistakes while trying to help me deploy this small app on Argo. (And they've done the same dozens of times at this point.)
Do with that info what you will but I'd strongly disagree if your takeaway is "they are not that good" -- because they have done quite a lot very successfully (with and on k8s).
> It's the abstraction.
My 22+ years of programming have taught me that people enjoy abstractions too much and make huge messes with them. I am not convinced that having an "abstraction" at all is even a good selling point anymore.
You originally said "I don't see how it's useful", and I observed that some people find it useful.
I'm sorry you've had a bad experience, but it's a bit short-sighted to extrapolate from that to "this is universally useless".
> you also come across as a bit blind about things outside the k8s holy land.
You're not the only one with multiple decades of experience in writing code and managing infrastructure. I've used a lot of the tools in the toolbox, I know when each is likely to be more or less appropriate.
Ah, sorry if I sounded like saying it's universally useless. I know it helps people and more seriously speaking I'm aware of most of what it abstracts. But from where I'm standing it didn't do it well. Top much YAML, and I have to hand-hold it too much. It absolutely can do better at inferring or generating names / IDs, for example.
> Really? So when do I get to describe my singular app that needs Postgres and Kafka in 7-10 lines in a .txt file
Could you post those 7-10 lines needed to fully manage a Postgres AND Kafka deployment? I'm by no means a master, but I have a decent amount of experience outside the "k8s holy land" and I have no idea how to accomplish that.
We didn't need anything fancy at all, 99% can just be the defaults. That's what I meant. The other 1% is basically: user, password, port, topic name, replicas, partitions. Could have been two lines, namely two URLs.
No backups? Never do major version upgrades? Neither of these things are covered by the Postgres defaults. And even without them, I don't think I can get Ansible to install Postgres in < 10 lines. Yet all of these are covered in my ~30 line YAML running Postgres on Kubernetes. Plus of course recovering from pod and node failure, load balancing and more.
I think for most places, most of the time Kubernetes is overkill. Cloud in general isn't great bang for the buck. But speaking to your anecdote, having actually deployed these things in the past 3-4 hours to deploy a custom application, AND Postgres AND Kafka is a pretty compelling use case FOR Kubernetes. It would certainly take me a lot longer doing a proper job of it managing the system directly or using something like Ansible.
I as a dev should not waste days on this -- something I have no experience in. I can add the basic service / ingress / whatever and move on with life.
The infra / platform guys can add HA and backups later, right?
> I think for most places, most of the time Kubernetes is overkill.
Then we agree. I am not sure this app in particular was a good fit for k8s though; having it on a DO droplet + managed Postgres + managed Kafka would have taken me 1 hour, 40 minutes of which would be me cursing because I forgot to put a host name and port combo somewhere. :D
> I think for most places, most of the time Kubernetes is overkill.
True, but as you yourself said, backups and high availability are much harder on bare metal -- for devs anyway. I can do an amazing job at it because I do it at home for my stuff buuuuuuuut, it's going to take me days, maybe even two work weeks. Not a good time investment.
That's why I am resenting it when I have to do it: it's wasting my time with something that I never learn properly so I have to relearn it every time from scratch because I can't engrave it in my memory and that is because I never do it for long enough stretches of time to indeed remember it.
I already complained to my manager about it and he took it seriously, by the way. I am not raging at you or others about a deficient process in my employer's company (that I plan to help fixing, or at least mitigate somewhat). I am displeased with the fact how everyone just settled on a very, VERY low maxima -- and somehow nobody wants to rock the boat.
We can do better. We should do better. But alas, guys like myself have worked 22+ years on stuff they don't love -- and I already have burnout and hate working programming for money, which is another tragedy entirely -- and will NEVER EVER get the chance to change anything at all because I have no choice but work for the man... for now. Though at 40+ I am not sure I have the strength to manage to somehow command huge commissions for consulting or w/e else. Anyhow, off-topic.
Of course there's tradeoffs, but I think it's a specific perspective that says that Kubernetes is any more complex than virtualizing the hardware and scheduling multiple VMs across real hardware.