What did your internal discussion conclude for the question "Why did we not take a step back earlier and think, why are we doing it this way?"
Im genuinely curious because this is not singling out your team or org, this is a very common occurrence among modern engineering teams, and I've often found myself on the losing end of such arguments. So I am all ears to hear at least one such team telling what goes on in their mind when they make terrible architecture decisions and if they learned anything philosophical that would prevent a repeat.
Oh we had it coming for quite some time and knew we would need to rebuild it, we just didn’t have the capacity to do it unfortunately.
I was working on it on and off moving one endpoint at a time but it was very slow until we hired someone who was able to focus on it.
It didn’t feel good at all. We knew the product had massive flaws due to the latency but couldn’t address it quickly. Especially cause we he to build more workarounds as time went on. Workarounds we knew would be made redundant by the reimplementation.
I think we had that discussion if “wtf are we doing here” pretty early, but we didn’t act on it in the beginning, instead we tried different approaches to make it work within the serverless constraints cause that’s what we knew well.
I have had CTOs (two in my career) tell me we had to use our AWS credits since they were going to expire worthless. Both experiences were at vc-backed startups.
Not sure what the different takeaways would be though?