I worked somewhere that had a lot of this sort of thing going on once. You cannot overestimate how hard it is to get anything done: politics and organisational dysfunction, not to mention that you probably don’t have access to half of what you need to in order to fix any given problem and are even more unlikely to be able to get it, mean there are just huge scads of problems that, on the face of them, look relatively straightforward to solve but which, in practice, are organisationally impossible to solve.

> you probably don’t have access to half of what you need to in order to fix any given problem

I’ve found this to be one of my largest day-to-day problems even in a relatively functional organisation. Particularly when it involves something I can’t run on my own machine, like an AWS service.

In a previous role I often found myself constructing elaborate hypotheses about what was going on inside systems I couldn’t see into. I’d then need to try to verify it with someone on another team, in another timezone, who had the access but not necessarily the development background. Which usually meant getting on a screen share and asking them to click various things I wasn’t allowed to. If I was wrong, back to the drawing board and start again.

Yep. I don't work with data engineering, but from hearing war stories from them, this could perfectly describe the last four companies I worked at. :/

> you probably don’t have access to half of what you need to in order to fix any given problem and are even more unlikely to be able to get it

I once sat down with a data engineer to try to fix a specific problem they had and that was 100% accurate. They were left to die by Ops and CISO.

A framing/question I like to ask use is: "Look for a root problem that ought to be fixed through a change in policy, politics, or incentives, and the wasteful use of time/money is how the company tries to avoid or defer facing it."

For example, Operations might demand that Engineering develops an increasingly-byzantine approvals process, to stop Sales from over-promising impossible or unprofitable projects.

I inherited a pipeline like this. It is as if everything is a global variable. You cannot "just fix" one thing in isolation, because some spooky action at a distance of which you were unaware relies upon this insane behavior. Each and every hack is the expected input somewhere else in the chain. You have to carefully inspect everything downstream of any kind of minor adjustment because your cleanup is quite likely to break something else.

Immensely frustrating and draining where you can have accomplished ~nothing in a full day of work to fix what should have been a five minute change.

I'm the author. You are exactly correct. Everything was so heavily interwoven that it was impossible to tell what would happen downstream without making an edit and then tracking the changes through dozens of steps through the architecture diagram.

`The purpose of the system is what it does.’

In this situation I’d make a second copy of a bad thing and then try to make the copy good, instead of changing anything in place. But yeah, I totally get the huge upstream fight it must be and I’m not trying to backseat drive… just marveling at how the organizational fuckups & constraints make it hard to fix obvious problems

You're right. I actually did propose this, but the other difficulty is getting stuff put onto the Jira board. My belief, yet to be verified, is that it was possible to deliver everything the executives promised AND perform the refactoring without any drop in service, but this would have required convincing management that a well-run team could 3x productivity. Eventually everyone got tired of talking to people that were just used to bad performance, so they couldn't envision smooth CI/CD and happy workers (as opposed to contented to get paid to hang around workers, which is what happy means in many cases).

Again and again it comes up in many scopes and contexts. Avoid global mutable state. But some people do not learn.

The typical problem is a culture where if you say

"I would like to make this systemically better by addressing our second order problems that are causing our very visible first order problems"

You will be told that we absolutely do not have time for that. The only actions you're allowed to take are fighting the fires closest to you, not turning off the pumps that spray the gasoline everywhere.

Typically this is only even possible because nothing that you're doing is actually used or scrutinized since if it was, someone would have immediately noticed that nothing works. Usually this is at places running on varying levels of investment dollar three-card-montey.

You're operating in the same company that let this happen in the first place.

And — not to take away from the piece – If noone noticed the logs have been garbage for two years, they can't be that important.

I wrote a piece last year on how they were running a $500K Snowflake bill due to typing one number in wrong, and how I noticed within my first few weeks there by literally eyeballing something and going "queries take milliseconds so you really have to justify everywhere a minute appears in the configuration".

The logs are stupid (sorta) but imagine how many other issues can exist if we uncover something like this every time we open something.

It all seems easy to fix until you end up in a place where management process (and thus politics) has become more important than outcomes.

The people with experience and knowledge get pushed aside in favor of someone who can talk management-speak and (often) looks the role. Suddenly meetings where Things Get Decided only include managers who don’t know what they’re doing.

The best is when they make a mess and then get tremendous accolades for half-fixing it.

I’ve even seen it happen within relatively small startups. It’s a sign of rotting culture, but sometimes you have a mortgage to pay and have to get comfortable with the situation until you can find a better gig.

The logs are unimportant until suddenly they're critical.

Irreducible complexity and load-bearing bugs. Kludge built upon kludge built upon kludge. An engineer with the go ahead and all the support required would still struggle because the knock on effects of any one change cascade out in unpredictable ways. Not to mention working in an active environment where, although the other engineers support the goals in theory, they still need to deliver business requirements for a fickle management that doesn't truly understand what lurks beneath the facade - they don't have time to do it the right way, and if they try they'll just break both the old system and the new.

Sadly it sounds like most AU corporate companies I've worked at.

Why are we so crummy? Is it because our talent finds it generally easy to migrate to the states for better opportunities, because of no language barrier and the ‘coalition of the willing’ visa (iirc)?

This must be one reason (not Australian, but have friends there), but more importantly it seems to me the "better opportunities" are much, much better. Australia seems to have worker rights that make most of the US look decent by comparison while wages remain far below the latter---and I don't mean just US hot spots: an Australian friend went from Sydney to Salt Lake City (!) to get a much better job with much higher compensation and better rights/environment. In Utah!

> Australia seems to have worker rights that make most of the US look decent by comparison

This is the first I've heard anyone say that. Can you elaborate? We've got more leave (separate holiday and sick leave), none of the "at-will" stuff, right to disconnect, less hours, etc.

US software engineer. I have 24 days of PTO, 15 company holidays, and 9 sick days. 10 weeks for parental leave (16 for moms). $240K salary, $400-600K in annual vesting equity. That’s private paper equity, but I’ve already been able to cash out $700K and buy a house with cash.

Fully remote. I can expense $120/month for phone and internet, and a few lunches each month, too. I can get a new laptop and/or monitor sent to me just by asking.

When I do visit the office, the trip is fully expensed. Free daily lunch. Coffee and drinks and snacks everywhere, free. Private desks in a semi-open office with couches scattered around. Lounges with hundreds of board games, nearly all of which have seen table time during work hours.

Primary projects are tracked in a knowledge-sharing system, but I can mostly work on what I want to. I’m encouraged to merge small fixes and refactors without any ticket-pushing at all. Yelling by managers or anyone would not be tolerated.

“At-will” is more FUD than reality in my experience. Most companies, when firing or laying someone off, give something like 2 weeks of severance for every year of service.

That compensation is better isn't in dispute. My question is about worker rights, how much of the stuff you posted is just your company vs US law? Are you protected better by law than someone here would be?

What is more important, the law, or the typical employee experience? If you're in a strong market where workers are in demand (evidenced somewhat by higher compensation), then workers will tend to be treated better for fear that they will leave for another opportunity (remember: "at will"). If workers have fewer opportunities, then how much can the law really help?

Are there specific protections that are lacking in the US that you would expect to result in worse employee outcomes?

I'm not sure what any of that has to do with my question. The person I replied to said that from what he knew worker rights were somewhat better in the US so I asked for examples. How an employer treats you is of course important, but can't really be considered worker rights if they can decide to change that depending on market conditions.

Again, I'm not arguing about who has it better. I am asking a very narrow question about worker rights differences between the two countries. If you want me to say that US software workers have it better, then sure. But that's not really what I'm asking.

I work at more typical software job for an AUS company operating in the US ... AUS workers make less money than the US office but they can rollover PTO indefinitely, right to log off, etc. I think the main reason people come to the US to work from aus is the cost of housing anywhere near the cities is exorbitant and the australian version of the american dream is unbelievably dead.

How do I get such a job as someone from Europe? Where do I search for such a job? Is it Bay Area only?

I wish I could suggest something for you. My path was moving to the Bay Area 13 years ago to work for a small startup, helping to grow it, then going remote after a few years. Startup is a B2B with an ethical technical founder, and it had a credible business model from day 1.