There's nothing wrong with this in principle, every type system must reject some valid programs. There's no such thing as a "100%" type system that will both accept all valid code and reject all non-valid code.
I'm not questioning the principle, I'm critiquing the implementation versus the competition. Swift doesn't do a good job, it throws up its hands too often.