I've had this before and it can be "peering", which is much more obvious when you have a couple of different connections with different ISPs to test... i.e where one ISP will be much slower on the VPN, and only when using servers with a certain ISP on the other end due to peering arrangements between the two ISPs.

Mullvad tend to have a couple of options of ISPs across the servers in one location, and the couple of times I've come across peering I've always managed to find a combo that fully utilises my connection so far.