Huh? Why can’t you assign a port/s to the CPE? You can even implement a port knocking scheme if you’re worried about some service/s on your home network being wide open to the world.
The point is that outside traffic isn't even reaching anything you even have control over, because you don't have a public IP (i.e. the ISP won't set up port forwarding for you). Let's say you wanted to directly send a packet to my phone. There's no way we could make that happen even with both of our cooperation because my phone doesn't get a publicly addressable IP.