Obviously the criminals get their firearms from somewhere else since they can't be purchased legally inside the city, but you'll never eliminate every possible source. Even if you put up a massive wall and have 100% success in eliminating smuggling (which is obviously impossible) these things just aren't that hard to manufacture locally using commonly available and benign tools and materials. Criminals will still have them, and regular citizens will have an even harder time defending themselves.