To build on the stochastic parrots bit -

Parrots hear parts of the sound forms we don’t.

If they riffed in the KHz we can’t hear, it would be novel, but it would not be stuff we didn’t train them on.