Now try to add a submodule X (which is a bare repository) to your repository Y

Good job, now you can't add it nor remove it, without manually removing it in .git folder.