One of the interesting things when weights are open sourced is the community can often improve the results. See all the bugs fixed in Gemma for an example.
Doubtful, for purely information theoretic and memory capacity reasons. It may outperform on some synthetic metrics, but in practice, to a human, larger models just feel “smarter” because they have a lot more density in their long tail where metrics never go
