Pretty much every Python + scientific package (and these days, why else would you buy a P9 cluster but for that kind of work?)

Some of the Linux distros (like Debian) already include that sort of stuff. For eg the Debian Science Team have packaged a lot of difference types of scientific software. The Debian Python modules team packages a lot of different Python modules too.

https://blends.debian.org/science/tasks/ https://wiki.debian.org/Teams/DebianScience

Edit: of course a lot of more modern science and ML stuff is notoriously difficult to package, with pre-generated files, vendored code, lock files and so on.

Yeah, I am aware of that, but Linux distros package very old versions, and we need to be able to support multiple for users. Realistically, academic researchers like to use 'bleeding edge' type things. Generally a big HPC cluster comes with RHEL too and a support contract, because you need the drivers for the hardware - things like Infiniband networking, etc. so switching distro isn't an option really.

Linux does seem to have some open source Infiniband drivers, but maybe not for the ones you own, or maybe the proprietary ones are faster.

https://wiki.gentoo.org/wiki/InfiniBand https://wiki.archlinux.org/title/InfiniBand https://pve.proxmox.com/wiki/Infiniband