I understand the attack but my point is that the next step after installing the popular package is probably to import/require it in my code, then run my unit tests, which will allow the evil package to run whatever code they intended to run.
Interestingly enough, this actually isn't always the case! For example, install scripts provide the unique ability to affect build machines, which often don't run the code, but do run install scripts. Build machines can still have sensitive keys though, and will often automatically start building in response to a PR being submitted. So in these cases, the attack targets a part of the system that is inaccessible to attacks that require importing or actually running code, and only requires submitting a pull request, not even having it accepted!
Another example is when npm is used for front-end code (which is a lot!). It's true here that the code will eventually be imported or required, but in the browser sandbox. Install scripts in this instance do meaningfully change the attack surface to not just be whatever cookies you have set up or whatever, but the entire contents of your filesystem.
Yes I answered a sibling comment on front end code, I'll ask here again: doesn't browser code have tests that are run with nodejs?
And my build / CI machines generally run the tests too (though it's possible to separate the test runners from the build runners, I would not bet it's a very common thing).
1. With regard to build / CI, I certainly don't know the percentages worldwide, but this example actually comes from my own experience as a company I used to work at I believe had this setup of separate build and CI machines (for a variety of reasons, including the resource requirements being quite different).
2. At least with our frontend tests, no, we don't run them "in" node-js. Well, to clarify, the "tests" are run in node-js, but the frontend code in question is run in Puppeteer. Hence those libraries do not usually have any chance of touching our filesystem (except for their install scripts). This is a less complicated answer for non-"isomorphic" companies -- for example if your backend is Ruby and you use puppeteer-ruby, then I think we can agree that your npm packages should never run on your machine outside of a browser.
Just to give you a third case, again from my own experience, install scripts are also a good vector for typo attacks. If you type `npm install lodsah` instead of `npm install lodash`, and you don't notice and hit control-C fast enough, that mere installation can be sufficient to compromise your system.
Just an extra thought: as soon as you are running anything from an npm "script" it's also lost, because any package is free to add anything in node_modules/.bin which then gets added to the PATH of those npm "script". So I guess you will want to fix that next :)
