Npm has always had the ability to do npm link. This utility is meant for developers who are working on a module that is designed to be used in other apps or modules. Basically it allows you to symlink the module you're working on into the
node_modules folder for another app or module so that you can make changes to the module without having to keep reinstalling it over and over again into the parent app just to test the module. The process is a quick 2-step process.
$ cd path/to/yourModule
$ npm link
This creates a symlink from your module's directory to the npm's global
node_modules directory. If you were developing a CLI tool then this step is all you need to do and your CLI tool should now be usable in the terminal. However if your module is intended to be installed into another app then there's one more step.
$ cd path/to/yourTestApp $ npm link yourModule
This creates a symlink from the global symlink you created in step 1 to yourTestApp's
node_modules directory. In npm 2 and lower this worked almost flawlessly. You could continue working on your module and then just restart your test app and instantly test the changes you made to your module without having to install the module again. You could even run
npm install in your test app and npm would detect symlinks in your app's
node_modules and skip over them so as not to overwrite the symlink.
I've been following an issue on the npm bug tracker for a while now. Basically, when npm 3 released they made a somewhat major change to the way dependencies are installed. By default npm 3 and higher now flatten the dependency tree as much as possible. So if you have a dependency and that dependency has five other dependencies npm would hoist those five other dependencies up into your app's root
node_modules directory instead of installing them in your dependency's
node_modules directory. There are exceptions to this, such as when two dependencies depend on different versions of the same dependency, but for the most part the dependency tree gets flattened into the top-level
This flattening solved a whole lot of problems in other areas but it introduced one new problem. If you
npm link a dependency and then run
npm install in your test app, your test app will steal your linked module's dependencies and hoist them into its own
node_modules directory. This causes all sorts of headaches because since the linked dependency isn't actually inside the test app's
node_modules it has a hard time requiring its own dependencies because from the module's perspective there is no parent
node_modules directory. What used to be a really convenient way to test a module in tandem with an app became a really tedious and frustrating hassle.
The issue persisted through npm 4 but now we are on npm 5. After reading through the issue and all its comments I began to realize something had changed in npm 5 so I decided to experiment on my own.
First I recreated the original issue using firstname.lastname@example.org.
You can see in the clip above that running
npm install within module B does indeed steal module A's lodash dependency and hoists it into its own
Since people were still complaining in the thread about
npm link being broken in npm 5 I went ahead and tried to reproduce the error in npm 5.
At first the results I got were very bizarre to me. Running
npm link ../A worked fine; a symlink was created and A's dependencies seemed in tact. But then running
npm install ../A after that surprised me. It kept the symlink intact but it nuked A's
node_modules folder. I thought I had duplicated the original issue but then I looked in B's
node_modules folder and
lodash was nowhere to be found. At this point I was very confused, but I noticed something different in npm 5 that I hadn't seen in npm 4 or earlier.
I noticed that when we ran
npm install ../A it modified
package.json, but with something I hadn't seen before:
"A": "file:../A". See, in npm 4 and earlier if you ran
npm install ../A it wouldn't treat it any differently than it treated a package from the registry. The only difference would be that it would copy the module from the file system instead of the npm registry. I remember this because if you did
npm install followed by a file path it would add the fully qualified file path to
The fact that it now adds a relative file path to
package.json gave me a hunch.
Sure enough, running
npm install ../A by itself in npm 5 without running
npm link created a symlink to the relative path. It did not copy the files like npm 4 and earlier would do. That's when the light bulb clicked on for me. They've change the way they expect
npm link to be used, they just haven't told anyone. They haven't even updated the documentation. For the record, I do think the behavior of A's
node_modules folder getting deleted when the symlink is overridden by the new symlink created by
npm install ../A is a bug.
After a few more experiments I realized that they now intend
npm link to be used for temporary symlinks. If you need to debug a module used by your app real quick then you can go into your app and run
npm link ../path/to/module. It will overwrite the installed module with the symlink, leaving you free to start debugging. However, if you run
npm install after that then npm will read from
package.json and install the original package from the registry again.
npm install no longer detects if a module in
node_modules is a symlink and skips over it like it used to in npm 4 or earlier. There are only two possible conclusions from here. Either this is a bug, or it's intended behavior.
Normally I'd totally say that this new behavior of
npm link is a bug, but after discovering the changes to
npm install ../file/path I now think it's intended behavior. I think they meant for
npm link to become a quick way to create a temporary link to a module for debugging. If you want a permanent link to a module then you just run
npm install ../path/to/module and the link reference is stored in package.json. I believe the thinking is you would only have these links stored in there during active development and once its time to release you'd change them to actual registry references when your associated modules are published to npm.
I tried to convey these findings in the comments on that issue but I felt like people weren't completely understanding me and that we were talking past each other quite a bit. Some people did bring up some good criticisms that I agreed with though so I thought I'd finish this post up with a list of pros and cons for this new behavior.
Storing the permanent symlink references in
package.jsongives us the ability to completely delete our
node_modulesfolder and run a fresh
npm installand have those symlinks restored. In earlier versions of npm your symlinks created by
npm linkwouldn't get blown away by
npm install, but if you had to completely delete your
node_modulesfor some reason then you'd have to redo them all after install.
Similarly, storing relative path symlink references in
package.jsonprovides the ability for others to clone your project and associated modules for an instant development setup. This would be great for actively developing a module with a plugin architecture as the in-development plugins would automatically be symlinked into the main module's dependencies so long as the relative paths were maintained.
There is a risk of accidentally committing these references in
package.jsonand cause your build to fail. Previously, links created by
npm linkwould just live in your
node_modulesforever as long as you didn't delete them, but
node_modulesis in most projects'
.gitignorefiles which conveniently left those symlinks out of the repository so you don't taint other peoples clones with them. Now there is the risk of accidentally committing those references when you didn't intend to.
If you don't intend to commit these relative path symlink references but you want to maintain permanent local symlinks then you have to add a build step to remove them.
Personally I feel like the pros outweigh the cons with these changes. At first I was one of those complaining that the issue wasn't fixed, but thanks to Paul Sanchez pointing out this symlink creation on
npm install I now think I like the new behavior better. Though I do think they could, you know, document such changes so that people are aware of them, but I digress.
The biggest complaint I've seen from the few who understood what I was saying is that they don't want to override their package.json references just to have symlinks that don't disappear when you run
npm install. I don't disagree with that as it is kind of annoying, but there are definitely workarounds. You could write a build script to swap them out with the real references on commit or build.