The simplest solution is to stop requiring the top level package to be a module. Allow es modules to be require()-ed. It's be synchronous and slow and ideologically impure, but it'll make it so that millions of projects (mostly private!) will be able to start adopting es modules without a massive refactor. Anyone with a project of significant age or size looks at ES modules now and thinks "fuck it". There's no return on investment to convert (other than less pain while trying to upgrade or adopt new libraries). It's a big undertaking with loads of risk (modifying import order isn't safe!) and the payoff is "it's the shiny new thing".
I had been using adminjs at work. Their new major version was ESM-only, and it was easier to _write a new admin panel from scratch_ than it was to refactor our entire codebase to be ESM just so we could upgrade one library. I expect that's the situation at hundreds of thousands of other companies.
Like for all the belly aching that happened over async functions (and the whole function color rant), synchronous and asynchronous functions work together just fine through plain old promises. You can easily use async functions alongside functions that use old fashioned callbacks. ESM vs CJS is a file coloring (versus function coloring) problem, but there's no interoperability. There's no escape hatch when you just need one file to use another file but their colors are incompatible.
> Allow es modules to be require()-ed. It's be synchronous and slow and ideologically impure, but it'll make it so that millions of projects (mostly private!) will be able to start adopting es modules without a massive refactor.
This is already available in Node.js 22 [1] with ` --experimental-require-module` flag.
It's wild to think about: esm has taken nearly a decade to be feasible for folks to use, without heavy encumbrance; it was the key feature of es-2015/es6!
It took many more years for import-maps to become a thing, such that we could start using modules in a modular fashion; we've had to use bundlers/codemods to re-resplve dependencies, unless you've been going to go Deno's original path of just hitting absolute urls wherever they be. There's still no standing proposal for how we get import-maps on service-workers & shared workers; modules still are not a ubiquitous feature of the core web & JavaScript ecosystems. 9 years latter. https://github.com/WICG/import-maps/issues/2
It's been hard being a webdevs, when the future remains so partially implemented.
The article's exploration of the bomb ecosystem is telling. And is much of the reason I hope JavaScript Registry (jsr) project might succeed. Npm will never escape it's past. CJS is going to be in there for a long time. Being on a registry where everything is esm & typescript, that sounds divine. https://deno.com/blog/jsr-is-not-another-package-manager
Did anyone actually want ES modules, and are anyone using ES modules in production today? I don't think so except for transpiling languages that compile and bundle the JS. The only argument against lazy loading is that single threaded UI's will freeze for a few ms when the code loads, but if you compare that to the time it takes to fully load a modern website that point is moot.
There are a few hiccups, but I've found ESM to be a huge win overall. The killer feature for me is the ability to write code that can be imported in a browser without any sort of build step. You don't even need node/npm. Just download the file/directory from somewhere and import. It's perfectly feasible to build a complex browser app with many dependencies using nothing but git submodules.
The main thing you're missing out on is the optimizations that come with bundlers. But they should be treated as just that: optimizations. You shouldn't be required to use a bundler just to develop/distribute JS programs.
(Most) everyone was already using bundlers at this point. Tree shaking was much easier to do after the dependency graph became computeable. Which it wasn't with CJS; require could happen anywhere with any parameters.
And require happening anywhere while being sync was kind of a non-starter. That freeze is amplified by most projects having hundreds of not thousands of requires across it's dependency graph.
There's definitely some truth to your gripes. Back in 2015 there was a ton of latent hope http/2 push and ESM would let us get away from bundling & perhaps even needing build steps at all. That hasn't panned out.
Indeed, push was removed entirely! https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYL... Except it's still the core backend for Push API. Alas, fetch, in spite of many years of asking for it, never got the ability to observe a Push request, so devs literally never had a chance to explore the possible content update/content discovery use cases that make so so much sense in Web Push Protocol. Crying waste & shame; Push had promise outside of asset delivery but it never got a chance (except the deeply browser-intermediated Push API).
There are a variety of tools & options for bundlers to emit bundled esm. Which would nicely make your bundle usable by multiple different consumers.
Bundler developer here: some bundlers (like Parcel) use esm imports within their generated runtime output to facilitate loading dynamic bundles.
It's a bit simpler (and I'd assume faster) than the traditional approach of dynamically creating a script tag at runtime.
Additional features like access to import.meta lets you get the path to the current script which also helps bundlers with resolving the paths to dynamic imports without requiring the developer to specify the base path explicitly.
There are problems though, for example you cannot retry an import() if the internet drops. The failed import is cached so the browser will fail the import again on retry, even if the connection is restored https://github.com/fatso83/retry-dynamic-import
You might just be using the node-fetch example as a stand-in to make your point about esm/cjs, but the fetch api is built in to Node (since v16.15) and is no longer even considered experimental as of Node 21+.
By god do I agree with this. I feel like all my most diabolical ESM issues are beating the tsconfig.json, webpack.js, and package.json into shape to get ESM and legacy working.
Nothing except doing things the standard way so that standards-compliant language implementations (and other tools) can work with them instead of targetting IE^H^H NodeJS's non-standard quirks.
> the payoff is "it's the shiny new thing"
NodeJS's reality distortion field has claimed another.
Ecmascript modules are going on 10 years old at this point. (For comparison, that's more than the amount of time that NodeJS existed when modules weren't a part of the language.)
> Ecmascript modules are going on 10 years old at this point.
False. Ecmascript defined the syntax for imports but not the mechanism for module loading. I've been using ES6 imports with CJS modules for almost as long as the syntax has existed (without problems!); the incompatibility you're describing is ones of years old.
> Nothing except doing things the standard way so that standards-compliant language implementations
Node's way of handling modules isn't the same as the way the browser works (and it can't be: see node_modules). There's no singular all-encompassing "JavaScript ESM" standard. There's not even consistency between server-side JS runtimes.
But more importantly, saying "the way your code worked for fifteen years is now shit and won't work going forward, you need to rewrite it because there's a new thing now" is an incredibly fucked way to run an ecosystem. See: Python 3.
> Node's way of handling modules isn't the same as the way the browser works
That's a Node problem. 100% foreseeable (and avoidable).*
There's a reason why I compared NodeJS to Internet Explorer in my first comment—because it's extremely apt. Too bad most NPM programmers have blinders such that their entire worldview consists of NPM+NodeJS (not unlike the way programmers who were used to exclusively targeting Microsoft tech 20 years ago lived in their own parochial world) and an expectation that everyone else should bend over backwards to accommodate their way of life.
If you as a programmer want not to be burned by IE-/NodeJS-like shenanigans, then don't run arms outstretched into those flames.
* And, no, it's not an anachronism to say that. There's a reason why the notion of "forward compatibility" exists—at least among the wise. For an example of how to approach things sensibly (read: avoid getting burned exactly like one should otherwise expect to), have a look at the position/approach to forward compatibility that the TypeScript team (initially a Microsoft undertaking, funnily enough) has taken. There's no reason anyone should expect to be able to throw caution to the wind, YOLO their way forward with their hands covering their ears while shouting "la-la-la" and for everything to turn out okay. That is profoundly unwise.
> an incredibly fucked way to run an ecosystem
Right. Which makes it doubly crazy the way that folks underneath the NodeJS carnival tent have taken to approaching things for the last 15 years—and the last 10 years in particular. (Again, to emphasize: 10 years!)
But of course I'm being facetious, because when you said "an ecosystem" you didn't narrowly mean "the NodeJS ecosystem". You meant "the JS ecosystem"—leaning heavily on the very same merger doctrine of the-NodeJS-ecosystem-is-the-JS-ecosystem-which-is-the-NodeJS-ecosystem that I mentioned NPMers believing in before.
> See: Python 3.
Sure, and Python is proprietary—with a BDFL, a community rallying for the most part behind a single implementation (where you get what you pay for if you go astray), and no strong commitment to standards/compatibility for most of its existence; the Python project can do whatever the Python project wants to do. Likewise Perl. Likewise, you know... Dart. That's how those things go.
JS is fundamentally different, and that's what you and countless other NPM programmers seem not to be getting. See also my earlier comment:
The difference is the language/standard in question neither originated with NodeJS nor is NodeJS now nor has it ever been led by the people behind the language/standard
The node_modules ecosystem is, in fact, not the whole world, nor is it synonymous with "JS". (Heck, it's not even synonymous with CommonJS[1].) NPMers can ignore this fact at their peril, or they can wise up and accept that they can't simply will things to go the way they apparently think they should be able to make them go despite mountains of evidence and wisdom to the contrary.
The whole thing feels like IPv6 adoption to me: Make v2 of a thing that is undeniably better, but downplay/ignore the migration strategy and then just complain that people are lazy if they haven't migrated yet.
Although on a payoff note, it's not even the shiny new thing, it's almost a decade old. We really need a Python 2.7 style "no: we're done with this. Don't uplift your code as much as you deem necessary, but we're moving on. You've had a decade by now" event for Node.
The syntax is a decade old, and people have been using CJS with ES6 import syntax this whole time without major issues. Node ESM support is an entirely different self-inflicted problem that's been around for just a couple years. Don't gaslight folks into thinking this was something that the ecosystem was planning for a decade. It simply wasn't.
If by "just a couple years" you mean five years. There is no gas lighting here: I am directly blaming Node for hurting the uptake of ESM by not planning to switch to ESM-first (with CJS second) as part of the ESM support work, and I'm blaming Node even more for then also not starting that planning work when ESM in Node was considered mature enough to not live behind an experimental feature flag anymore.
Node was, from the get-go, strongly tied to V8's development roadmap. They (like everyone else in the JS language space at the time) were watching the ESM debate, and had every opportunity to plan for a "few years out" switchover, starting the moment TC39 accepted ESM into the official language spec, with V8 following suit.
They didn't, but thankfully it's never too late: Node should absolutely still go "we're finally deprecating CJS. You should use ESM, and for the next 2 years it's business as usual but after that we won't support CJS unless your project marks itself as such in package.json, and 2 years after that, we're removing CJS support entirely".
> Node should absolutely still go "we're finally deprecating CJS. You should use ESM, and for the next 2 years it's business as usual but after that we won't support CJS unless your project marks itself as such in package.json, and 2 years after that, we're removing CJS support entirely".
So this has two consequences:
1. NPM is split in two. All the packages that don't upgrade are useless. If you rely on one, you either need to do the work to port it (and all of its dependencies) before you can even consider migrating your own code. Or pray that someone else will do it.
2. Companies simply won't upgrade. The last version of Node to support CJS will persist for a decade or more. It might get forked and make it harder for Node to actually drive the ecosystem forward. If you think io.js won't happen again as a result of a decision like this, you're mistaken.
1) no they aren't, just like Python 2.7 libraries still work just fine with python 2.7 and that's fine and entirely on you (HI GRAPHICS INDUSTRY!).
If a 5 year old library doesn't work with current Node, that's perfectly fine. Kind of already the case right now anyway, so nothing new under the sun there. Packages that are still actively used, though, get 4 years to uplift. that is plenty of time.
2) how is that different from companies that still use Node 14 or even 10 in production today because their dependency chains make it impossible to uplift even without taking CJS vs. ESM into consideration?
Everyone who's already stuck in the past is going to stay stuck in the past, anyone who can afford to uplift their codebase over two to four years will be better off, including the entire JS dev landscape. Let's not pretend that the bad habits of others should hold everyone else back. That's how we got here in the first place.
> it was easier to _write a new admin panel from scratch_ than it was to refactor our entire codebase to be ESM just so we could upgrade one library
A middle ground answer would be using import() in CJS to asynchronously require the ESM module. It would require some hacking around your loading flow to ensure the module loads before you try to do anything with it but would still be preferable to rebuilding the entire thing.
I have a custom loader that can operate in both sync and async mode with the same syntax; it started off as a single loader to load AMD and CommonJS but I’ve been meaning to add ESM support as well (it was more opaque than I liked when I last tried, required too much browser magic).
I don't disagree! But is there really an argument (other than an ideological one) for why it needs to be an awaited import? Why does it matter that it's async?
ESM (as standardized, not necessarily as implemented in Node) treats import specifiers as URLs. That alone would inherently be async because network requests are async across basically all JS runtimes. ESM was also later extended to support top level await, making module execution itself async.
Node has recently taken a more pragmatic approach, by treating local specifiers as potentially synchronous, allowing sync require() of ESM… which then fails if the required module uses top level await.
Fetching code at runtime over the network makes sense in a browser, it doesn’t make sense outside of a browser and would actually be a huge security risk without all of the security features that browsers have. So it just reinforces the perception that ESM was only designed for browsers.
Both TC39 and node utterly cocked this up. The design of ESM is bad, async imports are a stupid idea, and node’s implementation has been beset by ideology.
"Sometimes works, but can also blow up at runtime after any upgrade" seems to me like a "pragmatic cure" that is worse than the existing "disease" of needing to treat all ESM imports as async and/or migrating all uses of `require()` to `await import()`. If incrementally moving a CJS code base to use `await import()` in more places is too hard then you probably have other problems you've overlooked for too long (bad promise usage, callback waterfalls, etc).
(But I'm a hardline "all you need is type=module" sort and think a CJS=>TS emitting ESM "rip off the bandaid" approach to CJS legacy libraries works and is fundamentally easier than most of the "pragmatic" solutions to CJS/ESM interop. It is past time to jettison CJS out the airlock.)
I'd say it's exactly the opposite. Neither require() or import require an async code structure. The biggest reason I avoid ESM like a plague is that many of my projects are not, and do not need to be, async. Adding import() is a huge hassle because it is async.
Adding an importSync() function would be a perfect solution.
That was my first thought, and what I attempted first in my project. But it turned out that rewriting everything, while being incredibly tedious, was significantly easier to reason about and ensure continuity than developing my own bespoke dependency loader in a large project where not all portions are well understood.
Thank you for saying this. I got 'flagged' when I said it more irritably. The solution is so simple and the payoff so great that it seems to me that the standards bodies are a bad process.
Me? I'd prefer importSync() to having require() cover ESM files. I think it would be more clear but, either way, it would do exactly what you say, open the CJS world to migration.
> I'd prefer importSync() to having require() cover ESM files.
My only criticism is that in order to do incremental conversion, both need to work. At that point, you'd might as well just eliminate the cognitive overhead and have one function.
> It's be synchronous and slow and ideologically impure
Sure. It also "just works" at the CLI. Sometimes I just want to test code quickly at the CLI. I'm not into "architecting" this part of my code as much as ES thinks I want to.
Why should I need to go out of my way to do this? Why do I need to care what kind of module I'm importing and reach into the stdlib to run code? It should just work.
To be frank I don't care about these questions, perhaps you're right and it should, but whatever. It's a solution I used in the three cases it was necessary that blocked moving my codebase to ESM.
I had been using adminjs at work. Their new major version was ESM-only, and it was easier to _write a new admin panel from scratch_ than it was to refactor our entire codebase to be ESM just so we could upgrade one library. I expect that's the situation at hundreds of thousands of other companies.
Like for all the belly aching that happened over async functions (and the whole function color rant), synchronous and asynchronous functions work together just fine through plain old promises. You can easily use async functions alongside functions that use old fashioned callbacks. ESM vs CJS is a file coloring (versus function coloring) problem, but there's no interoperability. There's no escape hatch when you just need one file to use another file but their colors are incompatible.