Micro-libraries should never be used
bvisness.meMicro-libraries are really good actually, they're highly modular, self-contained code, often making it really easy to understand what's going on.
Another advantage is that because they're so minimal and self-contained, they're often "completed", because they achieved what they set out to do. So there's no need to continually patch it for security updates, or at least you need to do it less often, and it's less likely that you'll be dealing with breaking changes.
The UNIX philosophy is also build on the idea of small programs, just like micro-libraries, of doing one thing and one thing well, and composing those things to make larger things.
I would argue the problem is how dependencies in general are added to projects, which the blog author pointed out with left-pad. Copy-paste works, but I would argue the best way is to fork the libraries and add submodules to your project. Then if you want to pull a new version of the library, you can update the fork and review the changes. It's an explicit approach to managing it that can prevent a lot of pitfalls like malicious actors, breaking changes leading to bugs, etc.
Micro-libraries anywhere else are everything you said: building blocks that come after a little study of the language and its stdlib and will speed up development of non-trivial programs.
In JS and NPM they are a plague, because they promise to be a substitute for competence in basic programming theory, competence in JS, gaps and bad APIs inside JS, and de-facto standards in the programming community like the oldest operating functions in libc.
There are a lot of ways for padding a number in JS and a decent dev would keep an own utility library or hell a function to copy-paste for that. But no. npm users are taught to fire and forget, and update everything, no concept of vendoring (that would have made incidents like left-pad, faker and colors less maddening, while vendoring is even bolt in npm and it's very good!). They for years copy-pasted in the wrong window, really, they should copypaste blocks of code and not npm commands. And God helps you if you type out your npm commands because bad actors have bought the trend and made millions of libraries with a hundred different scams waiting for fat fingers.
By understanding that JS in the backend is optimizing for reducing cost whatever the price, becoming Smalltalk for the browser and for PHP devs, you would expect some kind of standard to emerge for having a single way to do routine stuff. Instead in JS-world you get TypeScript, and in a future maybe WASM. JS is just doomed. Like, we are doomed if JS isn't, to be honest.
The whole web stack must die and be replaced. JS, CSS, HTML, HTTP are huge cost center for global economy.
Strongly disagree, but for JS.
Current web stack is very complicated, HTML and CSS DOM is a rats nest and that's a superficial example. Adding asynchronous rpc pushes that way over the top. Luckily HTTP has been through the production crucible for 30 years. What I see is not a cost center but a reflection of basic truth:
- UI and data representation is hard.
- Developers use the tools they know about.
But about micro-libraries, the first point isn't very important because this is a social problem. There is no standard library for NPM that does what people need. There should be a curated function library of this crap.
In my imaginary vision there are several, and people would look askance when you don't use one without an obvious reason. I very much sympathize with the desire to burn-it-all, but I like that I can use LetsEncrypt and am cognizant that there is a lot of thought and raw technological progress bound up in all this.
Remind me which part of web has ===? I don't think it's HTML or CSS.
HTTP, HTML and to some extent CSS are solid technologies that are very well designed and has been standing thr test of time.
How people use them is another matter. But don't blame that on the technology.
They are not designed for modern use cases and certainly not well-designed according to modern understanding of this word. In fact they did not stood the test of time as something worthy of preservation, they only survived and mutated, because it is extremely hard to replace them.
And of course we should blame the technology: the sole purpose of a standard is to be used by people. If people struggle with it the standard is unfit for its purpose.
You are 100% right but it's no reason to pick on web in particular! When you hit a site, you may have a virtual HTML environment running a JIT virtual machine from a server that's running a virtual python environment on a virtual python environment on a virtual machine on a virtual machine, and all that runs on a virtual x86 processor which in reality is a series of microcode processors. Yes, yes, it would be so much simpler to have your web go straight to the source, microcode, and yes, things would be simple and fast... .. but ... but ABSTRACTION!!!! LETS ABSTRACT EVERYTHING EVER AS MANY TIMES AS POSSIBLE WITH AS MANY ABSTRACTION LAYERS AS POSSIBLE IT GENERALIZES TO THE GENERAL CASE ABSTRACTION GENERALIZES ALL CASES WOOOOWWWW THE COMPUTER SCIENCE OF IT ALL
That is the state of things.
> And of course we should blame the technology: the sole purpose of a standard is to be used by people. If people struggle with it the standard is unfit for its purpose.
'People' is equivocal here. While complex, and (as you point out, in so many words) an evolved standard, the HTML/CSS/JS stack is arguably one of humanity's greatest achievements, up there with the invention of paper, or perhaps cuneiform.
It's imperfect, like all living standards, but it manages to be ubiquitous, expressive, and _useful_. And for those of us who grew up with these standards, they are second nature.
Like piano, mastering these standards gives you the ability to express complex UI concepts with grace and alacrity.
Don't smash up your parents' piano simply because practicing scales is a chore :)
> Don't smash up your parents' piano simply because practicing scales is a chore :)
I witnessed evolution of UIs from Turbo Vision to modern web and mobile frameworks. My first commercial website went live in 1999. No, UI is not hard and doesn’t require any mastery. As a matter of fact, building decent UI for a client-server application is a simple task with the right tools and processes. Modern Web is not parents‘ piano - you can call it elegant only if you have seen nothing else. It’s a fridge with mostly expired cheap food, from which you have to cook a decent meal for a party. It is possible, no doubt. Without food we will die, so we have to cook it. Instead we could just go shopping.
> HTML/CSS/JS stack is arguably one of humanity's greatest achievements
We deserve to go extinct.
The HTTP-centrism is the worst.
QUIC is hilarious because they ended up fitting everything and the kitchen sink for any need in the proto, just because 15 years ago firewalls blocked this or that and the reimplementation with websockets of the thing that was blocked runs only in a browser.
> The HTTP-centrism is the worst.
> QUIC is hilarious
It's apples to oranges. QUIC is actually used _with_ HTTP/3.
Consider not mix transport level (QUIC) and application level (HTTP) protocols in the same comparison to oppose them.
First draft of HTTP/3 was called "HTTP/2 Semantics Using The QUIC Transport Protocol". I'm sorry, i should've laid that more shoddily and call that "GoogleHTTP".
I don't see why you feel the need to drag HTTP, HTML and CSS through the mud
The four abstraction layers reflect the reality of technological drift over time.
Even stipulating a Wand of Internet Technology (WIT) that could produce the One True Stack (OTS), two things remains undone:
- fixing all the old stuff (or does OTS emulate it all?)
- precluding further drift (or does OTS end all that?)
Those are not the Problems, just common problems with known solutions (Migration Path and Let-It-Go).
You can use Flutter now. Go use it. Stop whining.
I think that it's ugly but okay-ish right now. What is very very bad is the tooling, and someone should remember people that Facebook and Google do things that serves Facebook and Google-scale needs (billions of users, thousands of devs working asynchronously, no time).
What I end up thinking (maybe i'm wrong) is that node.js must be nuked out of backend and on frontend maybe some of the devs should use either a number of libraries under 15 and write custom code for the rest, or use a language that transpiles to JS like TS, flutter, nim, Go or what have you.
Maybe JS should be nuked out of tooling too, sometimes it's actively damaging and sometimes dead slow. Use something else if wrangling asset files are a problem.
If you want a DX where backend is frontend, you must use the only three mantained languages that can do that without trying to actively damage you or users, which are a Smalltalk (like Pharo), a Lisp (like Clojure/Clojurescript) or Java.
I‘d prefer to see a completely new renderer backed by containerized JVM with some DSL for UI as a replacement of modern browsers and application-first encrypted by default binary protocol replacing HTTP (one reference implementation for debugging it would fe fine).
Could you link to somebody who is teaching npm users to "fire and forget?" Someone who is promising a substitute for competence in basic programming theory? Clearly you and I do not consume the same content.
This is just a discourse based on "I need to churn out something, I need that fast and I didn't start in the web game when Backbone and E4X were solid corporate choices". If you are not in a hurry, work in a solid team and have a good attention span, a lot of clickbait idiocy around JS may not happen. It's just that the lone inexperienced guy is one of millions inexperienced guys who are taught the wrong ways everyday.
I'm presenting you one of countless examples: a lot of coding bootcamps teach React, maybe with TS, maybe with JS.
Enter react-create-app.
https://github.com/facebook/create-react-app
The docs are a link, while the commands you can copy and paste are laid out at 9th row in the README. That will become a habit for a junior.
I don't really see the problem with the CRA docs.
You will never see it, blinded by a sound experience in UNIX.
The UNIX philosophy is being a bit abused for this argument. Most systems that fall under the UNIX category are more or less like a large batteries-included standard library: lots of little composable units that ship together. UNIX in practice is not about getting a bare system and randomly downloading things from a bunch of disjointed places like tee and cat and head and so on, and then gluing them together and perpetually having to keep them updated independently.
They ship together because all of those small composable units, that were once developed by random people, were turned into a meta-package at some point. I agree with you that randomly downloading a bunch of disjointed things without auditing and forking it isn't good practice.
I'm also not arguing against a large popular project with a lot of contributors if it's made up of a lot of small, modular, self-contained code that's composed together and customizable. All the smaller tools will probably work seamlessly together. I think UNIX still operates under this sort of model (the BSDs).
There's a lot of code duplication and bad code out there, and way too much software that you can't really modify easily or customize very well for your use case because it becomes an afterthought. Even if you did learn a larger codebase, if it's not made up of smaller modular parts, then whatever you modify has a significantly higher chance of not working once the library gets updated, because it's not modular, and you updated internal code, and the library authors aren't going to worry about breaking changes for someone who's maintaining a fork of their library that changes internal code.
> all of those small composable units, that were once developed by random people, were turned into a meta-package at some point
No they weren’t. Every UNIX I used in the 80s and 90s shipped with those little composable building blocks as part of the OS, and GNU bundled them in things like coreutils forever. It’s not like there was some past time when there were independent little things like cat and wc and so on written by random people on the internet that somehow got bundled into a meta-package after they existed. That didn’t happen.
They were developed by different authors...
They have evolved together, though, not in isolation from each other.
So are different functions and modules in the standard library of a large batteries-included language like Python, Java or Go.
But in fact, most of the traditional utilities you know and love were first implemented by a small team in Bell Labs, with the most "core" of what we now call coreutils (i.e. "ls") being written single-handedly by Ken Thompson and Dennis Ritchie. The other significant chunk of utilities (commands like vi, more, less, netstat, head) were developed by a group of people at UC Berkeley and were released later as the Berkeley Standard Distribution (BSD).
GNU Coreutils, as we known them, are mostly re-implementations of the original Unix commands (incidentally, a lot of them have been initially implemented by two guys as well - Richard Stallman and David Mackenzie). This is no small feat, but the GNU coreutils authors took good care to maintain compatibility with existing UNIX tools (and later with the POSIX standard when it was released). They didn't just randomly implement commands in the vacuum and waited for someone to come up and bundle them together. It's worth noting that if you're using a Mac, you're not even using GNU coreutils. Most of the core commands in macOS are derived from FreeBSD which traces its lineage back to the original UNIX implementation.
The fact is, most of the individual commands that are now in coreutils were never released individually by themselves - they were generally released as a distribution and usually developed together and specifically to interact with other tools. The Unix Philosophy could not have been developed if Unix did not have a basic decent set of core utilities to begin with, and in fact the core utilities like cat predate Unix pipe support (only introduced in version 3).
The availability of a reliable core system commands which are guaranteed to be available and follow a strict IEEC/ISO standard(!) was pretty important for the development of an ecosystem of non-core commands that are built on top of them. Imagine what would have happened if some commands used fd 0 for stdin and fd 1 for stdout, but others used a different file descriptor or perhaps even an entirely different output mechanism, such as using a system call or sending data through a socket. Interoperability would be much harder.
But this is exactly the case in JavaScript, especially within the NPM ecosystem. The lack of a proper standard library means that every developer has to cobble together the most minuscule of utilities that are usually available as part of the standard library in other ecosystems. But what's worse is that all the various libraries don't play very well with each other. You don't have enough standard base types that you can pass between the different libraries.
For instance, there are no reliable types for describing time (besides a crappy built-in date), input/output streams (until recently), HTTP client, sockets, URLs, IP addresses, random number generators, filesystem APIs, etc. Some of this stuff (like Fetch API and Subtle Crypto) exist in Node.js and have been locked behind feature flags since forever, but that effectively means that you cannot use thme. In the earlier days of the JS ecosystems things were much worse since we didn't even have a standard way for specifying byte arrays (Uint8Array) or even Promises.
Who often worked together, sometimes, gasp in the same building! Definitely not random people on the internet not met or even emailed.
Then only use the software that Bob wrote in the same building you work at, I don't know what to tell you.
I don't think you can necessarily trust Bob's code without looking at what he wrote, just because you see him in person.
And coreutils has 200 contributors, many random people and strangers, from all over the world now.
> randomly downloading things from a bunch of disjointed places like tee and cat and head and so on, and then gluing them together and perpetually having to keep them updated independently.
I have distressing news about my experience using Linux in the '90s
We should totally have a system like that, though. It'd be such a great learning environment.
It's called Linux From Scratch.
> So there's no need to continually patch it for security updates, or at least you need to do it less often, and it's less likely that you'll be dealing with breaking changes.
Regardless of how supposedly good or small is the library, the frequency at which you need to check for updates is the same. It doesn’t have anything to do with the perceived or original quality of the code. Every 3rd party library has at least the dependency on platform and platforms are big, they have vulnerabilities and introduce breaking changes. Then there’s a question of trust and consistency of your delivery process. You won’t adapt your routines based on specifics of every tiny piece of 3rd party code, so you probably check for updates regularly and for everything at once. Then their size is no longer an advantage.
> Copy-paste works, but I would argue the best way is to fork the libraries and add submodules to your project. Then if you want to pull a new version of the library, you can update the fork and review the changes.
This sounds “theoretical” and is not going to work at scale. You cannot seriously expect application level developers to understand low level details of every dependency they want to use. For a meaningful code review of merges they must be domain experts, otherwise effectiveness of such approach will be very low - they will inevitably have to trust the authors and just merge without going into details.
They don't need to understand the low level dependencies. People can create metapackages of a lot of a bunch of self-contained libraries that have been audited and forked, and devs can pull in the metapackages. The advantage is the modularity, which makes the code easier to audit and is more self-contained.
When's the last time ls, cat, date, tar, etc needed to be updated on your linux system? probably almost never. And composing them together always works. This set of linux tools, call it sbase, ubase, plan9 tools, etc, is one version of a metapackage. How often does a very large package need to be updated for bug fixes, security patches, or new versions?
> When's the last time ls, cat, date, tar, etc needed to be updated on your linux system? probably almost never.
Bad example: http://www.slackware.com/security/viewer.php?l=slackware-sec...
They find stuff like this fairly often in GNU coreutils even to this day.. it’s the main reason there’s a Rust coreutils effort.
It's probably still a good example. Looking up the CVEs for various search terms:
coreutils: 17 results
linux kernel: 6752 results
x11: 184 results
qt: 152 results
gtk: 68 results
docker: 340 results
rust: 455 results
python: 940 results
node: 110 results
javascript: 5657 results
firefox: 3268 results
chrome: 3763 results
safari: 1465 results
webkit: 1346 results
The large monolithic codebases have a lot more CVEs. I'd also argue that patching a fix on code made up of small, modular parts is much easier to do, and much lower hanging fruit for any casual developer to submit a PR for a fix.
The large ~~monolithic~~ codebases have a lot more CVEs
Who would've guessed. Also the older ones also got more CVE's than newer ones, even if they aren't that big.
What you call an audited metapackage is nothing other than a non-micro-library. The property that it has been audited/assembled/designed/tested in conjunction and would be updated as a whole is exactly the benefit that non-macro-libraries provide.
It's not the same as a non-micro-library, because a non-micro-library is a bunch of internal code that isn't as well-documented and isn't self-contained, and maybe come and go as the non-micro-library continues to churn. I can't easily change a large monolithic library or project do better optimize for my use case. I could do that much easier though if the large thing was composed of a bunch of small self-contained things.
I was assuming that the constituents of the metapackage aren’t completely independent. TFA is about micro-libraries the size of is-number and left-pad. If you only consider completely independent libraries of that type, you won’t get very far. A more realistic take would be hundreds of such micro-libraries that aren’t self-contained, but instead build on each other to successively provide more powerful functions. And then the more transitive dependencies become more like “internal code” in regular libraries, and you’ll get the same churn, and you’ll be similarly unable to change things due to the interdependencies. I don’t see how you can end up with a comparable functionality a regular library provides without the same potential of running into the issues you note.
What if you're using a larger library and you wanted to swap out a sorting algorithm for one that's optimized for your use case?
I would say that the API boundary being more modular and explicit makes it possible to actually do those kinds of swaps if the larger library is composed of smaller modular code, in ways that you wouldn't be able to if it's buried in a bunch of internal code -- you would have to fork the library in that case.
> How often does a very large package need to be updated for bug fixes, security patches, or new versions?
I don’t think you understand my comment, because you are asking the wrong question again. It is not how often you need to actually update one dependency, but how often you need to check for updates that matters. That has to be done frequently no matter what and must be automated. E.g. despite low number of CVEs in coreutils you have to check them very often, because the impact can be very high. Once you have this process in place, there’s no advantage in using micro-libraries. I’d actually expect that in a micro-library environment most breaking changes happen when a library becomes unsupported and you need to choose an alternative, making the migration process more complicated than in case of a bigger framework which just changed the API.
Maybe I'm not understanding your argument. Are you saying that if all these larger programs wrote all those utilities from scratch, it makes it so that if someone messed something up, the rest of the large programs are unaffected?
All of those larger programs can mess things up in different ways, opening up all sorts of attack vectors and bugs. And a smaller utility that's much more easily auditable and easier to contribute fixes to is arguably less likely to have attack vectors in the first place.
I'm not sure you have to check large libraries less often. I would argue at least as much if not more often, because it's harder for people to audit larger codebases and to also contribute fixes to them. A significantly smaller number of devs can (or could, if they had bandwidth/time) understand a large self-contained codebase than a very small self-contained codebase.
I think that if a larger library is made of many smaller, modular, self-contained parts, that are composed together, and you can swap out any smaller building block for something that fits your use case (inversion of control), then that's a really good way to write a large library. Sloppier code might find it's way in due to time/resource constraints though, or the authors might not notice that some of the code isn't entirely modular/composable/swappable.
> I would argue at least as much if not more often
The same. The frequency of checks is not dependent on the library size and depends on the risk profile of your application, so library size cannot be considered an advantage with regards to updates. In most cases upgrade to newer version can be done automatically, because it is unrealistic expectation that developers will review the code of every dependency and understand it. Breaking changes likely occur with the same frequency (rare), albeit for different reasons, and impact can be on the same scale for tiny and large depenencies.
If these libraries are so small, self-contained and "completed", why not just copy-paste these functions?
Submodules can work too, but do you really need these extra lines in your build scripts, extra files and directories, and the import lines just for a five line function? Copy-pasting is much simpler, with maybe a comment referring to the original source.
Note: there may be some legal reasons for keeping "micro-libraries" separate, or for not using them at all though but IANAL as they say.
As soon as source code is in your repo it's way more probable to getting touched. I'd never open that box ever because I don't want to waste time with my team touching code that they shouldn't when reviewing.
If you want the same functionality, build it according to the conventions in the codebase and strip out everything else that isn't required for the exact use case (since it's not a library anymore)
">The UNIX philosophy is also build on the idea of small programs, just like micro-libraries, of doing one thing and one thing well, and composing those things to make larger things."
The Unix philosophy is also built on willful neglect of systems thinking. The complexity of system isn't in the complexity of its parts but in the complexity of the interaction of its parts.
Putting ten micro-libraries together, even if each is simple, doesn't mean you have a simple program, in fact it doesn't even mean you have a working program, because that depends entirely on how your libraries play together. When you implement the content of micro-libraries yourself you have to be at the very least conscious not just of what, but how your code works, and that's a good first defense against putting parts together that don't fit.
It's not a willful neglect of systems thinking. Functional programmers have been able to build very large programs made primarily of pure functions that are composed together. And it makes it much easier to debug as well, because everything is self-contained and you can easily decompose parts of the program. Same with the effectful code as well, leveraging things like algebraic effects.
> The UNIX philosophy is also build on the idea of small programs, just like micro-libraries, of doing one thing and one thing well, and composing those things to make larger things.
They have small programs, but that are not of different project. For example all the basic Linux utilities are developed and distributed as part of the GNU coreutils package.
It's the same of having a modular library, with multiple functions in them, that you can choose from. In fact the problem is that these function like isNumber shouldn't even be libraries, but should be in the language standard library itself.
> I would argue the problem is how dependencies in general are added to projects
But you need the functionality anyway, so there are two dependencies: on your own code, or on someone else's code. But you can't avoid a dependency, and it comes at a cost.
If you don't know how to code the functionality, or it will take too much time, a library is an outcome. But if you need leftPad or isNumber as an external dependency, that's so far in the other direction, it's practically a sign of incompentence.
If incompetent it provides a way to be sure?
Could you for laughs explain for which cases these are, why they are needed and why they did it this way?
1) num-num === 0
2) num.trim() !== ''
3) Number.isFinite(+num)
4) isFinite(+num)
5) return false;
6) Why this specific order of testing? Why prefer Number.isFinite over isFinite?
https://www.npmjs.com/package/is-number
I would have just....module.exports = function(num) { if (typeof num === 'number') { return num - num === 0; } if (typeof num === 'string' && num.trim() !== '') { return Number.isFinite ? Number.isFinite(+num) : isFinite(+num); } return false; };
Why is that not precisely the same? (it isn't)isNumber = num => isFinite(num+''.trim());how about...
Is there a difference?function isNumber(num){ switch(typeof num){ case "number" : return !isNaN(num); case "string" : return isFinite(num) && !!num.trim(); } }IMHO NPM should have a discussion page for this. There are probably interesting answers for all of those looking to copy and paste.
I don't care. The problem in this case is that if you install "isNumber", you use someone else's definition of what a number is. If you ever have to check if something is a number, you should do that according to your own specs, not hope someone else got it right. In this case, strings with all kinds of weirdness seem to be allowed, and perhaps that's not acceptable.
Only the fact that you can write a bunch of slightly different versions because of Javascript's flaws/features should be a sign that you just can't grab an arbitrary implementation and hope it matches your use case, and especially not if you don't/can't pin the version.
The UNIX philosophy is also build on the idea of small programs, just like micro-libraries, of doing one thing and one thing well, and composing those things to make larger things.
This year I started learning FORTH, and it's very much this philosophy. To build a building, you don't start with a three-story slab of marble. You start with a hundreds of perfect little bricks, and fit them together.
If you come from a technical ecosystem outside the Unix paradigm, it can be hard to grasp.
Yeah, exactly! FORTH looks really awesome, I haven't gotten around to learning it much though. I heard it's addictive and fun.
Yeah, it's all concatenative programming: FORTH, unix pipes, function composition as monoids, effect composition as kliesli composition and monads, etc.
It makes it super useful for code readability (once you're familiar with the paradigm), and debugging, since you can split up and decompose any parts of your program to inspect and test those in isolation.
This has nothing in common with the UNIX approach. Awk, grep, sort, less and the like are perhaps small, but not that small and not that trivial.
Unix has yes, tr, cut, true, false, uniq, nl, id, fold, sort, sleep, head, tail, touch, wc, date, cal, echo, cat...
These are tiny programs.
I mean, sort has put on some weight over the years, sure. But if it were packaged up for npm people would call it a micro-library and tell you to just copy it into your own code.
Yet, they are still a lot bigger than most micro-libraries. And more complex. And most of them tend to be parts of the same package (coreutils). So no, they have nothing in common with microlibraries. Not in concept, not in how they are used, shipped or maintained.
> Yet, they are still a lot bigger than most micro-libraries. And more complex.
Some of them are incredibly trivial. I made this exact same comment months ago, but the yes command is basically a one line bash function [1]:
[1] https://news.ycombinator.com/item?id=38799808function yes { while true; do echo "${1:-y}"; done }Yeah, the thing is that `yes` isn't a stand alone project, it is usually part of a bigger project such as coreutils (https://github.com/coreutils/coreutils/).
For the comparison to be valid you would have to split up coreutils into roughly 100 individual repositories and replace many of the implementations with ones that are trivial, buggy, and/or unmaintained that pose a supply chain attack risk because it gets hard to keep track of what's maintained, by whom and how. Coreutils is close to 100kLOC and its programs aren't packaged individually. It is far, far from the random mess that are microlibraries in NPM.
less (17kLOC), awk (43kLOC) and grep (4kLOC) are separate projects, but some of those require a bit more insight than much application code these days, so it makes sense that they are individual projects.
> For the comparison to be valid you would have to split up coreutils into roughly 100 individual repositories and replace many of the implementations with ones that are trivial, buggy, and/or unmaintained that pose a supply chain attack risk because it gets hard to keep track of what's maintained, by whom and how.
You could paraphrase that as: core utilities that ship with my operating system should obviously be more reliable than random code fetched from the internet.
> For the comparison to be valid
I was responding to the limited scope of your statement "Yet, they are still a lot bigger than most micro-libraries. And more complex." Utilities like `yes` and `true` are neither big nor complex. The man pages are longer than the source code necessary to replace them.
Right! So if it is indeed so easy to understand what is going on, why would you need to make it an external dependency that can update itself behind your back?
If you understand what is going on, paste it into your tree.
> Micro-libraries are really good actually, they're highly modular, self-contained code
Well I think that is the point, they're not self-contained. You are adding mystery stuff and who knows how deep the chain of dependencies go. See the left-pad fiasco that broke so much stuff, because the chain of transitive dependencies ran deep and wide.
NPM is a dumpster fire in this regard. I try to avoid it - is there a flag you can set to say "no downstream dependencies" or something when you add a dependency? At least that way you can be sure things really are self-contained.
There is a "no downstream dependencies" option; it's called writing/auditing everything yourself. Everything else -- be it libraries, monolithic SaaS platforms, a coworker's PR, etc. -- is a trade off between your time and your trust. Past that, we're all just playing musical chairs with where to place that trust. There's no right answer.
The article doesn't claim to have an answer for everything. The article argues that tiny i.e. micro libraries are a almost always a bad tradeoff.
Yeah there's a way to do that, yarn and pnpm can flatten the dependency tree. You can add the fork directly too:
yarn add <path/to/your/forked/micro-library.git>
pnpm add <path/to/your/forked/micro-library.git>
I remember adding a random date picker that pulled in a copy of React with it to a non-React project. NPM is a dumpster fire at a nuclear facility.
Do you know what else is all of that? Writing the five lines of code by hand. Or just letting a LLM generate it. This and everything else I want to reply has already been covered in the article.
Nothing wrong with that either, like I said copy paste works too. A lot of minimalistic programs will just copy in another project.
Forking the code and using that is arguably nicer though IMO, makes it easier pull in new updates from the code, and to be able to track changes and bug fixes easier. I've tried both and find this approach nicer overall.
Micro libraries are ok - TFA even says you can use self-contained blocks as direct source.
Mirco dependencies are a god damn nuisance, especially with all the transitive micro-dependencies that come along, often with different versions, alternative implementations, etc.
If you're writing micro libraries, without intending to reuse them, why are you making it a library?
>I would argue the problem is how dependencies in general are added to projects
I haven't done anything with this myself (just brainstormed a bit with chatgpt) but I wonder if the solution is https://docs.npmjs.com/cli/v10/commands/npm-ci
Basically, enforce that all libraries have lock files and when you install a dependency use the exact versions it shipped with.
Edit: Can someone clarify why this doesn't work? Wouldn't it make installing node packages work the same way as it does in python, ruby, and other languages?
I'm not sure why you're getting downvoted. The left-pad incident on npm primarily impacted projects that didn't have lockfiles or were not pinning exact versions of their dependencies. I knew a few functional programmers that would freeze the dependencies to an exact version before lockfiles came around, just to ensure it's reproducible and doesn't break in the future. Part of what was to blame was bad developer practice. I like npm ci.
These days with LLMs, doing leftPad yourself is incredibly easy, I would just do that.
With LLMs? I don't think something like leftPad was every difficult to create.
Well if you think about it then yes, but thinking and making sure is a waste of energy and time. Also typing the boilerplate.
This energy could be spent elsewhere.
Like similarly it is not that hard to pick clothes for the day, but it is much easier if you always have the same clothes easily available and can move on with your day.
Why even stop at micro-libraries? Instead of "return num - num === 0" why not create the concept of pico-libraries people can use like "return isNumberSubtractedFromItselfZero(num)" ? It's basically plain English right?
You could say that if all the popular web frameworks in use today were rewritten to import and use hundreds of thousands of pico-libraries, their codebase would be, as you say, composed of many high modular, self contained pieces that are easy to understand.
/s
The primary cause of the left-pad incident was that left-pad was removed from the npm registry. Many libraries depended on left-pad. The same could have occurred with any popular library, whether micro or not.
To reformulate the statement made in the intro of this post: "maybe it’s not a great idea to outsource _any critical_ functionality to random people on the internet."
It has long been a standard, best practice in software engineering to ensure dependencies are stored in and made available from first-party sources. For example, this could mean maintaining an internal registry mirror that permanently stores any dependencies that are fetched. It could also be done by vendoring dependencies. The main point is to take proactive steps to ensure your dependencies will always be there when you need them, and to not blindly trust a third-party to always be there to give your dependencies to you.
> To reformulate the statement made in the intro of this post: "maybe it’s not a great idea to outsource _any critical_ functionality to random people on the internet."
Well everything is critical in the sense that a syntax error could break many builds and CI systems.
This is what lock files are for. If used properly, and the registry is available, there are no massive issues. This is how things are supposed work – all the tooling is made this way.
In short, I think the lessons from the leftpad debacle are (1) people don’t use existing versioning tooling, (2) there is a surprising amount of vendors involved if you look at dep trees for completely normal functionality and (3) the JS ecosystem is particularly fragmented with poor API discipline and non-existent stdlib.
EDIT: Just read up on it again and I misremembered. The author removed leftpad from NPM due to a dispute with the company regarding an unrelated package. That’s more of a mismanaged registry situation. You can’t mutate and remove published code without breaking things. Thus NPM wasn’t a good steward of their registry. If there’s a need to unpublish or mutate anything, there needs to be leeway and a path to migrate.
The key point is "If ... the registry is available", and the dependencies contained therein. We take on risk by relying on NPM to always be there and always provide us the dependencies we have already invested in. I'm arguing that organisations should take a more defensive stance against dependencies becoming unavailable. If you depend on it, keep a copy of it somewhere that you control.
The problem with micro is 100 micros is 100x more surface area and chances than 1.
Micro libraries are worse than no libraries at all - but I maintain they are still better than gargantuan "frameworks" or everything-but-the-kitching-sink "util"/"commons" packages, where you end up only using a tiny fraction of the functionality but have to deal with the maintenance cost and attack surface of the whole thing.
If you're particularly unlucky, the unused functionality pulls in transitive dependencies of its own - and you end up with libraries in your dependency tree that your code is literally not using at all.
If you're even more unlucky, those "dead code" libraries will install their own event handlers or timers during load or will be picked up by some framework autodiscovery mechanism - and will actually execute some code at runtime, just not any code that provides anything useful to the project. I think an apt name for this would be "undead code". (The examples I have seem were from java frameworks like Spring and from webapps with too many autowired request filters, so I do hope that is no such an issue in JS yet)
> but I maintain they are still better than gargantuan "frameworks" or everything-but-the-kitching-sink "util"/"commons" packages, where you end up only using a tiny fraction of the functionality but have to deal with the maintenance cost and attack surface of the whole thing.
Indeed. Several toy projects I've done were blown up in size by four orders of magnitude because of Numpy.
I only want multi-dimensional arrays that support reshaping and basic element-wise arithmetic, maybe matrix multiplication; I'm not even that concerned about performance.
But I have to pay for countless numerical algorithms I've never even heard of provided by decades-old C and/or FORTRAN projects, plus even more higher-math concepts implemented in Python, Numpy's extensive (and fragmented - there's even compiled code for testing that's outside of any test folders) test suite that I'll never run myself, a bunch of backwards-compatibility hacks completely irrelevant to my use case, a python-to-fortran interface wrapper generator, a vendored copy of distutils even in the wheel, over 3MiB of .so files for random number generators, a bunch of C header files...
[Edit: ... and if I distribute an application, my users have to pay for all of that, too. They won't use those pieces either; and the likelihood that they can install my application into a venv that already includes NumPy is pretty low.]
I know it's fashionable to complain about dependency hell, but modularity really is a good thing. By my estimates, the total bandwidth used daily to download copies of NumPy from PyPI is on par with that used to stream the Baby Shark video from YouTube - assuming it's always viewed in 1080p. (Sources: yt-dlp info for file size; History for the Wikipedia article on most popular YouTube videos; pypistats.org for package download counts; the wheel I downloaded.)
Sometimes importing zombie "undead code" libraries can be beneficial!
I just refactored a bunch of python computer vision code that used detectron2 and yolo (both of which indirectly use OpenCV and PyTorch and lots of other stuff), and in the process of cleaning up unused code, I threw out the old imports of the yolo modules that we weren't using any more.
The yololess refactored code, which really didn't have any changes that should measurably affect the speed, ran a mortifying 10% slower, and I could not for the life of me figure out why!
Benchmarking and comparing each version showed that the yololess version was spending a huge amount of time with multiple threads fighting over locks, which the yoloful code wasn't doing.
But I hadn't changed anything relating to threads or locks in the refactoring -- I had just rearranged a few of the deck chairs on the Titanic and removed the unused yolo import, which seemed like a perfectly safe innocuous thing to do.
Finally after questioning all of my implicit assumptions and running some really fundamental sanity checks and reality tests, I discovered that the 10% slow-down in detectron2 was caused by NOT importing the yolo module that we were not actually using.
So I went over the yolo code I was originally importing line by line, and finally ran across a helpfully commented top-level call to fix an obscure performance problem:
https://github.com/ultralytics/yolov5/blob/master/utils/gene...
Even though we weren't actually using yolo, just importing it, executing that one line of code fixed a terrible multithreading performance problem with OpenCV and PyTorch DataLoader fighting behind the scenes over locks, even if you never called yolo itself.cv2.setNumThreads(0) # prevent OpenCV from multithreading (incompatible with PyTorch DataLoader)So I copied that magical incantation into my own detectron2 initialization function (not as top level code that got executed on import of course), wrote some triumphantly snarky comments to explain why I was doing that, and the performance problems went away!
The regression wasn't yolo's or detectron2's fault per se, just an obscure invisible interaction of other modules they were both using, but yolo shouldn't have been doing anything globally systemic like that immediately when you import it without actually initializing it.
But then I would have never discovered a simple way to speed up detectron2 by 10%!
So if you're using detectron2 without also importing yolo, make sure you set the number of cv2 threads to zero or you'll be wasting a lot of money.
This is mortifying. This should not be acceptable implicit behaviour for imports to implicitly run code by simply existing
I know, right??! I was alternating between hitting my head up against the wall in despair, and jumping for joy I accidentally found a way to speed up detectron2 by 10%. I'll take the win.
YOLO: You Only Load Once
That’s just the way Python works though, an import reads a script line by line defining the functions and executing calls. I think that’s true of any scripting language?
How it works and how it should be used are different.
Say “no” to import side‐effects in Python:
https://news.ycombinator.com/item?id=7536246
https://chrismorgan.info/blog/say-no-to-import-side-effects-...
>I must be able to import your module— any module— at any time without anything breaking.
Or even the beneficial side effect of some other module running 10% faster. ;)
What side-effects, if any, are okay when importing a python module?
https://softwareengineering.stackexchange.com/questions/4540...
>Q: Is this a better design pattern or does it just kick the issue down the road?
>A: Kicking the issue down the road is the basic idea of the "functional core, imperative shell" design pattern.
Functional Core, Imperative Shell (2012):
https://news.ycombinator.com/item?id=34860164
https://www.destroyallsoftware.com/screencasts/catalog/funct...
Well yeah, I wouldn't hold it that way either. The number of PRs I've commented on to take side effects out from imports is like , 75% of all the PRs I've seen in my current job.
Maybe Python 4 can make anything except a function definition or import be a syntax error in a module haha.
C libraries compiled with -ffast-math would like a word…
Seems a lot like the classic "I put only a couple of the strong advantages and enumerate everything I could think about as disadvantage". While I'm bias (I've done a bunch of these micro-libraries myself), there's more reasons I/OSS devs do them! To name other advantages (as a dev consuming them):
- Documentation: they are usually well documented, at least a lot better than your average internal piece of code.
- Portability: you learn it once and can use it in many projects, a lot easier than potentially copy/pasting a bunch of files from project to project (I used to do that and ugh what a nightmare it became!).
- Semi-standard: everyone in the team is on the same page about how something works. This works on top of the previous two TBF, but is distinct as well e.g. if you use Axios, 50% of front-end devs will already know how to use it (edit: removed express since it's arguably not micro though).
- Plugins: now with a single "source" other parties or yourself can also write plugins that will work well together. You don't need to do it all yourself.
- Bugs! When there are bugs, now you have two distinct "entities" that have strong motivation to fix the bugs: you+your company, and the dev/company supporting the project. Linus's eyeballs and all (yes, this has a negative side, but those are also covered in the cons in the article already!).
- Bugs 2: when you happen upon a bug, a 3rd party might've already found a bug and fixed it or offered an alternative solution! In fact I just did that today [1]
That said, I do have some projects where I explicitly recommend to copy/paste the code straight into your project, e.g. https://www.npmjs.com/package/nocolor (you can still install it though).
[1] https://github.com/umami-software/node/issues/1#issuecomment...
Every team should eventually have some internal libraries of useful project-agnostic functionality. That addresses most of your points.
Copy-paste the code into your internal library and maintain it yourself. Don't add a dependency on { "assert": "2.1.0" }. It probably doesn't do what you actually want, anyway.
I think the more interesting point is that most projects don't know what they actually need and the code is disposable. In that scenario micro-libraries make some amount of sense. Just import random code and see how far you can get.
That's what I do for personal projects, I just run "npm publish"[1] on those and BAM it's managed and secured by npm and versioned instead of having to copy/paste or search old versions/new versions in Git history.
[1] I lied, I don't even run npm publish, I made my own tool for easy publishing so I just run `happy "Fixed X bug" --patch`
> Micro-libraries should never be used. They should either be copy-pasted into your codebase, or not used at all.
I would prefer them to be built straight in the languages.
Yes, everyone seems to take the wrong lesson from left-pad. The reason left-pad happened on NPM isn't that there's something uniquely wrong with how NPM was built, but that JS has a uniquely barren standard library. People aren't writing their own left-pad functions in Java or Go or Python, it's just in the stdlib.
At the same time Go is quite barren when it comes to list (slice) functions, but I largely agree with Java and Python
I was about to jump into the comment section and say something along the lines of "but no one really thinks they're actually good, right?", only to see the top comment arguing they're good.
Astounding that this is as polarizing of a take as it seems to be
and because it updates fairly frequently
I fail to comprehend how a single-function-library called "isNumber" even needs updating, much less "fairly frequently".
The debate around third-party code vs. self-developed is eternal. IMHO if you think you can do better than existing solutions for your use-case, then self-developed is the obvious choice. If you don't, then use third-party. This of course says a lot about those who need to rely on trivial libraries.
>I fail to comprehend how a single-function-library called "isNumber" even needs updating, much less "fairly frequently".
If someone uses isNumber as a fundamental building block and surrogate for Elm or Typescript (a transpiler intermediate that would treat number more soundly I hope), this poor soul whom I deeply pity will encounter a lot of strange edge-cases (like that one stated in the article: NaN is a number or not?) and if they fear the burden of forking the library they will try to inflict this burden upstream, enabling feature or conf bloat.
I insinuate that installation of isNumber is, like most of these basic microlibs, a symptom of incompetence in usage of the language. A worn JS dev would try isNaN(parseInt(num+'')) and sometime succeed.
> [...] and sometime succeed
Nothing is ever certain when you program in javascript.
I think the updates are more for bugfixes around edge cases than feature additions.
> I fail to comprehend how a single-function-library called "isNumber" even needs updating
Never underestimate the complexity and footgunny nature of JS' type system.
Until I read the comments here, I thought from the title that this was about those small neighborhood "libraries" that are basically a box the size of a very large birdhouse, mounted on a post, with a bunch of donated books inside that passersby are free to borrow. I was really wondering why someone would have a problem with these, unless they work for a book publisher.
Micro libraries are fine (well... not really), the problem starts when each of those depends on 10 more "micro" libraries and so forth. The branching factor quickly leads to bloat. Libraries have a duty to minimize their footprint in ways that applications do not.
I suspect this might be seen as trolling, but why isn't there a standard lib of stuff like this?
surely it can't be beyond the wit of programming kind to have a standard lib, or even layers of standard lib for Node?
What is the argument for not having a standard lib, apart from download speed?
I'm not taking an absolute position either way -- the devil is in the details -- but here's my steelman for the opposing view:
When you put something in the standard library, it's harder to take it out, meaning that you're committing development resources to support the implementation. Furthermore things change: protocols and formats rise and fall in popularity and programming style evolves as the language changes (e.g. callbacks vs. promises in JS). Therefore the stdlib becomes where libraries go to die, and you'll always have a set of third party libraries that are "pseudo-standard", like NumPy in Python.
Having a minimal stdlib lets you "free-market" the decision, letting the community effects take care of what is considered standard in the ecosystem, and lets you optimize its minimal surface, like what happened with C.
But people don’t have to spend social capital to get it used at the next place they work.
Why isn't there some kind of 'compiler/linker/stripper' that would collect the functions actually used and compile them into an application specific library? Yes I know that dynamic dispatch makes that difficult but the programmer does surely know which functions he wants to call.
I sometimes hanker for a return to Fortran IV where every routine was separately compiled and the linker only put into the object code those that were referred to by something else.
Do you mean tree-shaking? I think basically all JavaScript "compilers" do this these days so that the code you serve is only the code you actually use.
Zig handles this with lazy compilation: code is parsed, but not even type checked, until the compiler reaches it.
This can lead to the occasional rude surprise when finally reaching code I've been working on for awhile, but haven't yet connected to the rest of the project. But it means there's no need for tree shaking, because nothing gets in until it gets used. One of my favorite things about the language.
its such a useful technique. tiny binaries ftw!!
There are many ways to look at this. Maybe the standard lib of such functions should be implemented as native functions in the language (1). Or as a standard external function library (2). Or people should copy-paste the functions they need and keep in their organization's function library (3). I am sure there are a few more options.
I moved to option 3: in all my apps I include a function library that I build over the years, so I don't start from scratch every time. I deeply hate ("hate speech" example here) dependencies to libraries from all over the Internet due to security reasons, but I copy-paste code when needed to my library after I read, understand and check the code that I copy. The biggest advantage is that some of this code is better than what I could invent from scratch in a busy day and I save the time of doing it right. The disadvantage is there is no way to reward these authors that contribute to humankind.
PS. My function library has functions mostly written by me, over 80%, but it includes code written by others. In my case, every time I need a function I check my existing library first, then analyze whether to write or copy.
I assume you correctly obey the licensing when you copy and paste it in....
Yes. It is not worth not doing it.
Every non-trivial Java project I've worked on ends up depending on Google's Guava and some combination of Apache Commons libraries.
If your only examples are leftPad and isNumber, I can’t take this article seriously. There are so many really useful micro libraries.
There are plenty of other one-liners: is-odd, is-promise, etc.
p-limit
P-limit is amazing. So good it kinda is a proof that most libs aren't as good as it.
Missing benefit: Libraries are a way to distribute and share the costs of labor, resulting in a more efficient ecosystem.
This doesn't apply to micro-libraries, but it looks like that cost/benefit list is intended to cover libraries in general.
I would argue that "They should either be copy-pasted into your codebase" would cause more code liabilities and maintenance required further down the line. I've personally seen codebases with a ton of custom code, copy-pasted code, inspired implementations before and it was horrible to get them up to speed with the latest functionality / best practices. I agree that having too many micro-libraries might not be beneficial though, but perhaps look for larger, more well-established libraries that encompasses those functionalities :)
What‘s an example for „latest functionality“ or „best practices“ that should or could possibly change for a function like leftPad and that would not automatically happen by virtue of being in the code base (such as formatting)?
You don't even have to talk about hypotheticals when it comes to this "vendor everything instead" philosophy. This is basically how the C world works, a lot of which is driven by a general allergy to dependencies by embedded developers - partially out of necessity (space and overhead are MUCH more important constraints in embedded land) - but also partially out of cargo culting.
I guess the opinion I'll share here is that I don't hear too many people arguing that the way embedded developers manage C libraries is at the forefront of how we should be handling and distributing code.
Many industries, like AAA game development, work this way.
There's a good reason for it -- code quality and performance is important. Understanding what your code actually does and how it does it, is important.
And as a result, AAA game software is able to push the boundaries of consumer compute performance. Micro-library-built software is generally just bloated crap that barely works. But it is fast to churn out, so there's that.
Fair points. But for every bad library there are 10 rock solid ones that are going to do a better job handling edge cases and being performant than you can for almost no additional effort.
Really what this comes down to is the pickiness of package repo maintainers and general skill level of engineers within the ecosystem. This kind of crap and cruft discussed in the article happens in ecosystems where the barrier to entry is tiny. You don’t really hear about left-pad like fiascos in Java land or Rust land. Not saying all devs are great that use these technologies, but in Rust case you really don’t contribute packages without jumping over some serious learning curve hoops and in Java land the canonical repository (Maven Central) is difficult enough to contribute to that it also staves off fly-by-night contributors
> One breaking change simply upgraded the minimum supported Node version from 0.10.0 to 0.12.0 and changed nothing else.
Well, that's a proper use of SemVer, not sure why you put it against the library's author. I've personally been burned enough times by libraries that for some reason think that literally being unable to compile them is somehow a backwards-compatible change, so it's refreshing to see that some people actually understand that.
There's nothing especially wrong with small libraries if you carefully manage them and don't allow for supply chain attacks. I don't think updates are a serious concern compared to not using a library, because your own code could easily have vulnerabilities too. It is harder to update lots of small libraries versus one big library, but you pick your battle.
In my Laravel projects, there are a few packages of much more niche/hobbyist origin without corporate backing, some haven’t been updated for a while, and others are perfectly fine and don’t need much maintenance.
Normally, packages are listed in my composer.json and stored in vendor/. For those packages, I created a separate folder called vendor_private/ which is part of my Git tree, put copies of these weird little packages in it, and set up my composer.json to consider that folder a repository.
Works like a charm. My big important packages are still upstream. I can customize the little ones as needed to fit better, or have better code, and not worry about them going unmaintained. It’s also way quicker than copying the files individually out of the package and into the right places (along with updating Namespaces, configuration, etc.) Once in a while, I’ll go back and see if anything worthwhile has changed upstream - and so far, it never has.
Personally I prefer sharing one level up from function to conceptual module, ie. instead of "left-pad" function, "string" module, ie. a bit like this [0] (`${authorshipDisclaimer}`).
I'm also an advocate, against crowd, of qualified imports as they help with refactoring (renames are propagated, especially in monorepos), readability/reviews (functions are qualified, you know where they're coming from) and overall coding experience – qualified module name followed by dot gives good autocompletion, imports look neat in larger projects etc. The codebase written like this resembles extended standard library. It also helps with solving problems by encouraging first principle thinking, bottom up coding that produces auditable codebase with shallow external dependencies etc.
These arguments aren't very convincing IMHO.
Using SNS as an example when it's neither micro nor a library but a service (and a huge abstraction over native push notifications, whereas most micro-libraries provide simple utilities that aren't very abstract), saying that complex libraries are harder to audit and hence a security risk (which should be a point in favor of micro-libraries that are small enough to audit in minutes), saying libraries might have large footprints (which is surely another reason to go for micro-libraries over all-you-could-possibly-need-libraries), saying transitive dependencies are bad, (yet again, this points towards an advantage of micro-libraries, which are less likely to have many dependencies), ... I don't know.
I think the JS library ecosystem is a debacle, but there's really only one point in this post that grabbed me:
"Would future updates be useful? No. The library is so simple that any change to the logic would be breaking, and it is already clear that there are no bugs."
Maybe what you want is a library ecosystem where things can be marked "this will never change". Something crazy happens and you actually need to update "is-number"? Rename it.
Of course, you can simulate that with a single large omnibus dependency that everyone can trust that pulls all these silly micro-libraries in verbatim.
> You can write isNumber(foo) instead of typeof foo === "number".
Indeed you can, but it depends what isNumber does. This is more like what it should do IMO:
function isNumber( foo ) { return ( (typeof foo === "number") && (foo == foo)) || ((typeof foo === 'object') && (foo instanceof Number) ); }
And that is I think the value of micro libs, at least in JS, you don't want to think about all the edge cases when you only want to check if something is a Number.
This library is a hilarious example of a huge problem with this kind of package. "Number" is in the eye of the beholder. A string containing numeric characters is, in my view, in no useful way "a number". A package that treats it as such just perpetuates weakly-typed nonsense.*
But the broader point is, you can't outsource understanding to a package. There will be places in your code where NaN is a perfectly valid number, or Infinity. And other places where you absolutely need to be sure neither of the above make their way in.
By pretending that a package can capture the universal essence of "numberless", and that this will broadly apply across the entire JS ecosystem (see reported benefits like "different libraries can all rely on is-number instead of rewriting duplicated helper functions!") is naive.
I wrote more about this in a post linked in a top level comment. The is-promise library is another great example.
* Personal pet theory is that the package author would have been embarrassed to publish a 1-line package, so included "numeric strings are numbers" as a fig leaf to justify the package's existence. They should have instead created two new packages, is-actual-number and is-numeric-string, so the implementation of is-number could be nice and clean:
I can feel the power of webscale coursing through memodule.exports = function(n) { return require('is-actual-number')(n) || require('is-numeric-string')(n); }This is an argument for having a library that provides that function, but it is not an argument that it should be a micro-library.
Doesn't that exclude NaN (which you probably want despite the name)? I think this really highlights that you probably do want to think about those edge cases...
In any case this is a bad example because Typescript exists.
Accepting NaN as a number can potentially crash your app, that's why I reject it in isNumber.
I only tried to highlight some edge cases that I personally don't like to spend energy on, trying to get it right, when writing code. Btw, isNumber is a dynamic call in the example and unrelated to TypeScript. TypeScript doesn't exist at runtime.
Reminds me of the 2ality blog post on that:
the entire nodejs ecosystem needs to die. You all are just keeping it alive.
Have fun convincing all major browser vendors to no longer support JS scripts :)
(At this point Nodejs is the defacto tooling ecosystem for even JS destined to run in a browser. You can't separate the two.)
You can start by ignoring it
It really depends on the case. Some folks use the left-pad library for aligning, which can be done in 10 minutes. In C++, we have header libraries for thread pools, etc. I don't think implementing a fully functional thread pool with waiting and other features is an easy task. In conclusion, it really depends on the situation.
No it doesn't. The answer to needing a string manipulation function would be to use a string manipulation library that include that function, not one that is nothing but that finction.
If you don't need anything else, and having the linker not include unused code isn't good enough, then just vendor the single function.
There could still be some special case but that will need a lot of explaining to justify and will be such an exception that it is silly to talk about. There are legitimate one time freak exceptions to every principle. It means nothing.
Partially disagree, JS has unique features that require small libraries: https://bower.sh/my-love-letter-to-front-end-web-development
However, if I can inline a small function, I will, so in that sense I agree.
> Javascript has a very unique set of challenges that differentiates itself from the rest of the programming world. The primary driving factor for its unique position is javascript is downloaded on the client's browser. Languages that run on the server, for example, don't need to send code that it runs onto client machines, they merely respond to requests made from the browser.
This is profoundly true. JavaScript written for the frontend has different "physics" to backend code.
It's not only code size that is significant. It's the fact that when you ship code over the wire to a client, you don't know what browser or even JS engine version will be interpreting it. Platform incompatibility has been a huge driver of issues in the JS/NPM ecosystem and has caused JS's culture to develop the way it has.
I wrote more about this, link in a top level comment.
Not checking dependency updates need to die already. Not choosing better dependencies needs to die already.
Thought experiment: if a LLM can correctly produce the code for a micro-library like, say, leftpad... Should you call leftpad as a dependency or should you have the LLM generate that leftpad function for you?
And if the LLM ain't good enough to write leftpad, how can I trust it to write anything at all?
Where do you draw the line: is 10 lines "micro"? 50? 100? It is never quantified within, yet the very click-bait-y title relies on the term. How many bugs can hide in 50 lines or 100? And you really want to copy-paste that code at a static point in time?!
I don't think they need to draw an exact line for us to know what they're talking about.
The fact that we're even debating it shows that they do. And I didn't ask for "an exact line". I asked for anything. What is your qualification for a "micro" library? I'm betting it's different from 9 other respondents. That's a problem.
Furthermore, that's not even the main contention I was highlighting. Without a proper definition, the advice of "just copy/paste these" is dangerous. Someone will draw their line at something too large, copy/paste that in and inherit bugs/vulnerabilities they never fix. That's a big problem.
At no point does this article attempt to define the term "micro-library".
Perhaps we should start there.
I have found that micro libraries are being replaced by ChatGPT generated functions.
I think discussions like this mostly miss the point.
Obviously you want basic, stable and well documented functionality in your programming language.
But JavaScript does simply not have it. So how do you solve this dilemma?
1) the everything is an import way: use NPM and create a dependency hell from hell (requires Satan) made by Lucifer (same as Satan but different) using lava with fire (requires node v <= 9.42.0815) and heat (deprecated) requiring brimstone (only node v > 10.23) with a cyclic dependency on the Devil (incompatible with Satan).
2) the Golang way: copy paste ALL the things, only for your co worker to copy paste all the things again, only for your co worker to copy paste all the tings again, only for your...
Way 1 wastes your time when it breaks (sooner than later) but is necessary for non trivial functionality. Way 2 works only for trivial packages so choose your poison.
JavaScript (apart from not being a good programming language in general) is sorely missing a std lib.
One could argue that having a bad std lib is even even worse (PHP anyone?) but it is really hard to decide.
Sadly JavaScript is just unfit for the purpose it is being used for.
This seems better articulated like this:
Applications should never have trivial, tiny libraries as moving-target external dependencies.
If you must use a small library, bring it into the program.
The bigger principle is that atomising everything into small parts separated from their context obfuscated them and the bigger picture.
So, everybody can contribute an npm package.
The advantage: - everybody can contribute an npm package
The disadvantage: - everybody can contribute an npm package
Why don't people just copypasta the library into their codebase with the license header intact and keep it out of npm
>Micro-libraries should never be used
Passive voice. WHO should never use micro-libraries?
You, specifically.
A little copying is better than a little dependency.
"The library may be a bad fit for your problem."
How is this the fault of the library? You chose the wrong one!
"This often cancels out the primary benefit of libraries. No, you don’t have to write the code, but you do have to adapt your problem to fit the library"
You evaluated the library, found is unsuitable and yet, it is somehow their fault.
Why on earth would you project your own failures on to someone else's code? You do you!
If we had perfect knowledge, we could choose perfect libraries. But we don't. Maybe someday.
While I mainly agree with the author's substantive point, though I find some of the ways it's presented in this post not entirely convincing or fair, I am interested that someone else has identified this:
> I have talked a lot about the costs of libraries, and I do hope people are more cautious about them. But there’s one factor I left out from my previous discussion. I think there’s one more reason why people use libraries: fear.
> Programmers are afraid of causing bugs. Afraid of making mistakes. Afraid of missing edge cases. Afraid that they won’t be able to understand how things work. In their fear they fall back on libraries. “Thank goodness someone else has solved the problem; surely I never would have been able to.”
I think this is true, but why does the JS ecosystem seem to have "more fear" than for example the Python ecosystem?
I wrote about this a while ago. I think that actually JS does (or did) cause more fear in its developers than other programming languages. I described it as paranoia, a more insidious uncertainty.
Quoting myself[1]:
> There are probably many contributing factors that have shaped NPM into what it is today. However, I assert that the underlying reason for the bizarre profusion of tiny, absurd-seeming one-liner packages on NPM is paranoia, caused by a unique combination of factors.
> Three factors have caused a widespread cultural paranoia among JavaScript developers. This has been inculcated over years. These factors are: JavaScript's weak dynamic type system; the diversity of runtimes JavaScript targets; and the physics of deploying software on the web.
...
> Over the years there has been rapid evolution in both frontend frameworks and backend JavaScript, high turnover in bundlers and best-practises. This has metastasized into a culture of uncertainty, an air of paranoia, and an extreme profusion of small packages. Reinventing the wheel can sometimes be good - but would you really bother doing it if you had to learn all the arcane bullshit of browser evolution, IE8 compatibility, implementation bugs, etc. ad infinitum?
> And it's not just that you don't understand how things work now, or how they used to work - but that they'll change in the future!
[1] https://listed.to/@crabmusket/14061/javascript-s-ecosystem-i...
I think the culture of JS has been reinforced over time, and the result is a novel form of paranoia. npm makes package-sharing easy, developers share trivial packages, people use trivial packages, people rationalize trivial packages, people teach beginners never to write code, beginners think they can never write code, beginners grow up and here we are.
Certainly the language is quirky, but it really doesn't change that much. Frameworks have come and gone but JavaScript itself is still the same. is-number would have looked much the same 15 years ago, if anyone was crazy enough to actually distribute it.
I agree, I think. The language issues, and NPM technical issues, are small contributing factors. I'd argue the culture was more a result of the physics of deploying code over the internet to varied and unreliable browsers than it was to do with anything NPM did.
> “Thank goodness someone else has solved the problem; surely I never would have been able to.”
No, it's much more mundane: "Thank goodness someone else has solved the problem because I surely as hell don't want to solve it myself because I don't have either time or brain power/will/motivation for that". What is a number is JS? I don't even want to start thinking about it, just give me an isNumber() function. Why is it not in the standard libary in the first place?
> why does the JS ecosystem seem to have "more fear" than for example the Python ecosystem?
Perhaps it's because so many JS developers - quite rightfully - suffer from impostor syndrome?
It's the language with the largest proportion of people who didn't set out to be programmers but somehow got mission-crept into becoming one.
My instinct would have been that this applies to Python! I'm only working from an anecdotal dataset of 1, a friend who works in insurance and is becoming necessarily more and more familiar with Python for data processing.
The title of the article is "Micro-libraries need to die already". Renaming the submission to "Micro-libraries should never be used" is pathetic, Daniel. I'm not surprised though.
TL;DR most micro libraries should be gists