Show HN: Pgsh – branch Postgres like Git
github.comThis looks really handy. Quite frequently I want to iterate on a migration without having to write the "down" migration until the "up" migration is done. This will save a lot of DB dropping and re-seeding.
Just a note for anyone trying to download this -- please first try `pgsh@0.10.1`. The `.2` bugfix release caused a regression that I'm working through now. Thanks :)
0.10.6 is released with a fix :-)
Kinda like aquameta[0]?
Nothing like aquameta, but I guess the title could apply to both...
I installed this, just to see:
"added 242 packages from 179 contributors and audited 1098 packages in 13.598s"
how can I trust that this doesn't contain malicious code?
Such is life in the npm world, unfortunately.
I've tried to be minimal with my dependencies while providing a rich user experience, but I'm sure we could do better still. Any dependencies in particular you'd like me to look into removing?
> Such is life in the npm world, unfortunately.
I don't really think npm is to blame in this specific case. It's merely acting as an index to a set of tarballs, e.g. the same way PyPI or crates.io do.
It would be fairer to point the blame at the ecosystem and mindset of developers of popular JavaScript libraries.
I've recently been taking my time evaluating dependencies for a few small node projects, and I've actually found I've been impressed at how few dependencies are used by a subset of the Javascript package ecosystem.
I find, anecdotally, that rather than the broad spectrum you might expect, packages tend to either have a huge number of dependencies, or very few. Meaning if you do want to slim down your project's dependencies, it's usually simply a matter of identifying the whales and substituting/eliminating them with something saner.
Take for example two of the most popular package bundlers:
- parcel: direct dependencies: 59, total dependencies: 1575
- rollup: direct dependencies: 3, total dependencies: 9
Completely agreed. If auditibility of your dependency tree is something you're after, npmjs.com makes it pretty easy to find particular packages that also care about that, keeping your dependency list as small as possible.
The downside is that the most popular libraries and frameworks do not really do this; babel being a particularly egregious example, but I understand why they choose that path even if it isn't the choice I would make!
It’s actually not that straightforward. A library could bundle their dependencies as part of their own build step/publish. They would appear to have zero/less dependencies, but really they just bundled them.
Possible but I haven't seen a library that does this. Seems like a very unusual way to do things.
Something to worry about if it ever became prevalent I guess, but doesn't seem to be rn.
knex seems to be the big offender here: 277 sub-dependencies.
Other than that, I have to commend you on the rest of your dependency management; most of the rest of your chosen dependencies have between 0 and 3 sub-dependencies.
Sorry, I have no advice to give, the whole npm dependency nightmare horrifies me. Totally understand that this is not a problem of your making, though.
I guess you can use packages that use minimal dependencies, and pin those to specific versions.
Maybe look to convert to a language that doesn't do this? I realise that's a tough call, though.
And this is why I refuse to use any npm or similar code. Period.
I was looking at a webpack-based project I'm working on the other day; one of the packages was literally just a single line exporting a regex literal.
Well the obvious way is to look through every line of code.
The package uses functions from multiple packages, if they were to instead write them all themselves you may end up with 50,000 lines of code.
It's basically just split up amongst a bunch of different folders and files with a bunch of extra "garbage" and 99% unused code.
So if you don't trust it, read it all.
But at some point you got to trust something.
The part you leave out of that explanation is that for those files and folders there are 179 authors to trust for all future changes (including adding more authors via granting access to their repo or adding more deps).
Sure, you can do locking, but that does not go deep well, and also turns into a hell of trying to determine if every (for your use-case) pointless release of a sub-dep is worth updating to.
This. am I really expected to review every change that 179 authors make to 242 packages?
And if I don't, am I responsible for the malicious code insertion, or is NPM going to take responsibility for that?
You are responsible for malicious code insertion either way.
If you delegate trust in any way, you are responsible for how that trust was (mis)used.
This is why the JavaScript ecosystem is broken as a whole
You might find this interesting: https://news.ycombinator.com/item?id=21515725
I'm pretty sure WebAssembly is just a friendly name for the next government spying program.
How do you normally trust something?
PostgreSQL has 1257 source files [1], 1,250,000 lines of code [2], and ~400 authors [3].
So...however you'd normally audit that.
Probably look at the number of users, check for CVEs, and cross your fingers.
[1] find . -name '*.c' | wc -l [2] find . -name '*.c' | xargs cat | wc -l [3] git log | grep -o 'Author: \S\+ \S\+' | sort -u | wc -lPostgreSQL is one project.
These dependencies are 242 projects, privately managed and unvetted.
I can choose to "trust" the PostgreSQL project, as it's a mature project with high visibility. I can't practically choose to "trust" those 242 projects and gauge the trustworthiness of 179 separate contributors; doing so would take vastly more time than most people have.
Dependency bloat is a huge and frustrating problem, especially for people who want to know where their code is coming from and don't want to have to decide to trust hundreds of people for a random cli tool.
good point. Oh wait, no, that's total bullshit.
So, firstly, Postgres takes responsibility for contributions. The fact that someone has submitted a patch or PR is no guarantee that it has been accepted.
Secondly, they combine all those contributors to release specific versions with specific features. I can trust that someone at postgres has reviewed all those contributions to ensure they make the grade and included them in the release for a reason.
Thirdly, Postgres as an institution takes responsibility for the code it releases. If I find something messed up in a postgres release, and report it to postgres, they will take responsibility for that and manage the rest of it.
None of this is true for NPM.
> If I find something messed up in a postgres release, and report it to postgres, they will take responsibility for that and manage the rest of it.
> None of this is true for NPM.
Not as it pertains to bugs, but as it pertains to malicious code (the grandparent's actual question), npm accepts and acts on reports. In fact, their CLI even reports security vulnerabilities in your packages, a feature lacking in almost every other package manager.
If all code were in on source code repo, that all authors contributed too....is that really so big a difference as splitting our the areas of responsibility? What are the chances that Tom Lane looks at every single line of every single patch?