Atomicwrites' old versions have been purged from PyPI
github.comThe whole package has now been deprecated by the maintainer:
'PyPI wants me to enable 2FA just because I maintain this package, and both that and the mess resulting from a stunt of mine, I thought it'd be a good time to deprecate this package. Python 3 has os.replace and os.rename which probably do well enough of a job for most usecases.'
https://github.com/untitaker/python-atomicwrites
Edit:
From the bug report
'I decided to deprecate this package. While I do regret to have deleted the package and did end up enabling 2FA, I think PyPI's sudden change in rules and bizarre behavior wrt package deletion doesn't make it worth my time to maintain Python software of this popularity for free. I'd rather just write code for fun and only worry about supply chain security when I'm actually paid to do so.'
I can see the maintainers point, even if it may be inconvenient.
That sounds like a best of both worlds. PyPI sets a minimum bar for developer responsibility and you can opt out of publishing to PyPI if you don't want to be that responsible.
The system works.
I wonder how people who maintain CI pipelines feels about it on monday if they're recalled from vacations because the pipelines broke.
they are being paid for being recalled from vacations while this developer is doing it for free... that was his point...
Oh I agree, my point was rather that the haphazard way of handling this by the PyPi folks.
This is a bizarrely emotional response to me. PyPI offered to provide a security key to make the maintainer's life easier so it's hard to see this as an "entitled" act. When I see the core infrastructure for open source software ecosystems improve I cheer that effort on.
While I am in full support of not asking too much of open source maintainers a cooperative stance makes the overall situation better for everyone involved. This could have been handled in a better way.
> PyPI offered to provide a security key to make the maintainer's life easier
It's even easier to just leave 2FA disabled and stop maintaining the project. Which is what they did.
Are maintainers obligated to support their projects indefinitely?
There’s a moral obligation to mitigate harm caused by your project.
I recently ran into a situation where a very old package caused terrible damage.
I contacted the pypi maintainer. He apologized and promised to fix it. Six months later, no changes.
This was a very unusual situation, as the package was the same name as a module later adopted in the standard library.
The author was under the impression the package was literally uninstallable since the code hadn’t been valid Python for over two decades, including the setup script.
Still wish they would delete it.
What was the license of this package?
I just checked - it doesn’t include any kind of license.
Note that that means you had no right to use it at all.
Not quite. When a package is submitted to PyPi, there’s legalese that says people can download and use it.
> If I upload Content other than under an Included License, then I grant the PSF and all other users of the web site an irrevocable, worldwide, royalty-free, nonexclusive license to reproduce, distribute, transmit, display, perform, and publish the Content, including in digital form.
The paragraph following that one is more important in this context:
> For the avoidance of doubt, my warranty or license as set forth above applies to the Content exactly in the form provided to the PSF, and does not, other than as may be provided in the Included License, grant the PSF or any users of the website a license to make or distribute derivative works based on my Content.
So yes, you have a right to download and run the package (which I didn't know about), but you do not have a right to bundle it with your software and distribute that.
It is even debatable if a package that explicitly depends on the initial one, even without bundling it, would be legal - I think it probably wouldn't be, since as long as it is dependent on that package, it is arguably a derivative work of that package, which the PSF terms of service do not authorize you to make.
It should go without saying, but IANAL. I'm basing my opinion on 3rd party dependency legal reviews that I've gone through at my company (from the software engineering side), and here using a package without an explicit license was explicitly prohibited.
Having a specific license, and not depending on a platforms default legalese is absolutely a best practice.
This package was installed by a novice programmer. It’s archaic (pre-git, pre-Python-2.5) and has no business being used by anyone, anywhere, for anything.
In which country?
Almost all of them. All rights are reserved by default without you having to do anything in most countries. Hence open source licenses.
I'm not a lawyer or a smart individual but oh boy that's a red flag, just like several other details you offered. With the information available, it sure seems like that's a dependency that should never be brought into a project.
Everyone agrees nobody should use this module. It’s archaic.
It was being installed by a brand new programmer.
Maybe some people use their side projects to develop software without the bureaucratic crap full time jobs have. And any amount of bureaucracy is too much for his free side project
So why publish to PyPI? I have tons of personal projects that never leave my laptop, or, at most, github.
I had a package which I didn't publish on PyPI, just my web site as a "if you break it, you get to keep the pieces" sort of thing. I didn't even have a PyPI account.
Someone else added it to PyPI without telling me. And people started using it from PyPI.
I started getting messages about it, like PyPI developers asking maintainers to upgrade package metadata to include if it supported Python 3. That's when I realized it was on PyPI in the first place.
I had to contact the original uploaded to get access to the account.
One user even emailed me a question and said I had an obligation to support it, since I put it on PyPI.
Damned if you do, damned if you don't.
I don't really see the problem. You put the code online and someone published it to PyPI. That you got what effectively amounts to spam emails because of that doesn't seem pertinent. Just block the emails. Unfortunately, putting out public communication addresses like emails does indeed invite all kinds of unwanted, unsolicited messaging.
Why not just ignore that like any other spam?
My point was that a personal project that you have on github could still be put onto PyPI by someone else, without you knowing. Even if you actively want to avoid PyPI.
What you want to do about it is a different topic.
Unlike most spam, I can't figure out how to select interesting email about my projects that I want to answer, from emails I don't want to read at all because they make my blood boil, like those asserting that because the project is on PyPI I'm obligated to help them.
It's rather moot now as I haven't gotten emails about it for 8-10 years.
Huh. As a supply-chain issue, is it important to PyPI that the person in charge of the PyPI entry be affiliated with the project, and share reputational risks should the PyPI packager add malware?
That seems like an interesting vector. Find a potentially useful Python package which isn't distributed via PyPI, add an entry using a new account which looks like it's part of the project, add malware, and upload.
> I had to contact the original uploaded to get access to the account.
That's your mistake there. Others would have left it alone or added a note somewhere.
But that begs the question: why doesn't pypi verify that uploader and developer coincide?
> Others would have left it alone or added a note somewhere.
I doubt I'm unique in this regard.
> why doesn't pypi verify that uploader and developer coincide?
How would that verification process work?
I have failed to find a PyPI requirement that they coincide.
It appears that if you have a public repo with a FOSS project but no PyPI entry then anyone is free to use your repo to create a PyPI entry. It's not quite namesquatting given that it's (at least at the start) the same code base.
I'm not sure if PyPI even allows a name transfer to you, if I read https://peps.python.org/pep-0541/#name-conflict-resolution-f... correctly:
] None of the following qualify for package name ownership transfer ... User A owns a project X outside the Package Index. User B creates a package under the name X on the Index. After some time, User A wants to publish project X on the Index but realizes name is taken.
EDIT: Someone who avoids submitting to PyPI because of philosophical objections to the PSF Code of Conduct appears to have no recourse should this happen, as the resolution process requires following the PSF Code of Conduct.
Forcing people to use a token that can be lost is not an improvement. This shit is going to hit the fan when Github turns on mandatory 2FA.
Well, you need more than one.
And I locked myself out of the first one while (before finishing!) setting up my second, so IMO you need more than two.
(It's not a great story, the tl;dr is I used a different passphrase for the second one, mixed them up, and ploughed through my 3 tries at the passphrase on my first one confident I was getting it right.
I also think that default (Yubikey's) of 3-tries is insanely low, getting it wrong just once is nerve-wracking; how much easier is it to brute-force in 30? That's more guesses of pet names et al. sure but you're not brute forcing it in that. Just don't use a pet name.)
From the GitHub README:
> PyPI wants me to enable 2FA just because I maintain this package, which I don't care for. So this package is now unmaintained.
Just set up a KeepassXC file and put your 2FA info in there? You don't need to give PyPI your phone info, PyPI takes TOTP[1]. 2FA is pretty normal; I don't see why the author has a problem with it. It doesn't violate privacy (since it's not actually tied to any PII like a phone number), it takes like 10 seconds to set up, and it protects your packages from hackers. Perhaps the author simply doesn't see the point of 2FA, since he implies the PyPI authors only did it for compliance reasons (and not for normal bolt-your-doors security reasons, which is more likely)?
He calls setting up 2FA "an expense of my free time" when surely it took more time for him to delete and re-add his package than it would have to just set up 2FA.
EDIT:
To be fair, the maintainer owes us nothing[2], sure. But it's not unreasonable to protect the larger community with basic security practices, either.
1: https://pypi.org/help/#twofa
2: https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...
>but his response feels more so
If we keep treating open source maintainers like they owe us anything, we will have fewer open source maintainers.
We can say the same thing about maintainers of PyPI. They host your libraries and serve it to anyone who wants, free of charge. The only thing they ask in return is to maintain a minimum level of security so that they have less headache in the future.
I think they also deserve some respect.
Yep. Both parties here are within their rights. It's the HN comment about entitlement of the maintainer that I was responding to.
I think both parties are within their rights, but I also think this is a stupid move on PyPI's part. Maintainers are already working for free; start making them jump through hoops and some will decide it's all too much work and leave.
I think it would be much better to throw up a warning (potentially a loud one) when a dependency is maintained by someone without 2FA.
Packages being taken over because of stolen credentials creates a maintenance nightmare, and bad publicity, for PyPI itself. As such, they have every right, and a reasonable need, to require 2FA. In contrast, the maintainers of PyPI don't lose too much if a few projects choose not to use the platform anymore. Remember, you're not paying PyPI anything either, so the fact that it may inconvenience your own projects, whether free or proprietary, is not their problem.
> we will have fewer open source maintainers.
That isn't necessarily a bad thing. I would be happy to lose every developer who is unwilling to enable 2FA. I am glad to see that that's what happened here. The developer has no responsibility to maintain their code, and PyPI has no responsibility to let them publish their code. Both sides discussed this and an agreement was reached - the developer will no longer publish their code to PyPI.
No one acted maliciously. Everyone wins.
That's fair, he owes us nothing[1]; I agree with that. But it's not unreasonable to protect the larger community with basic security practices, either.
1: https://gist.github.com/richhickey/1563cddea1002958f96e7ba95...
I am not objecting the 2FA deployment - it's a good idea. I am objecting the attitude towards maintainers which disagree - they have the right to disagree. They owe us nothing.
I can't blame them, I would have done the same. I assume their priorities are not aligned with pypi and have no incentive to jump through those hoops.
Or you just maintain it?
2FA hardly adds any security if you already use password manager with long random passwords.
This is clearly not true. Having a second factor helps maintain security in the situation where your password is compromised (phishing is just one scenario). It isn't perfect, and can itself be defeated. However, compromising an account with 2FA is demonstrably more difficult than one without.
While there are scenarios where 2FA can maintain security where a password is compromised, it's absolutely true that for a large swath of practical threat models, almost the entire benefit of 2FA comes in the form of assigning the shared secret instead of letting the user pick a weak and/or widely-reused password and the "having a second factor" bit doesn't really factor into the picture in any meaningful way.
These weaknesses are implementation specific. FIDO2/U2F is unphishable, requires proof of presence, and is a significant security win over a strong password.
Is PyPI requiring maintainers to use a hardware key? If not, I don’t understand how this policy is helpful.
Anyone who hadn’t already turned on 2FA is going to use the most frictionless so-called second factor they can.
They've been offering people hardware keys for free.
That's a great initiative, but I expect the maintainers who are interested to be the ones who've already turned on (some lesser form of) 2FA voluntarily.
Anyone who is turning on 2FA because of this requirement is going to select the most frictionless method of complying with the mandate. Which will not be a hardware key.
OK? So more people use TOTP and there's a marginal security win. And maybe a few use a token, and there's a significant win.
TOTP is a minuscule security win in exchange for a significant amount of inconvenience, versus using a good password manager. If you want to prevent password reuse, add an option to use a pre-generated password as an alternative to 2FA.
I think the statement "these weaknesses are implementation specific", while true, is irrelevant when 99% of people affected by this mandate (and 99.9% of 2FA users in general) are going to use an implementation with these weaknesses. And, I think it really sucks that PyPI is loosing maintainers due to a policy that won't increase security in a meaningful way.
The difference between a password manager and TOTP is that TOTP is something PyPI can enforce and a password manager is not.
Yes, TOTP adds very little advantage when you have an already safe password. But there is no way for PyPI to know if you're doing that or not, and they can know if you're using 2FA.
> 99% of people affected by this mandate (and 99.9% of 2FA users in general) are going to use an implementation with these weaknesses
Time will tell. PyPI is giving away free keys, presumably to encourage adoption of the safer option.
I'm actually very happy to see maintainers go. If they weren't willing to enable 2FA I worry about what other issues their software poses.
> Yes, TOTP adds very little advantage when you have an already safe password. But there is no way for PyPI to know if you're doing that or not.
But there is! Pre-generate a password for the user, instead of letting them supply one. This adds no extra inconvenience if you're already using a password manager, but it makes password reuse impossible.
So make a proposal for them to do that.
Okay, this is me proposing it! :)
I'm not the first person to come up with this idea. I think if PyPI had taken the approach of "here are our concerns, here is what we're protecting, here are some different options to make this work in the most convenient way possible", that would have been received very differently. Instead, they issued an ultimatum.
I think you'll find that in the course of making an actual formal proposal there are more caveats than you might imagine, and not necessarily as many upsides.
For example, just off the top of my head;
2FA as a general solution allows for U2F, which is significantly safer than your solution, even if it also allows for TOTP, which is "only" as safe as your solution.
It's also a change in user experience. Migrating users with existing passwords over to these new passwords may be more difficult for any number of reasons.
That's just a few seconds of thought. A real proposal might explicitly address these concerns. Like I said, feel free to make that proposal. A post on HN is not going to cut it, and it isn't going to help anyone. If you want to be more involved in how PyPI is maintained I suggest you do so.
I'm not calling it a weakness. I'm saying that the alleged advantages of the 2F in 2FA don't normally matter to people who just want to their shit to work.
If they can phish my password, they can trivially phish my OTP as well. The one thing I can see actually protecting against that is a physical hardware key, but that's a lot of extra inconvenience.
You know which modules I'm not using for my critical projects? Ones whose maintainers refuse to enable 2fa. We already know how supply chain security problems have plagued npm and pypi. Dependabot should alert you when your dependency comes from a package maintainer that doesn't use 2fa.
That's entirely reasonable. However, it is also reasonable for the author, who is working for free, to ignore your concerns.
I think it's completely insane to not use 2FA when available... but I also support the freedom to not maintain a piece of software unpaid. One person projects are pretty miserable.
I have ADHD. I lose things. I once had to restore access to a 2FA protected account I’d lost the token to. It took weeks of back-and-forth and involved sending personal information (selfies with identity cards) the service had no business knowing.
Never again. Especially for an unpaid personal project for which I owe nobody anything. If PyPI sent me this email, I’d immediately nuke all versions of all packages I maintain, replace with a blank/no code “upgrade” version that contains nothing but a readme explaining what happened, and close/deactivate my account.
I lose things incredibly often too(Like, losing my wallet twice and keys once, all within a 12 month period, going inside and leaving keys in the front door, needing GPS to get home 3 blocks away, etc).
If 2FA was token based as people seem to want it to be, I'd have an issue, but SMS based is enough to keep out the majority of opportunistic attackers while being recoverable. Plus, there's always printable recovery codes with Google at least.
> If 2FA was token based as people seem to want it to be, I'd have an issue, but SMS based is enough to keep out the majority of opportunistic attackers while being recoverable.
But so is using a long, unique, random password stored in a password manager! In fact, a strong password is more secure because it's not vulnerable to SIM swapping.
Admittedly, you could use both, but many/most services will let you use SMS for password recovery once it's set up, so it ends up becoming a single factor!
I'm also really nervous about loosing access to my phone number some day due to some screwup or other.
> Plus, there's always printable recovery codes with Google at least.
But I loose things. Especially slips of paper which I usually don't need to access. There is absolutely no way in hell I will be able to find a printout of backup codes when I actually need them.
Also got this letter of happiness. I don't mind 2FA, already had it set up. But PyPi is weird. I wanted to add a secondary 2FA device for backup, but they would not just let me do it. I had to download recovery codes first. But what am I going to do with them? Unlike 2FA tools there's no convenient way to store them. But because they insisted (and they really did by immediately asking me to burn one of them) I just saved them into a random file on my local disk. I suppose I could delete them, but I would rather not have gotten them in the first place.
PyPI identifies a package as critical and asks the maintainer to enable 2FA.. but allows them to simply delete the package to get around this requirement?
His right
I dunno, I think if you publish a copy of your code to a registry then it would be both desirable and reasonable for that copy to be immutable. Allowing the deletion of published libraries can have huge downstream impacts and ultimately makes the registry less trustworthy.
Edit: to be clear, not trying to shame the author here - it sounds like they tried to avoid this situation: "what i didn't consider is that this would delete old versions. those are apparently now gone and yet it's apparently not possible for me to re-upload them. i don't think that's sensible behavior by pypi, but either way i'm sorry about that."
I think this is a bad design on PyPI's part though.
If you want immutable, use a blockchain.
Life is not immutable. There could be a claim about IP, malware, whatever.
Versions should be immutable, but possible to delete
I agree. The logic is similar to why you can't delete an HN comment once someone replies.
Well sure. But what happens if your post contained some confidential data? It gets redacted
Yes, but you have to email the HN mods, so there's a form of review built into the process. It can't be done unilaterally.
Apparently when the 2fa requirement is actually implemented (this was just an announcement which triggered this) deleting a package would require 2fa as well.
Other registries go further and make it harder or impossible to delete once certain criteria are met (pretty sure this was put in place after leftpad broke the whole ecosystem): https://docs.npmjs.com/unpublishing-packages-from-the-regist...
I assume/ hope that this is PyPI's first step in rolling out mandatory 2FA? Otherwise the whole "you're critical so you have to enable it" seems a bit silly in that you're going to have developers who get critical decide they don't want to do this, and at that point pull packages/ stop maintaining.
Just having a 2FA requirement from the start (or some grace period like 7 days) seems like the way to do it.
Someone on Reddit [1] ran their own version [2] of the query PyPi used to make this determination. Over the last 6 months, atomicwrites was downloaded 38,497,903 times, good for just under #400 by rank.
[1] https://old.reddit.com/r/Python/comments/vuh41q/pypi_moves_t... [2] https://gist.github.com/jack1142/efe5c89b861a41616aaf8587838...