Announcing the data.gov archive
lil.law.harvard.eduGreat to see there's some resistance. What I'm missing from this announcement though is any mention of how they intend to secure this "vault" against the current government. I'm assuming good intentions on the part of Harvard, but keeping this data online against the express will of the government is gonna cost (political) capital. And from what I can see, the archive is hosted by US entities on US-controlled servers on US soil?
This is the same thing that's been bothering me with archive.org lately, by the way. I haven't found a good way to simply (for some reasonable definition definition of "simple") contribute 10 TiB or so of redundant storage on my (european) home server either. That kind of thing might (have to) serve to ensure tamper-resistance for that data, given the current political climate on both sides of the pond. Any pointers welcome.
> What I'm missing from this announcement though is any mention of how they intend to secure this "vault" against the current government.
Maybe this?
> In addition to the data collection, we are releasing open source software and documentation for replicating our work and creating similar repositories. With these tools, we aim not only to preserve knowledge ourselves but also to empower others to save and access the data that matters to them.
https://github.com/harvard-lil/data-vault
And since the data lives here: https://source.coop/repositories/harvard-lil/gov-data/descri...
Combined with this:
> To download an individual dataset by name you can construct its URL, such as:
> https://source.coop/harvard-lil/gov-data/collections/data_go...
> https://source.coop/harvard-lil/gov-data/metadata/data_gov/f...
> To download large numbers of files, we recommend the aws or rclone command line tools:
> aws s3 cp s3://us-west-2.opendata.source.coop/harvard-lil/gov-data/collections/data_gov/<name>/v1.zip --no-sign-request
So one could "easily" mirror the whole thing, making it distributed.
And 16 TB isn't what it used to be (though it is a big transfer). I'm hopeful that it's already being mirrored in multiple places including overseas.
I think a fully distributed storage system must be the way here. There must be some IPFS type system where Harvard could say "we designated a set of data that we can add to as needed but only delete from with a critical mass of storage providers' consent, here are some instructions for you to add your spare capacity to become a storage provider".
I was just thinking IPFS or possibly even some blockchain solution for this. Bram Cohen (the creator of bittorrent) also has a new project I think in the DePin sector.
Sensitive information just can't be hosted on a centralized server anymore, it has to be distributed for the good of the project.
This sounds an awful lot like Pied Piper!
It might not be ironclad, but the ease with which federal workers can fiddle with data they're hosting themselves vs. fiddling with data in Harvard's library is a pretty big difference. And if it ever came to demands for censorship, it wouldn't be Harvard Library's first rodeo.
>how they intend to secure this "vault" against the current government
Is there any risk of the government ordering them to take it down? That seems unlikely to me. The US has strong free speech protection, stronger than European free speech protection.
>keeping this data online against the express will of the government is gonna cost (political) capital.
Costing them political capital (aka the government is unhappy) is different from the government ordering them to take it down. Also, when you say "express will", are you saying the government has explicitly publicly stated that they don't like that Harvard is hosting this data?
> The US has strong free speech protection
The US literally just told (e.g., [0]) all scientists working for it they are not permitted to publish papers or speak at conferences or travel. What the US "has" is irrelevant when laws are being ignored.
[0] https://www.science.org/content/blog-post/revised-and-extend...
That's an employer telling employees what to do? Google can tell the same thing to it's employees and does. I don't see the relevance to the government telling other entities what to do.
The federal government isn't just "an employer", it's an extension of the public will funded with public money and should work toward the public good.
The topic is if the federal government is ignoring free speech laws in this specific instance. I'm arguing it's not.
As for public good, you'd probably find it depressing what the majority of Americans view as good.
The employer is the US federal government, in this case.
And employees signed contracts which allow the government to tell them how to do their jobs or fire them. Just like any other employer. Harvard did not.
Sure. But is it a good thing that their employer did this in this situation?
Is it a normal thing for the US government to do? Why or why not?
Do other government do this? Which ones and for what reasons?
> Is there any risk of the government ordering them to take it down? That seems unlikely to me. The US has strong free speech protection, stronger than European free speech protection.
In other words, you don't think we have to worry about Congress bringing in university presidents to grill them over political activities against the government's current policies occurring on their campuses? Certainly we wouldn't see any of those whose didn't give testimony that the government liked would end up being forced to resign in the aftermath; we're not in the dark ages anymore like we were in 2023...
I hadn't heard of that case before. However, some government officials saying that someone should resign is a bit different from the government actually forcing someone to resign or be fired.
The comment I replied to said "What I'm missing from this announcement though is any mention of how they intend to secure this "vault" against the current government.", which I believe is talking about a case where the government orders Harvard to take the info down. If Harvard will delete the data because a few government officials say they should (but don't order them to do so), then I don't see what can be done to secure the vault against the government. E.g. hosting it in Europe won't be any help, because Harvard could just delete it from the European hosting.
Whether the US still has this protection of free speech remains to be seen
> The US has strong free speech protection, stronger than European free speech protection
Citation needed. "European free speech protection" doesn't exist, each country has its own rules and freedoms. Hungary is drastically less free than e.g. France... but overall, as a rule of thumb, unless for you freedom of speech includes freedom to be a Nazi, European countries are pretty free. And don't take stances such as "spending money is free speech which is sacred so no campaign financing rules". You might get arrested for a Nazi salute though... which, if you think is a bad thing, you haven't been paying attention in history classes nor modern news.
The US values free speech higher than privacy. The EU values privacy higher than free speech.
>Opinions on the right to be forgotten differ greatly between the United States and EU countries. In the United States, accessibility, the right of free speech according to the First Amendment, and the "right to know" are typically favored over removing or increasing difficulty to access truthfully published information regarding individuals and corporations. Although the term "right to be forgotten" is a relatively new idea, the European Court of Justice legally solidified that the "right to be forgotten" is a human right when they ruled against Google in the Costeja case on May 13, 2014.[20]
>The European Union has been advocating for the delinkings requested by EU citizens to be implemented by Google not just in European versions of Google (as in google.co.uk, google.fr, etc.), but on google.com and other international subdomains. Regulators want delinkings to be implemented so that the law cannot be circumvented in any way.
https://en.wikipedia.org/wiki/Right_to_be_forgotten#European...
Right to be forgotten isn't related to free speech, it is indeed about privacy.
Free speech and "right to know" are different things. The second one is about transparency, and is very important... but as you say, the EU draws the border with "your rights stop where others' begin". We have a similar thing in many EU countries where people arrested/on trial still have a right to privacy until conviction, to avoid their names being forever plastered everywhere, with mugshots, based on nothing proven.
Right to be forgotten is related to free speech. It's a situation where one person's speech is at odds with another person's privacy. That's why this is mentioned many times in the article I linked.
The first paragraph I quoted earlier.
>Those who oppose the right worry about its effect on the right to freedom of expression and whether creating a right to be forgotten would result in a decreased quality of the Internet, censorship, and the rewriting of history.[5]
>Wikimedia is based in the United States, where the First Amendment protects freedom of speech and freedom of the press. In Germany, the law seeks to protect the name and likenesses of private persons from unwanted publicity.[63] On January 18, 2008, a court in Hamburg supported the personality rights of Werlé, which by German law includes removing his name from archive coverage of the case.[64]
>There is opposition to further recognition of the right to be forgotten in the United States as commentators argue[who?] that it will contravene the right to freedom of speech and freedom of expression, or will constitute censorship, thus potentially breaching peoples' constitutionally protected right to freedom of expression in the United States Constitution.[96] These criticisms are consistent with the proposal that the only information that can be removed by user's request is content that they themselves uploaded.[clarification needed][96][97]
>Major criticisms stem from the idea that the right to be forgotten would restrict the right to freedom of speech.[111][112][113] Many nations, and the United States in particular (with the First Amendment to the United States Constitution), have very strong domestic freedom of speech laws, which would be challenging to reconcile with the right to be forgotten.[114] Some academics see that only a limited form of the right to be forgotten would be reconcilable with US constitutional law; the right of an individual to delete data that he or she has personally submitted.[96][97][115] In this limited form of the right individuals could not have material removed that has been uploaded by others, as demanding the removal of information could constitute censorship and a reduction in the freedom of expression in many countries.[116] Sandra Coliver of the Open Society Justice Initiative argues that not all rights must be compatible and this conflict between the two rights is not detrimental to the survival of either.[117]
>There were concerns that the proposed General Data Protection Regulation would result in Google and other Internet search engines not producing neutral search results, but rather producing biased and patchy results, and compromising the integrity of Internet-based information.[119] To balance this criticism, the proposed General Data Protection Regulation included an exception "for the processing of personal data carried out solely for journalistic purposes or the purpose of artistic or literary expression in order to reconcile the right to the protection of personal data with the rules governing freedom of expression."[70] Article 80 upheld freedom of speech, and while not lessening obligations on data providers and social media sites, nevertheless due to the wide meaning of "journalistic purposes" allows more autonomy and reduces the amount of information that is necessary to be removed.[2]: 9 When Google agreed to implement the ruling, European Commission Vice-President Viviane Reding said, "The Court also made clear that journalistic work must not be touched; it is to be protected."[125] However, Google was criticized for taking down (by the Costeja precedent) a BBC News weblog post about Stan O'Neal by economics editor Robert Peston (eventually, Peston reported that his weblog post has remained findable in Google after all).[126][127] Despite these criticisms and Google's action, the company's CEO, Larry Page worries that the ruling will be "used by other governments that aren't as forward and progressive as Europe to do bad things", though has since distanced himself from that statement.[128] For example, pianist Dejan Lazic cited the "Right To Be Forgotten" in trying to remove a negative review about his performance from The Washington Post. He claimed that the critique was "defamatory, mean-spirited, opinionated, offensive and simply irrelevant for the arts".[129][130] and the St. Lawrence parish of the Roman Catholic church in Kutno, Poland asked Google to remove the Polish Wikipedia page about it,[131] (in Polish) without any allegations mentioned therein as of that date.
>Index on Censorship claimed that the Costeja ruling "allows individuals to complain to search engines about information they do not like with no legal oversight. This is akin to marching into a library and forcing it to pulp books. Although the ruling is intended for private individuals it opens the door to anyone who wants to whitewash their personal history… The Court's decision is a retrograde move that misunderstands the role and responsibility of search engines and the wider Internet. It should send chills down the spine of everyone in the European Union who believes in the crucial importance of free expression and freedom of information."[132]
>In 2014, the Gerry Hutch page on the English Wikipedia was among the first Wikipedia pages to be removed by several search engines' query results in the European Union.[133][134] The Daily Telegraph said, on 6 August 2014, that Wikipedia co-founder Jimmy Wales "described the EU's Right to be Forgotten as deeply immoral, as the organisation that operates the online encyclopedia warned the ruling will result in an Internet riddled with memory holes".[135]
> Volokh, Eugene (2000). "Freedom of Speech, Information Privacy, and the Troubling Implications of a Right to Stop People from Speaking about You".
>Portals: Freedom of speech
>Categories: Freedom of expression
> The US has strong free speech protection, stronger than European free speech protection.
For this to be said in the current circumstances shows either a political viewpoint or a lack of knowledge.
See this comment: https://news.ycombinator.com/item?id=42983898
Additionally:
* In some European countries newspapers cannot publish the names of people on trial. In the US they can.
* In some European countries you cannot take pictures of people in public and publish them. In the US you can.
What's the name of the law in Germany that puts you in jail for Holocaust denial?
From the announcement: "This work is made possible with support from the Filecoin Foundation for the Decentralized Web and the Rockefeller Brothers Fund."
> how they intend to secure this "vault" against the current government
Definitely a concern, if they want to harass Harvard and the other universities they could, but I don't think they'll bother. They know the data will be backed up, that's not the point.
Taking it off of data.gov accomplishes two things:
1) Makes it look like they're doing something, playing to the base. Easy to do
2) Delegitmize any insights the data might have. "Sure you have 'data', is it official data? I don't see it on data.gov. How do we know its not fraudulent?" It makes it harder to use it to justify policy changes. It adds one more tool to the denial crowd.
> ...how they intend to secure this "vault" against the current government.
Yup, I was about to ask whether Trump could still force them to delete what he doesn't like. Time will tell, I guess.
In general, no. By withholding federal funds, and with owning congress and scotus, yes.
I'll admit I'm not privy to US educational system etc, but isn't Harvard a private university ? Why would they be receiving significant federal funds in the first place ?
We subsidize everything here. If you're important enough that the government might want to influence your behavior it has very little legal authority to demand things. Instead, we encourage behavior with tax breaks, contracts, and grants. Then we add stipulations to said breaks/funding, threatening to revoke them should the business/org in question not comply.
It's kinda like passing laws that require certain behaviors from businesses and such, except also taxpayers get their money funneled into private orgs for the privilege.
"Federal student loans allow [private universities] to raise fees much higher than they otherwise would be able to charge, as do tuition tax credits and Pell Grants. The tax-deductible treatment of private donations helps fund new buildings. … State and local government exemption of facilities from property and sometimes sales taxes provide further assistance. The federal government hands out research grants, with generous (probably overly generous) provision for overhead expenses. Endowments are also advantaged enormously by tax privileges."
https://www.forbes.com/sites/richardvedder/2018/04/08/there-...
Research grants. That said, Harvard has the largest university endowment in the world, they'd be just fine operating at their current size off their endowment proceeds effectively forever.
For a while it was controversial whether they would even allow themselves to draw from the account itself versus merely funding things from interest.
To put that into perspective of what it means, conservatively they can spend $2.14B every year and never touch the principle of their $53.6B[1] endowment following the "4% rule".
[1]: https://www.forbes.com/sites/michaeltnietzel/2024/10/17/harv...
They already fund a fraction (1/3, I think) of their annual operating budget (>$6B) doing something similar to that. Ah, the article you linked says more or less that.
IIUC, The controversy was that some folks wanted to draw down the principal to do capital improvements (new buildings, etc) and other folks wanted to be as conservative as possible (potentially not even using the interest for operations).
Yeah, I'm not sure what Trump could really do here. Harvard gets < 1B in federal funds every year, mainly for research. Harvard also has a ~50B endowment. Short of sending in the military or confiscating Harvard's private funds, it seems like Harvard can wait him out.
With that said, the current administration appears to have a war on education in general so who knows what ideas they could cook up.
Quoting Trump directly, from Agenda 47:
> Furthermore, I will direct the Department of Justice to pursue federal civil rights cases against schools that continue to engage in racial discrimination. And schools that persist in explicit unlawful discrimination under the guise of equity will not only have their endowment taxed, but through budget reconciliation, I will advance a measure to have them fined up to the entire amount of their endowment.
Public data. License violations might cause some consternation but anything on data.gov is public and intended for the public. Deleting these are potentially legal violations requiring their publication.
If I remember correctly, Harvard has immunity to eminent domain under the Massachusetts constitution. Maybe it has a similar right which would make it immune to such attacks?
I beg you, please stop applying rule-of-law mindset against might-makes-right adversaries. It creates blind spots giving the illusion that the attack surface is way smaller than it actually is.
Muskolites are taking on the SSN system without any Congressional oversight as we speak. The President is attacking ius soli which is a Constitutional right. If they decide that sending their sleuths to Cambridge MA to physically destroy this data is in their best interest, they will do so and handle the courts later. Just stop pretending they will play by the book.
How do you recommend engaging with the situation then? Throw out the book too?
Take these new attack vectors into account in your threat analyses. In this specific case, perhaps start by shipping copies to non-US jurisdictions.
> Take these new attack vectors into account in your threat analyses.
Exactly. And while it might not be possible to protect against all the new vectors day one, no one should be surprised.
read curtis yarvin, find the weaknesses in his arguments, and use those weaknesses against their ideology.
for start, he thinks FDR was a dictator but the only reason FDR had such power was because the support of labor and some parts of capital. he would know that if he was a historian instead of LARPing as one.
so organizing and supporting the labor movement would be a good place to start, since a organized labor could become a powerful poltical force. otherwise everybody is fragmented on both sides which further enables their ideology.
a recent blog of his gave me this idea: https://graymirror.substack.com/p/the-pleasure-of-error
he legit thinks trump is about to be FDR 2.0 but his mistake is ignorance about how the power worked during that time. you can even use his "differential" idea in that essay against him. the differential now is labor against wealth. always has been, especially when FDR wielded all the power yarvin thinks was a dictatorship since a lot of that power was given to him by the support of the massive labor movement at the time.
more practically, this blog has some good insight into the "what do i dooooo?????" question https://wagingnonviolence.org/2024/11/10-things-to-do-if-tru...
On the technological side torrents, IPFS, and the like, urging international collaborators to make copies of everything public. On the political side mass protests, strikes, whistleblowing, general unrest that goes beyond mere social media activism.
Not OP but torrent would be my go to for a short term solution.
Torrent would do it but 16TB is still not that cheap to just have running in your house nas (otherwise I would).
Torrent would definitely be a great approach, 16TB is a commitment, but there's no problem with partial seeding also. I would choose selective chunks which consist of completed files.
European chiming in here. We need to backup, hash, and distribute this data. I bought four 12 TB HDDs (second hand enterprise) for about 360 EUR from USA (included tax & S&H). I could buy more, but I am feeling financially insecure now, and not buying anything from the USA anymore. I do have 3x 4 TB drives spare though. Might use these instead of going to some great parties.
You can download and share only selected parts of a torrent. With enough people you're going to get full coverage anyway.
Assuming that people randomly select the portions.
how do you coordinate who downloads which part?
Torrents track availability of pieces, you can seed ones with least availability if you want that.
Potential attack vector: seed excessive copied of a portion from many sock-puppets. Wait for non-sock-puppet seeds to dwindle. All the existing copies are now under your control.
Of course this is more of a job for responsible universities/libraries/research centres around the world, rather than single individuals. I would be surprised if Harvard didn't already go through their contact list to ensure as many copies are being made as possible.
Im predicting no more elections for you guys in 4 years. Something makes me think theres gonna be some "reason" to turn them off.
I'm not one of the "guys" as I am from the EU. There is a narrow way through for the US, but treating the incumbent as Just Another Guy is not that.
I am sure there will be elections. Question is how fair they will be. Have a look at Russia's recent presidential election. Putin "winning" it was a given from the start. US voters need to be mindful of all the ways an election can be rigged.
IMO US elections have already been less-than-fully-democratic for a while, what with the voting registration and ID nonsense, as well as obvious gerrymandering and voter intimidation. At the very least, significantly less democratic than a number of other western countries.
Let’s stop applying copium to recent events and making excuses. The American people knew exactly what they were getting and voted for him anyway. He won a majority of the popular vote.
The Senate is 2 Senators per state regardless of the population and isn’t as susceptible to gerrymandering.
That being said, I don’t feel bad for anyone who voted for Trump and is having buyers remorse.
We are already seeing it with Arabs in Minnesota who thought Trump would be better for Arabs in Gaza and are now having regrets seeing how Trump wants to use US troops to “clear out Gaza” and Latin Americans who are appalled that he tried to remove birthright citizenship.
Next up will be all of the “rural American” voters who are going to bare the brunt of inflationary tariffs.
He did not get a majority of the vote. He won with a plurality of 49.8%.
Does that somehow make you feel better about your fellow Americans? Yes I’m a born and bred US citizen.
The crap that Michelle Obama and the rest of the DNC spew about “going high when they go low” and “this is not who we are” is just that.
This has always been who we are.
This is exactly who we are. Andrew Jackson was the most popular president of the 19th century.
When national guard troops shot and killed student at Kent state university protesting Vietnam, 58% of Americans were in support of it.
Yes this is who we are.
And Jim Crow laws…
60% of the people in the south supported them and 60% of people nationwide were opposed to interracial marriages.
No it does not make me feel better. I’m rather disgusted by 70 million of my fellow Americans, including a few family members and newly former friends. However, I think it’s important that we pushback on the idea that the fascists have an overwhelming mandate to do what they are now doing. The truth still matters.
Putin's meddling all over the place including here.
To the extent that he has a role in this (certainly has publically claimed to!), he will in no way seek to make America a powerful allied country to Russia. There's no reason to assume he intends Trump or even Musk to actually be powerful in their own right.
Especially Musk. Musk must be mad to put himself in the position he's in. The only reason Putin would allow Musk to do what he's doing, is because it's patently obvious that Musk is undermining his own potential political base in every way, with every action. A widely hated man who's gone full Bond villain is no threat to Putin, politically. He's setting himself up to be demonized (Musk is) and doesn't even seem to register it.
also, many of musk's actions are illegal. Which means he is now owned by Trump. Play nice or go to jail. For a long time.
Don't worry! There are a lot of fine patriots with firearms that keep them just in case of such a thing!
My current thinking for 2028/29:
- 20% likelihood of no election at all
- 70% "managed democracy" with outcome decided in advance
- 10% normal transfer of power if Trump or his successor loses
I think they're racing against the mid-terms next year. If they can lock it in by then then they're home free.
(disclosure: Brit not USian)
> I think they're racing against the mid-terms next year.
This is the race. Historically the party in power loses the mid-terms, and I think we're already seeing huge portions of the population having buyers remorse. But, Trumps team knows this. I was waiting for them to attack the FEC, and here we are.
https://www.nytimes.com/2025/02/06/us/politics/federal-elect...
That may be so, but given what Trump and Musk have been up to, the situation of the courts, and how they blatantly don't give a f*k about what's constitutional or not, I wouldn't rely on this so-called "immunity".
I really hope I am wrong, but I'm planning on seeing some headlines about Musk shutting down Harvard next week over this.
For anyone who still thinks the existing laws, constitutions and policies mean anything to this current regime, prepare to get some whiplash. They are proving that none of that matters if they simply ignore it and do what they want to do anyway.
It wouldn't be the first time that Musk has called for Harvard to be defunded. [0]
[0] https://www.businessinsider.com/elon-musk-calling-for-harvar...
Fourth paragraph
>I'm assuming good intentions on the part of Harvard, but keeping this data online against the express will of the government is gonna cost (political) capital.
Harvard like other liberal institutions has little to no political capital in a Republican white house in the first place. Why would it cost them any to host data that is in the public domain?
This sort of silly overreaction is part of why people voted for Trump: it is genuinely funny to see people overreact to Trump because theyve been told by the legacy media that he is a "threat to democracy" over and over.
Last time he was elected his government also removed a bunch of climate data from govt websites which was quickly mirrored by third parties. Nobody was taken away by the Gestapo then and there is no reason to think things will be any different this time.
> Last time he was elected his government
I'm not sure pointing out 'last time' is such a great idea considering it ended with a violent insurrection attempt which I watch in real time. I don't need the news or anyone else to tell me that a threat to democracy is exactly what he is.
Maybe. He's already shipping illegal immigrants to a new concentration camp at Guantanamo bay.
> Nobody was taken away by the Gestapo...
https://www.vanityfair.com/news/2020/07/trump-administration...
> Nobody was taken away by the Gestapo
Someone forgot the "kids in cages" episodes in 2018 when Trump was only testing the waters...
EDIT: https://en.wikipedia.org/wiki/Trump_administration_family_se...
https://www.aclu.org/news/smart-justice/president-obama-want...
Nothing to do with Trump. Also Trump's policy was completely sensible, and no different to what happens when adults are sent to prison anywhere else: you cant put their kids in prison with them obviously.
That’s exactly who Trump is and what he wants, as a tool of cruelty to startle others.
You’re denying evidence. You’re deflecting. It’s documented in the press and in the legal proceedings that it was Trump policy.
You’re deflecting too by stating a different matter. Here kids were grouped and put in cages, from all ages, without relatives, without adults other than the guards.
We know deflection is part of the strategy… you’re not fooling anyone.
Someone didn't like to be reminded of things. :)
Wow, amazing. There _are_ people that do not want history to be documented and reminded...
For the record, my previous comments being upvoted, then largely downvoted, for a matter of facts.
Is anyone out there archiving USGS/NOAA datasets ? It sounds ridiculous, but this appears to be where we are now. There is a submission about NOAA on the frontpage now: "Scientists on alert as NOAA restricts contact with foreign nationals" [1]
ArchiveTeam is working on everything US-Government and you can help https://tracker.archiveteam.org/
I find it assuming that the might of the American government -- in trying to take a bunch of data offline -- is being resisted by a digital "militia" of hobbyist archivers and non profits.
Theres something that about this that just rings second amendment. Personally I think the concept of civilians having weapons to be a check on a nation state is absurd, but in this case it feels pretty empowering.
Well I wouldn't really call it the "American Government" per say... Its a Geriatric former reality TV show host elected to the presidency by offering to do for America what he did for steak or private education. That guy and his cronies really aren't the American Government. They were just elected to be in charge of the American Government.
Larping the larpers.
Will does this include all USGS data?
This is a topic that came up at work today as we rely on this data and are considering backing up most of the Lidar data from there ourselves (100s of TB probably)
EDIT: no, looks like it is only the footprints
Very happy this is happening. There's a ridiculous amount of incredibly valuable data, scientific documents, etc. "out there" that are at risk.
I haven't had much time to look at this yet and see what all is there, but whether currently included or not, a couple of things I really hope get archived are the contents of the DTIC (Defense Technical Information Center) document repository (lots of really interesting older scientific publications) and the NASA TRS (Technical Report Server).
I'm working on my own archive of at least some portion of the DTIC stuff just to be on the safe side. So far everything I've tried to access is still there, but who knows how long that will last.
Honestly a shame it has to come to this. Sure, people elected this administration and I guess with that comes with a bunch things I disagree with. But the removal of years of scientific research and data from the web (paid for by citizens with their taxes) is absolutely unacceptable. Ravaging CDC data, climate data, etc is horrendous and unforgivable.
It's today's equivalent of book burnings.
"where they burn books, they will ultimately burn people as well."
Those who delete research will ultimately delete people as well.
this archive is going to disappear before the summer comes
From the post: Today we released our archive of data.gov on Source Cooperative. The 16TB collection includes over 311,000 datasets harvested during 2024 and 2025, a complete archive of federal public datasets linked by data.gov. It will be updated daily as new datasets are added to data.gov. This is the first release in our new data vault project to preserve and authenticate vital public datasets for academic research, policymaking, and public use.