Lawsuit claims Ancestry.com gets preferential access to New York public archives
buzzfeednews.comIf you read the article, Reclaim The Records filed a FOIL (NY state version of FOIA) request for the records, which were stored on microfilm. The state quoted $150k for the request -- you still need to pay to have copies made. The group thought the state's quote was too large. Then Ancestry.com comes in, pays for the microfilm to be digitized, and the state tells the group "nevermind, we can do it for free now."
And this is the evidence that bad things are going on, somehow. The records group got the copies faster than if Ancestry.com hadn't gotten involved, and didn't have to pay for the.
Hi, I’m Brooke Schreier Ganz, founder and president of Reclaim The Records. Your comment is misleading as to the substance of this lawsuit and the issue at hand. I would urge you to read this backstory, and the actual lawsuit text:
https://www.reclaimtherecords.org/freedom-of-information-req...
and
https://us11.campaign-archive.com/?u=5f700fdc65a51d3813e67da...
The short version: we had many onerous requirements put upon us by the New York State Department of Health when we tried to get the records through FOIL, only ONE of which was a crazy-inflated $152,000 price tag that had nothing to do with the actual cost of digitizing the microfiche. The state made up the price. This is illegal under FOIL’s requirement that a requestor pay only the actual costs of digitization. In the links above, you can read the story of how we on our own calculated the actual costs, and provided multiple price comparisons from commercial firms and from the National Archives (NARA), which were a tiny fraction of that inflated estimate.
Getting the records for free after a seventeen month long fight was unexpectedly awesome. But apparently-preferential treatment by government agencies for commercial entities instead of non-profit or individual records requestors is really disturbing. We’re trying to figure out what happened and why. And that’s where this lawsuit, for copies of agency e-mails and contracts and meeting notes and such, is important.
Is it possible that Ancestry's secret sauce was simply threatening legal action?
A couple of years ago, I was trying to get some public records from the University of North Carolina. I submitted a request by email, as their website said I could. They responded a week later and said they needed a letter. I sent a letter. They responded to the letter and said that the letter needed to include my cell phone number and email address to process the request. (There was no mention of this before. Records could have easily been mailed to me at my address, and they already had my email address from the aforementioned email incident.) So, I provided the info. I heard nothing for months. I asked for a status update. They said the request was "delayed." I heard nothing for months. I followed up, told them they were in violation of North Carolina state law, and that if they weren't going to provide the records, I'd pursue legal action. Boom, got the records within a couple of days. Like magic.
Also, during my years as a state employee, I noticed that one of the few things that could get the lumbering bureaucracy moving (as an entity, there were plenty of hard-working well-meaning employees) was a threat of a lawsuit or a fine handed down by the feds. The institutional terror around that sort of thing was almost comical at times.
Hi Brooke, one thing that isn't clear from reading the links you provided, did Ancestry pay the fee requested by the state? From what I can tell, you refused to pay the fee (rightly or wrongly, I have no ability to assess the cost quoted to you). But if Ancestry paid it, would that not explain why their request was fulfilled first?
Hi! That’s exactly what we’d like to know! Our lawsuit is asking the New York State Department of Health for copies of “all correspondence, e-mails, proposals, drafts, notes, agreements, contracts, meetings and calendar entries, phone logs, meeting minutes, budget items, receipts, vendorization forms or data, bids, evaluation materials, Freedom of Information Law (FOIL) records requests and their associated correspondence and any appeals, and any other documentation or communications between the New York State Department of Health and Ancestry.com, or such materials within the New York State Department of Health about Ancestry.com."
Basically, we’re asking NYS DOH “what happened here?”
So here’s the thing: under New York’s law, either the state can quote you the actual costs of making the copies and you then pay it (paper photocopies are capped at $0.25/page, other media must be actual costs of duplication) OR the requestor can physically make the copies themselves, onsite, during business hours. Using methods of duplication like a cellphone camera (flash turned off and battery-powered) has been held by the state Committee on Open Government (COOG) in their Advisory Opinions on the law, to be totally free.
So initially we asked the state for a cost estimate for these microfiche and they pulled one out of their ass, $152,000 for what should have been a few thousand dollars at most (see lawsuit text for the estimate calculations). OR we could just make the microfiche scans ourselves onsite, and that should have been free.
So we were willing to do option B, make the copies. But the state made us jump through all kinds of hoops to prove that the magical microfiche machine was high quality, that we would do it hand-fed instead of sheet-fed, that we work around their employees’ schedule, and so on. We said yes to all of that.
And then blammo, 800 lb gorilla company gets the records first, literally using our own words to do it.
So either Ancestry paid the fee, or Ancestry took Option B, as we were going to do, and did the work themselves. As per their PR Department in this BuzzFeed article, they did the work themselves.
But then they’re a for-profit company, acting as a de-facto vendor for the state. So shouldn’t there be a contract for the work?
Folks - this is what makes government nervous. You have irreplacable records, and then a group comes in and considers all the rules "stupid" and "silly" and calls the microfiche machine "magical".
I don't think folks realize how risk averse government is. Literally, they probably use ONE brand of machine they KNOW works to deal with these records that go back over 100+ years. This is not some big conspiracy, this is literally how government works.
They've got an outfit come in, no budget, unwilling to pay for anything, can't be bothered to do things right. And then they get a corporation in like ancestry, probably can point to tons of experience doing exactly this, with lots of references, with deep enough pockets that if they screw up they can be persued to make it right.
Seriously, I've contracted with the goverment, the references thing alone is 100% how they operate, they want to see that you've done this same thing you will do for them 3x without problems for other agencies they can call up.
Seriously, if you did a high quality product, insured, check all the boxes, are totally 100% on all the "stupid" paperwork, the govt will fall over in love with you and you will have more work then you can handle, especially if you do it for free, and have 3 other places you've done it for who like your work.
You seem to be under the mistaken impression that we are a company or were bidding for a government contract. We are not and were not: governments are required to provide copies of public records like these, or else required to allow us access.
We’ve sued and won settlements in Freedom of Information Lawsuits like this one three times in the past four years (and won attorneys fees twice!), and currently have five more cases pending in various jurisdictions, both state and federal. Your apparent experiences as a paid or contracted vendor are not quite analogous to this situation.
You seem to be under the mistaken impression, that burdened under 100's (literally) of rules, the govt can do a one off special project like this for $3,000. Seriously, I've fought the goverment too, and won, on some stupid requirements that were explicitly revoked at higher levels of govt but hadn't filtered down.
That one issue (went through chain to their legal team, and back a few times) cost more than $3,000 of the governments time.
You are constantly making the claim that the government is charging 50x what it should cost. You are wrong. Period. I've paid a relative fortune to have the government run a select query on a database. And they still messed it up, I could probably have run it in 30 seconds.
If scanning records is something you do, then I would STRONGLY suggest rather than trying to get the government to figure out how to scan records, you, without calling the rules silly, develop some experience scanning the records yourself. If you offer to scan, with experience, insurance, good equipment etc and make the stuff available for free - you will be welcomed in many places.
Now you are suing because they gave you the records free that someone else either paid the govt to prepare (unlikely - that would have been a SLOW process) or prepared themselves and gave a copy to the government (which was probably not required)? I find that pathetic. NY did a good thing by having the first requestor make available a copy for others.
With reference to the last paragraph, the point is that nobody seems to know exactly what happened here and something seems a bit fishy. Isn’t it reasonable to follow this up and find out what happened?
> But then they’re a for-profit company, acting as a de-facto vendor for the state. So shouldn’t there be a contract for the work?
I assume that you are familiar with the contracts NARA has entered into with its "digitization partners"?
Oh yes. The ones where the records are supposed to become public (instead of “exclusive”) five years later, but in practice have not been made public...
Does it matter? The point is that the state made the cost prohibitively high.
It matters for "does Ancestry.com get preferential access" if they did, in fact, jump through all the same illegal hoops and just had the money to pay for them
In that circumstance, the state was able to grossly overcharge a private company for access to public records. I imagine that would also be worthwhile information to both that company's shareholders and voters in the state.
How on earth are there so many people on HN defending this? It's clear that _something_ untoward happened, and somehow the bad actor is not-for-profit archivists?
"grossly overcharge"?
Seriously - these tech companies have burn rates in the millions PER MONTH. I frankly wouldn't be surprised if ancestrery is doing over $1B per year.
Please note - these are records that CAN NOT be lost by the state. Some shmucks who can't even afford $100K want to take possession of this stuff? Forget it. Seriously, the van crashed on the way to the cheap place that scans these, the van burned up on the way to our "super cheap" scanning place - is no excuse to permanently lose irreplacable records. That's how half these startups and fly by night groups work.
I've often looked at these quotes. The cost to government, with salaries, pensions, supervisors and admin to do something it doesn't normally do, make sure it is safe, get in the scanners, do the quality control etc is higher than you think.
Just to do the RFP is a HUGE PAIN in goverment. You have to write it, get it approved, publicize it, get bids, find a scoring panel, score them, check they meet all the city and goverment requirements (redwood purchasing, mcbride principles for northern ireland, health care accountability - the list goes on). Then do all the adjustments for preference groups (now mostly local and micro biz and small biz, and WME etc). Then you have to fight the appeals. Every politicans rule drives the cost UP in govt. Voted in by the people. By the time you are at contract stage, getting city attorney approval etc - you are burnt out. Trust me, if someone comes knocking and wants all this for free - forget it.
If ancestry paid, then the real issue is these schmucks got preferential treatment by not paying and free-riding off the work the company paid for. And this is considered preferential treatment for ancestry?
I'm not fan of them (and don't use them), but folks, get a grip. If one requester has to pay $150K or insure, demonstrate prior experience doing this etc etc, and another get's it free because the first paid, that is NOT preferential treatment for the first who paid. What planet are these guys on?
Seriously - these tech companies have burn rates in the millions PER MONTH. I frankly wouldn't be surprised if ancestrery is doing over $1B per year.
This is not about how much money "ancestrery" has to burn. I have also bid government jobs; almost none of the expenses you mention have anything to do with safely handling vital records. In fact, they are inefficiencies that the government is not legally permitted to pass along to records requestors.
I'm not sure what your dog in this fight is, but you clearly have one. I would certainly be curious to know what would make you think that records required to be legally accessible to the public should only be available to those with burn rates in the millions PER MONTH.
Burn rates of tech companies are irrelevant. The question isn't whether or not ancestry paid the 150k but rather if it was appropriate for NY to charge 150k for preliminary access to these records.
If they are asking NY to do the work, then yes, that price sounds fair. And this illustrates the problem here, they have no clue.
What they are asking is for NY to bid this, fight the bid appeals, contract this, manage this, supervise this, etc. Govt does not do well with new stuff, and everything goes through the big bureaucracy.
And vendor side, all the requirements (the forced source hiring so you can't use your own staff, but have to wait forever for someone to send you some folks for the "entry level" positions you are required to fill from the pool they have, plus a billion political football things that you need to jump through that have no real impact on the actual project etc) means the bids are high.
As someone who has worked both sides of this - the lack of knowledge here is shocking.
I'm not in NY, but looked up just the fringe overhead rate - 62%. That's probably double the private sector as a point of reference.
So to digitize 100 years of irreplacable records, be on-site, probably have to offer to leave the machine and equipment, work around a potentially unionized workforce (trust me - no fun). That get's costly.
If Ancestry shows up with insurance, money, references and experience doing this, they definitely would have moved to front of line.
People also don't realize that most of the gadflys involved in bothers government wear everyone out there. There are tons of overblown drama from folks with no budget who are totally convinced the state is out to get them. That makes for some very jaded govt workers.
> If they are asking NY to do the work, then yes, that price sounds fair.
What exactly does the work entail? You appear to be speaking from a place of knowledge, even though you got the basic facts of the case incorrect as posted above.
Actually, I can kinda see his argument.
They have this data which is not digitized. Someone requests the data in a digitized form.
So, they go down their normal process for digitizing. They have certain standards for digitizing, and it hits the minimum threshold for going to tender (Maybe $10k).
It's a government organisation so they have to create a tender which organisations would bid for, and everything has to be certified to correct standards, insured, etc.
Reclaim The Records might be able to do it cost for $3k, but it still has to go to tender, and Reclaim the Records would have to prove they can meet all the requirements and such, which would probably quickly increase the cost.
Once it goes to tender, all the costs are waaaay higher, because you have to cover the costs of bidding and such.
But it doesn't matter what their "normal process" for digitizing is, because there's a law that says that they have to provide the requested information and restricts the kinds of expenses they can pass on to them.
So, sure, their "normal process" would say, "If this is to get digitized, we need to make sure that it will go into our existing system of digitized records, and it has to meet these standards of quality, and it has to have three people review it to make sure the OCR got done properly" etc etc.
But that's what it would take to get it into their system. Not what it would take to provide the records to the requestor.
If the requirements of the law can be satisfied and the requestor is happy to accept a simple scan costing a few thousand, the state doesn't get to say "we'll only do it by our full six-month six-figure process."
ancestry might not have gotten preferential treatment specifically but generally corporations did because charging that amount is a gate to keep out people who can't afford it
Which, to be clear, isn't a bad thing. You can't have randos coming in and ruining your documents. There must be an element of "are you seriously sure you can treat these things with care?" and 150k isn't a lot for a serious document digitizing entity.
The law is very clear: either the state hires an approved vendor, but only charges the actual costs of duplication plus hard drive or other media (no mark-up allowed), or the state allows the requestor to digitize the files themselves onsite for free. This is long-established law in most states, and more than forty years old in New York.
The state’s quote of $152,000 was more than fifty times the actual costs of digitization. We provided them with a breakdown of the costs, using both commercial and government sources as comparisons, even though that is supposed to be the state’s job, not the requestor’s.
The state still denied our request and our appeal.
So, no. Inflating estimates of the costs of public records by fifty times is flat-out illegal.
Do you have a detailed cost estimate?
New York government entities are required to use preferred source vendors (usually workshops for disabled people) for certain items when an item or service is available, including scanning. Preferred source vendors aren’t competitively bid and must be used by law.
When I was in school, paper copies of fiche from the state archives cost like $1-1.50 per fiche.
If you do some googling you can find various descriptions of issues, including cost, associated with that in the context of scanning.
”Do you have a detailed cost estimate?”
Indeed we do! Several pages, including methodology. Here you go: https://www.reclaimtherecords.org/wp-content/uploads/2015/10...
Just a quick sample: ”By way of background, our research would indicate the industry standard for fees for a job of this size – turning existing microfiche sheets into scanned digital images – would be approximately $3,000, not $152,000. This would indicate that DOH’s requested fees to process this FOIL request are nearly fifty times higher than actual costs...”
I admire your mission and understand your approach.
My only comment for what it's worth is that you may want to pursue a different path for these historical records. Using FOIL to access archival records is not a cost-efficient strategy for any stakeholder.
In my opinion, as an observer with no skin in the game is that investing the legal resources you have and working with someone like the library association to lobby the legislature to fund digitization would perhaps be a more productive avenue.
When I look at your website, and find the headline "CORRUPTION'S SUCH AN OLD SONG THAT WE CAN SING ALONG IN HARMONY, AND NOWHERE IS IT STRONGER THAN IN ALBANY" with respect to the city clerk office in a tiny place like Albany, NY. I take pause when I see that.
A handful of old ladies processing dog licenses in a broke upstate city isn't clutching to old death records to help big companies, particularly for data they may not even possess. Albany is not NYC, and the approach your org seems to take is like bringing a stereotype to life.
The costs to a city, county, or state agency for digitization under state Freedom of Information laws is zero dollars.
The costs of records requests are entirely borne by the requestor. We pay for the copies (but only the actual duplication costs, no mark-up allowed), we pay for the labor (at the government worker’s actual salary), we pay for the physical media like a USB hard drive or DVD’s, we pay for the shipping.
All the clerk needs to do is agree that the Freedom of Information request is legitimate and not prohibited by any exceptions to the law. And for that, the government employee has free legal counsel available to them in New York, in the form of the Committee on Open Government (COOG), which provides free Advisory Opinions of questions of the law. They answer questions by both e-mail and phone within 24-48 hours, from both requestors and agencies. Other states have similar programs available.
So yeah, this digitization is actually totally free to the government. We bear the costs, not them.
...unless they ignore the law entirely, in which case they get the pleasure of paying for their attorneys, and our attorney fees too. We’ve won those twice so far.
(by the way, the “corruption” in Albany line is a song lyric from the musical “Hamilton”)
100% false, you literally cannot even write the RFP to bid a scanning job for $3,000.
This is classic from folks like this. They are always like, here - you do the work, it will only cost $3,000.
The state should seriously offer them this deal. You properly bid, contract, insure, fight the appeals, do all the specialized hiring, handle all the HR and disability and other claims and scan the 100 years of records for $3,000 while working weird hours with a unionized workforce.
Again, you seem to be under some misapprehension that we are a company or are “bidding” on a project. Please educate yourself about state Freedom of Information Laws or Sunshine Laws.
We appreciate hearing that our work is “impossible” from people like you, especially since we’ve been doing it very successfully for four years now, and have so far put over 25,000,000 records online for totally free public use. This includes acquiring and publishing millions of records in New York City alone: marriage license indices from two different agencies, voter lists, and so on...
And we did it at a very reasonable $35 or $37 per microfilm roll. Because that’s what the law requires.
And thanks to the generosity of the Internet Archive, we don’t even have any server storage bills.
If you are telling a government agency what it will cost them to do the work, you are "bidding" the project. Does that not make sense?
I don't understand why you want them to do the work. How is it more cost effective to have the government do the work then to just do it yourself?
I'll answer this question, it is not. I 100% guarantee that ancestry, if they brought in their own staff or standard partner to do this, paid a LOT less then whatever the goverment would have charged you.
And then you got things for free. And are suing over this.
Let me make a suggestion. Follow the ancestry model. Offer a free service to agencies, backed with some quality, and if you then give them even a hard drive with the images you will make them very very happy. Seriously.
On the other hand, if you tell them to do something they don't know how to do, and then sue them when they don't do it the way you want, you will be helping to make the world worse off.
I did look at the fiche count, I will say, that's a very small job. That's where you will most likely see the biggest cost difference between doing it yourself and having them do a one off.
> Let me make a suggestion. Follow the ancestry model. Offer a free service to agencies, backed with some quality, and if you then give them even a hard drive with the images you will make them very very happy. Seriously.
This advice is terrible, and Asparagirl has every right to hold government to the standard the law demands, as well as ensuring certain requestors are not receiving preferential treatment.
If you enjoyed reading this story, which is about our fifth Freedom of Information lawsuit, you might also enjoy reading about our eighth and newest lawsuit, which we just filed against the parent agency of the New York City Municipal Archives a few days ago:
https://www.reclaimtherecords.org/records-request/22/
Teaser: “Welcome to the single stupidest lawsuit that our organization has ever had to file.
Alternately, we could subtitle this story let’s all gather round and watch the NYC Department of Records light taxpayer money on fire!”
Never heard of reclaim the records until now. What they are doing is amazing!
I was even moved to donate, even though I have no US ancestry ("Donation Confirmation Thank you for supporting Reclaim The Records!")
Reclaim the Records has done amazing work. HN user Asparagirl is the founder and runs it (https://news.ycombinator.com/user?id=Asparagirl).
waves hello
Ancestry developed relationships with many archival sources for a damn good reason. They have developed non trivial processes, equipment, and trained staff that treat the records with care. When I worked there it was a badge of honor for their document processing division that they were the only entity essentially allowed to just walk into the National Archives and take whatever they wanted to digitize, because they were known and trusted to return the extraordinarily fragile documents unharmed. Reputation matters. Presumably the cost that the plaintiff incurred was a form of insurance or "are you really seriously sure you have the capital to treat this stuff with care?"
I am not a lawyer and I might be making it all up.
I’m willing to wager Ancestry does digitization better, faster and cheaper than any non profit would.
The counterexample to your claim is FamilySearch.org, which worked with Reclaim the Records on digitizing a set of records. (It's a nonprofit run by the LDS church, which is a behemoth in genealogy records.)
Couldn't upvote this enough. I have done significant genealogical research over the last 6 or 7 years, using primarily Ancestry and FamilySearch.org. FamilySearch, particularly for non U.S. records, offers access to an order of magnitude more resources than Ancestry, though not all of them are digitally indexed. Ancestry, with its larger budget, seems to have invested heavily in digitizing and making available a small subset of commonly used resources, resources that will enable more casual users to make quick progress. So records like the 1940 U.S. Census and state-based death certificates (usually up to 1960) are digitized, and thus made widely available to keep and draw new subscribers.
Read what I wrote and try again. I said nothing about which records were scanned or what they did with them after scanning. I merely spoke of the scanning itself which, buoyed by a profit motive, will be optimized for speed and efficiency, unlike a non profit.
or that ancestry has a low-friction data ingestion pipeline that is many less steps than a standard data request.
cheaper for whom? their basic subscription is $200/y
"suspects".
I did not read the article because of this.
The internet slanders many based on only suspicion.
I suspect that we are not alone in the universe. I suspect that Elvis might still be alive. But I have no proof.
Get some proof.
I'm not going to write some article stating Elvis is still alive and pass it off as fact.
The article explicitly states they are doing to get information as to how Ancestry was so promptly serviced, and for how much.
They are trying to get some proof. That's the purpose of the lawsuit. You should try reading the articles from now on.
Trying to get proof is absolutely NOT proof.
I'm not going to read articles where someone has made something up.
You may as well start preparing for the forthcoming zombie apocalypse.
Trying to get proof is called Discovery and is quite literally one of the key points in filing a lawsuit. You can't argue something invalid because it doesn't have proof when the entire purpose of the lawsuit is to get proof.
Dude, trying to get proof is not proof.
For all they know ancestry didn't even ask the state to do the work but did it for them for free, and then after doing the work themselves gave it to these guys (not required).
So no cost estimate needed, no fight about cost needed.
You've got one group demanding detailed costs estimates, demanding names of vendors (the government probably doesn't even have yet because they've not bid it yet).
You've got ancestry who shows up, willing to work for free, willing to give images back, with past experience.
Do we seriously want government to turn group #2 away and battle away with group #1?
Or do we want govt to let group #2 do the work, no battle on price needed, then give the items to group #1. After exercising some common sense, the government is now going to be sued by these guys.
You have made the same misleading comments over and over again and it seems unreasonable.
The situation we have here is that something seems suspicious. From the perspective of this group, who have a fair bit of experience at this, either Ancestry stumped up what they consider an unreasonable sum, or they performed digitisation under what they consider unreasonable circumstances. Why is it unreasonable to require the state to explain what happened?
> Why is it unreasonable to require the state to explain what happened?
Because nothing requires them to explain what happened.
With a few notable exceptions, public records statutes only provide a right of access to extant records. They do not require agencies to answer questions or provide information.
That's just the way it is.
Reclaim the Records:
I read all of the publicly available court filings in your lawsuit. I'll reserve my comments on its merits for now and will simply recommend that you focus your PR efforts on the actual violations you are alleging in your lawsuit, and not on the possibility that Ancenstry.com was treated favorably vis-à-vis Reclaim the Records.
To be brutally honest, your focus on the latter sounds like whiney sour grapes at this stage, as you have zero solid evidence indicating that anything nefarious happened.
I completely understand the 'hot news' value of some public records. (I earn my living from it.) I also understand your suspicions about what happened as well as your interest in obtaining all records relating to the Ancenstry.com records request and its processing.
But until you have those records, your only potentially legitimate complaint is that they have not fulfilled their obligations under the FOIL. That's it.
Focus on that, otherwise you are going to see this thread repeating itself.
(For examples of what I mean, your website's page linked to in this very post literally highlights that Ancestry.com "received their copies of the records first" and that Ancestry.com "cut-and-pasted" "the exact words" of your organization's FOIL request. Unless you are suing Ancestry.com for copyright infringement, come on. People "steal" my request language all the time. It's only significant if something funky happened regarding preferential access.)
Good luck.
Their communications regarding this are now FOIA-able, no?
That’s the whole point! We’re not asking questions of the New York State Department of Health, we’re asking them for copies under FOIL of their e-mails, contracts, bids, evaluation materials, meeting notes, and such.
And the NYS DOH was like “nah”. Which is shady, and possibly illegal. So here we are.
> Because nothing requires them to explain what happened.
Actually that's the point of discovery in a lawsuit, and discovery absolutely compels both New York and Ancestry to provide explanations via evidence.
So you managed to be wrong on multiple accounts in a single sentence. You keep talking about "evidence". The point of discovery in a lawsuit is to discover evidence of wrong-doing.