OpenSCHUFA – Reverse-engineering the algorithm of the German credit score agency
okfn.deReading the funding goals[0] I don't understand why there is a funding goal of 30k to develop... an OCR app?
Wouldn't it make more sense, from both financial and technical standpoint to accept raw images then do the OCR server side? Server side OCR probably has more resources, and can utilize better OCR software. And you can re-run it after some time on every image if the results aren't that accurate.
just raw images would cause major privacy & data protection issues
Fair enough. I've donated, even though I still think that developing OCR app might be a bit waste of resources.
I still find it to be lesser "evil" than schufa - which, I'm pretty sure, is also extremely discriminatory towards people who move to Germany and giving them worse score by default. Hopefully you'll get enough optional data to be able to prove that too.
Not defending the Schufa, I strongly believe that any kind of aggregated credit history should never be in the hands of private companies.
But when I moved to Germany, I was under the impression that potential landlords just want to make sure you don't have negative entries, and didn't care much for the actual score. Luckily I managed to find an apartment belonging to a Russian investor, who didn't ask for my record at all. In the end I never got myself to even request a free report.
Yes, that's what's important for landlords. You get a letter from scufa saying "There are no negative entries for this person". That's enough.
However if you try to get a phone contract, credit card, internet contract, or anything else that requires payment in the future then your score is pretty much the only relevant part.
And that score is (maybe it varies by country) completely abysmal by default if you've moved to Germany - and it improves very slowly (<1%/yr).
I had no issue signing up for internet with 1&1 for two years, and got a free SIM card in the process.
As for credit cards, that's pretty common, traditional banks want to see your salary landing on your checking account for a few months before they issue you a credit card. Comdirect gave me a prepaid VISA in the meantime.
Anyway I ended up using a debit card (the MasterCard from N26) for almost everything, and later on I got a Gold credit card from Advanzia (I believe they do get information from the Schufa, but you also start with a pretty low monthly limit and need to prove them you pay your bills before they raise it).
You really shouldn't use N26. I don't know if they have improved the security of their app during the past two years, but here is a talk from 33c3 https://media.ccc.de/v/33c3-7969-shut_up_and_take_my_money
It's worth watching. They show how they where able to take over accounts and such.
I work in the credit finance space. The first thing that jumps out to me is the security and privacy of people's credit data details. There is a lot of personal information that could be used for stealing one's identity. What assurances are there that this data is protected and anonymous?
Stealing someone's identity is much harder with the data that schufa has.
And stealing identity isn't really a thing here. Like to open a bank account you still need to go to the post office with your ID which they then confirm, so knowing the things that schufa knows won't get you far.
As far as I understood it credit data companies in the US have a lot more info. Schufa has very basic things: "Opened account at X bank", "Got approved loan of 10k", "Signed a phone contract".
They don't have any insight into loan payments, individual transactions, or anything else really.
The data and credit scores are sent from the credit score agencies to each person directly (via snail-mail). After that they can decide to upload their "anonymized" data or not. In order to develop a privacy focused uploading process we started the crowdfunding campaign.
So you don't just need my data, you also want my money. Great! (sarcasm in case it's not clear)
It's almost as if some things are hard and need funding AND help. Who would've thought?
You are welcome!
If this is really ‘open’, how we can access submitted data?
Otherwise comments here about this project funded by competitor is making this a bit fishy.
1. date is not open yet, first people have to request the data (will take probalby 4-6 weeks until they get the credit scores)
2. see below https://news.ycombinator.com/item?id=16391885
If it will be open there is no problem at all.
Is Bertelsmann a competitor?
Nevermind, just read it below: https://en.wikipedia.org/wiki/Arvato
btw. we are requestion infoscore aka Bertelsmann - data see stats https://okfn.de/blog/2018/02/openschufa-english/#update
Why don't they just deliver them to you? I'm sorry but this is highly sensible data and the "where does the money come from" caution is the first line of defense a normal user has. Most of the time it is the only one.
Partnering with Bertelsmann here was a huge mistake. It questions not only data security it questions this first line of defense.
Think about it, how am I supposed to tell somebody to give their data to you when I already told them to look for the money first? I'd make myself and my methods unbelievable, incoherent and probably corrupted.
This is really terrible because this research is necessary and important but I hope you understand that I will neither support it nor recommend it...
we, the OKFDE, have no partnership with Bertelsmann our partner, algorithmwatch, has - independently from this project - funding from Bertelsmann Stiftung
again the data is first delivered only to the user, afterwards he/she can decide what to do
glad we agree that the issue is important
FYI in case you speak German, this project is currently all over the news in Germany, causing quite a stir.
TV http://www.tagesschau.de/inland/schufa-101.html https://www1.wdr.de/mediathek/video/sendungen/wdr-aktuell/vi... https://www.n-tv.de/ratgeber/Aktivisten-wollen-Schufa-Code-k...
Online https://www.wired.de/collection/life/openschufa-macht-bonita... http://www.spiegel.de/wirtschaft/service/kreditwuerdigkeit-w... https://www.golem.de/news/openschufa-reverse-engineering-der... https://www.heise.de/newsticker/meldung/OpenSCHUFA-Projekt-w... https://www.gruenderszene.de/allgemein/openschufa-algorithmu... https://www.futurezone.de/netzpolitik/article213448091/OpenS... https://netzpolitik.org/2018/jetzt-mitmachen-wir-knacken-die...
most motivational video, ever https://youtu.be/HBsD8BdXSCY
2 current related article worth reading http://www.tagesspiegel.de/wirtschaft/regierungsberater-gerd... http://www.faz.net/aktuell/politik/inland/dass-wir-ueberwach...
(disclaimer I'm part of the team)
Did you also cite the part where you are being financially supported by Bertelsmann Stiftung who have a competing product (Arvato Infoscore)? Though I am not sure if it is as bad as they claim...
a. is disclosed by our partner algorithmwatch b. the funding is not for this project c. see https://twitter.com/OpenSchufa/status/964408780162150400 "Die SCHUFA ist die bekannteste Auskunftei & Marktführer; wir fordern in unserer Projektbeschreibung u.a.: "Die SCHUFA & andere Auskunfteien müssen öffentlich & permanent darlegen, wie ihr Score funktioniert bzw. welche Modelle/Annahmen ihm zu Grunde liegen (Nachvollziehbarkeit)" d. read https://okfn.de/blog/2018/02/openschufa-english/#update (we already requesting infoscore Consumer Data GmbH data)
For everyone not fluent in German:
c. "The SCHUFA is the best known credit reporter & market leader. In our project description we ask: The SCHUFA & other credit bureaus must publicly and permanently explain how their score works and which models / assumptions are based on it (traceability)"
Infoscore is a <censored> company. They have a debt collection and a rating arm which are officially independent.
They also have a legal arm (Haas & Partner) in the same building (again "independent"), who collect legal fees on top of Infoscore's debt collection fees -- a practice that has been deemed illegal by several federal courts.
The legal arm is largely automated, so the "lawyer" fees are questionable.
Good luck convincing Germans to surrender their data. I would be very surprised seeing this working out.
Also, you will get data from a very skewed demographic.
AFAIR there is an upcoming EU law which requires (at least some) transparency on how scores are evaluated, in expectation of algorithms making life changing decisions. We'll see how that's panning out.
EDIT: to clarify, I'd love to see this working out. But I never would give data of this kind to an unknown organization.
We are trying to convince them.
Here are the live stats from the requests https://selbstauskunft.net/statistiken
Skewed demographics are on the radar (we know we are in bubble). EU regulation 2016/679 GDPR with May 25th is also on the radar.
For more related projects see http://mydata.org
I'd love to understand what you expect to find (given that you might have some special insight into credit scoring). I'd guess that the "algorithm" could be an embarrassingly simple sum of thumbs up/down for a handful of items. Like: If a person lives in a thumbsup/thumbsdown/nothumbs neighborhood etc.
wrt algo - afaik the biggest credit rating agency in Germany is "just" using a logistic regression. details will follow.
that is interesting! I'm very curious about the more :)
http://openschufa.de will be place for updates will take a couple of weeks, project will run for months to come
Next target: Fair, Isaac. Perhaps the OpenSCHUFA project can be convinced to take on Fair, Isaac next.
17 years living in Germany, never heard of Fair, Isaac. Creditreform and Arvato are much likely next targets.
Might be worth noting that OpenSCHUFA has some ties to the Bertelsmann Foundation, which owns Arvato Infoscore, a SCHUFA competitor.
Might be worth to tell the full story, as well:
- OpenSCHUFA was initiated by Open Knowledge Foundation Deutschland and Algorithmwatch, two independent and established NGOs.
- Algorithmwatch is supported by a foundation/trust called "Bertelsmann Stiftung".
- There is also a company called "Bertelsmann", which owns Arvato Infoscore, a competitor to SCHUFA.
- The Bertelsmann Stiftung (which is supporting Algorithmwatch which is co-initiator of OpenSCHUFA) owns a majority of shares of the company (that owns Arvator Infoscore, a competitor to SCHUFA), and some people say the foundation and company are tightly tied together.
- And last but not least Schufa now places in the press, that this is a big conflict of interest.
As stated before, we are not only asking for SCHUFA but also arvato data, see blogpost https://okfn.de/blog/2018/02/openschufa-english/#update
I think this is properly disclosed. And definitely does not reduce the value of the effort.
SCHUFA is demanding a passport copy from EU citizens to obtain the annual unpaid data overview report. A citizen of an EU country residing in Germany is under no obligation to hold a passport. I don't understand how until now no one torn the SCHUFA credit bureau into pieces over it.
> A citizen of an EU country residing in Germany is under no obligation to hold a passport.
Any EU citizen needs a passport or equivalent in Germany. One needs it already when entering Germany. Though you don't need to carry it all time with you. But you need to have one.
>passport or equivalent
there is a big difference between passport and normal ID, EU citizens dont need passports.
A citizen of EU country has no obligation to hold a passport while in another EU country in case their country issues other document confirming both identity and nationality of a citizen e.g. a national ID.
lipsm wrote "passport or equivalent" and you are jumping to "passport".
Here's the law: http://www.gesetze-im-internet.de/freiz_gg_eu_2004/__8.html
EU citizens in Germany have to carry a passport or equivalent.
> EU citizens in Germany have to carry a passport or equivalent.
..and no one has the right to demand explicitly passport if one is able to produce an equivalent and valid document e.g. national ID.
No, by law you are required to have a proper ID issued by an EU country.
No passport required, but it might be useful, since many online services require a passport, for example activating a SIM card.
> many online services require a passport
This is notorious and infuriating violation by many service providers in Germany.
Which ones? I never came across any. They may require a "Personalausweis" (like banks) but not a passport (which is a different thing). An EU citizen can come well by without ever needing one, as he can travel the EU by personal id alone. I only need a passport for international travel.
And that makes sense. You can be denied a passport, but not a personal id.
I have definitely experienced the case multiple times online where I have been denied something without a German Personalausweis or a passport. In my case, as a UK citizen I don't have a national ID card, so Passport must do, but the option is never "generic ID card" vs passport, but instead the German Personalausweis. Schufa is guilty of this, and iirc so is ImmoScout.
Well, the only document internationally recognized are passports. Not every country has identity cards like the german Personalausweis, let alone there being an international standard for them like there are for the passports.
The passport is simply the international baseline.
Nobody would complain if they demanded a a recognized identity document (like you need to identify yourself to state representatives). There is no good reason for Schufa to demand stricter documents than the state.
> There is no good reason for Schufa to demand stricter documents than the state.
There is - they are nosy, invasive, predatory fuckers.
no
You have to identify youself somehow. I mean this is Germany, not the UK where you collect utility bills to justify your existence /s.
In Germany as German you are obliged to have a valid identification document and as EU citizen just the same in case you are being asked for it. Wikipedia hast the links to the relevant laws https://de.wikipedia.org/wiki/Ausweispflicht#Deutschland
Why you're immediately teasing UK? They are exception in this regard. I'm talking about national IDs which most other EU (all but UK?) countries are issuing. Yet SCHUFA clearly demands passport in their instructions. In 6 different languages, 4 of which are official languages of the EU.
Maybe because they opted to scrap ID cards for stupid, vague reasons such as "reducing bureaucracy" and "civil liberties". Meanwhile, the de-facto method of ID is a driving license (which is useless for travel), or the national insurance number (similar to a social security number in the US) + other stupid forms of "ID" such as utility bills are used instead ubiquitously. The problem with the NIN (like the SSN) is that it is assigned once per person, cannot be changed, has severe consequences if stolen for identity theft reasons, and, oh yeah, it isn't identification because it has no picture, yet somehow every terrible company or bank needs it. Utility bills are even easier to fake or procure.
The farcical reason to scrap them comes from a government who's police force holds a large percentage of their population's faces - who are innocent - in a huge database and refuses to delete them. So yeah, it's dumb on all levels. Tease away at those muppets.
[0] https://www.theguardian.com/politics/2010/may/27/theresa-may...
Well at least the decent, law-abiding people have their power handed back to them:)... how come IBM is always involved... public IT project? IBM is already there, granted. They are Google/Facebook for public sector, i.e. they manage personal profiles (in fact very sensitive data) on behalf of authorities in too many countries across the world.
Because passports are the international standard, issued under international guidelines while identity cards aren't.
The international standard is completely irrelevant here. The parent poster specifically mentioned the case of EU citizens.
As a EU citizen you can settle in Germany without owning a passport. It's not up to private companies to redefine what consists in a recognized official form of identification.
EU national identity cards are issues under EU guidelines and recognized by the authorities of all member states.
They say that, but there is really no need. Just send them a letter with your name, DoB and address and request the information. They'll process it just fine, they actually have to because the law doesn't require a passport for identification.
I guess it is just a dark pattern to sell their "premium" products.
Show me a thing on their website which is not a dark pattern tricking users into their scammy pseudo-products.
When applying for a loan, mobile phone contract, or even trying to rent an apartment in Germany, the Schufa score - Germany's credit rating - is decisive. If you have a few "points" too little, your application is refused. (Computer says "No" to your new smartphone or apartment.) However, the calculation of these credit scores - done by the private Schufa company - is fully intransparent. The formula is a trade secret, and as such not open to the public.
We want to change this intransparency with the project OpenSCHUFA. Open Knowledge Foundation together with AlgorithmWatch want to reconstruct the Schufa algorithm with "reverse engineering".
How big is your dataset?
I'd expect the logic backing a legacy/old school company like this would turn out to be an expert system. If that's the case, and you have a LOT of data, you'd likely get very far by just feeding a bunch of SCHUFA applications and resulting scores into SKL's decision tree classifier.
They are easy to visualize, they likely model what's going on behind the scenes, and it takes very little effort to give it a try.
I cannot answer, I'm not affiliated with the project. Better ask @vavoida.
„...even trying to rent an apartment in Germany...“
While this MAY be true in a lot of cases it certainly is not all the time. From my experience 50/50.
Really? Schufa was the first thing anyone asks when I was interested in a new apartment. My schufa score is abysmal, but luckily the score is not included in this document, only the negative entries.
50/50 may hold true if you are renting directly from the owner. If a real estate agent is involved (or something like degewo) it is somewhere around 100% of all cases.
+1 for owner. Luckily you can choose (still).
Not all the time probably, though in my experience, especially if it's not a flat share but a proper apartment, it's definitely closer to 100% (Berlin).
I guess the likelihood increases if you go through one of the newer screen-tenants-first-online-platforms or the building has hired a property management company. Going through newspaper-classifieds or owner-let apartments yielded 100% no Schufa for me in Munich + Berlin.
see this information https://www.settle-in-berlin.com/what-is-schufa/
How do you want to reconstruct it exactly? The SCHUFA scores are only accessible for free once a year, so it would take quite some years to reconstruct them.
If you're going one at a time. You can improve this by increasing your number of datapoints (people's scores)
> For this purpose we call for data donations:
> Everybody can request their free SCHUFA-Score from Schufa under selbstauskunft.net/schufa and donate the data to our project.
How do you plan to do that? Which bits of information can you gather that allow you reverse engineer significant parts of their process?
Consumer's have a right to get an overview of all their collected data as well as the calculated (category) scores once each year for free. These could essentially serve as input output pairs.
I know what the reports contain. And I do not see how you could gather any meaningful insights from it.
It is very likely that there is relatively simple formula that maps the input data to credit scores.
Given enough inputs and outputs, you can figure out the formula.
I don't see them collecting enough relevant inputs.
We will see.
This is pure advertisement. Please stop misusing our forum for ads.
I am not affiliated with OpenSCHUFA.
It might be that on the wave of rage against SCHUFA someone wants to crowdsource huge library of personal data:)