Ask HN: Can you sell alternative data to financial firms as a side hustle?
* What major challenges did you encounter?
* What revenue model can you use?
* Generally who are your customers: eg small hedge funds, big banks, other data vendors?
I'm curious why this doesn't seem to appear in the usual side hustle threads, I must be failing to consider some things.
I’m sure a truly novel source of alpha is unlikely to occur to little old me, but as far as side projects go, this seems relatively well-defined in comparison to designing an app and finding product-market fit. In one of my previous jobs (financial services firm), we needed some specific data from census/bls etc. We looked around to see if anyone is selling the data - they were, but they quoted tens of thousands of dollars for what amounted to be downloading and parsing of few CSV files from the gov sites. We did it ourselves. That same client was paying tons of money for other data sets, which were also public datasets. The value add here is collating, cleaning etc. What I am trying to get at - there is a market even for easily and freely available data, as long as the seller is dependable. The only problem is, you need to get your foot in the door. That might be much harder than the technical work. How’d they got on your firm’s radar? i.e. how’d they market their services to u? This is one reason why we made Splitgraph [0]. We index 40k+ datasets from government data portals, and we give you one Postgres endpoint where you can query all of them together. Any table in a data “product” (live or versioned) is addressable in a SQL query with `namespace/repository:tag.table`. Splitgraph parses the query and forwards it to the appropriate data source(s), translating it to whatever upstream query language(s) defined in a Foreign Data Wrappers (FDW). Here’s an example of a query across coronavirus tables at the external data portals for Cambridge, MA and Chicago, IL. [1] In this case the FDW implements the Socrata query language (the data portal vendor for 200+ gov agencies or municipalities). I like this Ask HN thread, because it’s almost word for word the question we asked ourselves three years ago. How can we make it easy for anyone to sell data? Reasoning backwards, we realized what needed to exist: a single “database” where anyone can connect or upload a “table” of data. We talked more about this in 2018, in a presentation [2] emphasizing three core ideals: data should be composable, maintainable, and reproducible. This is kind of like the idea of treating “data as a product” within a “data mesh,” [3] as described by Zhamak Dehghani at ThoughtWorks. The same ideas apply, whether sharing data within an organization or across the web. The question is, how do we connect data publishers and consumers? The answer should be roughly the same, whether the data consumer is a paying customer or a colleague. Think GitHub.com vs GitHub Enterprise. It’s the same software. Part of our vision for the analogous Splitgraph.com is a place for developers to make money by selling data with minimal effort. That could be as simple as entering the read-only credentials to an existing database of data they want to sell. Or even (eventually) scraping the data, inserting it into a “virtual database,” configuring billing and calling it a day. It becomes a part of the “global database” that any authorized consumer can query along with all the other data on there. [0] https://www.splitgraph.com/explore [1] https://www.splitgraph.com/workspace/ddn?layout=hsplit&query... [2] https://www.slideshare.net/splitgraph/splitgraph-ahl-talk [3] https://martinfowler.com/articles/data-monolith-to-mesh.html This is just too much jargon, too much complexity. Hedge funds don’t care about data meshes. They do care about: - whether your data has a long history of at least 3 years
- whether it has some predictive value
- whether it can consumed easily thru an API or FTP.
- whether it’s cleaned, formatted and accurate
- whether it hasnt been tampered with historically. Yeah, that’s why we’re building the fabric for vendors to deliver exactly that. As a vendor, you just worry about acquiring your data and dumping it into your favorite database. Give us the read-only credentials and we’ll take it from there. We handle distribution and give you tools to configure automatic versioning. (Ultimately we can reduce this even further, so you don’t need to maintain a database and can write directly to your virtual Postgres database at Splitgraph). As a consumer, you just need to connect to one database and you can query any table you’ve been granted access to, using your existing Postgres client. Or you can use a REST API that exists for every version of every dataset. We’ve gotta add all the billing, access controls, and some web UI, but otherwise this all works now. We’re a long way from focusing on any marketplace aspect though. We’re prioritizing the intra-organizational use case for now (where they over-rely on jargon like this, btw). This is super impressive. There is so much one can do with your service, my imagination is running wild. Can you give some examples of how splitgraph is used in the real world? Throwaway account. I sell altdata on the side. 1. Biggest challenge: Generating awareness and attracting the first few sales were tough. Some customers ask for names of existing customers and want to buy from trusted vendors. Unlike any other b2b business, you probably won’t get many leads from blogging, or SEO. 2. Revenue model is subscription, usually annual payments. Not usage based just a flat fee per year. 3. Mostly hedge funds. A few research services. I wouldn’t recommend starting this type of business unless you have a partner with a lot of connections. There’s very little word of mouth in this industry since it’s secretive by nature. Once you get 1 customer, they won’t tell anyone else about you (unless they move to another company perhaps) Sounds off topic, but there is something that always bugs me about the data market: -=-=- What about the idea of billing ANY company that is selling my data. And control over my data. For example, FB should pay every single user it makes ad money from, some % of the revenue their data brings them. Apple automatically charges 30% of any app on their platform as a service fee to the app developers to access the audience - so the audience should also be able to say "if you are selling my information into a set of demographics, I should be entitled to a fraction of what that ad demographic produces in ad revenue for FB" -- You may opt in for providing more information in exchange for a larger little slice of what your data is worth - or you may opt out to receive nothing based on the data you already provide in exchange for using whatever the platform is. An example would be "I opt into providing access to my demographics (the data graph on who you are)and choose to be compensated in some manner by the use of the information." -=-- To me, the idea that we are automatically "products" without compensation is pretty lame. Now, I would NEVER use facebook - but lets assume one is using such an exploitative thing: there should be classes of how one may want to use it... FREE (meaning they get profile data), enhanced (meaning I opt to a more extensive profile, to allow them to use - but am compensated for it) etc... FB disgusts me. I would never use it - and there needs to be an act similar to "the right to be forgotten act" which says "the right to be compensated act" > For example, FB should pay every single user it makes ad money from, some % of the revenue their data brings them. Why would FB do that when its users happily do that for free? There's no incentive for them to pay users. You could say users would proffer more detailed data, but I doubt it, most people's data is not conscious, it's the unconscious actions that are valuable, such as going to a store and looking at various items. ive never done this, but i know people who have. from what i understand, having someone on the team that has worked in finance is key. they have the relationships in the industry to be able to actually find buyers and get paid. the second thing i learned is that you have to be able to provide consistent data, regularly, for a specific topic, commodity, company, etc. random, self-curated data-sets won't get you very far, but if you can provide data monthly about something specific, then you might start to get some traction. good luck!