Why You Need a Strategic Data Service
jasonkolb.comNow I understand the value of data, but what most companies miss is the value of consumer privacy. We like our privacy and will pay for it if we need to.
Typical data capture and analytics isn't like Prism. Seriously, you overestimate your own importance. Facebook doesn't care what you individually do, it's all about collecting statistics about everyone so they can better reach the groups you're a part of.
I know, and I'd rather the service be paid for by myself than by a secret service.
The author says that different data stores are good at different things, so we need to use multiple data stores in our application. He proposes that a data service layer can abstract these implementations, making it easier to swap out data stores as needed. I think that's a good idea. Separating our applications from data store specifics is a big reason why we use ORMs and ODMs today.
However, I think there are challenges with this polygot data store architecture that he doesn't address. Each addition requires due diligence to understand its CAP trade-offs (which the author mentions briefly), scalability and performance characteristics, how to configure, etc. These are non-trivial concerns even for a single database. It's important to consider these challenges when building out a data store or data service.
I'd propose an architecture where the data services layer itself exposes different data models to the application, all of which are persisted in a single data store. Given that many data stores use a key-value store under the covers anyway, translating the specific data model down to a single, persistent data store would simplify operations while exposing the desired data model to the application. (As a caveat, this multi-model approach requires ACID transactions to ensure strict consistency when translating between data models.) This approach provides operational simplicity with just one data store and application efficiency by exposing the "right" data model API.
Full disclosure: I'm an engineer at FoundationDB, a database that provides ACID-compliant polygot data models on a single key-value data store (http://www.foundationdb.com).
Predictive analytic is the thing I'm most interested in right now. Trying to use past data to predict the success of future product launches. I can say that having your data decoupled will save you some major headaches later on.
Great topic Jason. It seems the footnote links [2], [3], etc are missing something...I'm getting "File Not Found"/custom 404.
I had the same issue and was coming here to post the same thing.
Where did the footnotes go?