Treeline (YC W15) Wants to Take the Coding Out of Building a Back End
techcrunch.comI've always thought building a web service is the easy bit. In my experience, back end systems fit more closely with a conceptual model than the front end. As soon as things get to the user interface, you have to deal with real world problems, unexpected workflows, interactions, user expectations, variety of platform specifics, etc. To me, UI implementation seems so much harder.
We're essentially in agreement--backend development should be the easy part, but it often ends up being needlessly complicated and broken. So many backend systems are made up of the same basic components, but they end up being rewritten all the time because they don't work well together or are poorly documented. Treeline aims to make that 80% of your app that's always the same and basically automate it for you, leaving you to focus on the creative part (the special sauce that makes your app great).
I laud the effort of making it easier to build apps, but in my experience the backend stuff is mostly building the rest api and cron jobs, which are relatively straightforward. Imho, bulk of the complexity is in user interfaces with lots of moving parts.
Reading this, I am curious about the systems you've dealt with? If you are talking the back end basically being crud apps, sure, but most services, the bulk of the work is behind the scenese (rest apiece, etc)
outside the rest api, mostly crawling 3rd party apis, basic map-reduce aggregations, data sanity checks, the occasional s3 dump. No fancy distributed systems or machine learning stuff everyone else seems to be doing. How about yourself?
It's trivial if your app is just CRUD with domain specific fields, but there can still be some tricky parts;
* Sync'ing data between the frontend and backend if your app has any sort of offline capabilities. Conflict resolution is hard if you want anything better than just a dumb overwrite of what's there.
* Data munging for anything that isn't coming from a well defined API is a horrible and difficult task.
* Scaling and replication on a global scale is still a challenge. It doesn't affect many apps though.
You can knock up a decent REST API in Laravel/Express/Django/whatever in a few hours, but getting the data that you push through it still represents some fun and challenging aspects of app development.
Agree, those are all challenging problems to solve, but I haven't had the chance to work on them
1. we don't support offline mode in our app. Though, quite a few databases have a version number in their documents (es, couch etc), so the user can be shown conflicts if the server responds with a version mismatch
2. Luckily, I haven't had to work with any overly terrible APIs. The documentation is usually bad, but fiddling around with the req params/body usually gets me what I want
3. We're selling to businesses, so our traffic isn't at that scale.
You're absolutely right. Which is why we should make the backend even easier :)
I build whole systems, and have had specialized jobs ranging from 3D game programming to Machine Learning to Rapid Web App Dev. I've observed many the opposite. I would agree that interfaces have lots of moving parts, but part of what you try to do there is to reduce the complexity because it often means you are passing on too much to the user. I suppose there's a bit of Apple philossphy there.
Just cron jobs and REST APIs? Regarding cron jobs, we generally try to avoid them because communicating with them, getting good information, maintaining, and so on sucks. While I've written tons of cron jobs and still do, it's probably 1% of what I've observed happening in the back-end and usually a sign of a duct-taped system. This excludes baked-in unix tools that are actually supposed to work that way of course, rather I am talking things directly involving apps I build. As for REST API, not everything needs to be or should be REST. It introduces all kinds of extra layers of auth, slowdown, caching issues, and so on. For some domains, REST is a bad idea. Of course we use it where it makes sense, but it's not just some magic thing that is and should be used everywhere and anywhere to be representative of the bulk of back-end. Most REST services are built very quickly, it's rather the logic behind them that takes a long time.
As for concrete things I've encountered that are actually complex and happen in traditionally the back-end, I'd say none of these are straight-forward:
1. Caching - As it goes, other than naming, this is one of the most difficult things in computer science and happens everywhere in a good application, front, back, let, right, data, service, whatever.
2. Machine Learning, Sentiment Analysis, Analytics, OLAP, etc. - People forget that a big part of building a system is getting information back out of it.
3. Orchestration - this is hugely complicated. So much so, Microsoft created a very complicated and misunderstood product for it (BizTalk). Doing orchestration is still something no one gets. It happens in millions of forms and related manifestations.
4. Deployment - Versioning, machine provisioning, cluster management, and so on are all very complicated. There are tons of these things that fall under what many people call Dev Ops now too as well. I have yet to see magic here in any app. I challenge you to deploy a running system that is accepting live data transactions, then seamlessly roll it back without data loss, down time, corruption, weird states, unforseen bugs, and so on. No one has done this in full which is why this area is becoming increasingly full of solutions and start-ups. Whether it's Google App Engine, Azure, Heroku, your in-house AWS scripts, Chef, anything, I am telling you no one really does this 100% right. Supporting a real app is just as important as the app itself, and often the two have to communicate and be written in ways to make this work.
5. Streaming - Building streaming related back-ends is anything but simple and plug-and-play. No, I am not talking about streaming Tweets.
6. Data Migrations/Updates/Changes/Flow - Very important part of dev, and usually one of the top 5 time sinks. Complete nightmare and again products and libraries that help, but no magic bullets to be found here other than doing the work and putting in the time. As for simple cases, almost everyone deals with getting data in X form, transforming it to Y, and having it change form again after getting a reply from Z service. A huge part of people's time is simply manipulating data structures, at any place in the system and a reason why I tend to favor Clojure for web apps that don't need to be speed demons. As for migrations, ask someone to do one, then double or quadruple their estimate, sit back, and watch.
7. Authentication/Authorization - The world runs on more than just OAuth and Social Auth. Try dealing with people that have secure systems or many other systems that you need to communicate with, especially across network boundaries.
8. Legacy Integration - Happens all the time. Especially when you build a shiny new app very fast, then have to roll out v2 that actually scales. Beyond that, try any business that has been around 5+ years and actually makes money to see just how horrid this can be.
9. Data modeling - If the scenario of ABC person created XYZ app which now struggles to do Foo new thing without huge pains, you haven't worked on apps long enough. Data modeling mistakes and limitations are one of the single biggest problems and time sinks. More obvious examples include simple facts like having to dupe or dump data into other places to consume it the ways people actually want. While your shiny normalized tables with a REST service might work well for returning a simple list of customers, as that list scales and you realize you now have a new place where you need 2 fields from here, 8 from there, and so on and it needs to happen in 100ms or less in total back to the user, then you are in for a treat.
10. Logging - in big systems, the code here can be non-trivial. Often this means more than writing to a log file, but rather providing visibility into parts of the system in real-time or close to it.
11. Scaling - Everyone thinks they are doing this right until they don't do it right. This can involve anything from servers to message transport format to compression to memory management to clustering.
We could do this all day, but hopefully this makes a point.
Great list. Would love to see that expanded into a blog post :)
Agreed- the frontend is where the magic really happens. Our goal is to arm developers like us with a clean, transparent, and secure abstraction layer so we can focus on building awesome user experiences.
How to know if you might be a customer: Did you make a mockup that needs a backend but have little backend coding experience and are sick of including a bunch of the same libraries? If yes, maybe.
In short, they're integrating some commonly used API's and backend programs (Elastic Search etc) and putting a graphical programming model on top of it.
Crappy backends are pretty easy to throw up on a VPS or Heroku these days. It's so easy that I'm likely to select an app server(s) and language based on what library (if any) is most critical to that particular request type.
At first I wasn't sure who the target customer could possibly be. Was it a PaaS service? BackendaaS? A service like this lives in a weird space. It's almost only good for early product dev and when the team's backend devs are green.
Things like having unit tests, scale, completely custom capability etc will drive everyone who makes a successful product out at the very time that they can start to afford paying a lot.
Don't burn all your money yet ;-D
It's true that when this has been tried in the past, there's always been a point where the "magic" ends--where there's some functionality that you need which isn't included. Treeline is different because you can always create new components when you need them. The Machine specification (http://node-machine.org) is open-source, and you can upload new machinepacks to NPM (Treeline imports them automatically), or create them from right within the app. So the magic never has to end :)
Or as PG put it, "You get to make your own blocks!"
I'd bump that point and the integration with API's way up in the site copy. My first thought as an engineer when I see graphical programming tools is, "oooh...another one of those." It's the most controversial selling point. If I know I can do 90% of my backend work fast enough to slap an MVP together, the tradeoff might make sense. My default plan is a VPS with Flask or some other cheapo web app server that can pull 300req/s. That's a step beyond just using Heroku, but also a valid competitive option.
How would you position your product relative to Heroku in terms of power and how much schlep you trade for how much ease-of-use? I think it's very okay to be clear for now in order to convert well with your core customers.
We're not a production hosting platform--at least not at the moment. We do spin up a preview server for you to play around with your app while you build it in Treeline, but it's not meant to compete with Heroku or any such provider. Instead, we give you a way to download your project as a fully-formed SailsJS app which you can deploy anywhere that supports Node. You can also use the same tool to preview your app on your local computer while still keeping it synched up with Treeline in real time--hooray for sockets!
Okay, cool. Now that I've finished being uninformed enough to get questions out of the way, I'm guessing you chose Sails because it has some kind of JS DB for prototyping. I would spend time asking if Sails is the desired output format because, although I've only used it during a Startup Weekend fling, it feels like a framework, and don't forget that it's said that while you can plug your code into a library, in Soviet framework, code plugs into you -- I imagine that although a backend dev might be thinking, "Great, Node, kill me softly with 'undefined'" they would still kill if the implementation was really minimal with nothing in their way; if they like another JS framework or want to write a common backend for web and mobile apps, not having Sails' distractions in the way could be big for legitimately doing some of their work for them.
Another big feature would be doing the work to plug persistence into other DB's to make the output DB agnostic to some extent. Migrations would be the next step, but too much of a chore. All you need are to define data models for XYZ ORM/DB adapter to make the machine-generated step hit the right starting point for hand-off. If the ideal use case is in prototyping, I don't think migrations is in the right vein.
Armchair-CTO, signing off.
We chose Sails because we built it :)
Definitely don't burn any money until you try the thing out :)
So graphical programming counts as not programming these days.
Yep, I have something vaguely similar to this which I've built over the years for my own projects, essentially generators for boilerplate for back end stuff, written with preferred libraries, coding styles, testing approaches, etc. From who I know, I think it's common for back end devs.
Yet I've never really built 'a simple' app... It does always requires substantial time tweaking, customizing, and building things that just can't be automatically generated. I can't imagine for anything that's not just dead simple that you could take the generators all the way.
But I see you're targeting front end folks, which makes sense. If they have simple use cases this could work nicely for them.
After building one of the largest backend frameworks for Node.js I completely understand where you're coming from. It's taken us a very long time to solve this problem, and I can say with complete intellectual honesty that I believe we've solved it. Check out http://michaelmcneil.com/post/112978579967/mikermcneil-outco... for a more technical explanation of how the compiler works.
I think this service should have one-click deploy to other Node.js platforms, at least Heroku, Beanstalk and Bluemix. I have a feeling that their concept would be most useful on ipad/mobile. But for that to be useful there needs to be very easy deployment options that work with my hosting of choice.
I mean, imagine finding a bug, being able to fix it on your phone, and deploying it immediately. If it works that way, I'm sold.
Also, if this is aimed towards frontend devs primarily, then it should be noted that it is unlikely that a frontend dev would struggle with writing JS on Node. They are more likely to dislike the deployment.
> They are more likely to dislike the deployment.
Deployment is the worst, for everybody. We're definitely excited about the possibilities there--done right, you could wire up your app in Treeline and set it live without your feet ever touching the ground (or muck, as it were.)
Deployment system relies on sails deploy- Microsoft actually just contributed an open source deploy strategy for azure and Heroku expressed interest in contributing to theirs. AWS has also been very helpful. Will keep everyone updated as things progress.
The one-click deploy to heroku button is in the works.
This looks very very cool. But, who is the target audience here?
IMHO, this doesn't have the best Product/Market fit. I'm guessing the target is a non-programmer or just front-end guy who quickly wants to put together a backend.
If that's the target, then why not provide a set of generic backend functions. For the mentioned target.
I think you can limited to a simple set like: registration, authentication, user profile, follow (friend), add content (text, image, video), like, list content, list all liked, list friends, etc.
This way you can become "Wufoo for backend".
There's [0]hull.io, doing a lot of the things you've mentioned here.
[0] http://hull.io
Another person suggested, http://stamplay.com/
Great question! Open source drag and drop functionality currently exists for things like sending emails (via mailgun or mandrill) payments (via stripe) sending SMS (via Twilio) social auth (via twitter, Facebook, or github) password encryption (via bcrypt) analytics (via segment.com) and over 100 other common functions
But who is the target audience?
How do we try it out? I've been in beta list for a few days! :)
We are working on getting everyone in tonight or tomorrow!
Thanks for your patience! Working on it as fast as we can.
co-founder here if anyone has any questions
This looks great! I can imagine this lowering the barrier for building a quick side project significantly. I have a FT job so working outside of that stack means I have to spend a lot of time figuring out configs and other non-business logic related work.
Now a question - say I have a Treeline app that works great for a little side project. But eventually the project grows really big, both in traffic and scope. Is there a way to get off of Treeline (export code?) or do you think Treeline can scale well in this case.
You can take the code right now and download it and deploy it to any infrastructure provider you want! We provide a command line tool to sync code down and work locally or to package up and deploy.
Treeline compiles your apps to Node.js apps using the SailsJS framework. Your code is available to you at any time via a command-line tool--we offer the hosted version as an added bonus.
Great question :) code is recompiled in realtime as a node app you can deploy anywhere.
Also just as a note, I would shorten that demo video by just skipping over that Twilio setup part.
Can you handle apps with background tasks? In my experience, the backend systems basically end up being a large repository of jobs that need to get backgrounded, and handed off to other systems.
I would be even more excited if you could find some way to create and manage those effectively.
Machines can be triggered via kue, cron, or any other process of your choice (since they're effectively just augmented JavaScript functions)
Would love to see it integrated with a front end building platform like Webflow. I just built a quick site with Webflow, but because it lacks any integration with DB/Backend it was only used as a landing page. Having this ability would let users create and end to end web product with only basic knowledge of programming.
Cool idea, will check it out!
As a front end dev, this is a game changer. So pumped.
Suspicious account, a little...
Is this for free? i cant find pricing information
Free for open source- still working out pricing for private projects, but will be affordable for indie devs. Basically it will be something I could have afforded back in 2010 when I was in school.
Yahoo Pipes 2.0?
I don't normally comment here, but I'll bite on this one. ;tldr - I find the goal and customers here confusing, questionable, and not well thought-out. Additionally I find the founders insulting and naive in the article.
I still don't get who is the current and future customer even after reading the other posts here and other available material. I fail to see line of business users writing anything in this as it is too complicated for them and too close to programming. I cannot see an experienced dev introducing this in a stack and investing the time in learning the tool/libraries, building a system visually (super slow), and tying themselves to a 3rd party unknown start-up this tightly. As for the indie developers mentioned in the article who cannot build a scalable back-end, who are these people? Why are they developing apps? Maybe I am a grouching aging developer, but I really don't understand who these people are - "Indie developers comfortable building the user-facing side of an app themselves, but who would need assistance to build out a scalable backend."
I would argue that if you believe you are a person who builds apps, you should be capable of actually building the app you are building. Beyond that, you are just another person with an idea, which in our world means absolutely nothing. While I can see how people can build a prototype not knowing exactly what they are doing and become successful, the keyword here is prototype. A prototype doesn't usually need to be scalable. If you want and need scalability, you need to build something carefully which matches your app's use cases and real-world usage. This is a difficult problem and not one that can be solved by plugging in anything and waiving a wand.
Listen, I've worked as a developer since the late 80s, consultant since the 90s, spent many years at Microsoft, and worked for a few startups. I suppose I am one of those weird people who doesn't care about the tools, language, whatever, just give me the task and I will find the best tools to do it, and learn them in hours if I must to get the job done so I am open to anything to help, but this app makes me feel like I live in the Twilight Zone. I have seen this exact product in some shape or form so many times whether it was Visual XXX or ABC Builder or even some layer on a product like SharePoint, Dynamics CRM, Wordpress, whatever. The closest tools that have had at least practical success I've seen have been Excel and Hypercard. My hats off to you if you can make it work, but really I have my doubts. There is nothing written so far that is convincing me or more importantly, that would convince any of my previous customers (including private, government, and start-ups) to use this. And I assure you that my past customers would like nothing more than to take shortcuts, not pay for my hours, and/or find some way to in-house many development tasks. There are just so many things wrong here I don't know where to start. I'm not saying this to be mean, but rather I am genuinely curious who would use this and why. On top of this, you have tons of related competition, and hype around everything from Wolfram Language to Eden in this space if we broaden the scope to visual builders in general.
I think it's a bit telling that one of the first comments is, "Treeline is different." I am pretty sure everyone who builds a product like this says exactly this, including my former colleagues who built real, very large, and even well-selling products. I have never seen any product genuinely succeed in its goal here, at best these products just sold the sucker's line and did not retain their customers long-term. Alternatively, these products succeeded in something else, but failed in the "app" or "back-end" builder category.
I really also have to question the attitude of the founders. I found a lot of the quotes out there about this product insulting. "Building the backend, however, is like stacking wine glasses: it’s not that hard, but if you make a mistake it’s a bigger issue to deal with." If building a back-end is not hard, then why do I need your product? It should just be easy as long as I test it and do a lot of checking to make sure the wine glasses don't fall. What makes me think that the founders of all people are the ones I should trust to make these decisions when the pitch for the product is insulting the people who actually build solid back-ends? I've built a lot of back-ends that people actually use and not once has it been an easy task. Even the jobs that took me less than a day I would never say were easy; perhaps I simply discovered humility and actually lived long enough to see what happens to systems 10+ years after they are used by real people.
I would also apply the above to front-ends - they are never just "easy." But front-ends you can mess up and fix them without too much catastrophe beyond perhaps annoyed/lost users/etc. When back-ends fail or are designed badly, we lose data, ability to gain insight into our business, can't satisfy company/client requirements, and so on. These are much more serious consequences than your app not being responsive or shiny, or looking like it came from a wannabe Craigslist fan. Both are important to success, but let's not pretend that our apps could be the next Google if we could only sweep away that pesky back-end. It seems on some level the founders agree here that back-ends going wrong are really a problem, but why does this start-up suddenly guarantee success here? Just the fact of using node.js + <insert db> does not make a back-end good. Using things the community generates also does not make something good. Back-ends are anything but one size fits all, or even one size fits more than a few people.
Producing a good back-end is actually very hard for anything beyond a prototype and an often misunderstood art form. I don't get this notion I see a lot that xyz programmer is a "front-end" developer and abc programmer is a "back-end" developer. You are not a developer at all if you can't learn anything you put the time and effort into I would argue. At best, you are a beginning programmer who knows a few libraries that are focused on one or more aspects of a system. In practice, you don't always get to work in jobs in all roles, but you still have to know about all of them to be an effective developer. I would never hire someone for my team beyond a junior-level that was not strong in all areas. There are of course specialists as well, but these are more for things like crypto/security, embedded programming, and so-on rather than your general application code. Simply put, if you can't learn front or back-end development to meet the requirements of your development tasks, you have no business doing them. As a corollary, it is important to know your strengths and hand-off to people who might be more artistic or more familiar with a language, but it does not absolve you of learning about the other parts of your system. So I also ask, what part of this protects a programmer who is inexperienced, clueless, or otherwise too busy to care about the back-end to not structure or combine otherwise solid parts (for the sake of argument) of this tool from creating a back-end that is a complete disaster because of the sum of its parts?
I think this app discounts the close relationship of a system as a whole between the front and back-end. A few things were mentioned already like syncing data offline, but I have additional concerns. Note for example that caching, data modeling, and asynchronous programming for example are all very closely related to both databases/working with data and building user interfaces. I cannot tell you how many times I have had to build little tricks to makeup for technology or real-world issues that involved doing things like encoding some bits of information in a database key, field, whatever to make something more cacheable, O(1) lookup, or whatever to ensure the front-end experience was better, or even just deal with deficiencies from other parts of the code. I think the notion of all parts of the system needing to work with a consistent approach, philosophy, and quality is highly demonstrated by the current approaches in Clojure using ClojureScript, React, and Datomic for example where the notion of using immutable data makes it both easier to cache information nearly indefinitely, and to create UI that renders quickly due to the ability to use deltas and caching as bi-products of immutable data. There are many other systems and frameworks that share this view of "turtles all the way down."
Another problem with the product here is I don't see a good story of how someone can legitimately develop a solid back-end with this tool. The problem is not necessarily the tooling, but the customer that presumably uses this already lacks the skills to create a back-end. What about the product being visual suddenly mitigates this? It seems from the literature available that the implication is that a lot of back-ends are super simple and require merely basic CRUD operations. In my experience, this is anything but for most people. Business users for better or worse like to throw tons of rules, conditionals, corner cases, and so-on when developing a system. It is your responsibility if you are a developer to stop this, but I hardly think the customer here is one of those people who knows enough to do this correctly or at all. The tool looks like it can give you the ability to add all these rules to be something more than super simple CRUD, but the problem is not the rules working or not, but the fact that adding them and using them usually causes countless bugs and severe consequences without someone who knows what they are doing actually checking with proper thought. Assuming you are building a CRUD app with none of the aforementioned complexity, I hardly see why someone would not just use something that is either stupidly simple or closely matches their domain to do this like Salesforce, Sugar CRM, Excel, whatever. This product seems to give no immediate value proposition over a naive CRUD app over existing tools, and takes away just enough power to not make it something that can compete with a more complicated solution like existing web or app frameworks.
Additionally, I have encountered the truth is that there are few things in an application that matter more than data. One could argue the end-user functionality is king and while this is often true, you can still get away with not the greatest user experience and product but be successful. Judging the front-end is very subjective at times, but the back-end is rarely anything but subjective. I have seen countless front-ends that made millions for companies based on nothing more than Access, Excel, FoxPro, VB, Salesforce, Drupal, Wordpress, whatever. One thing they would all tell you though is their data (and even logic) was super important to them. Another reality is a lot of these people already have the data in abc form and need to transform it to xyz form(s) to take the next step. This solution does not seem to have any obvious story here. That seems to discount a lot of people with existing businesses and products. Doing ETL or any sort of transformations is exceedingly complex, and the person using this tool would hardly be the person to do it. I just can't imagine everyone else doing the heavy lifting of getting existing data into a form to use with this tool, then stop the heavy lifting to use an unknown quantity in an online visual builder.
I have many more concerns and arguments here, but to cut a long rant slightly shorter, putting everything else I said aside, what about the tree structure itself? Trees in web browsers seem like an awful fit. They are even a particularly awful fit in desktop UI beyond a few levels. Programming "back-ends" is notorious for deeply nested logic and complicated flows. I don't see how modeling this in a tree is a good fit. A DAG is a bit better, but my real issue here is that it is so visually literal. I can already imagine being driven crazy with identation. Undoubtedly I am sure there is some work-around, solution, whatever just like in normal programming of hiding the nesting complexity by naming sub-trees, collapsing things, etc in the way we collapse things into files, classes, objects, namespaces, etc. This though seems to work a lot better in a text editor with some visual capabilities rather than in a purely visual editor. I've seen structural editors with Lisp that try to do fun things here too, but that's another topic. Anyway, my point is that the UI itself seems like it won't scale well and introduce complexity in groking the code it is supposed to simplify.
Generally, the problem with this product and most in the same space is that as you take away control from what someone can do here, you get diminishing returns. The reason some things work with bad back-ends is because they are super simple. This looks anything but simple and I have no nice way of saying this, but it seems targeted at an odd group. Are you going after high school - early 20 year olds beginning programming? If so, did it occur to you that these people are often fickle with spending money in this area, and often don't have the money anyway? Who is your real customer beyond what is mentioned in the article. Sorry, but I don't see a future for this tool and I can't see why any logical person would invest the time and money to couple themselves to it. Shame on YC for investing in this one as well, you've been duped I think.
Washing clothes is not hard, but yet we still buy expensive washing machines to save us time. Please take a closer look- would love to hear your feedback after trying the platform out.
> Producing a good back-end is actually very hard for anything beyond a prototype and an often misunderstood art form.
Lol you didn't read his comment at all.
I fail to see how this is an actual answer to anything I have written. It feels like you are dodging with an analogy.
Can you please elaborate?
I understand where you're coming from, but I must respectfully disagree. However I'll say one more thing in defense of our users: modern front-end developers do data manipulation all the time. The data munging that's happening on the client is just as complicated and important as the coercion and marshaling on the backend. In many cases, the barrier preventing client-side developers from just doing the same thing is a different language, a lack of confidence, or organizational pressure and bullying.
Let's see what they can do :)