Ask HN: What should I build an iOS API in?
The system will be a CRUD back end for an iOS app, later offering a web app, then otehr clients, etc. I'm outsourcing it, so the choice of language/appserver will be up to the lucky (!) programmer.
Will be RESTful with JSON data exchange. Linux on EC2.
Other than that - what would you say would be the best route for:
- Low system overhead - Fast performance - Low resource usage - Scalability to multiple appservers (eventually)
Ruby on Rails? Pyton / Django? PHP on a framework?
Or doesn't it matter? I wish I could provide a strong opinion on a solution good for I/O. I'd need to do research and it's probably what you've already covered, from the sounds of the conversation already here. If I were faced with the decision knowing what I know today, I'd use some queuing system RabbitMQ or other AMQP protocol with a pool of DB servers sucking up requests and processing them. You can maintain a synchronous response with the requesting server via some frontend code (Python w/ Django or Node). With this setup, you can easily throw more DB instances in the pool to grow with your needs. The frontend should be able to handle as many requests as needed (especially Node which has VERY lightweight threads, I hear). You'll just need to reduce the turnaround time via the queue. Mike this is great, thank you. Good angle for me to think on. Doesn't matter what you use for front end. Ruby used to be a POS VM in terms of performance, but who knows now a'days, but just about anything can return json, so just grab what you can. Your real pain point is going to be your back end. If immediate consistency isn't important, run a job queue with works in the back end hitting some kind of fast access nosql and have a worker whose job it is to write to a mysql or postgress store. raid the ebs volumes and keep a monolithic volume sister box to take snapshots off. you can have the entire environment scripted and repeatable. I asked ravaa to stop by and give his .02. He's well versed in the world of infrastructure and I highly value his opinion. There is no right answer here. It really depends on what kind of purpose your backend serves. Is it processor bound or I/O bound? How frequently is your app going to hit the backend? How fast do you expect to grow? I/O Bound - very little other than db write/read operations. Going to hit the backend every transaction as its very stateful so minimal caching can be possible. Unknown on growth! Hope for the best, I want to take 1000 r/sec on an EC2 instance. If it's write heavy, more than the app server, it will be the DB that will be more important to you. If it's purely I/O bound, node.js might work out fairly well for you. Perhaps you could hook it up with mongodb and it will be pretty. :) I had thought about MongoDB, but bit worried about its global write lock! Surely that can't be a good idea!? http://www.mongodb.org/display/DOCS/How+does+concurrency+wor... Or does that only apply if "safe mode" is on, rather than fire-and-forget unverified writes? The global write lock, while important, is probably the most over-written-about and misunderstood issue. The writelock is only held while the db is actually in the middle of an write and tends to only be held for a very brief period (about 10s-100s of microseconds or less) at a time. For example, it is never held across multiple user actions and therefore is closer to a latch than a lock in common RDBMS terminology. The main cases where this caused issues for users are when some data we needed wasn't already in memory so we went to disk to fetch it (~10ms rather than 10us). One of the major improvements to 2.0 was a framework to yield the lock to allow other work while going to disk that is used in the places where this was seen most frequently in production. 2.2 will plug this in more places with the goal to never hold the lock when going to disk. This work is of course in parallel to work to move to DB, collection, and possibly extent-level locking replacing the global lock in almost all cases. As for safe-mode, the waiting that it does is always outside of the lock. It uses a cached datastructure that is protected by a secondary mutex so that it never has to interact with the global dbMutex. If you choose to wait for replication or journalling of your writes, that will block the connection and therefore your client thread so single-threaded tests will show much worse performance with the db mostly idle. If you use more client threads or asynchronous I/O connections you should see roughly identical throughput in aggregate (see mongostat) although much higher latencies compared with not waiting. If you have any questions about this feel free to shoot me an email. Replace the _ with an @ and add a .com to my user name. Thanks Mathias, great advice. Yes, the global write lock essentially means that you can't parallelize your write, though I hear that collection level locking is on the horizon. Unverified writes are generally bad for transaction data. If you _really_ think you need 1,000 writes, only a complete in-memory DB like Redis can offer that... My suggestion would be to go with a simple set-up and consider scaling issues when the need arises - they are always a good headache to have. Good info, thanks. MySQL it is then. :) Just keep in mind that MySQL doesn't scale easily. Master/Slave relationships get tricky to manage. I've only had minor experience with this, but it was enough to know to avoid it. It probably does not matter a lot. Find a programmer that has written a server back end to an iOS app before and use what she wants to use. Node.js