This story is available exclusively to Business Insider subscribers. Become an Insider and start reading now. Have an account? .
Pinterest engineers Yashwanth Nelapati and Marty Weiner recently shared some insights and lessons learned while developing and scaling the company.
Pinterest is a site for collecting and sharing photos of interesting objects around the web.
In a little over three years, Pinterest has grown from zero page views a day to billions per month. The most recent tally: 3.4 billion monthly page views from its 25 million members worldwide.
Today, Pinterest is valued at $2.5 billion following a $200 million financing round earlier this year.
This slideshow talks about the software and hardware Pinterest used to get where it is today. Warning: It's for geeks only!
Here we go!
You can follow pins and boards from people you know.
Here, you can see the pins from everyone you follow.
So users have boards and relationships.
Here's Pinterest's page view count at the beginning.
In March 2010, the team wasn't working with much from an infrastructure point of view.
Nine months later...
The product and architecture evolved.
Pinterest started doubling page views every month and a half, but everything was breaking.
So they ended up with five major technologies just for the data alone.
So they started dropping off technologies and did a massive restructuring of the architecture.
Here's what they changed the architecture to.
Pinterest's web traffic continued to increase.
Pinterest started to put more resources into its architecture to handle its growth.
Pinterest uses Amazon EC2/S3 for a few reasons. The main one: you can have new instances ready in a matter of seconds.
But there is limited choice.
Open source database MySQL has proven to be solid choice for Pinterest. It's incredibly mature and you can hire for it, as lots of engineers know MySQL.
Memcache is also incredibly mature, and it never crashes.
Redis isn't very mature, but it's simple.
Pinterest realized that during its rapid growth, it needed to spread the data evenly to handle the load. So they defined a spectrum of options between clustering and sharding.
With clustering, everything is automatic.
But sharding is a completely manual data placement process. It's used to separate databases into smaller, faster, and more manageable data pieces called shards.
If there's a massive bug, it will impact every single node. A SPOF brought down Pinterest four times.
With sharding, everything is manual. And that's a good thing.
If your project has a few terabytes of data, you should shard as soon as possible. When Pinterest's Pin table reached one billion rows, the indexes ran out of memory. That's when the company decided to shard.
So Pinterest froze some of its features to start the transition from clustering to sharding.
The less data you move, the more stable your architecture will be.
Since they wanted to shard on MySQL, they projected growth for next five years.
Pinterest initially put their databases on 8 physical servers.
For high availability, Pinterest ran MySQL in multi master replication mode.
With an increased load on a database, Pinterest replicated a server to handle some of the data nodes.
Since Pinterest is on AWS and MySQL queries took about 3 milliseconds, they decided to build the location into the ID.
If Pinterest has 50 IDs, for example, they split them up and run them in parallel. This is what Pinterest's lookup/rendering structure looks like.
All of Pinterest's data falls into two categories: objects or mappings.
How Pinterest brings up a user profile. Most of the calls are served from the cache (Memcache or Redis)
Pinterest built a huge scripting farm to move 500 million Pins and 1.6 billion follower rows. Scripting is what happens when you need to move from the old, unsharded system to the sharded one.
Need to give your brain a rest?
Read next
Megan Rose Dickey formerly covered tech startups focused on the shared economy and music industry. Dickey previously wrote for LAUNCH Media, where she covered the startup industry and organized tech conferences.She graduated from the University of Southern California in 2011 with a degree in Broadcast and Digital Journalism.