Hiring is one of the most time-intensive and critical things we do.
This means time is precious. We can’t waste any on interviewing or hiring the wrong people. That’s why every engineering candidate we talk to goes through a set of coding challenges.
We know there are different schools of thought on this (and some pretty passionate opinions) but it works for us.
If someone is going to get weeded out of the hiring process, best to weed them out early. It seems to be better for everyone involved.
We’ve been through several iterations of coding challenges to get where we are now. It’s definitely still a work in progress. But this is what’s working for us right now.
Test 1: Can you write any code?
Early on in the interview process, we set candidates up with our first test.
Quick sidebar: As much as we’d love to show you the actual tests here, we’re not. If someone read this and practiced ahead, that would be pretty unfair to people who didn’t.
Basically, we need to see if the candidate can write any code at all. You’d be surprised how many candidates fall short when it’s time to put the cursor to the editor.
For people with any sort of decent programming chops, it will take a little effort but shouldn’t be super challenging. Consider it a “price of admission” test. The work on this test has no practical application beyond showing us if you can write any code.
If someone’s banging their head on the desk and can’t get through it, there probably isn’t a relationship in store for us. Most reasonable coders we’ve given this to have gotten through in about 20 minutes.
What we like about this problem is there’s no hidden answer, or “leap of inspiration” here. We wanted a problem that someone can brute force their way through if need be. You don’t need to know some obscure algorithm or something to get the answer to “click.”
When they’re done, we receive a recording of the candidate doing the problem. We get to see the different steps they take and any comments they left along the way.
We don’t just learn if they can solve the problem or not. We get a lot of interesting insights into how the candidate works.
Some things we learn:
- How do they approach a problem?
- What do they do when they get a little stuck?
- Can they brute force their way through this?
- Are they going to be “done” as soon as it’s running, or are they going to clean up any inefficiencies once it’s working?
- Are they slow and methodical?
- Or do they hammer through a bunch of approaches and see what sticks?
- Do they leave good documentation and comments on their code?
- There aren’t really right or wrong answers to a lot of this. It’s just really helpful and interesting to us to see how candidates work.
Test 2: The take home project
If they aced the first test, we’ll do a few more rounds of interviews.
If everything is still going well, we move them onto the second and final coding challenge, the Take Home Project. This is a longer and more intense project. When someone makes it this far, both us and the candidate are pretty committed to making this work.
Unlike the first one, this is one is more a practical application test. It’s throwing them into the kind of situations they’ll face on the job.
Basically, we want to see how well they can build and deploy a thing for the web.
We ask that they spend no more than 8 hours on the test. It doesn’t have to happen all at once. We understand this is an interview exercise and people have other commitments.
We also give them free reign to use whatever services or code libraries they’d like.
All we’re really looking at is the finished product they send us.
You’d be surprised how much we learn from it.
Some things we learn:
- Does the candidate run tests? Do they do enough testing?
- Does the thing actually work when they say it works? (Yes, people have sent us things that don’t work.)
- Do they pay attention to the requirements?
- What technologies do they use?
- Are they using technologies they’re competent working in? Or are they using some odd or shiny things in order to impress us?
- What kind of design decisions do they make when presented with the freedom to do so?
- Like I said earlier, this is all definitely a big work in progress. Hopefully we keep getting better at testing and learning from candidates.
The successful candidates show us a lot about their skills. They also have fun with the problems. They get excited about the opportunity to show us what they can do.
We’ll certainly chat afterward a lot about coding and technology. But, thanks to these tests, we also have a lot more time to learn other things about the person. How they work on a team, what drives them, how they fit with our culture and values. Things that won’t be on the test.
Bonus: An engineering team’s secret weapon
We like to think Statuspage is an important piece of any developer team’s toolbox.
What does Statuspage do? We make it easy to keep customers informed when things go wrong. Engineers have a ton of work to do during things like system downtime. We don’t think you should spend that time answering emails.
We could tell you more, but we’d rather you see for yourself. Our trial is free and unlimited, so there’s no harm in setting up a page and seeing if it works for your team.
Note: This piece was originally published in 2016. We’ve learned a lot about our process on hiring since then and made some interesting improvements. Stay tuned for a upcoming piece on what we’ve learned and changed since.
This post was originally published on the Statuspage blog in 2016.