I feel compelled to write about interviewing as I’ve been guilty of doing things what I now consider to be the “wrong way”, previously in my career. I was even part of the original interviewing team at Google in 2000 and yes, we asked those ridiculous “windows in New York City” and “golf balls in car” questions. As an aside, Google typically no longer asks those questions. The issue at hand is how to properly interview engineers, however before we dive into that, let’s talk about how other candidates outside of software engineering are typically interviewed.
How do we interview non-engineering positions?
When one wants to hire a salesperson, what does a typical interview look like? Typically there are 3–4 interviews, most of it focused upon previous experience. “Tell me about a time when” is the gist of nearly every question when interviewing this person as far as experience. There are usually also some culture questions, some questions around why the candidate is leaving, and why does the candidate want this role. The “other” category for questions is usually around something that’s happening or will happen at the place they are interviewing and how would the candidate handle such a situation. These are good questions for software engineers as well. One can rinse and repeat for marketing, business development, finance, etc. The only difference in sales, is there is often a “convince me of X question”. The reason here is to see if a sales person can close and deal with counterarguments which can happen often in a sales role. This sales exercise is designed to show raw, on the spot, sales skills. Nearly every software engineer reading this will know where I’m eventually going, the dreaded code exercise, where one “demonstrates” raw coding skills.
Before we jump into the code exercise, let’s see what it would look like if we were interviewing other positions outside of software engineers. First we will tackle a typical hiring criteria, then a criteria as if these positions were interviewed like software engineers. If we were to try to find a good money manager we would typically look at their past performance to see what we could glean from their track record. Even though a famous and legally obligated phrase is “past performance is no guarantee of future results”, this is the most common method for choosing a money manager. When hiring an artist, we would look at their portfolio, decide if we like their work and make a call as to hire or not, again a common method. When choosing a doctor, we find the type of medical doctor we require, and often times go online to check their results from people posting about the care they’ve received. What if we asked the money manager to detail out how much they remember from their initial certification (in the USA, typically a series 7) and chose based upon their recall of said exam? No examining their past portfolio results. What if we asked the artist to paint us a picture right then and there, of an image of our choosing, having never seen their portfolio and make a decision based upon that? And finally what if we asked the doctor to practice medicine on a crude drawing of patient with arbitrarily chosen symptoms, upon a whiteboard(NO GOOGLING). I would argue these are ridiculous criteria for hiring anyone, welcome to a software engineering whiteboard code exercise.
What’s the coding exercise?
The interview coding exercise is something nearly all software engineers are familiar with. In case you are not, it boils down to solve problem X on the whiteboard using code. Don’t use a computer, don’t Google for help, just recall stuff you learned in CS201, or Coursera, or for that matter have never ever had to use in your career, and put it upon said whiteboard with a marker. This is a farcical method for interviewing engineers. First off, Googling is one of the most commonly used techniques for software engineers regardless of years of experience. Secondly, not allowing someone to do their job using the tools of the job is also absurd, much like the doctor practicing medicine on a whiteboard. Additionally, I’ve never in my career had to construct a hash table (a common structure in software) from scratch, isn’t it enough that I know when to use one, can describe how it works, and that the bulk of languages come with this structure by default? Or that I can Google how to build one in case the language doesn’t have it included? Apparently, this isn’t good enough for many software interviews. Imagine if your doctor couldn’t consult a textbook/website to diagnose you, and if the diagnosis is wrong, they can’t bill you. By the way, 1 out of 5 people are misdiagnosed on the first attempt, but apparently that’s ok as long as my engineer can write code with a marker on a whiteboard to perfection. And this is the trouble with the whiteboard coding interview, it’s fundamentally unrealistic.
Anecdotally, I’ve worked with and known many software engineers who didn’t/don’t do well on whiteboard coding interviews but are the highest order of coders I’ve ever seen. Coding in front of people isn’t something that’s common in software engineering. Pair coding, coding with someone else, is common, but not someone scrutinizing your code on a whiteboard with little/no contribution. It’s a contrived exercise that requires an overhaul. Software interviews being broken is common knowledge in the community, but a major shift of the process has yet to take hold in the mainstream.
What’s the fix?
We already know the fix, we discussed it above. The fix is to ask for non-proprietary, original code the candidate has written and question them about it. Typically this would require them presenting on their laptop. Ask them to explain the code, the problem it solves, what would they change, etc. This would be viewing a software engineer’s “portfolio”. I’ve used this method and garnered tremendously positive feedback for a number of years now. This allows for a software engineer to be comfortable, you to probe fairly deeply, and know what you were looking for in the first place, e.g. can they pass your coding bar. In my experience, having done both interviewing methods, this method has not only improved the candidate experience as evidenced by candidate feedback, but also netted even more (read: headcount) high quality software engineers than the old method. My teams and I managed to hire 100 highly experienced(10+ years minimum) software engineers in just under 12 months using this method, 2 years later there was sub 3% turnover.
Why aren’t most companies doing this?
Viewing a portfolio for a money manager, artist, or patient feedback for doctor is normal, so why is this not normal for a software engineer? I have some theories on this, unfortunately none of them definitively provable. The first is that the old method was normalized before code was portable and viewable easily. In the 90s it would have been difficult to submit code, as easily accessible code on the web didn’t really exist in the way it does today, and most engineers coded on workstations not laptops, so transporting the code and setting up for an interview would be highly abnormal for the time. Another theory is that academia crept into the interview process through some tech company influencers during the same time period, thus coding a well-known academic data structure/algorithm on the whiteboard was a reasonable test for academic focused hiring. Another theory is that if you they code on a whiteboard, then using the normal tools would be a breeze for them and thus they’re solid enough and know the craft well. In my opinion, this is the weakest of the theories as all coding on a whiteboard shows is that they can memorize syntax and have done a number of common coding exercises to prepare, it shows none of what they would do in a production scenario, or code they’ve written that’s in the real world being used. The whiteboard also limits the complexity of code they can write, and as such is limited in showing off a software engineer’s skill set.
My last theory, the one I believe to be true, is laziness. Laziness is why most companies aren’t fixing their interviewing process. Even though massive companies with more resources and cash available than some mid-sized countries can’t find enough engineers, they refuse to admit fault with their hiring process. Not every software engineer is skilled enough to obtain a given position, of course, but whiteboard coding exercises as a criteria are the equivalent of claiming there’s a shortage of engineers you can hire because you want all of your engineers to be left-handed. You can find left-handed coders, but it’s an arbitrary bar that’s not relevant to the job. In 30+ years of coding I’ve never had to write code on a whiteboard for any reason other than an interview, which is why I refuse to subject candidates to it today. We can do better, so let’s do better.