Google's “Director of Engineering” Hiring Test
gwan.comThese are the prescreen questions that were used for senior SRE. It's not a hiring test- this person didn't even get to the actual hiring interviews (which would have asked you, for example, to implement quicksort in code on a whiteboard).
The recruiter-screeners in this case don't know anything, they're just looking for you to answer some questions against a table of "right answers", and they wouldn't know it if you answered in a way that was technically correct, but not in the answer key.
The reasoning behind this, sadly, is that Google thinks it tuned its hiring questions to reduce the rate of false positives- hiring an unqualified person into a role- at the expense of false negatives (not hiring a qualified person). Eventually, I stopped referring people to Google as the process was quite capricious. I told anybody who wanted to apply to read everything online about the process, memorize CLR and leetcode, and then tell the interviewers what they wanted to hear.
This is also what I tell people, including students, unfortunately. Interviewing at Google seems to do a reasonable job of preventing them from hiring anyone hopeless, but not a good job hiring people who aren't hopeless.
Most Googlers I know are not at all confident that they would get their job back if they had to reinterview for it, including those with top performance review ratings.
Heh.
I once helped author the hiring guidelines and questions used for recruiting and interviewing. After months of effort, the committee proudly, euphorically called the effort good.
I then asked the committee "Who here could pass the high bar we just set?"
Total buzz kill.
The upside was I was never asked to help again.
There’s also the story of one hiring committee that was proud of its high bar. It got pranked by someone feeding them their own packets, which they denied. It was in a tech talk somewhere, don’t recall where.
This is a valuable experiment to run for any hiring process.
Generally, you want to select for successful employees in your company, which hirers normally are. If they can't get selected, something is very very wrong with your process.
As a former SRE with over 25 years experience at companies small and large, I’ve never understood why an SRE would need to implement quicksort, or how the skills needed by a competent SRE - specifically, to guide developers in making their applications adhere to well-known reliability and scalability tenets, and capture and evaluate the data to show where problems may lie — are evidenced by such rituals. And those rituals tell you nothing about your ability to influence developers through your social skills to adapt the business and its processes to have successful outcomes.
Obviously Google has had some success in this space as their opinions are highly influential, but sometimes I can’t help but wonder if they were successful in spite of their interview practices—especially in the early days—because jobs at Google were much-coveted: talent from all over the world sought out a job there because the money and the prestige was just so damned good.
Google has two kinds of sres - the one you describe and just swes that are assigned to sre department that mainly write code.
What is CLR in this context? Common Language Runtime (Microsoft's .NET VM) doesn't seem applicable.
CLRS - the intro to algorithms book
Afaik this prescreen questionnaire is there bc it was statistically determined it fairs better than a resume screen at selecting candidates that would pass subsequent interviews
This exchange happened around 2016 or so, if I remember correctly. Already posted here a couple of times.
And it happened to me also. Those recruiters were hilarious, you cannot make that up. I got 8 of 10 by luck, but refused a 2nd exchange with those bots.
It’s great that they still don’t get how pointless it is to memorize things like protocol numbers for esoteric stuff that is trivial to google and find in a few seconds if the need should ever arise, which it rarely would. IIRC they asked me for the one for GRE.
I also had one (I forget the exact question) where my answer was technically correct but “wasn’t what [they] had”, and my explanation about why it was the same thing went way over the recruiter’s head.
Whoever thought this would be a fun little trivia game to have people play for their careers…yeah, no.
I still got through to the technical interviews wherein the first technical screener told me he doubts he’d be able to get his current position again now if he had to go through the interview for it, and he told me seriously not to feel bad if I don’t pass. I did pass that one.
Second technical screen was with a person with a thick accent and bad VoIP who was getting obviously frustrated due to both of us having to repeat stuff a lot.
This was all after a 6+ month dance of scheduling and waiting and hearing nothing from their recruiters. Got a rejection weeks later and never tried re-applying. They still reach out like every 6 months.
I had similar questions on 2016, although not quite as wrong/bad as presented here. E.g. "why is quicksort the best" wasn't asked. Instead, something more like "What is the average case time complexity of quicksort?"
Thanks for this - now I know what link to reply with for recruiters that keep spamming me as if I’d be interested in working at whatever company they’re spamming me for.
I had a similar experience a few years ago. Although I soon realized what the deal was, and what the expectations of the screener were (so I could adjust and ‘pass’), it discouraged me from moving forward.
It seems nowadays the process is much more reasonable, but Google is still far behind other FAANG companies in regard to their interviewing process, in my opinion.
The reason for this is that resume review is really ineffective as a screen. You end up having engineers wasting time on technical interviews that are clearly a pass in the first 10m (but you keep going for 45 to stay professional)
IMO, a better approach now are basic on line code screens designed to take 15m for a qualified candidate. The goal of which is to show you are serious about the tech stuff and give people a way to bow out gracefully.
> You end up having engineers wasting time on technical interviews that are clearly a pass in the first 10m (but you keep going for 45 to stay professional)
If someone is a pass in the first 10 minutes, the next 35 minutes are me trying to sell the company to the candidate. This is not, even slightly, a waste of time.
I have Allied Mastercomputer levels of HATE for whichever HR drone decided that "interviewing candidates" needed to be a bullet point in the "corporate evaluation process" and thus made the interviewing process even shittier than it needed to be.
I think the post you were replying to was using "pass" to mean "pass on the candidate" i.e. not move forward in the interview/hiring process. The point of such recruiter screens is to avoid an engineer seeing in a coding screen that the candidate has no chance in hell of success.
Hrm. Good point. I didn't catch that might have been the implication.
However, being professional is also important and I don't consider it a waste. I've experienced the reverse. It was clear after one interview that I was not a fit for the position.
I still ran the interview out even after pointing out that I really wasn't a fit because that's what professionals do. I later got a contract out of that company simply because I was professional about things and so many other people aren't.
Resume screening is very effective when done by someone who is intelligent. The tech industry refuses to take hiring seriously and apply such people to the problem.
I've read many resumes that seem solid. Then the candidate knows almost nothing. They just lie on the resumes.
I've also talked to people with terrible resumes who turn out to be great. They are just bad at resumes.
I second that. It's not necessary about "lying", many candidates may drop a keyword as something they know, when really they have only heard about it instead of mastering it.
Some people also under-rate their skills, because they are very demanding to themselves (the bad ones don't know "sort" and want to re-implement their own sorting method in Python).
I pre-screen using practical problems that I have to solve every day, and when candidates know their UNIX shell commands that's usually a really good sign.
I think the point of GP comment to yours is that by reading resumes skillfully -- that is, between the lines -- and comparing with other factors (e.g. their portfolio) it's pretty easy to tell the "probably solid" from the keyword-stuffers and buzzword droppers.
By "probably solid" I don't mean "immediate hire" but rather "It's a safe bet this person isn't utterly incompetent and I won't be wasting 10 minutes of my time talking to them."
I do not think it is possible to weed out all or even most of the completely incompetent programmers on the basis of resume screening. My reasoning is, I frequently conduct technical interviews, and a good percentage of those are with candidates lacking basic coding skills, and many of those candidates have impressively written resumes.
I would reckon (based on my own observation, combined with some research that has come out recently -- see below) that "a good percentage of this good percentage" probably failed due to nervousness, rather than than truly lacking basic coding skills.
If they failed your test -- you just know they failed your test. That doesn't mean you now that they're incompetent.
It is absolutely true that a good percentage of my candidates fail due to stress. As an interviewer do everything you can to decrease the level of stress, but there's no way to eliminate it. I often note, in interview feedback, my subjective belief that the candidate's poor performance is likely to have been a fluke due to the particularities of interview conditions. But in addition to candidates who make errors under pressure, there are many candidates who are unable to perform very, very basic tasks. There are many candidates, for example, who mess up basic syntax; who adopt bizarre naming conventions which suggest a complete unfamiliarity with functioning codebases; who confidently propose solutions which make zero sense, and are unable to answer basic questions about those solutions, but simultaneously are unaware that they are not answering the questions you pose. Many of these candidates have sterling resumes. At a certain point, you realize that resumes are not a good predictor of performance.
Indeed, the point was that resume screening can have >50% accuracy when done right. Unfortunately I’ve seen 5% as the accepted norm in the industry.
What is your bar for “terrible”? Resumes are a demonstration of written communication skill — they should at least be competent at expressing certain basic things about the candidate. An underwhelming list of accomplishments or lack of significant prior experience need not be showstoppers, of course.
I frequently administer technical interviews. For candidates who have passed an initial recruiter screen, I have ascertained no correlation between resume quality and code production. Many, many candidates with extremely impressive resumes are unable to produce basic, working code.
The reason for this is that resume review is really ineffective as a screen.
No is saying they should rely solely on resume review as a screen.
What is under question here is the perceived wisdom (on Google's side) of screening for Director positions using badly programmed chatbots (or recruiters, whatever) instead of having them be asked the same questions by an actual human.
IMO, a better approach now are basic on line code screens designed to take 15m for a qualified candidate.
Perhaps, but companies like to screw that up as well -- having people code into a Word doc, for example (try it sometime), or assigning problems that are either brain teasers or essentially willingness-to-cram filters rather than what they should be -- a simple, straightforward test of minimal programming ability (to determine whether further time investment is merited).
I've used (I think) hackerrank, and just done really basic stuff. Like freshman CS basics, starting with working code and extending it (eg a class that calculates the average of its contents, and you add min, max and median). Surprisingly it screens out about half.
Finding the median with or without sorting first?
If it's the latter (essentially a test of whether they've memorized quickselect) then that sounds like a horrible filter, actually. Unless it's for the position of "Senior Quicksort Engineer".
They are free to use the standard library, and use whatever language they like (eg python). It's not about how to sort
If some asks me to "calculate" something I assume they mean from scratch (without using a built-in). That's probably what a fair number of these people did.
Also, my sense is that Python is something of an outlier in having a mean and median functions in it standard library. I could be wrong, but AFAIK Go and JS do not, for example -- so people using those languages would surely bomb (at least on the median calculation part -- again, assuming they interpreted your question as "calculate from scratch").
I don't mean to be pedantic or split hairs. The point I'm trying to make is that even simple-seeming problems can have gotchas to them, depending on the context.
Interviewers could do better by either thinking just a bit more about the problems they select, or just communicating better. But many do not, unfortunately -- I have the sense they just pull problems out of the air, and see what sticks. Meanwhile counting the high number of fails as a success signal.
The site has a detailed spec and test input/output, you can build and run the tests
The standard library is to sort, not take the median
Well it does seem to be a cute filter for Python programming ability in that they definitely need to have taken a tour of the modern standard library to come up with
But still - no 'for' loops? No if-then-else? You might be giving some of these candidates a pass.min(x) max(x) statistics.mean(x) statistics.median(x)It's really just to make the point that you will need to write real code and answer technical questions in this process. And it provides an easy way to for them to drop out if they decide they don't want to do that.
Well put. I appreciate your clarifications.
"No [one] is saying ..."
Not surprising, the candidate did fail at acknowledge who the interviewer was to adapt their answers. It was a recruiter reading answers from a sheet, the simpler you keep it the highest the chance of passing the stupid test.
Some of those questions don't even have a single "will definitely match what's on the sheet" answer in the first place.
Not everyone wants to work at a place that needs you to dumb things down just to get your foot in the door as a Director of Engineering
Not everyone wants a director of engineering that can't communicate with the rest of the company.
Don't do that. Don't conflate having to dumb yourself down for recruiters that reached out to you for a director of engineering role with communication skills.
Yes sometimes you need to be able to simplify things for non-technical shareholders, but no that's not what this should be.
In an engineering focused organization if recruiters are reaching out for a position this high up they should either be solely acting as a data gatherer while a highly technical person reviews, or they should be technical enough themselves.
We're not talking about screening internships here, the person in the article didn't ask Google for an interview, it was on Google to provide competent points of contact. That doesn't necessarily scale if we're talking about applications they receive, but that's why the article states almost immediately it was an unsolicited interview.
At the same time, don’t conflate engineering skill with director level management skill, they are two vastly different things.
I don't see how this follows what I said in the slightest...
If the person can't tell the difference between "dumbing down" and communicating appropriately for the audience in order to get their message across then I think the process ending in "no hire" is working fine.
There's no level above which this stops being important. If anything, it only increases as you go up.
> for a position this high up
There are probably thousands of 'director of engineering' or similar roles at Google. If it's anything like other companies, they will be managing something like ~50 people. It's a middle management role, not that high up - except in compensation relative to the median...
Are we really downplaying being an L8 at Google now?
The point is that for a role with 1000 seats, at a turnover of 10% and the usual acceptance rate of 0.1-1%, they will be interviewing thousands of people for this role every year. It's not gonna be a white-gloves process like hiring a C-level.
See sibling comment, and also there is a gulf between C-level hiring loop and throwing clearly non-technical recruiters at L8s.
They didn't even need a technical recruiter, having the recruiter do their thing when it comes to soft aspects, like lightly checking dates, seeing if the person seems interested, etc. Then bringing in someone technical to review the technical stuff... it's really not asking that much.
L7 is < 3% and L8 is < 1% about of all of the employees specifically in the engineering department. Should be much less than 1000.
I'm going by headcount numbers found online. 4% of 27000 is >1000. And HR is probably dealing with hiring for director roles among the other 100k non-engineering employees as well.
> In an engineering focused organization if recruiters are reaching out for a position this high up...
How high up is this? What's a director in FANG-land? In some places, a manager of managers is a director - which isn't very high, and can be closer to the ICs than to the CEO on the org chart.
This isn't about saying a director is a 1 of 1 position, it's not.
But there are orders of magnitude fewer people are hired yearly into director positions than something like an L5.
So the expectation is past a point you're not going to reach out to someone and waste their time like this.
-
Also a manager of managers is plenty high, some places are just so bloated it doesn't seem that way.
You'd think someone you're trusting to have that level of influence is worth the extra effort
> But there are orders of magnitude fewer people are hired yearly into director positions than something like an L5.
That is true, but gives no indication of how many Director candidates they have to go through as Google has "a lot* of L5s. I would expect the C-Suite to get the high-touch approach, as they probably screen less than a dozen interviewees for CxO.
Let's ballpark this (simplified). Say there's a 1:3 manager to L5 ratio, and 1:5 director to manager ratio. We'd roughly expect 1 director to be hired for every 15 L5s. If Google hires 1500 L5s a year, then it would hire 100 corresponding directors. If it averages 20 applicants per post, that's about 2000 screens per year. Would that justify getting specialized recruiters? I don't know
Aside: this used to be the kind of "estimate/puzzle" question Google used to ask it's interviews, allegedly
> I don't know
Allow me to inform you it is. Your ratios are unrealistic even for estimating unless you really think there's 1 L6 for every 3 L5s at Google. Also we're talking about around an L8 for director:
Yet it still holds for those numbers that yes, if you can afford to reach out to them, you can afford technical staff to be in the loop. Because it's not just about the company, it's about respecting the engineer's time. Not that many people are qualified enough to be getting calls to be hired in as directors, it's not a small feat even if it's 100 hires a year. You don't want them having experiences like this if only for the image it gives off to other potential hires..
> Aside: this used to be the kind of "estimate/puzzle" question Google used to ask it's interviews, allegedly
Used to use about a decade ago? And I believe the moment you said "1:3 manager to L5 ratio" your interviewer would give you a subtle but questioning look and expect you to revise that.
You may have misread my comment - I said 1 director[1] to every 15 L5s.
> And I believe the moment you said "1:3 manager to L5 ratio" your interviewer would give you a subtle but questioning look and expect you to revise that.
I deliberately chose a high number to show that the number of Director interviews is still large after steelmanning your PoV (at the lower boundary of director-to-L5 ratio): I don't imagine many teams have 3+ L5s on them.
Also, in the spirit of learning, could you perhaps plugin numbers that you think are more reasonable (L5 per team, and number of teams a director is responsible for)?
1. I'm calling the L5's skip-level manager a Director - I'm assuming that's what they are called at Google, but I could be wrong. That's the ratio I'm calculating
You didn't misread it, I said 1 manager (L6)to 3 L5s.
L6 is Engineering Manager 1. That's how far up an L8 is.
And L6s are usually estimated to be 10% of Google. You can assume that there are order of magnitudes fewer devs each time you go up from there.
Also to be clear, it's extremely to be hired into Google higher than an L6. Even getting hired in as an L6 takes more experience than it does to join as an L5 and become an L6.
Most L6s at other FAANG companies would be downleveled without a very very strong performance, so getting reached out to as an L8 is not a small deal...
Either way, the point stands.
L8, above senior managers that are L7.
Yes - after “Recruiter: wrong, not "attributes", it's "metadata".” - it’s clear enough what’s happening. “stat() return an error code, not an inode” - I know this isn’t what’s on the paper, and the author should have also. “stat() let you get the inode number” would have let you stay pedantically correct yet still satisfy the recruiter.
“What is the type of the packets exchanged to establish a TCP connection?” - as soon as I read the question, I knew right away the author would give the exact hex, and the recruiter would want the three letter acronyms.
Perhaps this was a brilliant engineer disguising a personality interview as a technical one. Personally I would want my coworker or boss to be someone with enough awareness to have passed this interview. Compassion, empathy, social awareness, understanding someone else’s role and perspective, setting aside your ego to help a conversation go smoothly - important for a director to have.
Send them an invoice for your wasted time.
These questions would make for a fun tech pub trivia night.
I wonder how many teams of working engineers would get them right without cramming?
This reads like it was written by Marvin from Hitchhiker's Guide to the Galaxy.
"Here I am, brain the size of a planet, and they tell me to take you up to the bridge"
It is so typical to interview very senior people who get really arrogant about their skills, but when it comes down to it, they fail at this rapidfire, and in a very non graceful way. This rapidfire is very coarse and weeds out most incompetent people. It's likely not your answers, but your attitude that failed you.
Unlikely, unless the interviewer was just trolling them.
If it was just their attitude, they’d not tell them they were wrong when they weren’t and tell them to work on x y and z to be qualified in the future. They’d just complete or end the interview and say something like, “Great! You should hear back in x days” and then reject in their ATS.
If not, then the recruiter is an asshole and gaslighting the candidate.