For many organizations (mine included), November is performance review season. I know it’s not everyone’s cup of tea, but I quite enjoy both getting and giving of constructive feedback. So much so that apparently I like writing about how I like performance reviews every November as well.
Today’s post is about specific questions for collecting feedback from stakeholders about a person you’re reviewing. Giving such feedback is uncomfortable for many people; it can feel like tattling or speaking ill of someone behind their back. But as a manager, my perspective is inherently limited (and usually biased). I require input from those who are working with my team day-to-day in order to fairly evaluate them and give them the feedback they need (both growth opportunities and especially praise they may not otherwise hear).
I’ve used a variety of approaches in the past, with mixed success. But I recently formulated a pair of questions in conversation with a career coach that I think can coax out the desired information without making stakeholders have too icky a feeling about giving it. They’re delightfully simple, and can be answered quickly and discreetly via BCC-ed email or even DM:
Spent my morning debugging a data extraction pipeline that’s been failing to properly update a dashboard, despite it completing without error (lovely). Digging through the output of the pipeline software got me thinking about logging. There’s a real art to getting it right: too much is impossible to parse (and potentially leaks sensitive information), but too little is useless. Sometimes the difference between finding the source of the problem and not is having logs in a Goldilocks Zone.
In today’s case, the log looked something like this:
- Calling the API
- The API call succeeded
Better than nothing, at least it would have let me know if the API call was failing altogether. But this would have been so much more helpful:
- Calling the API with parameters X=a and Y=b
- The API call succeeded and returned N items
I updated the data pipeline to provide the above output, and will see how it runs tonight.
Detailsmatter, y’all! I wish I knew a way to teach the above to software engineers, but I fear there’s no real way to learn except through the experience of having to do this sort of debugging. It’s not fun work, but it’s oh so valuable.
Sometimes certain bits of writing resonate in such fundamental ways that you come back to them time and time again. I have a couple of those in my personal brag document, for example. And these two articles, which I share regularly when describing my profession:
I bet I read both of these at least 4 times a year. They do not get old.
Speaking of re-reading, it’s a pretty rare thing for me (for reasons). Off the top of my head, the only books I know for certain I’ve read more than once (and in all cases still only twice) are The Lord of the Rings trilogy, The Hobbit, The Chronicles of Narnia, the Foundation Trilogy, and (somewhat randomly) Thinking in the Future Tense.
Had to fill out an online form yesterday to register for the chance to upgrade my season tickets to the San Diego Wave. The first draft of this form had some issues…
Misunderstanding of checkboxes and radio buttons is a classic, but inadvertently applying phone number validation to what is definitely not a phone number field is a new one.
Add this post to the cacophony, I guess. Today I want to cover a number of common interview mistakes, and approaches both interviewees and interviewers can take to mitigate them. That’s right, it’s a both/and responsibility. I know there have been times I’ve come to an interview unprepared to give the candidate my best, and the result was possibly missing out on a good hire. Don’t do that.
Mistake 1: Regurgitating resume content
As an interviewee, if you’re asked to introduce yourself, give an overview of your background, or describe the projects you’ve worked on, be sure to bring something unique to your answer beyond what’s on your resume. Otherwise you’re just wasting time that’s better spend on other topics. One idea: have an elevator pitch for yourself.
And interviewers? Avoid this by 1) reviewing the candidate’s resume ahead of time, so you don’t feel the need to ask for an overview, 2) gently interrupting if an answer veers into rote regurgitation, or 3) avoid these kinds of questions altogether. Better to just jump into the meat of the conversation, because interview time is precious. Speaking of…
Mistake 2: Wasting too much time giving introductions
While there’s value in easing into a conversation that’s high-stakes, it does no one any favors if a big chunk of time is spent on introductions that are not data-rich. That goes both for the interviewee and the interviewer. When I led interviews at Amazon I had a script so that I could get through my boilerplate in a crisp 60 seconds. Say hello, set some ground rules, describe the objective of the interview, and then jump in.
Mistake 3: Too much “we” and not enough “I”
Interviews are not the time for false humility. We know doing anything of reasonable complexity requires a team, but what was your role specifically? It’s easy to fall into this trap; if you’re giving the interview, don’t let your candidate go too long before redirecting them to talk more about themselves. If they struggle to give details about their own actions, that’s a concern! Closely related to this next mistake:
Mistake 4: Talking in generalities
This is a fatal mistake of which I’ve written before. Twice. Seriously, tell real stories that actually happened. As an interviewee, this is where preparation is key. Stories are much easier to tell when you’ve captured the details ahead of time. The sorts of behaviors employers are going to ask about aren’t rocket science (unless you’re interviewing for NASA, I suppose). Do a bit of research on the company’s values and prep stories that align to them.
As an interviewer, this is another case where’s it’s totally okay to interrupt and redirect if you think a candidate is drifting into theory. But help them along by phrasing questions effectively (i.e. “tell me about a time when…”) and perhaps even setting expectations in your intro that you want real examples. If you actually do care about a theoretical answer, say so explicitly.
Mistake 5: Telling the same story twice
Good interview teams are going to talk about the stories they heard during a debrief, so there’s no reason to tell the same story to two different interviewers. This gets easier if you’ve done your prep work. At absolute worst, if a story is rich enough to answer multiple questions, make sure you tell it from different angles.
And interviewers, coordinate ahead of time on the questions you’re going to ask. It’s a waste of time for there to be duplication between interview sessions. It’s a completely avoidable but all too common error that doesn’t speak well of your company when it happens.
Mistake 6: Telling stories not aligned to the level of the role
Candidates should understand the requirements of the role, especially as it relates to the degree of seniority, and be able to give examples that illustrate the required complexity and significance. If you’re asked about a time you failed, and you’re interviewing for a leadership role, you better be able to share a situation that involved real consequences. And if you’re aiming to be a senior engineer, I need tohear about more than just the cool code you wrote… how did you lead?
Interviewers, you need to be familiar with expectations for the role as well, so you can identify these “weaksauce stories” when you hear them. Sometimes follow-up questions can save them, but often I find that it’s better to step in and ask if the candidate has another example.
My favorite way to suss out the significance of a story is to ask a candidate about the people they were talking to at the time: only other engineers at their level? Related roles like product managers or salespeople? Managers? Directors? What about people outside their company, were they interacting with customers directly? Vendors? Other stakeholders? Answers to these questions will tell you a lot about where a person fit in their prior organizations and if they’ll be suited for the role you’re offering.
Mistake 7: Focusing too much on technology
Obviously having the technical skills required of the role is important, but just as critical is the leadership you show in applying those skills. So be sure to talk about those parts of the job too. Keep your stories balanced.
Interviewers also have responsibility here, to keep the questions they ask balanced between technical and behavioral. Amazon did a great job of this; in my experience they put more weight on behaviors aligned to their Leadership Principles during their interviews than any other tech company. Having done over 150 interviews during my tenure there (about half as a Bar Raiser), it’s a lesson I won’t soon forget.
Mistake 8: Failing to discuss results
I love the STAR method for structuring stories: Situation, Task, Action, Result. But each of these parts of the story are not equally important: they grow in significance from left to right. Results are the most critical parts of a story to tell, so make sure you don’t spend too much time describing the situation alone. What was the outcome? If you can share specific numbers, even better (you did prepare your stories, right?)
Interviewers can easily help their candidates here by always asking about results if they’re not shared proactively. Don’t commit this next common error:
Mistake 9: Rushing too quickly to the next question
Stories have layers; take the time to dig into them (10 minutes per story is my rule of thumb) with good follow-ups before moving on to your next prepared question. Sometimes the perfect follow-up emerges organically, but if it doesn’t, keep these classics at the ready:
Also, don’t be afraid of a bit of silence to be sure the candidate is completely done with their story. In graduate school I was trained to pause for 8 full seconds, a practice I still use today. Especially don’t fill silence by doing the following:
Mistake 10: Leading the witness
Interviewers, keep your questions open-ended, and resist the urge to prompt candidates on the “correct” (from your perspective) answer. Let them give the answer they want to give, because that’s what you’re there to evaluate. I recognize this can be difficult, especially if a candidate is struggling; human nature is to want to help. Gentle nudges are okay, but it’s easy to give too much.
Full disclosure: I’m bad at this one. It goes against what are normally healthy collaborative impulses. At least be aware of the tendency. Speaking of fighting against otherwise healthy tendencies:
Mistake 11: Providing too much real-time feedback on answer quality
This last one might be controversial, but I stand by it. Candidates sometimes want to get feedback on how they’re doing, but doing so is tricky territory. For one, it’s not a good use of time. Second, off-the-cuff judgments on candidates are rarely as helpful as thoughtful consideration, so an answer given in real-time might not be a good one. Finally, you don’t want to give a false impression that you might have to go back on later if you decide not to proceed with them.
I’m not saying being completely stoic, nor am I saying not to give some gentle redirections if you’re not getting what you need (as I’ve said throughout this post), but if asked directly, keep judgments on answer quality to yourself. Depending on the situation, I either use some version of “I heard what I needed to” and move on, or ask a specific follow-up if I think there’s more to discover.
Much digital ink is spilled on LinkedIn pontificating about hiring processes, what recruiters and hiring managers should and shouldn’t do, etc. There’s some truth amongst the noise, but I get a good chuckle when perspectives are shared with no basis in either data or experience.
This is especially true of management advice. If you haven’t actually been a manager, your opinions on how to do it are worth less than a byte stored in S3 (i.e. not much). Go give it a try and let’s talk again, eh?
Even if you don’t see yourself in such a role long-term, I recommend everyone do a tour of duty as a people manager if they can. You’ll gain valuable insights into what it takes do the very human work of running a business.
This week I got into a friendly debate about developer onboarding in two different fora (the Rands Leadership Slack and an in-person CTO Lunches gathering). My hot take: technical onboarding documentation is at best over-rated, at worst counterproductive, and most organizations shouldn’t give it much thought.
Before you pick up stones, hear me out. My logic is based partly on experience, and partly on a theory. The experience is that writing detailed onboarding steps takes considerable time, usually from well-tenured developers whose attention is better spent elsewhere. Keeping said documentation up-to-date takes even more time and is nearly impossible to do with a fast moving product. And having half-baked or incorrect instructions is worse than nothing for a new team member. Putting in all this effort just to save a little bit of onboarding time doesn’t make sense. The juice isn’t worth the squeeze.
But there’s more. My theory is that having an onboarding process spelled out in painstaking details actually robs the newcomer of the chance to build muscle through struggle, shortcuts creative exploration of how things could be better, and sets an expectation that “getting up to speed” means following a mechanized set of steps instead of self-directed discovery of a codebase and getting to know fellow team-members and their institutional knowledge through asking excellent questions.
Sure, this might mean some initial frustration. That can be managed. And it might mean a new developer is slower to begin delivering value. But my hypothesis is that once they do, they’ll be better prepared to provide broader and deeper value in the long run.
And besides, it’s likely your system isn’t as complex as you think it is, and you should be looking for the kind of people that can do their own problem-solving. If it takes someone many days just to get an app running, you’ve got more serious problems. Which makes me think this could make for an interesting interview exercise: “Here’s our code repository. Get it running in a fresh environment.”
I did tons of hiring during my time at Amazon. I haven’t done as much in my new role, but that’s starting to change (any open roles I’ll put on my LinkedIn profile if you’re interested in checking them out).
There’s a lot I could say about the work of both identifying good candidates and presenting yourself well when you’re looking for a job. But I’ll keep it brief today.
And second, during interviews, my number one thing: tell me what you did, not what you would do. Unless I specifically ask for a theoretical answer, I want to hear actual stories of actual work and actual outcomes. Experience is evidence.
And as it turns out, color is hard too! Did you know there are “imaginary colors”? I didn’t? How cool!
This is not an argument to run away from technology, but to say that we who do this work must be vigilant and realistic. The answer to “how long” is never “five minutes”. And we must engage across a broad set of disciplines, because our own perspectives are limited.
When confronted with complexity, the wrong answer is to retreat to comfortable simplicity. Read. Listen. Have an open mind and broaden your view of the world.
