What can we learn from the people we interview? Interviewing Software Engineers

A few months ago, I was given the chance to interview a potential Graduate Developer at my company. That interview mainly involved reviewing their code submission and asking them a few basic questions about their thought process and how they would improve their code. Overall, for graduates our expectations aren’t very high but we want to find people that demonstrate good beginner developer qualities and people that demonstrate a level of honesty about their code without any ego in their somewhat limited prowess (with all due respect).

Last week I had the privilege of stepping it up a notch. I paired with our tribe’s Principal Developer, Brett, in a technical interview for a mid-level candidate. For mid-to-senior level candidates at the company I work for, the process involves three candidate interviews: a cultural interview, a technical interview and a coding interview. Some teams slightly differ but this is what we do in our tribe.

We jumped into a meeting room and interviewed our candidate for an hour, asking the person various technical-focused questions. I’d like to share some of my personal learnings from the experience.

Plan Of Attack: How do you as the interviewer prepare for a technical interview?

Assessing someone against a ‘skills matrix’

Brett, my co-interviewer and I had slightly different approaches to the interview initially so we made the time to meet beforehand and align our thinking. Brett prepared a copy of our company’s ‘Developer Skills We Value’ matrix which has a whole bunch of topics from programming to design, architecture, operations, Computer Science fundamentals and the like. He wanted to ask questions targeted towards some of these areas from what he considers to be a ‘good’ mid-to-senior level developer.

Assessing someone against what the team needs right now

My thinking was more along the lines of what we want from a person who’s hired in our team. I was thinking in terms of a person who can learn from us and that we can learn from. They need to be genuine full-stack (and that’s extremely hard to find in Australia). We have services in Ruby on Rails, Java with Spring Boot, front-end services in React and we’re considering introducing some Kotlin, not to mention we work in a complex domain alongside some legacy systems.

However, it’s not always about the tech. Ideally, we want someone with experience working in a production environment with at least some of these technologies but we also want someone with a little bit of Ops experience. We want someone who follows XP practices: who consistently applies TDD and sees the value in Pair Programming. We want someone who writes ‘clean code’ and we want someone familiar with good CI/CD pipelines, not to mention a basic understanding of Domain Driven Design.

If you’re going for a Senior Developer role, have you mentored junior developers in a formal capacity? I mean genuine mentoring, helping upskill people, and not just explaining what you’ve worked on to junior staff. Are you attending meetups or are you a part of any communities? What do you care about, and why? In short - do you care about being a great software developer or are you focused on being a good delivery robot?

Hybrid candidate assessment approach: Low formality focus

Brett and I aligned our thinking and came up with some notes on our combined approach. Above all, we agreed that it would be most important to keep it conversational so that our candidate felt comfortable being in the room with us. To do this, we would keep it ‘Q&A style’ for the majority of the session and before asking a question we would incorporate our thinking or our experience at the company by summarising what had just been said and relating it to our own approach, naturally then leading into a follow-up question if possible.

Of course, before all of these considerations, we individually reviewed the candidates code submission to the company, the candidates CV and the candidate’s Github profile.

When the candidate arrived, Brett went to collect the person from reception and I went to grab a glass of water for the person. Details matter.

During The Interview

Keeping up the pace of questions

It was important for us to keep up the pace throughout the interview - we only had an hour. We started with a brief overview of what we work on in our tribe, a brief high-level overview of how my crew fits into the whole of the company and a casual introduction for everyone in the room. This gave us the chance to ask the candidate ‘So what have you been up to the past few years, what have you been working on, what kind of tech stacks?’. At least this way they can start with a brief overview of their CV. From that, Brett and I were able to naturally pick one of the things they mentioned and lead it into a follow-up question. ‘It sounds like you’ve done a lot of front-end work but as we a primarily a back-end team, tell us a bit about some of the back-end development work you’ve done’.

Asking questions by taking the candidate’s world view

We asked the candidate all sorts of questions based on their experience and attempted to tailor it more to their world view where we could. For example, if they’d worked more on front-end with React we’d tailor a TDD question more-so in the context of React with Jest/Enzyme, and maybe a mention of snapshot testing, rather than with something like JUnit.

Avoid asking narrow questions with a single, obvious answer

One mistake that I made was asking a very specific question where I should have kept it a bit more open-ended. I asked the candidate what other tests besides unit tests and integration tests they might see in a CI/CD pipeline; the candidate needed further clarification on what I meant and how that related to a CI/CD pipeline. I explained that in our pipelines we perform smoke tests, performance tests and security tests before each production deployment. I realised that this question was far too specific to the way we work and perhaps the candidate hadn’t seen this type of setup before; I could have phrased it more in the way of what they think a good CI/CD pipeline might look like.

Post-Interview

Treat the candidate like a proper guest

Brett offered to take the candidate around for an office tour. We often do this for candidates, even if we’re not sure about whether they’ve definitely made it to the next interview round. It’s a nice thing to do and shows how open we are and willing to share our environment with any visitor. I think it’s nice.

Come together with co-interviewers and discuss your impressions

Soon after, Brett and I sat down for a ten-minute briefing to compare notes. We talked about what we thought about the candidate’s responses to some of the more important questions we asked. It was concerning to us that the candidate hadn’t applied TDD consistently and was willing to write untested code due to delivery pressure. It was also concerning to us that candidate struggled to answer what ‘good quality’ code is in their view, in a language-agnostic way. We’re looking for things like:

  • Good separation of concerns
  • Good domain modelling
  • Test quality & test comprehensiveness / the test pyramid
  • Clear intent
  • Readability, naming, comment pollution
  • SOLID Principles
  • Design –> Extensibility

Unfortunately, the candidate only addressed a couple of these key areas very lightly. And because of that, we could see their lack of experience in the way they fumbled through a very narrow view of writing code. Brett and I agreed on this and for that and various other reasons, we had to decline the candidate from proceeding to further interview rounds.

A ‘Good’ candidate vs a ‘not so good’ candidate?

I find it interesting how different companies assess a ‘good’ person from a ‘not so good’ person. I feel that on any article you read, people mention things like open-source contributions (coding in your own time), having a portfolio to show your interviewers, or even having a lot of Stack Overflow and/or Github contributions that shows you’re active and out there. I feel that at my company we glance over their Github and we take it as a positive (I even asked the candidate about one of their personal projects), however, I feel that it wasn’t really that important.

…Does your Stack Overflow profile actually even matter?

I feel that it was more important that the candidate had learned to be a great developer, consistently applying the modern-software-engineering-thingys from your Kent Beck, Martin Fowler and Bob Martin type of people and then were they able to explain all of those things in a way that we could imagine them applying the same principles in our team?

My feeling is that it didn’t matter so much what they had worked on but it mattered whether they approached what they had previously worked on by following good software engineering principles and then explaining the application of those principles in a succinct way. That, in my opinion, is the challenge for us in finding a great polyglot developer. Many developers out in the world focus on the language, on the technology, on the framework, but really it doesn’t matter what you used at the end of the day.

My Key Takeaways (for myself and other developers)

  • This type of “technical Q&A” interview is very hard for a candidate to prepare for. I feel that it’s about knowing your thought processes in the way you go about your job every day. I think it is a skill in itself to be able to articulate this out loud.
  • Are you are under delivery pressure to ship code quickly? Explain the value that automated tests provide to your product owners. Stand for something. Defend a good quality codebase.
  • Focus on applying good software engineering principles to your code. Read books often and apply the principles of those books: Kent Beck, Martin Fowler and Bob Martin at a minimum.
  • Apply TDD 100% of the time; if you get stuck, spike first, then delete and TDD. Keep up the TDD. We cannot reduce ourselves to writing what Kent Beck calls ‘cowboy code’.
  • Don’t take a narrow (language or framework focused) view of explaining core software engineering principles.