Author Archives: Hugh E. Williams

Unknown's avatar

About Hugh E. Williams

Founder, Advisor, Professor, and Investor. Former Tech Exec.

Hiring Engineers

3 Replies

I’ve spent much of the past seven years helping recruit great candidates. I’ve probably interviewed over 1,000 people (wow!). In this post, I thought I’d share some of the experiences and beliefs I’ve built up along the way. Before I start, I should say that Ken Moss and Jim Walsh have shared with me their interviewing philosophies and experiences over the years, and much of what I say below was influenced by them.

Sourcing Candidates

The most successful source of candidates is personal referral. Why? The interview process only estimates whether an engineer is great – we all want the error bars to be small, but there’s only a certain amount of information you can gather in a day of interviews. Having prior knowledge of a candidate is invaluable – if you’ve worked or studied with them, you’re able to decrease the error bars significantly. Moreover, there’s the importance of the human side – if you know someone, and you’ve enjoyed working with them, and the feeling is mutual, you’re already ahead of your competitors and you’ve already given the candidate a reason to come work with you. (Sourcing through a recruiting team is also important, but in my experience the error bars are higher, the success rate is lower, and the engineering team’s knowledge isn’t directly applied to the sourcing problem.)

All up, one of the most important things you can do to help build a great team is make personal referrals. Take an old colleague out to lunch!

Great Interviews

I’m passionate about great interviews. I believe in two things: the candidate must have a great experience, and you must interview for core competencies (and not skills and training). Having a great experience is important: even if a candidate isn’t successful in the interview, they’ll talk to their friends, and spread the word about the interview experience – you want that message to be positive. Hiring for competencies is also important: if you focus only on skills, you may not hire people who can grow and change as your business and technology grows and changes.

What are core competencies? They’re inate traits and abilities, such as integrity, communication skills, and courage and conviction. There’s a company called Lominger that has worked hard on developing a list of the competencies, and ways to describe them and help you understand how to assess your profiency at each one. The only list in the public domain that I can find that’s similar is here.

I believe that all competencies are important, but that there are four that are critical to being a successful engineer at the companies I’ve worked at:

  1. Intellectual horsepower
  2. Problem solving skills
  3. Drive for Results
  4. Action-oriented

Interviews should focus on uncovering the capabilities of the candidate at those four compentencies. (Again, I emphasize that other competencies are important – none of us want to work with folks with low integrity, no sense of humor, or no interest in valuing diversity. But I’m much happier approximating the candidate’s skills at those after the interview, rather than making them the focus.)

Intellectual horsepower is basically being smart, and being able to learn and grow when presented with new knowledge. In interviews, I typically measure this by how fast the candidate understands the questions, the types of questions they ask, and how “fast paced” the conversation is. If I learn something from talking to the candidate, where I’m provoked to have a new thought, I’m usually satisfied that the candidate has intellectual horsepower.

We want problem solvers who can solve complex challenges in code. We don’t need software engineers who can only solve an organizational challenge, figure out a clever physics problem, or solve puzzles from NPR’s Car Talk. It’s therefore essential to present computer science problems, and ask the candidate to solve them with real code. I’m not a fan of pseudo code, and I tend to ignore the output of interviews that don’t have real, hard, problem solving problems with coding solutions. Some of my favorites questions for recent college graduates are: reverse a linked list, and write a program to shuffle a deck of cards. (The former either gets a recursive solution, or uses a stack; if they get it fast, ask them to try the other solution. The latter is best solved with a single pass through an array, where each element is swapped with another random element.)

Drive for results means getting things done for a reason, with zeal, and a strong desire to reach a conclusion. People with this competency are “finishers” and will deliver results for the customers and business, and they realize that getting it done is more important than making it perfect. These kinds of people are scrappy, and make the right tradeoffs, and they’re the ones who work the smartest. How do you figure this out? If I’m interviewing college graduates, I ask about their favorite project while they were at college – do they talk about the customer? The impact it had or could have? Was it actually a summer job that shows their passion for results in the real world? Or was it just a technology for technology’s sake? Is it esoteric or applied? Try asking the question, you’ll get an instant feel for what I mean. If it’s someone more experienced, I generally ask about a project or team they’ve enjoyed being on.

Action-oriented means getting started, and being decisive about starting (and often figuring out only what is needed before beginning). These folks are more action and less talk. They’re the ones you will perceive as hard working. This is a hard competency to explicitly understand in an interview – but you can often observe it in their approach to problem solving questions. Do they jump up to the whiteboard, grab a pen, and start solving? Do they make and state assumptions, just so they can get on with it? Or do they endlessly push back and ask questions? Do they criticize you and your question? Do they try and divert the interview somewhere else? Do you have to ask them to get up and use the whiteboard?

When you’re done with an interview, I believe it’s critical to write down the questions you asked and what you learnt. I typically write down what I learnt about the four competencies, and I always begin my writeup with a definitive statement of whether or not I’d hire the candidate. Great decisions are only made after considered thought – and writing it down makes you think, and makes you stand behind a decision. It’s also very useful – others will learn what you ask and how you interpret it, and it’s also a great record for when a candidate applies again at a later date (this happens more frequently than you’d expect). A good interview writeup is several paragraphs in length in my experience.

Great Experiences

We want to win the hiring race. Great candidates will typically interview with multiple companies, and often have multiple offers. When it comes to decision time, they’ll reflect on more than the position you’re offering and the compensation. They’ll think hard about the people they met, the questions they were asked, and how they were treated during the interview process.

Here are some basic tips:

  1. Don’t ask the same question as someone else – understand what’s been asked so far, and show that you know who they’ve already talked to
  2. Read their resume, show interest in them and their experiences. Often, I look for the unique thing and ask about it – “How long have you been playing guitar?” or “How did you enjoy living in London?”
  3. Leave 5 or 10 minutes to answer questions, sell your experience at the company, and give the candidate a chance to use the restroom or get a drink
  4. Show that you’re smart, great at problem solving, you’re action oriented, and driven for results. Be engaged, animated, excited, and passionate about what you’re doing – great folks want to work with great people
  5. If you’re the manager, make sure nothing slips between the cracks. Stay close to recruiting and the candidate, and make sure everything happens in a timely fashion – even when we don’t want to offer a role, make sure the “regret” experience is timely, in person (not an email!), and professional

The bottom line is it’s like running a retail business. Part of success is having a great customer experience – if you upset someone, trust me that you’ll upset five more through word of mouth. On the flip side, do it well, and you’ll have an enhanced reputation as a great technology company that’s well worth considering as a destination.

Thanks for reading. I hope this helps you hire more great folks!

Ideas and Invention (and the story of Bing’s Image Search)

23 Replies

I was recently told that I am an ideas guy. Probably the best compliment I’ve received. It got me thinking, and I thought I’d share a story.

With two remarkable people, Nick Craswell and Julie Farago, I invented infinite scroll in MSN Search’s image search in 2005. Use Bing’s image search, you’ll see that there’s only one page of results – you can scroll and more images are loaded, unlike web search where you have to click on a pagination control to get to page two. Google released infinite scroll a couple of years ago in their image search, and Facebook, Twitter, and others use a similar infinite approach. Image search at Bing was the first to do this.

MSN Search's original image search with infinite scroll

How’d this idea come about? Most good ideas are small, obvious increments based on studying data, not lightning-bolt moments that are abstracted from the current reality. In this case, we began by studying data from image search engines, back when all image search engines had a pagination control and roughly twenty images per page.

Over a pizza lunch, Nick, Julie, and I spent time digging in user sessions from users who’d used web and image search. We learnt a couple of things in a few hours. At least, I recall a couple of things – somewhere along the way we invented a thumbnail slider, a cool hover-over feature, and a few other things. But I don’t think that was over pizza.

Back to the story. The first thing we learnt was that users paginate in image search.  A lot. In web search, you’ll typically see that for around 75% of queries, users stay on page one of the results; they don’t like pagination. In image search, it’s the opposite: 43% of queries stay on page 1 and it takes until page 8 to hit the 75% threshold.

Second, we learnt that users inspect large numbers of images before they click on a result. Nick remembers finding a session where a user went to page twenty-something before clicking on an image of a chocolate cake. That was a pretty wow moment – you don’t see that patience in web search (though, as it turns out, we do see it at eBay).

If you were there, having pizza with us, perhaps you would have invented infinite scroll. It’s an obvious step forward when you know that users are suffering through clicking on pagination for the bulk of their queries, and that they want to consume many images before they click. Well, perhaps the simplest invention would have been more than 20 images per page (say, 100 images per page) – but it’s a logical small leap from there to “infinity”. (While we called it “infinite scroll”, the limit was 1,000 images before you hit the bottom.) It was later publicly known as “smart scroll”.

To get from the inspiration to the implementation, we went through many incarnations of scroll bars, and ways to help users understand where they were in the results set (that’s the problem with infinite scroll – infinity is hard to navigate). In the end, the scroll bar was rather unremarkable looking – but watch what it does as you aggressively scroll down. It’s intuitive but it isn’t obvious that’s how the scroll bar should have worked based on the original idea.

This idea is an incremental one. Like many others, it was created through understanding the customer through data, figuring out what problem the customers are trying to solve, and having a simple idea that helps. It’s also about being able to let go of ideas that don’t work or clutter the experience – my advice is don’t hang onto a boat anchor for too long. (We had an idea of a kind of scratchpad where users could save their images. It was later dropped from the product.)

The Live Search scratchpad. No longer in Bing's Image Search

I’m still an ideas guy. That’s what fires me up. By the way, Nick’s still at Microsoft. Julie’s at Google. And I am at eBay.

(Here’s the patent document, and here’s a presentation I made about image search during my time at Microsoft. All of the details in this blog post are taken from the presentation or patent document.)