The Product Data Science Interview Guide

The Product Data Science Interview Guide

Overview

I remember the first time I got asked a product data science interview question. I was interviewing for a data science role at LinkedIn a year out of college. The interviewer came in and nicely asked me how I would determine if a new feature on LinkedIn messaging would affect the user experience.

I was pretty confident in telling them that we could AB test this new feature. But they responded that they didn’t have the capacity to do so. So now what?

I froze because honestly, I had no idea.

No one is really good at product interviews unless they’ve seen them before. It’s a specific type of interview question that is meant to test your ability to understand how to build products, and yet is especially tough to learn given the foreign nature of interpreting a case question in under an hour.

This specific type of interview is increasingly common for analytical roles. Big tech companies, such and Facebook and Google, focus mainly on product metrics and analytics, so data scientists function as an intersection between product managers, analysts, and software engineers. Data science, by definition, is the combo between product management and software engineering.

But I digress, let’s dive into how to solve product interview questions.

Why do product interviews come up?

Product data science interview questions stem from companies testing how you would respond to the natural life-cycle of a product.

Product Lifecycle model

When looking at the general roadmap for any kind of product in technology, there’s a team. This team usually includes a group of engineers, a product manager that sets up the roadmap and controls the major decisions, and one or two data scientists that measure product performance.

A typical timeline of the team’s decisions may look similar to this:

  1. The product manager asks the data specialist for product insight
  2. The two collaborate on building this feature based on data
  3. The feature is built, tested, and launched
  4. Further work is done to improve and analyze its successes or failures.

The data scientist role collaborates quite often with major product decision makers and builders, so these product data science questions often test your business intuition and ability to influence the product. A data scientist acts as a guardian of the data and is tasked with conveying the internal processes within the product, which the product manager relies on heavily as a key component of contributing to the product roadmap.

Many times, you’ll have to know what you’re investigating and the methods to measure success of a product, so these product data science questions have become increasingly important in the interview process.

Depending on the role, product interviews will come up at various frequencies. For instance, for machine learning roles, product interviews will almost never come up. The inverse can be said about product analyst interviews, where you’ll probably encounter more than one product interview.

Nevertheless, if we’re talking about big tech companies, almost every single data science role at Facebook, Apple, Amazon, Google, etc.. will ask a product interview question. And many times multiple times. This is due the very nature of the role being one that works with product owners very closely.

Frequency of Product Interview Questions

Product Interview Questions frequency

Product Data Science Interview Prep

Believe it or not, the most important thing you can do to prepare for a product interview is to do what the recruiters tell you.

Use. The. Product.

Don’t believe me? Think of the behavioral psychology standpoint.

Let’s pretend I am instead the interviewer in the last scenario I posted where I flunked the interview. I work at LinkedIn and spend 8+ hours a day thinking about how to grow and improve messaging between users. I run AB tests, have growth benchmarks, tested different designs, ran thousands of SQL queries and analyses, and now have just been pulled into an interview.

What’s my expectation of a data science candidate?

Well I mean, they don’t have to blow it out of the water, but they do have to be someone I like to work with. All of my coworkers are smart and friendly people, and I tend to be able to bounce some good ideas on InMail and other LinkedIn messaging features off of them.

Is this expectation realistic that this data science candidate in front of me has thought about LinkedIn messaging as much as my coworkers have? Probably not.

But as an interviewer, I am judging you against all the other people I have interviewed and the only other people I know, my data science coworkers.

Whenever you interview as a candidate, remember that your interviewer is likely testing you on knowledge that you do not have as a non-employee of the company. That means that to catch up, you need to do as much product research as you can.

Thinking about the product as a person that works at the company is very different from just using the product on a regular basis. While I may use LinkedIn for browsing articles and connecting with other professionals, I don’t use it in a way in which I am cognizant about which LinkedIn features have potential for growth in Q3. Looking at a company and understanding their product from a business perspective is imperative to understanding and connecting with employees in technical and product discussions during the interview.

Some ways to do this would be to guide your thinking with a few questions. For example with LinkedIn we can brainstorm:

  • How does LinkedIn make money?
  • What are some new features LinkedIn released. Why do you think they released them?
  • Which features on LinkedIn drew you into the website? Which features drew you into a particular product?
  • Why would a person pay money to use a certain service on LinkedIn?

The faster you start treating an interview like a business consultation, the better prepared you’ll be for a product interview.

Different Types of Product Interview Questions

image

At Interview Query we have bucketed these interviews into about five distinct categories. Some get re-worded so they’re trickier, some are repeated almost word by word from the examples we’ve found. Here are brief summaries on each type of interview question before we dive into the course structure below.

1. Investigating metrics.

These questions often center on a certain metric going up or down, and how this metric may communicate the health of a product. A common question template is “why is feature x dropping by y percent.”

For example: “Why are Facebook friend requests dropping by 10 percent?”

With this question, we’d want to know information about the drop– is it important, does it affect the business, what may be the cause? An additional addendum to this question is on fractional metrics, which play into a concept on analyzing a metric that is a fraction, for example: “Why are Facebook friend requests per user dropping by 10 percent?”

2. Measuring success

These questions ask for methods to measure the success of the feature or product. Examples could be “How would you measure the success of Facebook marketplace?” or “How would you measure the success of Yelp reviews?”

These are all features of an overlying platform or product, which ties into the product roadmap timeline that we referenced above as analyzing the success of released features. The company wants to know if the feature has improved anything and if it was worth the time and effort to build. At this point, the data scientist steps in and thinks of methodology to investigate the data, and determines its success.

3. Feature changes.

A typical question is in the format “Let’s say we want to add/change/improve a new feature to product X. What metrics would you track to make sure it’s a good idea?”

At first glance, this may seem similar to the previous question– you would track a feature change and its metrics. The small difference of a feature change is that the variables at play change as well. Instead of analyzing the entire product, you’re tasked with analyzing how this feature may have affected users.

4. Metric trade-offs.

You can imagine this question to be along the lines of “If you are the product manager for Facebook and you see that comments are down by 10%, yet reactions are up by 15%, how would you deal with it?”

This question is extremely common especially in huge platforms such as Facebook, Google, and Amazon, in which there are definitely causal effects on other features when the sites updates or new features are released. A data scientist in this situation would need to decide which metrics matter, why this change might have occurred, and tackle the severity of the change.

5. Growth.

A possible question could be “we want to grow x metric on y feature; how would we do that?”

Primarily, only companies focused on growth would utilize this problem, thus this question may appear in interviews for startups in growth mode or for companies with a huge acquisition team.

Lastly, these companies want to gain a sense of your strategic skill and business intuition in tackling these questions. A piece of advice is to think big and always take a step back before you dive in, as a common mistake is to instantly dive in and get pigeonholed into one specific idea. If you concentrate too much on a specific area, it’s difficult to envision the full cycle of the product or your ultimate product goal.

To get a better sense of similar questions asked in a product data science interview, check out this article on Product Analyst Interview Questions and Answers.

Learn more about Product Interviews

This course is designed to help you understand how to use product metrics and analytics to influence decisions.