Prepare for and practice interview questions from Metromile across topics like SQL, Python, Machine Learning and more.

Metromile Interview Questions

Metromile Interview Guides

A/B Testing

How would you assess the validity of the result?

Experiment Validity

How would you determine if the new model predicts delivery times better than the old model

Delivery Estimate Model

A/B Test Power Size

How would you analyze the effectiveness of this new feature?

Green Dot

How would you test whether the feature was working successfully

Delivery Online

Machine Learning

What metrics would you use to track accuracy and validity of the model

Spam Classifier 

How does random forest generate the forest and why use it over logistic regression

Random Forest Explanation

Explain the difference between XGBoost and random forest and give an example where you would use one over the other

Xgboost vs Random Forest

Product Sense & Metrics

How would you measure the success of Facebook Groups

Group Success

How would you measure the success of the Instagram TV product

Instagram TV Success

Measure Facebook Stories success by tracking reach, engagement, and actions aligned with specific business goals

Facebook Story Success

Select the 2nd highest salary in the engineering department

2nd Highest Salary

Get the top 3 highest employee salaries by department

Top Three Salaries

Statistics

Explain what a p-value is to someone who is not technical

P-value to a Layman

Find a bound for how many people drink coffee AND tea based on a survey

Bounds for Coffee and Tea Drinkers

Let’s say that your company is running a standard control and variant AB test on a feature to increase conversion rates on the landing page. The PM checks the results and finds a .04 p-value.

How would you assess the validity of the result?

Write a function <code>data_stream_median</code> to calculate the new median from a stream of ordered integers and the new element which is not yet inserted into the stream.

The median is the middle value in an ordered integer list. If the size of the list is even, there is no middle value, so the median is the mean of the two middle values.

Example:

<pre tabindex="0" class="chroma"><code>new_value = 2
stream = [1, 2, 3, 4, 5, 6]
def data_stream_median(new_value, stream): -&gt; 3
</code></pre>

Write a function data_stream_median to calculate the median from a stream of integers.

Data Stream Median

Data Structures & Algorithms

Write a SQL query to select the 2nd highest salary in the engineering department.

Note: If more than one person shares the highest salary, the query should select the next highest salary.

Example:

Input:

<code>employees</code> table

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<code>departments</code> table

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
Output:

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>

Given the <code>employees</code> and <code>departments</code> table, write a query to get the top 3 highest employee salaries by department. If the department contains less that 3 employees, the top 2 or the top 1 highest salaries should be listed (assume that each department has at least 1 employee). 

Note: The output should include the full name of the employee in one column, the department name, and the salary. The output should be sorted by department name in ascending order and salary in descending order. 

Example:

Input:

<code>employees</code> table

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<code>departments</code> table

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
Output:

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>employee_nam</code>e</td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>department_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>

Let’s say we want to build a new delivery time estimate model for consumers ordering food delivery.

How would you determine if the new model predicts delivery times better than the old model?

When rolling out a new delivery time estimate model in production, what is an important consideration?

When rolling out a new delivery time estimate model in production, what is an important consideration?

Delivery Estimate Model I

What is one way to measure the performance of a new delivery time estimate model for food delivery?

What is one way to measure the performance of a new delivery time estimate model for food delivery?

Delivery Estimate Model II

Let’s say you’re a Product Data Scientist at Instagram. How would you measure the success of the Instagram TV product?

What is the primary goal of Instagram TV as a product feature?

What is the primary goal of Instagram TV as a product feature?

How would you measure the success of Facebook Groups?

Considering the purpose of Facebook Groups as a platform for connecting users through shared interests or real-life relationships, which metrics would be appropriate to measure the success of these groups?

Considering the purpose of Facebook Groups as a platform for connecting users through shared interests or real-life relationships, which metrics would be appropriate to measure the success of these groups?

How would you explain what a p-value is to someone who is not technical?

What does a p-value in a statistical test represent?

What does a p-value in a statistical test represent?

P-value to a Layman I

In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?

In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?

P-value to a Layman II

How does random forest generate the forest? Additionally, why would we use it over other algorithms such as logistic regression?

What happens when you average the output on multiple decision trees?

What happens when you average the output on multiple decision trees?

Average Trees

Three zebras are chilling in the desert. Suddenly a lion attacks.
Each zebra is sitting on a corner of a triangle with sides of equal length.
Each zebra randomly picks a direction and only runs along the outline of the triangle to either edge of the triangle.

What is the probability that none of the zebras collide?

Three zebras are chilling in the desert. Suddenly a lion attacks.

Each zebra is sitting on a corner of a triangle with side of equal length. Each zebra randomly picks a direction and only runs along the outline of the triangle to either edge of the triangle.

What is the probability that none of the zebras collide?

Three zebras are chilling in the desert. Suddenly a lion attacks.

Each zebra is sitting on a corner of a triangle with side of equal length.  Each zebra randomly picks a direction and only runs along the outline of the triangle to either edge of the triangle.

What is the probability that none of the zebras collide?



Three Zebras

Brainteasers

Let’s say that you work for an E-commerce store. A new marketing manager joins the company and redesigns the existing new-user email journey to try and boost customer conversion rates. A few weeks after launching the new email journey, the new-user to customer conversion rate goes up from 40% to 43%.

Before you can celebrate, the company’s CMO points out a troubling fact. Just a few months prior to the new manager starting the conversion rate already stood at 45%, before slowly dropping down to the “starting” 40%.

How would you investigate if the redesigned email campaign actually led to the increase in the conversion rate, and that the increase wasn’t instead the result of other factors?

What would be an appropriate method to investigate whether the redesigned email campaign led to an increase in the conversion rate of an E-commerce store, in the absence of infrastructure for an AB test?

What would be an appropriate method to investigate whether the redesigned email campaign led to an increase in the conversion rate of an E-commerce store, in the absence of infrastructure for an AB test?

Causal Email Journey

How would you find out if an increase in user conversion rates after a new email journey is casual or just part of a wider trend?

Causal Email Journey 

Analytics

Let’s say you’re a data scientist at LinkedIn where you’re working on a product that sends qualified job candidates to companies. The team has launched a new feature that allows candidates to message hiring managers at companies directly during the interview process to get updates on their status.

Due to engineering constraints, the company can’t AB test the feature before launching it.

How would you analyze how the feature is performing?

How would you analyze the performance of a new LinkedIn feature that lets candidates message hiring managers during the interview process

Recruiting Leads

Let’s say you are tasked with building a spam classifier for emails. Assume that you’ve built a V1 of the model.

What metrics would you use to track accuracy and validity of the model?

When an interviewer asks a question along the lines of:

<ul>
<li>What would your current manager say about you? What constructive criticisms might he give?</li>
<li>What are your three biggest strengths and weaknesses you have identified in yourself?</li>
</ul>

How would you respond?

When asked about your strengths in an interview, what is an effective way to respond?

When asked about your strengths in an interview, what is an effective way to respond?

Your Strengths and Weaknesses I

Which of the following is an acceptable strategy when discussing weaknesses in an interview?

Which of the following is an acceptable strategy when discussing weaknesses in an interview?

Your Strengths and Weaknesses II

What do you tell an interviewer when they ask you what your strengths and weaknesses are?

Your Strengths and Weaknesses

Behavioral

How would you measure success for Facebook Stories?

Let’s say we’re working on a new feature for LinkedIn chat. We want to implement a green dot to show an “active user” but given engineering constraints, we can’t AB test it before release.

How would you analyze the effectiveness of this new feature?

How would you decide whether or not to create a product, like a job board, for Facebook?

Imagine Facebook is creating a job board. How would you design an A/B test to evaluate its effectiveness?

Assessing the market potential and then use A/B testing to measure its effectiveness against user behavior

Facebook Job Board Design

A survey asked 100 respondents two questions:

<ol>
<li>Do you like tea?</li>
<li>Do you like coffee?</li>
</ol>

70% of respondents said they liked coffee, and 80% of the respondents said they liked tea. Only 10% of respondents said they liked neither.

Based on this survey, what are the upper and lower bounds for the proportion of the population that like tea and coffee?

Given that 70% of people like dogs, 60% like cats, and only 10% dislike both, what is the proportion of people who like both dogs and cats?

Given that 70% of people like dogs, 60% like cats, and only 10% dislike both, what is the proportion of people who like both dogs and cats?

Bounds for Coffee and Tea Drinkers I

In a survey with 100 respondents, 80% like reading, 70% enjoy sports, and 10% dislike both. Using a 95% confidence interval, what are the lower and upper bounds of the proportion of respondents who like both reading and sports?

In a survey with 100 respondents, 80% like reading, 70% enjoy sports, and 10% dislike both. Using a 95% confidence interval, what are the lower and upper bounds of the proportion of respondents who like both reading and sports?

Bounds for Coffee and Tea Drinkers II

Explain the difference between the XGBoost and random forest algorithms and give an example where you would use one over the other.

Why might a data scientist choose to use XGBoost instead of Random Forest for a particular machine learning task?

Why might a data scientist choose to use XGBoost instead of Random Forest for a particular machine learning task?

XGBoost vs Random Forest: Choice

Let’s say you’re analyzing an AB test that has both a test group and a control group.

<ol>
<li>How do you calculate the sample size necessary for an accurate measurement?</li>

<li>Let’s say that the sample size is similar and sufficient between the two groups. In order to measure very small differences between the two, should the power get bigger or smaller?</li>
</ol>

Let’s say we’re building a new feature that tells dashers when they should go online or not in order to meet delivery demand. 

How would you test whether the feature was working successfully?

Let’s say we want to build a new algorithm for Lyft Line.

How should we test it, measure success of the test, and eventually roll it out?

Calculate Moving Average	SQL	Easy
Predict Customer Churn	Machine Learning	Medium
A/B Test Significance	Statistics	Medium
Optimize Query Performance	SQL	Hard
Feature Importance Analysis	Machine Learning	Medium
Clean Missing Data	Python	Easy
Neural Network Architecture	Deep Learning	Hard
Calculate Cohort Retention	SQL	Medium
Bayesian Probability	Statistics	Easy
Recommend Similar Products	Machine Learning	Hard

Metromile Interview Questions

Metromile Interview Guides

Metromile Interview Questions

Challenge

Metromile Salaries by Position

Discussion & Interview Experiences

Discussion & Interview Experiences