Prepare for and practice interview questions from ADM across topics like SQL, Machine Learning, Statistics and more.

ADM Interview Questions

ADM Interview Guides

A/B Testing

Experiment Validity

Delivery Estimate Model

Button AB Test

A/B Test Power Size

Network Experiment Design

Machine Learning

Encoding Categorical Features

Bagging vs Boosting

What are the logistic and softmax functions? What is the difference between the two?

Softmax vs Logistic

Food Prep Features

How to model merchant acquisition in a new market?

Merchant Acquisition

Product Sense & Metrics

How do we measure the success of acquiring new users through a free trial

Netflix Retention

Group Success

WAU vs Open Rates

Instagram TV Success

Uber Eats Success

Data Structures & Algorithms

String Shift

Given two nonempty lists of user_ids and tips, write a function to find the user that tipped the most.

Biggest Tip

Moving Window

Given a json string with nested objects, write a function that flattens all the objects to a single key-value dictionary.

Flatten JSON

Write a function to find how many friends each person has.

How Many Friends

Select the 2nd highest salary in the engineering department

2nd Highest Salary

Download Facts

Write a SQL query to count transactions filtered by several criterias.

Count Transactions

Order Addresses

Over-Budget Projects

When an interviewer asks a question along the lines of:

<ul>
<li>What would your current manager say about you? What constructive criticisms might he give?</li>
<li>What are your three biggest strengths and weaknesses you have identified in yourself?</li>
</ul>

How would you respond?

When asked about your strengths in an interview, what is an effective way to respond?

When asked about your strengths in an interview, what is an effective way to respond?

Your Strengths and Weaknesses I

Which of the following is an acceptable strategy when discussing weaknesses in an interview?

Which of the following is an acceptable strategy when discussing weaknesses in an interview?

Your Strengths and Weaknesses II

What do you tell an interviewer when they ask you what your strengths and weaknesses are?

Your Strengths and Weaknesses

Brainteasers

What are the assumptions of linear regression?

Which assumption of the residuals of the standard linear regression model can not be overcome by increasing the sample size?

Regression assumptions

What are the assumptions of linear regression?

Assumptions of Linear Regression

Statistics

Let’s say that you’re training a classification model.

How would you combat overfitting when building tree-based models?

Let's say that you're training a classification model.   How would you combat overfitting when building tree-based models?

Overfit Avoidance

How would you handle the data preparation for building a machine learning model using imbalanced data?

Addressing imbalanced data in machine learning through carefully prepared techniques.

Data Preparation for Imbalanced Data

Let’s say we’re comparing two machine learning algorithms. In which case would you use a bagging algorithm versus a boosting algorithm? 

Give an example of the tradeoffs between the two.

In machine learning, when would you use a bagging algorithm over a boosting algorithm?

In machine learning, when would you use a bagging algorithm over a boosting algorithm?

Bagging vs. Boosting

Write a SQL query to select the 2nd highest salary in the engineering department.

Note: If more than one person shares the highest salary, the query should select the next highest salary.

Example:

Input:

<code>employees</code> table

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<code>departments</code> table

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
Output:

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>

Explain the difference between the XGBoost and random forest algorithms and give an example where you would use one over the other.

Why might a data scientist choose to use XGBoost instead of Random Forest for a particular machine learning task?

Why might a data scientist choose to use XGBoost instead of Random Forest for a particular machine learning task?

XGBoost vs Random Forest: Choice

Xgboost vs Random Forest

How does random forest generate the forest? Additionally, why would we use it over other algorithms such as logistic regression?

What happens when you average the output on multiple decision trees?

What happens when you average the output on multiple decision trees?

Average Trees

Random Forest Explanation

Let’s say that you work at a B2B SAAS company that’s interested in testing the pricing of different levels of subscriptions.

Your project manager comes to you and asks you to run a two-week-long A/B test to test an increase in pricing.

How would you approach designing this test? How would you determine whether the increase in pricing is a good business decision?

Testing Price Increase

A team wants to A/B test multiple different changes through a sign-up funnel.

For example, on a page, a button is currently red and at the top of the page. They want to see if changing a button from red to blue and/or from the top of the page to the bottom of the page will increase click-through.

How would you set up this test?

Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.

How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.

Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.

How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.

A/B Test Variants

Let’s say we want to launch a re-design of a landing page to improve the click-through rate. We can do this by implementing an AB test.
Given that we launch an AB test, how would you infer if the results of the click-through rate were statistically significant or not?

How can you accurately conclude if the results of an A/B test, conducted to evaluate the effectiveness of a landing page redesign, are statistically significant?

How can you accurately conclude if the results of an A/B test, conducted to evaluate the effectiveness of a landing page redesign, are statistically significant?

Statistically Significant Test

Precisely ascertain whether the outcomes of an A/B test, executed to assess the impact of a landing page redesign, exhibit statistical significance.

Let’s say that your company is running a standard control and variant AB test on a feature to increase conversion rates on the landing page. The PM checks the results and finds a .04 p-value.

How would you assess the validity of the result?

Let’s say you work at Allstate. Allstate is running <code>N</code> online ads right now. The table <code>ads</code> contains all those ads, ranked by popularity via the <code>id</code> column (e.g., the entry with <code>id = 1</code> is the most popular, etc.).

Create a subquery or common table expression named <code>top_ads</code> containing the top 3 ads (by popularity) and return the number of rows that would result from the following operations

<ol>
<li><code>ads INNER JOIN top_ads</code></li>
<li><code>ads LEFT JOIN top_ads</code></li>
<li><code>ads RIGHT JOIN top_ads</code></li>
<li><code>ads CROSS JOIN top_ads</code></li>
</ol>

Note: Please make the <code>join_type</code> column in your output have the values <code>inner_join</code>, <code>left_join</code>, etc. for each of their respective join types

Note: Please return only one query with each number in a different row

Example:

Input:

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
Output:

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>join_type</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>number_of_rows</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>

Size of Joins

Let’s say you are working on Google Docs. A product manager comes to you and asks how the product is doing. 

What are the top five metrics that you would start tracking to understand the health of Google Docs?

Docs Metrics

Let’s say you work as a data scientist at a bank.

You are tasked with building a decision tree model to predict if a borrower will pay back a personal loan they are taking out.

<ol>
<li>How would you evaluate whether using a decision tree algorithm is the correct model for the problem?</li>

<li>Let’s say you move forward with the decision tree model. How would you evaluate the performance of the model before deployment and after?</li>
</ol>

Decision Tree Evaluation

Let’s say you want to test the close friends feature on Instagram Stories.

How would you make a control group and test group to account for network effects?

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

Network Experiment Design I

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

Network Experiment Design II

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

Network Experiment Design III

Let’s say you’re given all the different marketing channels along with their respective marketing costs at a company called Mode, that sells B2B analytics dashboards.

What metrics would you use to determine the value of each marketing channel?

When determining the value of each marketing channel for the company Mode, which metric is considered the key metric?

When determining the value of each marketing channel for the company Mode, which metric is considered the key metric?

Marketing Channel Metrics

What metrics would you use to determine the value of each marketing channel?

Analytics

Let’s say you’re analyzing an AB test that has both a test group and a control group.

<ol>
<li>How do you calculate the sample size necessary for an accurate measurement?</li>

<li>Let’s say that the sample size is similar and sufficient between the two groups. In order to measure very small differences between the two, should the power get bigger or smaller?</li>
</ol>

Given two strings, write a function to return <code>True</code> if the strings are anagrams of each other and <code>False</code> if they are not.

Note: A word is not an anagram of itself.

Example 1:

Input:

<pre tabindex="0" class="chroma"><code>string_1 = &#34;listen&#34;
string_2 = &#34;silent&#34;
</code></pre>

Output:

<pre tabindex="0" class="chroma"><code>True
</code></pre>

Example 2:

Input:

<pre tabindex="0" class="chroma"><code>string_1 = &#34;banana&#34;
string_2 = &#34;bandana&#34;
</code></pre>

Output:

<pre tabindex="0" class="chroma"><code>False
</code></pre>

Valid Anagram

How would you explain the concept of linear regression to three different audiences: a child, a first-year college student, and a seasoned mathematician? Ensure your explanations are tailored to the understanding level of each audience.

Describe linear regression to various audiences with different levels of knowledge.

Explaining Linear Regression to Different Audiences

Let’s say we want to build a new delivery time estimate model for consumers ordering food delivery.

How would you determine if the new model predicts delivery times better than the old model?

When rolling out a new delivery time estimate model in production, what is an important consideration?

When rolling out a new delivery time estimate model in production, what is an important consideration?

Delivery Estimate Model I

What is one way to measure the performance of a new delivery time estimate model for food delivery?

What is one way to measure the performance of a new delivery time estimate model for food delivery?

Delivery Estimate Model II

How would you choose the k value when using k-means clustering?

choosing k value during k-means clustering

Choosing k

How would you measure the success of Facebook Groups?

Considering the purpose of Facebook Groups as a platform for connecting users through shared interests or real-life relationships, which metrics would be appropriate to measure the success of these groups?

Considering the purpose of Facebook Groups as a platform for connecting users through shared interests or real-life relationships, which metrics would be appropriate to measure the success of these groups?

Let’s say we’re testing a new UI with the goal to increase conversion rates. We test it by giving the new UI to a random subset of users.

The test variant wins by 5% on the target metric. What would you expect to happen after the new UI is applied to all users? Will the metric actually go up by ~5%, more, or less?

Note: Assume there is no novelty effect.

New UI Effect

What do you think are the most important metrics for WhatsApp?

What is the optimal strategy for reducing customer churn rate in a messaging platform like WhatsApp?

What is the optimal strategy for reducing customer churn rate in a messaging platform like WhatsApp?

WhatsApp Metrics

Let’s say you’re interning at a fintech startup that’s building a platform for online games of chance. During testing, one of your simulations flips a coin 10 times and gets 8 tails and 2 heads.

How would you determine if the coin is fair, and what statistical approach would you use to support your conclusion?

How would you assess if a coin is fair after observing 8 tails in 10 flips?

Fair Coin

Probability

Let’s say you’re a Product Data Scientist at Instagram. How would you measure the success of the Instagram TV product?

What is the primary goal of Instagram TV as a product feature?

What is the primary goal of Instagram TV as a product feature?

Let’s say you have to analyze the results of an AB test.

One variant of the AB test has a sample size of 50K users and the other has a sample size of 200K users.

Given the unbalanced size between the two groups, can you determine if the test will result in bias towards the smaller group?

Given an AB test where one variant has a sample size of 50K users and the other has a sample size of 200K users, what is the likely effect of the unbalanced sizes on the test results?

Given an AB test where one variant has a sample size of 50K users and the other has a sample size of 200K users, what is the likely effect of the unbalanced sizes on the test results?

Sample Size Bias

In an A/B test, how can you check if assignment to the various buckets was truly random?

In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?

In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?

Random Bucketing

Let’s say you have a categorical variable with thousands of distinct values, how would you encode it?

Which method can be used to extract communities from large networks, that does not require a pre-determined number of clusters like K-means?

Which method can be used to extract communities from large networks, that does not require a pre-determined number of clusters like K-means?

Calculate Moving Average	SQL	Easy
Predict Customer Churn	Machine Learning	Medium
A/B Test Significance	Statistics	Medium
Optimize Query Performance	SQL	Hard
Feature Importance Analysis	Machine Learning	Medium
Clean Missing Data	Python	Easy
Neural Network Architecture	Deep Learning	Hard
Calculate Cohort Retention	SQL	Medium
Bayesian Probability	Statistics	Easy
Recommend Similar Products	Machine Learning	Hard

ADM Interview Questions

ADM Interview Guides

ADM Interview Questions

Challenge

ADM Salaries by Position

Discussion & Interview Experiences

Discussion & Interview Experiences