Prepare for and practice interview questions from Instacart across topics like Takehome, SQL, Python and more.

Instacart Interview Questions

Instacart Interview Guides

A/B Testing

Experiment Validity

Delivery Estimate Model

Button AB Test

A/B Test Power Size

Network Experiment Design

Probability

First to Six

500 Cards

Raining in Seattle

Found Item

Given three uniform(0,4) random variables, what is the probability that the median of them is greater than 3?

Median Probability

Product Sense & Metrics

How do we measure the success of acquiring new users through a free trial

Netflix Retention

Group Success

WAU vs Open Rates

Instagram TV Success

Docs Metrics

Analytics

What metrics would you use to determine the value of each marketing channel?

Marketing Channel Metrics

Let’s say that you're in charge of an e-commerce D2C business that sells socks. What business health metrics would you care?

D2C Socks e-Commerce

Achieving maximum profits through dynamic pricing with real-time supply and demand fluctuations

Understanding Dynamic Pricing Strategy

How would you analyze how the feature is performing?

Recruiting Leads

Data Structures & Algorithms

Given a list of tuples featuring names and grades on a test, write a function to normalize the values of the grades to a linear scale between 0 and 1.

Normalize Grades

Stop Words Filter

Write a function to calculate the number of common items.

Common Item Frequency

Let’s say that you work at a B2B SAAS company that’s interested in testing the pricing of different levels of subscriptions.

Your project manager comes to you and asks you to run a two-week-long A/B test to test an increase in pricing.

How would you approach designing this test? How would you determine whether the increase in pricing is a good business decision?

Testing Price Increase

A team wants to A/B test multiple different changes through a sign-up funnel.

For example, on a page, a button is currently red and at the top of the page. They want to see if changing a button from red to blue and/or from the top of the page to the bottom of the page will increase click-through.

How would you set up this test?

Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.

How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.

Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.

How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.

A/B Test Variants

Let’s say we want to launch a re-design of a landing page to improve the click-through rate. We can do this by implementing an AB test.
Given that we launch an AB test, how would you infer if the results of the click-through rate were statistically significant or not?

How can you accurately conclude if the results of an A/B test, conducted to evaluate the effectiveness of a landing page redesign, are statistically significant?

How can you accurately conclude if the results of an A/B test, conducted to evaluate the effectiveness of a landing page redesign, are statistically significant?

Statistically Significant Test

Precisely ascertain whether the outcomes of an A/B test, executed to assess the impact of a landing page redesign, exhibit statistical significance.

Let’s say that your company is running a standard control and variant AB test on a feature to increase conversion rates on the landing page. The PM checks the results and finds a .04 p-value.

How would you assess the validity of the result?

Let’s say you are working on Google Docs. A product manager comes to you and asks how the product is doing. 

What are the top five metrics that you would start tracking to understand the health of Google Docs?

Let’s say you want to test the close friends feature on Instagram Stories.

How would you make a control group and test group to account for network effects?

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

Network Experiment Design I

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

Network Experiment Design II

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

Network Experiment Design III

Let’s say you’re given all the different marketing channels along with their respective marketing costs at a company called Mode, that sells B2B analytics dashboards.

What metrics would you use to determine the value of each marketing channel?

When determining the value of each marketing channel for the company Mode, which metric is considered the key metric?

When determining the value of each marketing channel for the company Mode, which metric is considered the key metric?

Let’s say you’re analyzing an AB test that has both a test group and a control group.

<ol>
<li>How do you calculate the sample size necessary for an accurate measurement?</li>

<li>Let’s say that the sample size is similar and sufficient between the two groups. In order to measure very small differences between the two, should the power get bigger or smaller?</li>
</ol>

Let’s say that you’re the data scientist for the marketing/advertising division of your company. The marketing executive wants to test multiple new channels including:

<ul>
<li>Youtube ads</li>
<li>Google search ads</li>
<li>Facebook ads</li>
<li>Direct mail campaigns</li>
</ul>

Given these new marketing channels, how would you design an a/b test to utilize the marketing budget in the most efficient way possible?

Marketing Dollar Efficiency

Let’s say we want to build a new delivery time estimate model for consumers ordering food delivery.

How would you determine if the new model predicts delivery times better than the old model?

When rolling out a new delivery time estimate model in production, what is an important consideration?

When rolling out a new delivery time estimate model in production, what is an important consideration?

Delivery Estimate Model I

What is one way to measure the performance of a new delivery time estimate model for food delivery?

What is one way to measure the performance of a new delivery time estimate model for food delivery?

Delivery Estimate Model II

How would you measure the success of Facebook Groups?

Considering the purpose of Facebook Groups as a platform for connecting users through shared interests or real-life relationships, which metrics would be appropriate to measure the success of these groups?

Considering the purpose of Facebook Groups as a platform for connecting users through shared interests or real-life relationships, which metrics would be appropriate to measure the success of these groups?

Let’s say we’re testing a new UI with the goal to increase conversion rates. We test it by giving the new UI to a random subset of users.

The test variant wins by 5% on the target metric. What would you expect to happen after the new UI is applied to all users? Will the metric actually go up by ~5%, more, or less?

Note: Assume there is no novelty effect.

New UI Effect

What do you think are the most important metrics for WhatsApp?

What is the optimal strategy for reducing customer churn rate in a messaging platform like WhatsApp?

What is the optimal strategy for reducing customer churn rate in a messaging platform like WhatsApp?

WhatsApp Metrics

Let’s say you’re interning at a fintech startup that’s building a platform for online games of chance. During testing, one of your simulations flips a coin 10 times and gets 8 tails and 2 heads.

How would you determine if the coin is fair, and what statistical approach would you use to support your conclusion?

How would you assess if a coin is fair after observing 8 tails in 10 flips?

Fair Coin

Let’s say you’re a Product Data Scientist at Instagram. How would you measure the success of the Instagram TV product?

What is the primary goal of Instagram TV as a product feature?

What is the primary goal of Instagram TV as a product feature?

Let’s say you have to analyze the results of an AB test.

One variant of the AB test has a sample size of 50K users and the other has a sample size of 200K users.

Given the unbalanced size between the two groups, can you determine if the test will result in bias towards the smaller group?

Given an AB test where one variant has a sample size of 50K users and the other has a sample size of 200K users, what is the likely effect of the unbalanced sizes on the test results?

Given an AB test where one variant has a sample size of 50K users and the other has a sample size of 200K users, what is the likely effect of the unbalanced sizes on the test results?

Sample Size Bias

In an A/B test, how can you check if assignment to the various buckets was truly random?

In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?

In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?

Random Bucketing

You are about to get on a plane to Seattle. You want to know if you should bring an umbrella. You call 3 random friends of yours who live there and ask each independently if it’s raining. Each of your friends has a 2⁄3 chance of telling you the truth and a 1⁄3 chance of messing with you by lying. All 3 friends tell you that “Yes” it is raining.

What is the probability that it’s actually raining in Seattle?

What is the main difference between the Frequentist and Bayesian methods of probabilistic analysis?

What is the **main** difference between the Frequentist and Bayesian methods of probabilistic analysis?

Frequentist Vs Bayesian Analysis

Let’s say you work at an eCommerce startup. You run an A/B test on the company’s checkout product page with the hypothesis that surfacing free shipping will increase conversions.

The experiment group specifies whether the product qualifies for free shipping or not. In the control group, there is no specification of free shipping.

Let’s say we run the experiment for two weeks and get these results.

Control:

<ul>
<li>3056 page visits</li>
<li>1413 conversions</li>
<li>46% conversion</li>
</ul>

Experiment:

<ul>
<li>2947 samples</li>
<li>1533 conversions</li>
<li>52% conversion</li>
</ul>

How would you evaluate the results and if the test was successful or not?

Free Shipping Mention Test

Let’s say that you’re in charge of an e-commerce D2C business that sells socks.

What business health metrics would you care about tracking on a company dashboard?

Let’s say you’re a data scientist at LinkedIn where you’re working on a product that sends qualified job candidates to companies. The team has launched a new feature that allows candidates to message hiring managers at companies directly during the interview process to get updates on their status.

Due to engineering constraints, the company can’t AB test the feature before launching it.

How would you analyze how the feature is performing?

Let’s say that you’re a data scientist on the engagement team. A product manager comes up to you and says that the weekly active users metric is up 5% but email notification open rates are down 2%.

What would you investigate to diagnose what’s happening?

Email Open Rate

Three zebras are chilling in the desert. Suddenly a lion attacks.
Each zebra is sitting on a corner of a triangle with sides of equal length.
Each zebra randomly picks a direction and only runs along the outline of the triangle to either edge of the triangle.

What is the probability that none of the zebras collide?

Three zebras are chilling in the desert. Suddenly a lion attacks.

Each zebra is sitting on a corner of a triangle with side of equal length. Each zebra randomly picks a direction and only runs along the outline of the triangle to either edge of the triangle.

What is the probability that none of the zebras collide?

Three zebras are chilling in the desert. Suddenly a lion attacks.

Each zebra is sitting on a corner of a triangle with side of equal length.  Each zebra randomly picks a direction and only runs along the outline of the triangle to either edge of the triangle.

What is the probability that none of the zebras collide?



Three Zebras

Brainteasers

What is an unbiased estimator and can you provide an example for a layman to understand?

What makes an estimator of a population statistic unbiased?

Unbiased Estimator

Statistics

What are the benefits of dynamic pricing, and how can you estimate supply and demand in this context?

Uber Rider deals with huge amounts of data but at Uber Fleet, the amount of data for experimentation is rather low. Let’s say you run an AB test for Uber Fleet and find out that the distribution is not normal.

What kind of analysis would you run and how would you measure which variant won?

Which of the following is NOT an acceptable way to deal with AB testing on a non-normally distributed population?

Which of the following is NOT an acceptable way to deal with AB testing on a non-normally distributed population?

Non-Normal AB Test

Non-Normal AB Testing

Let’s say that you work for an online media company. The media company is starting to monetize its web traffic and wants to experiment with adding web banners into the middle of its reading content to see if it can monetize effectively.

How would you measure the success of the banner ad strategy?

How would you measure the success of a banner ad strategy?

Banner Ad Strategy Success

Amy and Brad take turns in rolling a fair six-sided die. Whoever rolls a “6” first wins the game. Amy starts by rolling first.

What’s the probability that Amy wins?

Amy and Brad take turns in rolling a fair six-sided die. Whoever rolls a “6” first wins the game. Amy starts by rolling first.

What’s the probability that Amy wins?

Imagine a deck of 500 cards numbered from 1 to 500. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what’s the probability of each subsequent card being larger than the previous drawn card?

Imagine a deck of 2000 cards numbered from 1 to 2000. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what’s the probability of each subsequent card being larger than the previously drawn card?

Imagine a deck of 2000 cards numbered from 1 to 2000. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what's the probability of each subsequent card being larger than the previously drawn card?

2000 cards

Let’s say you’re working on a building a better search engine for Google. You build it and want to see if it serves better results than the existing one in production.

How would you determine which search engine performed better? Which metrics would you track?

Calculate Moving Average	SQL	Easy
Predict Customer Churn	Machine Learning	Medium
A/B Test Significance	Statistics	Medium
Optimize Query Performance	SQL	Hard
Feature Importance Analysis	Machine Learning	Medium
Clean Missing Data	Python	Easy
Neural Network Architecture	Deep Learning	Hard
Calculate Cohort Retention	SQL	Medium
Bayesian Probability	Statistics	Easy
Recommend Similar Products	Machine Learning	Hard

Instacart Interview Questions

Instacart Interview Guides

Instacart Interview Questions

Challenge

Instacart Salaries by Position

Discussion & Interview Experiences

Discussion & Interview Experiences