Prepare for and practice interview questions from Heetch.

Heetch Interview Questions

Heetch Interview Guides

Machine Learning

How would you encode a categorical variable with thousands of distinct values

Encoding Categorical Features

Write a function calculate_rmse to calculate the root mean squared error from y_pred and y_true

RMS Error

How would we build a model to detect fraud and text customers to approve or deny fraudulent transactions

Bank Fraud Model

How would we know if we have enough data to create an accurate enough model?

One Million Rides

In which case would you use a bagging algorithm versus a boosting algorithm

Bagging vs Boosting

Write a query to select the top 3 departments with at least ten employees and rank them according to the percentage of their employees making over 100K in salary.

Employee Salaries

Select the 2nd highest salary in the engineering department

2nd Highest Salary

Write a query that returns all neighborhoods that have 0 users.

Empty Neighborhoods

Write a query to return whether each user's subscription date range overlaps with any other completed subscription

Subscription Overlap

Last Transaction

Data Structures & Algorithms

Given two sorted lists, write a function to merge them into one sorted list.

Merge Sorted Lists

Write a function to create a single dataframe with complete addresses in the format of street, city, state, zip code.

Complete Addresses

Given a list of integers, and an integer N, write a function to find all combinations that sum to the value N.

Sum to N

Determine whether there exists a permutation of an input string that is a palindrome.

Permutation Palindrome

Write a function `sorting` from scratch to sort a list of strings in ascending alphabetical order

Sort Strings

Product Sense & Metrics

What would you investigate to diagnose what's happening?

WAU vs Open Rates

Calculate the formula for the average lifetime value

Revenue Retention

How do we measure the launch of Robinhood’s fractional shares program

Fractional Shares

Google Maps Improvement

How would you measure the success of Uber Eats?

Uber Eats Success

Probability

What is the probability that each subsequent card is larger than the previous drawn card when picking three cards from a shuffled deck of 500 cards

500 Cards

First to Six

What is the probability that it's actually raining in Seattle?

Raining in Seattle

What is the probability that item X would be found on Amazon's website?

Found Item

Given three uniform(0,4) random variables, what is the probability that the median of them is greater than 3?

Median Probability

<p>You’re tasked with building a model to predict if a driver on Uber will accept a ride request or not. </p>

<ol>
<li><p>What algorithm would you use to build this model? What are the tradeoffs between different classifiers?</p></li>

<li><p>What features would you use?</p></li>
</ol>


Building a model to predict if a driver on Uber will accept a ride request or not

Ride Requests Model

<p>We’re given two tables, a <code>users</code> table with demographic information and the neighborhood they live in and a <code>neighborhoods</code> table.</p>

<p>Write a query that returns all neighborhoods that have 0 users. </p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>users</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>neighborhood_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>
</tbody>
</table>
<p><code>neighborhoods</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>city_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>


<p>Given a <code>employees</code> and <code>departments</code> table, select the top 3 departments with at least ten employees and rank them according to the percentage of their employees making over 100K in salary.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>employees</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>departments</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>percentage_over_100k</code></td>
<td>FLOAT</td>
</tr>

<tr>
<td><code>department_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>number_of_employees</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


<p>A team wants to A/B test multiple different changes through a sign-up funnel.</p>

<p>For example, on a page, a button is currently red and at the top of the page. They want to see if changing a button from red to blue and/or from the top of the page to the bottom of the page will increase click-through.</p>

<p>How would you set up this test?</p>


<p>Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.</p>

<p>How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.</p>


Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.

How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.

A/B Test Variants

Button AB Test

A/B Testing

<p>Write a SQL query to select the 2nd highest salary in the engineering department.</p>

<p><strong>Note:</strong> If more than one person shares the highest salary, the query should select the next highest salary.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>employees</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>departments</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


<p>Let’s say that your company is running a standard control and variant AB test on a feature to increase conversion rates on the landing page. The PM checks the results and finds a .04 p-value.</p>

<p>How would you assess the validity of the result?</p>


How would you assess the validity of the result?

Experiment Validity

<p>How would you explain what a p-value is to someone who is not technical?</p>


<p>What does a p-value in a statistical test represent?</p>

What does a p-value in a statistical test represent?

P-value to a Layman I

<p>In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?</p>

In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?

P-value to a Layman II

Explain what a p-value is to someone who is not technical

P-value to a Layman

Statistics

<p>Given the <code>employees</code> and <code>departments</code> table, write a query to get the top 3 highest employee salaries by department. If the department contains less that 3 employees, the top 2 or the top 1 highest salaries should be listed (assume that each department has at least 1 employee). </p>

<p><em>Note: The output should include the full name of the employee in one column, the department name, and the salary. The output should be sorted by department name in ascending order and salary in descending order.</em> </p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>employees</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>departments</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output</strong>:</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>employee_nam</code>e</td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>department_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Get the top 3 highest employee salaries by department

Top Three Salaries

<p>Given a table of product subscriptions with a subscription start date and end date for each user, write a query that returns true or false whether or not each user has a subscription date range that overlaps with any other completed subscription.</p>

<p>Completed subscriptions have <code>end_date</code> recorded.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>subscriptions</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>start_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>end_date</code></td>
<td>DATETIME</td>
</tr>
</tbody>
</table>

<table>
<thead>
<tr>
<th><code>user_id</code></th>
<th><code>start_date</code></th>
<th><code>end_date</code></th>
</tr>
</thead>

<tbody>
<tr>
<td>1</td>
<td>2019-01-01</td>
<td>2019-01-31</td>
</tr>

<tr>
<td>2</td>
<td>2019-01-15</td>
<td>2019-01-17</td>
</tr>

<tr>
<td>3</td>
<td>2019-01-29</td>
<td>2019-02-04</td>
</tr>

<tr>
<td>4</td>
<td>2019-02-05</td>
<td>2019-02-10</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th><code>user_id</code></th>
<th><code>overlap</code></th>
</tr>
</thead>

<tbody>
<tr>
<td>1</td>
<td>1</td>
</tr>

<tr>
<td>2</td>
<td>1</td>
</tr>

<tr>
<td>3</td>
<td>1</td>
</tr>

<tr>
<td>4</td>
<td>0</td>
</tr>
</tbody>
</table>


<p>Amy and Brad take turns in rolling a fair six-sided die. Whoever rolls a “6” first wins the game. Amy starts by rolling first.</p>

<p>What’s the probability that Amy wins?</p>


Amy and Brad take turns in rolling a fair six-sided die. Whoever rolls a “6” first wins the game. Amy starts by rolling first.

What’s the probability that Amy wins?

<p>You are about to get on a plane to Seattle. You want to know if you should bring an umbrella. You call 3 random friends of yours who live there and ask each independently if it’s raining. Each of your friends has a <sup>2</sup>⁄<sub>3</sub> chance of telling you the truth and a <sup>1</sup>⁄<sub>3</sub> chance of messing with you by lying. All 3 friends tell you that “Yes” it is raining.</p>

<p>What is the probability that it’s actually raining in Seattle?</p>


<p>What is the <strong>main</strong> difference between the Frequentist and Bayesian methods of probabilistic analysis?</p>


What is the **main** difference between the Frequentist and Bayesian methods of probabilistic analysis?

Frequentist Vs Bayesian Analysis

<p>Given the <code>transactions</code> table below, write a query that finds the third purchase of every user.</p>

<p><em>Note: Sort the results by the <code>user_id</code> in ascending order. If a user purchases two products at the same time, the lower <code>id</code> field is used to determine which is the first purchase.</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>transactions</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>product_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>quantity</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>product_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>quantity</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Write a query to find the third purchase of every user sorted by user_id ascending

Third Purchase

<p>Given two tables:  <code>accounts</code>, and <code>downloads</code>, find the average number of downloads for free vs paying accounts, broken down by day.</p>

<p><em>Note: You only need to consider accounts that have had at least one download before when calculating the average.</em></p>

<p><em>Note: round <code>average_downloads</code> to 2 decimal places.</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>accounts</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>account_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>paying_customer</code></td>
<td>BOOLEAN</td>
</tr>
</tbody>
</table>
<p><code>downloads</code> table</p>

<table>
<thead>
<tr>
<th>column</th>
<th>type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>account_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>download_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>downloads</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>download_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>paying_customer</code></td>
<td>BOOLEAN</td>
</tr>

<tr>
<td><code>average_downloads</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>


Write a query to find the average downloads for free vs paying accounts by day, rounded to 2 decimals

Download Facts

<p>Given two sorted lists, write a function to merge them into one sorted list.</p>

<p><em>Bonus: What’s the time complexity?</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">list1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">5</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="n">list2</span> <span class="o">=</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">4</span><span class="p">,</span><span class="mi">6</span><span class="p">]</span>
</span></span></code></pre>

<p><strong>Output:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">merge_list</span><span class="p">(</span><span class="n">list1</span><span class="p">,</span><span class="n">list2</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">4</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">6</span><span class="p">]</span>
</span></span></code></pre>


<p>You have two sorted lists, one with m elements and one with n elements. What would be the time complexity of merging the two lists via merge sort?</p>


You have two sorted lists, one with m elements and one with n elements. What would be the time complexity of merging the two lists via merge sort? 

Merging Lists Complexity

<p>Given a table called <code>user_experiences</code>, write a query to determine the percentage of users that held the title of <code>&#34;Data Analyst&#34;</code> immediately before holding the title <code>&#34;Data Scientist&#34;</code>.</p>

<p>Immediate is defined as the user holding no other titles between the <code>&#34;Data Analyst&#34;</code> and <code>&#34;Data Scientist&#34;</code> roles.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>user_experiences</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>position_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>start_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>end_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>percentage</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>


User Experience Percentage

<p>Given the tables <code>users</code> and <code>rides</code>, write a query to report the distance traveled by each user in descending order.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>users</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>rides</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>passenger_user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>distance</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>distance_traveled</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>


Distance Traveled

<p>Imagine a deck of 500 cards numbered from 1 to 500. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what’s the probability of each subsequent card being larger than the previous drawn card?</p>


<p>Imagine a deck of 2000 cards numbered from 1 to 2000. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what’s the probability of each subsequent card being larger than the previously drawn card?</p>


Imagine a deck of 2000 cards numbered from 1 to 2000. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what's the probability of each subsequent card being larger than the previously drawn card?

2000 cards

<p>Let’s say we want to build a naive recommender. We’re given two tables, one table called <code>friends</code> with a <code>user_id</code> and <code>friend_id</code> columns representing each user’s friends, and another table called <code>page_likes</code> with a <code>user_id</code> and a <code>page_id</code> representing the page each user liked.</p>

<p>Write an SQL query to create a metric to recommend pages for each user based on recommendations from their friend’s liked pages. </p>

<p><em>Note: It shouldn’t recommend pages that the user already likes.</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>friends</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>friend_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>page_likes</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>page_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>page_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>num_friend_likes</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Write a query to get the number of friends of a user that like a specific page

Liked Pages

<p>Say you are tasked with designing a data mart or data warehouse for a new online retailer.</p>

<p>How would you design the system?</p>

<p><em>Note: Sketch a star schema to explain your design.</em></p>


Design a data warehouse for a new online retailer

Retailer Data Warehouse

Data Modeling

<p>Let’s say that you’re a data scientist on the engagement team. A product manager comes up to you and says that the weekly active users metric is up 5% but email notification open rates are down 2%.</p>

<p>What would you investigate to diagnose what’s happening?</p>


Email Open Rate

<p>Let’s say you want to test the close friends feature on Instagram Stories.</p>

<p>How would you make a control group and test group to account for network effects?</p>


<p>What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?</p>

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

Network Experiment Design I

<p>What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?</p>

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

Network Experiment Design II

<p>If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?</p>

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

Network Experiment Design III

How would you make a control group and test group to account for network effects

Network Experiment Design

<p>In an A/B test, how can you check if assignment to the various buckets was truly random?</p>


<p>In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?</p>

In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?

Random Bucketing

How can you check if assignment to the various buckets was truly random

<p>Given a table of bank transactions with columns <code>id</code>, <code>transaction_value</code>, and <code>created_at</code> representing the date and time for each transaction, write a query to get the last transaction for each day.</p>

<p>The output should include the id of the transaction, datetime of the transaction, and the transaction amount. Order the transactions by datetime.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>bank_transactions</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>transaction_value</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>transaction_value</code></td>
<td>FLOAT</td>
</tr>

<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


<p>Given a dictionary with weights, write a function <code>random_key</code> that returns a key at random with a probability proportional to the weights.</p>

<p><strong>Example 1:</strong></p>

<p><strong>Input:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">weights</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;A&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">:</span> <span class="mi">2</span><span class="p">}</span>
</span></span></code></pre>

<p><strong>Output:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">random_key</span><span class="p">(</span><span class="n">weights</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="k">return</span> <span class="n">A</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span> <span class="n">of</span> <span class="n">the</span> <span class="n">time</span><span class="p">,</span> <span class="n">B</span> <span class="mi">2</span><span class="o">/</span><span class="mi">3</span> <span class="n">of</span> <span class="n">the</span> <span class="n">time</span>
</span></span></code></pre>

<p><strong>Example 2:</strong></p>

<p><strong>Input:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">weights</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;A&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">}</span>
</span></span></code></pre>

<p><strong>Output:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">random_key</span><span class="p">(</span><span class="n">weights</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="k">return</span> <span class="n">A</span> <span class="mi">1</span><span class="o">/</span><span class="mi">2</span>  <span class="n">of</span> <span class="n">the</span> <span class="n">time</span><span class="p">,</span> <span class="n">B</span> <span class="mi">1</span><span class="o">/</span><span class="mi">2</span>  <span class="n">of</span> <span class="n">the</span> <span class="n">time</span>
</span></span></code></pre>


Weighted Keys

<p>Let’s say you work at Uber. You’re getting reports that riders are complaining about the Uber map showing wrong location pickup spots.</p>

<p>How would you go about verifying how frequently this is happening?</p>

<p><em>Note: Assume you have user location data but not external data sources.</em></p>


You're getting reports that riders are complaining about the Uber map showing wrong location pickup spots. How would you go about verifying how frequently this is happening?

Uber User Journey

Analytics

<p>Let’s say that you work for a software as a subscription (SAAS) company that has existed for just over a year. The chief revenue officer wants to know the average lifetime value.</p>

<p>We know that the product costs $100 dollars per month, averages 10% in monthly churn, and the average customer sticks around for around 3.5 months.</p>

<p>Calculate the formula for the average lifetime value.</p>


<p>If a software-as-a-subscription company has a monthly churn rate of 20% for a product that costs $100, what would be the average lifetime value of a single customer?</p>


If a software-as-a-subscription company has a monthly churn rate of 20% for a product that costs $100, what would be the average lifetime value of a single customer?

Average Lifetime Value

<p>Let’s say you work at a file-hosting website. You have information on user’s daily downloads in the <code>download_facts</code> table</p>

<p>Use the window function <code>RANK</code> to display the top three users by downloads each day. Order your data by <code>date</code>, and then by <code>daily_rank</code></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>download_facts</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>date</code></td>
<td>DATE</td>
</tr>

<tr>
<td><code>downloads</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>daily_rank</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>date</code></td>
<td>DATE</td>
</tr>

<tr>
<td><code>downloads</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Write a query to display the top three users by downloads each day

Top 3 Users

<p>Lime’s consumer app team wants to measure the number of users who return to the app after their initial login. Given a user_logins table that records each user’s login by date, write a query to produce a daily summary with retention and engagement metrics.</p>

<p>For each date that appears in the user_logins table, calculate daily retention metrics including cohort size, 7-day retention, retention rate, and rolling 7-day active users. <em>(A cohort for a given date consists of all users who logged in on that specific date (regardless of whether it was their first login or a subsequent login)</em>.</p>

<p>Notes:</p>

<ul>
<li>A user is considered retained if they log in again within the next 7 days after their cohort date (days 2-8), excluding the cohort day itself.</li>
<li>The rolling 7-day users includes all distinct users who logged in during the 7-day window ending on the current date (from login_date - 6 days to login_date, inclusive).</li>
<li>Each (user_id, login_date) pair is unique - users can only log in once per day.
The output should be ordered by login_date ascending.</li>
<li>Retention rate should be calculated as: (retained_within_7d / cohort_users) * 100, rounded to 1 decimal place.</li>
</ul>

<h3><strong>Schema:</strong></h3>

<p><strong>Input:</strong></p>

<p><code>user_logins</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>login_date</code></td>
<td>DATE</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>login_date</code></td>
<td>DATE</td>
</tr>

<tr>
<td><code>cohort_users</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>retained_within_7d</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>retention_rate</code></td>
<td>DECIMAL(3, 2)</td>
</tr>

<tr>
<td><code>rolling_7d_users</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>

<h3><strong>Example:</strong></h3>

<p><strong>Input:</strong></p>

<table>
<thead>
<tr>
<th>user_id</th>
<th>login_date</th>
</tr>
</thead>

<tbody>
<tr>
<td>1</td>
<td>2024-06-01</td>
</tr>

<tr>
<td>2</td>
<td>2024-06-01</td>
</tr>

<tr>
<td>1</td>
<td>2024-06-02</td>
</tr>

<tr>
<td>2</td>
<td>2024-06-08</td>
</tr>

<tr>
<td>3</td>
<td>2024-06-08</td>
</tr>

<tr>
<td>1</td>
<td>2024-06-09</td>
</tr>

<tr>
<td>3</td>
<td>2024-06-10</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>login_date</th>
<th>cohort_users</th>
<th>retained_within_7d</th>
<th>retention_rate</th>
<th>rolling_7d_users</th>
</tr>
</thead>

<tbody>
<tr>
<td>2024-06-01</td>
<td>2</td>
<td>1</td>
<td>50.0</td>
<td>2</td>
</tr>

<tr>
<td>2024-06-02</td>
<td>1</td>
<td>1</td>
<td>100.0</td>
<td>2</td>
</tr>

<tr>
<td>2024-06-08</td>
<td>2</td>
<td>1</td>
<td>50.0</td>
<td>2</td>
</tr>

<tr>
<td>2024-06-09</td>
<td>1</td>
<td>0</td>
<td>0.0</td>
<td>2</td>
</tr>

<tr>
<td>2024-06-10</td>
<td>1</td>
<td>0</td>
<td>0.0</td>
<td>2</td>
</tr>
</tbody>
</table>
<p><strong>Explanation</strong>:</p>

<p>Retention is calculated for users who return within 7 days after their first login on a given day. Rolling 7-day user counts unique users over the current and previous 6 days.*</p>

<p>Suppose the current <strong><code>login_date</code></strong> is <strong>October 10, 2027</strong>.</p>

<ul>
<li>The <strong>cohort</strong> for this date includes all users who logged in on <em>2027-10-10</em>.</li>
<li>A user is considered <strong>retained</strong> if they log in <strong>again between 2027-10-11 and 2027-10-17</strong> (inclusive).</li>
<li>The cohort day itself (<strong>2027-10-10</strong>) is <strong>excluded</strong> from this retention window.</li>
</ul>


How would you calculate daily user retention and rolling activity for Lime's app using login data?

Daily Retention Summary

<p>Let’s say we have a very naive advertising platform. There is an audience of size <em>A</em> and a limited amount of impressions <em>B</em>.</p>

<p>Each of the impressions goes to just one user, defined at random. Every user in the audience has the same random chance of receiving each impression. </p>

<ol>
<li><p>Compute the probability that a user has exactly 0 impressions. </p></li>

<li><p>What’s the probability that every person has at least 1 impression?</p></li>
</ol>


Compute the probability a user gets exactly 0 impressions, and the probability everyone gets at least 1 impression

Impression Reach

<p>Let’s say that you work at a bank that wants to build a model to detect fraud on the platform.</p>

<p>The bank wants to implement a text messaging service in addition that will text customers when the model detects a fraudulent transaction in order for the customer to approve or deny the transaction with a text response.</p>

<p>How would we build this model?</p>


<p>Let’s say that you work at a bank that wants to build a model to detect fraud on the platform.</p>

<p>The bank wants to implement a text messaging service that will text customers when the model detects a fraudulent transaction in order for the customer to approve or deny the transaction with a text response.</p>

<p>Which statement is true?</p>


Let's say that you work at a bank that wants to build a model to detect fraud on the platform.

The bank wants to implement a text messaging service that will text customers when the model detects a fraudulent transaction in order for the customer to approve or deny the transaction with a text response.

Which statement is true?

Calculate Moving Average	SQL	Easy
Predict Customer Churn	Machine Learning	Medium
A/B Test Significance	Statistics	Medium
Optimize Query Performance	SQL	Hard
Feature Importance Analysis	Machine Learning	Medium
Clean Missing Data	Python	Easy
Neural Network Architecture	Deep Learning	Hard
Calculate Cohort Retention	SQL	Medium
Bayesian Probability	Statistics	Easy
Recommend Similar Products	Machine Learning	Hard

Heetch Interview Questions

Heetch Interview Guides

Heetch Interview Questions

Challenge

Discussion & Interview Experiences

Discussion & Interview Experiences