Prepare for and practice interview questions from Calm.

Calm Interview Questions

Calm Interview Guides

Data Structures & Algorithms

Given two sorted lists, write a function to merge them into one sorted list.

Merge Sorted Lists

Write a function to create a single dataframe with complete addresses in the format of street, city, state, zip code.

Complete Addresses

Given a list of integers, and an integer N, write a function to find all combinations that sum to the value N.

Sum to N

Given an integer N, write a function that returns all of the prime numbers up to N

Prime to N

Determine whether there exists a permutation of an input string that is a palindrome.

Permutation Palindrome

Write a query to select the top 3 departments with at least ten employees and rank them according to the percentage of their employees making over 100K in salary.

Employee Salaries

Select the 2nd highest salary in the engineering department

2nd Highest Salary

Write a query that returns all neighborhoods that have 0 users.

Empty Neighborhoods

Write a query to return whether each user's subscription date range overlaps with any other completed subscription

Subscription Overlap

Last Transaction

Machine Learning

How would you encode a categorical variable with thousands of distinct values

Encoding Categorical Features

Write a function calculate_rmse to calculate the root mean squared error from y_pred and y_true

RMS Error

How would we build a model to detect fraud and text customers to approve or deny fraudulent transactions

Bank Fraud Model

How would we know if we have enough data to create an accurate enough model?

One Million Rides

In which case would you use a bagging algorithm versus a boosting algorithm

Bagging vs Boosting

Probability

What is the probability that each subsequent card is larger than the previous drawn card when picking three cards from a shuffled deck of 500 cards

500 Cards

First to Six

What is the probability that it's actually raining in Seattle?

Raining in Seattle

What is the probability that item X would be found on Amazon's website?

Found Item

Given three uniform(0,4) random variables, what is the probability that the median of them is greater than 3?

Median Probability

Product Sense & Metrics

What would you investigate to diagnose what's happening?

WAU vs Open Rates

Calculate the formula for the average lifetime value

Revenue Retention

How do we measure the launch of Robinhood’s fractional shares program

Fractional Shares

Google Maps Improvement

How would you measure the success of Uber Eats?

Uber Eats Success

<p>A dating website’s schema is represented by a table of people that like other people. The table has three columns. One column is the <code>user_id</code>, another column is the <code>liker_id</code> which is the <code>user_id</code> of the user doing the liking, and the last column is the <code>date time</code> that the like occurred.</p>

<p>Write a query to count the number of liker’s likers (the users that like the likers) if the liker has one.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>likes</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>liker_id</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>count</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Count the number of users that like each user

Liker's Likers

<p>Let’s say that we are the PM of the Calm meditation app. We launch the app in a new country but find out it isn’t performing as well as expected.</p>

<p>What would you investigate to find out why?</p>


What would you investigate to find out why the app isn't performing as well as expected in a new country

New Geography

Business Case

<p>We’re given two tables, a <code>users</code> table with demographic information and the neighborhood they live in and a <code>neighborhoods</code> table.</p>

<p>Write a query that returns all neighborhoods that have 0 users. </p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>users</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>neighborhood_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>
</tbody>
</table>
<p><code>neighborhoods</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>city_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>


<p>Write a SQL query to select the 2nd highest salary in the engineering department.</p>

<p><strong>Note:</strong> If more than one person shares the highest salary, the query should select the next highest salary.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>employees</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>departments</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


<p>Given a <code>employees</code> and <code>departments</code> table, select the top 3 departments with at least ten employees and rank them according to the percentage of their employees making over 100K in salary.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>employees</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>departments</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>percentage_over_100k</code></td>
<td>FLOAT</td>
</tr>

<tr>
<td><code>department_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>number_of_employees</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


<p>Let’s say that your company is running a standard control and variant AB test on a feature to increase conversion rates on the landing page. The PM checks the results and finds a .04 p-value.</p>

<p>How would you assess the validity of the result?</p>


How would you assess the validity of the result?

Experiment Validity

A/B Testing

<p>A team wants to A/B test multiple different changes through a sign-up funnel.</p>

<p>For example, on a page, a button is currently red and at the top of the page. They want to see if changing a button from red to blue and/or from the top of the page to the bottom of the page will increase click-through.</p>

<p>How would you set up this test?</p>


<p>Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.</p>

<p>How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.</p>


Your team wants to run an AB test on the color (red or blue) and position (top/bottom of the page) of a button that links to a promotion.

How many different variants of the test should you run to see the effect that a red button has on the bottom of the page, if currently the button is blue and on the top of the page.

A/B Test Variants

Button AB Test

<p>Given the <code>employees</code> and <code>departments</code> table, write a query to get the top 3 highest employee salaries by department. If the department contains less that 3 employees, the top 2 or the top 1 highest salaries should be listed (assume that each department has at least 1 employee). </p>

<p><em>Note: The output should include the full name of the employee in one column, the department name, and the salary. The output should be sorted by department name in ascending order and salary in descending order.</em> </p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>employees</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>first_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>last_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>department_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>departments</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output</strong>:</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>employee_nam</code>e</td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>department_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>salary</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Get the top 3 highest employee salaries by department

Top Three Salaries

<p>Given a table of product subscriptions with a subscription start date and end date for each user, write a query that returns true or false whether or not each user has a subscription date range that overlaps with any other completed subscription.</p>

<p>Completed subscriptions have <code>end_date</code> recorded.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>subscriptions</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>start_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>end_date</code></td>
<td>DATETIME</td>
</tr>
</tbody>
</table>

<table>
<thead>
<tr>
<th><code>user_id</code></th>
<th><code>start_date</code></th>
<th><code>end_date</code></th>
</tr>
</thead>

<tbody>
<tr>
<td>1</td>
<td>2019-01-01</td>
<td>2019-01-31</td>
</tr>

<tr>
<td>2</td>
<td>2019-01-15</td>
<td>2019-01-17</td>
</tr>

<tr>
<td>3</td>
<td>2019-01-29</td>
<td>2019-02-04</td>
</tr>

<tr>
<td>4</td>
<td>2019-02-05</td>
<td>2019-02-10</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th><code>user_id</code></th>
<th><code>overlap</code></th>
</tr>
</thead>

<tbody>
<tr>
<td>1</td>
<td>1</td>
</tr>

<tr>
<td>2</td>
<td>1</td>
</tr>

<tr>
<td>3</td>
<td>1</td>
</tr>

<tr>
<td>4</td>
<td>0</td>
</tr>
</tbody>
</table>


<p>Given two sorted lists, write a function to merge them into one sorted list.</p>

<p><em>Bonus: What’s the time complexity?</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">list1</span> <span class="o">=</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">5</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="n">list2</span> <span class="o">=</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">4</span><span class="p">,</span><span class="mi">6</span><span class="p">]</span>
</span></span></code></pre>

<p><strong>Output:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">merge_list</span><span class="p">(</span><span class="n">list1</span><span class="p">,</span><span class="n">list2</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">4</span><span class="p">,</span><span class="mi">5</span><span class="p">,</span><span class="mi">6</span><span class="p">]</span>
</span></span></code></pre>


<p>You have two sorted lists, one with m elements and one with n elements. What would be the time complexity of merging the two lists via merge sort?</p>


You have two sorted lists, one with m elements and one with n elements. What would be the time complexity of merging the two lists via merge sort? 

Merging Lists Complexity

<p>Amy and Brad take turns in rolling a fair six-sided die. Whoever rolls a “6” first wins the game. Amy starts by rolling first.</p>

<p>What’s the probability that Amy wins?</p>


Amy and Brad take turns in rolling a fair six-sided die. Whoever rolls a “6” first wins the game. Amy starts by rolling first.

What’s the probability that Amy wins?

<p>Given an integer <code>N</code>, write a function that returns a list of all of the prime numbers up to <code>N</code>.</p>

<p><em>Note: Return an empty list there are no prime numbers less than or equal to <code>N</code>.</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">N</span> <span class="o">=</span> <span class="mi">3</span>
</span></span></code></pre>

<p><strong>Output:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">prime_numbers</span><span class="p">(</span><span class="n">N</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">]</span>
</span></span></code></pre>


<p>Given two tables:  <code>accounts</code>, and <code>downloads</code>, find the average number of downloads for free vs paying accounts, broken down by day.</p>

<p><em>Note: You only need to consider accounts that have had at least one download before when calculating the average.</em></p>

<p><em>Note: round <code>average_downloads</code> to 2 decimal places.</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>accounts</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>account_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>paying_customer</code></td>
<td>BOOLEAN</td>
</tr>
</tbody>
</table>
<p><code>downloads</code> table</p>

<table>
<thead>
<tr>
<th>column</th>
<th>type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>account_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>download_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>downloads</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>download_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>paying_customer</code></td>
<td>BOOLEAN</td>
</tr>

<tr>
<td><code>average_downloads</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>


Write a query to find the average downloads for free vs paying accounts by day, rounded to 2 decimals

Download Facts

<p>Let’s say we want to build a naive recommender. We’re given two tables, one table called <code>friends</code> with a <code>user_id</code> and <code>friend_id</code> columns representing each user’s friends, and another table called <code>page_likes</code> with a <code>user_id</code> and a <code>page_id</code> representing the page each user liked.</p>

<p>Write an SQL query to create a metric to recommend pages for each user based on recommendations from their friend’s liked pages. </p>

<p><em>Note: It shouldn’t recommend pages that the user already likes.</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>friends</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>friend_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>page_likes</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>page_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>page_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>num_friend_likes</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Write a query to get the number of friends of a user that like a specific page

Liked Pages

<p>Given a table called <code>user_experiences</code>, write a query to determine the percentage of users that held the title of <code>&#34;Data Analyst&#34;</code> immediately before holding the title <code>&#34;Data Scientist&#34;</code>.</p>

<p>Immediate is defined as the user holding no other titles between the <code>&#34;Data Analyst&#34;</code> and <code>&#34;Data Scientist&#34;</code> roles.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>user_experiences</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>position_name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>start_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>end_date</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>percentage</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>


User Experience Percentage

<p>Given the tables <code>users</code> and <code>rides</code>, write a query to report the distance traveled by each user in descending order.</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>users</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>name</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>rides</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>passenger_user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>distance</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>name</code></td>
<td>VARCHAR</td>
</tr>

<tr>
<td><code>distance_traveled</code></td>
<td>FLOAT</td>
</tr>
</tbody>
</table>


Distance Traveled

<p>You are about to get on a plane to Seattle. You want to know if you should bring an umbrella. You call 3 random friends of yours who live there and ask each independently if it’s raining. Each of your friends has a <sup>2</sup>⁄<sub>3</sub> chance of telling you the truth and a <sup>1</sup>⁄<sub>3</sub> chance of messing with you by lying. All 3 friends tell you that “Yes” it is raining.</p>

<p>What is the probability that it’s actually raining in Seattle?</p>


<p>What is the <strong>main</strong> difference between the Frequentist and Bayesian methods of probabilistic analysis?</p>


What is the **main** difference between the Frequentist and Bayesian methods of probabilistic analysis?

Frequentist Vs Bayesian Analysis

<p>Given the <code>transactions</code> table below, write a query that finds the third purchase of every user.</p>

<p><em>Note: Sort the results by the <code>user_id</code> in ascending order. If a user purchases two products at the same time, the lower <code>id</code> field is used to determine which is the first purchase.</em></p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>transactions</code> table</p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>product_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>quantity</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Columns</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>product_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>quantity</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>


Write a query to find the third purchase of every user sorted by user_id ascending

Third Purchase

<p>Let’s say you want to test the close friends feature on Instagram Stories.</p>

<p>How would you make a control group and test group to account for network effects?</p>


<p>What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?</p>

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

Network Experiment Design I

<p>What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?</p>

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

Network Experiment Design II

<p>If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?</p>

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

Network Experiment Design III

How would you make a control group and test group to account for network effects

<p>Imagine a deck of 500 cards numbered from 1 to 500. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what’s the probability of each subsequent card being larger than the previous drawn card?</p>


<p>Imagine a deck of 2000 cards numbered from 1 to 2000. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what’s the probability of each subsequent card being larger than the previously drawn card?</p>


Imagine a deck of 2000 cards numbered from 1 to 2000. If all the cards are shuffled randomly and you are asked to pick three cards, one at a time, what's the probability of each subsequent card being larger than the previously drawn card?

2000 cards

<p>Tell me about a data project that didn’t go the way you expected. What did you set out to do, what surprised you, and how did you handle it?</p>


Describing a data project and its challenges

Hurdles In Data Projects

Behavioral

<p>How would you explain what a p-value is to someone who is not technical?</p>


<p>What does a p-value in a statistical test represent?</p>

What does a p-value in a statistical test represent?

P-value to a Layman I

<p>In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?</p>

In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?

P-value to a Layman II

Explain what a p-value is to someone who is not technical

P-value to a Layman

Statistics

<p>Given a dictionary with weights, write a function <code>random_key</code> that returns a key at random with a probability proportional to the weights.</p>

<p><strong>Example 1:</strong></p>

<p><strong>Input:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">weights</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;A&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">:</span> <span class="mi">2</span><span class="p">}</span>
</span></span></code></pre>

<p><strong>Output:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">random_key</span><span class="p">(</span><span class="n">weights</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="k">return</span> <span class="n">A</span> <span class="mi">1</span><span class="o">/</span><span class="mi">3</span> <span class="n">of</span> <span class="n">the</span> <span class="n">time</span><span class="p">,</span> <span class="n">B</span> <span class="mi">2</span><span class="o">/</span><span class="mi">3</span> <span class="n">of</span> <span class="n">the</span> <span class="n">time</span>
</span></span></code></pre>

<p><strong>Example 2:</strong></p>

<p><strong>Input:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">weights</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;A&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span> <span class="s1">&#39;B&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">}</span>
</span></span></code></pre>

<p><strong>Output:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">random_key</span><span class="p">(</span><span class="n">weights</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="k">return</span> <span class="n">A</span> <span class="mi">1</span><span class="o">/</span><span class="mi">2</span>  <span class="n">of</span> <span class="n">the</span> <span class="n">time</span><span class="p">,</span> <span class="n">B</span> <span class="mi">1</span><span class="o">/</span><span class="mi">2</span>  <span class="n">of</span> <span class="n">the</span> <span class="n">time</span>
</span></span></code></pre>


Weighted Keys

<p>Let’s say we’re comparing two machine learning algorithms. In which case would you use a bagging algorithm versus a boosting algorithm? </p>

<p>Give an example of the tradeoffs between the two.</p>


<p>In machine learning, when would you use a bagging algorithm over a boosting algorithm?</p>


In machine learning, when would you use a bagging algorithm over a boosting algorithm?

Bagging vs. Boosting

<p>In an A/B test, how can you check if assignment to the various buckets was truly random?</p>


<p>In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?</p>

In an A/B test, which of the following is a common method to determine if the assignment to the test groups was truly random?

Random Bucketing

How can you check if assignment to the various buckets was truly random

<p>Write a query that creates an attribution rule for each user. If the user visited Facebook or Google at least once then the attribution is labeled as “paid.” Otherwise, the attribution is labeled as “organic.”</p>

<p><strong>Example:</strong></p>

<p><strong>Input:</strong></p>

<p><code>user_sessions</code> table</p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>created_at</code></td>
<td>DATETIME</td>
</tr>

<tr>
<td><code>session_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>
</tbody>
</table>
<p><code>attribution</code> table </p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>session_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>channel</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>
<p><strong>Output:</strong></p>

<table>
<thead>
<tr>
<th>Column</th>
<th>Type</th>
</tr>
</thead>

<tbody>
<tr>
<td><code>user_id</code></td>
<td>INTEGER</td>
</tr>

<tr>
<td><code>attribute</code></td>
<td>VARCHAR</td>
</tr>
</tbody>
</table>


Write a query to create an attribution rule for each user

Attribution Rules

<p>Let’s say that you work for a software as a subscription (SAAS) company that has existed for just over a year. The chief revenue officer wants to know the average lifetime value.</p>

<p>We know that the product costs $100 dollars per month, averages 10% in monthly churn, and the average customer sticks around for around 3.5 months.</p>

<p>Calculate the formula for the average lifetime value.</p>


<p>If a software-as-a-subscription company has a monthly churn rate of 20% for a product that costs $100, what would be the average lifetime value of a single customer?</p>


If a software-as-a-subscription company has a monthly churn rate of 20% for a product that costs $100, what would be the average lifetime value of a single customer?

Average Lifetime Value

<p>You’re given a string that may contain the characters <strong><code>{</code></strong>, <strong><code>}</code></strong>, <strong><code>[</code></strong>, <strong><code>]</code></strong>, <strong><code>(</code></strong>, and <strong><code>)</code></strong>.</p>

<p><strong>Task:</strong> Verify that the string is balanced. A balanced string is one where every opening character, <strong><code>{</code></strong>, <strong><code>[</code></strong>, or <strong><code>(</code></strong>, has a corresponding closing character, <strong><code>}</code></strong>, <strong><code>]</code></strong>, or <strong><code>)</code></strong>.</p>

<p>Write a function called <strong><code>is_balanced(string: str) -&gt; bool</code></strong> which verifies the balance of a string.</p>

<p><strong>Example:</strong></p>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">is_balanced</span><span class="p">(</span><span class="s1">&#39;(())[]</span><span class="si">{}</span><span class="s1">&#39;</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">True</span>
</span></span></code></pre>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">is_balanced</span><span class="p">(</span><span class="s1">&#39;{([()</span><span class="si">{}</span><span class="s1">])()}&#39;</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">True</span>
</span></span></code></pre>

<pre tabindex="0" class="chroma"><code><span class="line"><span class="cl"><span class="n">is_balanced</span><span class="p">(</span><span class="s1">&#39;</span><span class="si">{}</span><span class="s1">[]())&#39;</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">False</span>
</span></span></code></pre>

<hr/>


Write a function that tests whether a string of brackets is balanced.

The Brackets Problem

<p>Let’s say we have a very naive advertising platform. There is an audience of size <em>A</em> and a limited amount of impressions <em>B</em>.</p>

<p>Each of the impressions goes to just one user, defined at random. Every user in the audience has the same random chance of receiving each impression. </p>

<ol>
<li><p>Compute the probability that a user has exactly 0 impressions. </p></li>

<li><p>What’s the probability that every person has at least 1 impression?</p></li>
</ol>


Compute the probability a user gets exactly 0 impressions, and the probability everyone gets at least 1 impression

Calculate Moving Average	SQL	Easy
Predict Customer Churn	Machine Learning	Medium
A/B Test Significance	Statistics	Medium
Optimize Query Performance	SQL	Hard
Feature Importance Analysis	Machine Learning	Medium
Clean Missing Data	Python	Easy
Neural Network Architecture	Deep Learning	Hard
Calculate Cohort Retention	SQL	Medium
Bayesian Probability	Statistics	Easy
Recommend Similar Products	Machine Learning	Hard

Calm Interview Questions

Calm Interview Guides

Calm Interview Questions

Challenge

Calm Salaries by Position

Discussion & Interview Experiences

Discussion & Interview Experiences