Prepare for and practice interview questions from IHME across topics like SQL, Python, Algorithms and more.

IHME Interview Questions

IHME Interview Guides

Machine Learning

Bias variance tradeoff and class imbalance in finance

Bias - Variance Tradeoff and Class Imbalance in Finance

Evaluate News

Design an ML system to predict the movie score from the review text

Score Based on Review

How would you structure your training, validation, and test splits to ensure your model generalizes to future patients? 

Training vs Validation vs Test Data

Creating a machine learning model for evaluating a patient's health

Risk Assessment Model

Behavioral

How would you answer when an Interviewer asks why you applied to their company?

Why Do You Want to Work With Us

How would you negotiate and resolve disagreements when a client rejects your proposed solution?

Client Solution Pushback

What do you tell an interviewer when they ask you what your strengths and weaknesses are?

Your Strengths and Weaknesses

A/B Testing

How would you assess the validity of the result?

Experiment Validity

How would you make a control group and test group to account for network effects

Network Experiment Design

Statistics

Explain what a p-value is to someone who is not technical

P-value to a Layman

Determine whether using an 83% cutoff overestimates or underestimates the actual population’s credit score

Credit Score Estimation

ML Ops

How would you investigate and address a decline in a model's production performance?

Model Product Performance Degradation

When an interviewer asks a question along the lines of:

<ul>
<li>What would your current manager say about you? What constructive criticisms might he give?</li>
<li>What are your three biggest strengths and weaknesses you have identified in yourself?</li>
</ul>

How would you respond?

When asked about your strengths in an interview, what is an effective way to respond?

When asked about your strengths in an interview, what is an effective way to respond?

Your Strengths and Weaknesses I

Which of the following is an acceptable strategy when discussing weaknesses in an interview?

Which of the following is an acceptable strategy when discussing weaknesses in an interview?

Your Strengths and Weaknesses II

Let’s say that your company is running a standard control and variant AB test on a feature to increase conversion rates on the landing page. The PM checks the results and finds a .04 p-value.

How would you assess the validity of the result?

Let’s say you want to test the close friends feature on Instagram Stories.

How would you make a control group and test group to account for network effects?

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

What could be a potential risk when Facebook segments the metrics of the user interface change by market or demographic groups?

Network Experiment Design I

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

What is the primary metric that Facebook should monitor if it decides to make the user interface of its posting feature more like Instagram's?

Network Experiment Design II

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

If Facebook changes its user interface to mimic Instagram's, which adverse effect should they anticipate and monitor?

Network Experiment Design III

How would you explain what a p-value is to someone who is not technical?

What does a p-value in a statistical test represent?

What does a p-value in a statistical test represent?

P-value to a Layman I

In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?

In a statistical test, how does a low p-value (less than 0.05) influence our decision about the null hypothesis?

P-value to a Layman II

You’ve been tasked with building a classification machine-learning model to predict whether a transaction is either fraud or not fraud for a credit card company. You have ten years of historical data on transactions, including a flag for whether a transaction was manually identified as fraud.

Describe how you might go about building a fraud detection model for credit card transactions. Be sure to mention the possible model types, discuss the bias-variance tradeoff in model development, and address any complexities that arise from the class imbalance.

Input features in the data include:

<ul>
<li>Transaction amount</li>
<li>Merchant category for the transaction</li>
<li>Zip code for the merchant</li>
<li>Zip code for the billing address</li>
<li>Average transaction amount for the account over the past six months</li>
</ul>

Output feature:

<ul>
<li>0/1 flag for fraud (0 = not fraud, 1 = fraud)</li>
</ul>

Note: Fraudulent transactions are (thankfully) a very small percentage of all historical transactions. Assume fraudulent transactions are 0.01% of historical data.While building the model to perform the classification, you need to consider the bias/variance tradeoff, and take into account the fact that there is a class imbalance (very few of the observations are “fraud”).

When an interviewer asks you a question along the lines of:

<ul>
<li>Why did you apply to our company?</li>
<li>What are you looking for in your next job?</li>
<li>What makes you a good fit for our company?</li>
</ul>

How should you respond?

When asked 'What are you looking for in your next job?' in an interview, how can you tie the company's employee benefits into your response?

When asked 'What are you looking for in your next job?' in an interview, how can you tie the company's employee benefits into your response?

Why Do You Want to Work With Us I

How can company values be used effectively in an interview when asked 'What makes you a good fit for our company?'

How can company values be used effectively in an interview when asked 'What makes you a good fit for our company?'

Why Do You Want to Work With Us II

When responding to the question 'Why did you apply to our company?' during an interview, what aspect should you highlight?

When responding to the question 'Why did you apply to our company?' during an interview, what aspect should you highlight?

Why Do You Want to Work With Us III

You work as a backend engineer at Glassdoor. The company relies on multiple legacy systems to aggregate and process job postings, salary data, and company reviews. To ensure reliable data ingestion and processing, you are tasked with building a monitoring service that tracks the health of these legacy systems via regular heartbeat signals.

The service must expose one endpoint to receive heartbeat signals from each legacy system, and another endpoint to return a list of systems that have not sent a heartbeat in the last five seconds. How would you architect this monitoring system? Write a technical design document describing your approach.

Design a monitoring service for legacy system health via heartbeat endpoints at Glassdoor.

Legacy System Heartbeat Monitor

DevOps

You’re building a machine learning model to predict hospital readmission risk using patient records from the past five years. The data includes patient demographics, diagnoses, treatment history, and outcomes, but hospitals periodically change treatment protocols and data collection standards over time.

How would you structure your training, validation, and test splits to ensure your model generalizes to future patients? What risks would arise if these datasets were not clearly separated, and how would you validate that your final evaluation reflects real-world performance?

Let’s say we’ve deployed a recommendation model for an e-commerce site, and after two months, we notice that the model’s click-through rate has dropped from 12% to 7%.

How would you investigate the cause of this performance decline, and what steps might you take to address it?

You work as a machine learning engineer for a health insurance company. Design a machine learning model, which given a set of health features, classifies if the individual will undergo major health issues or not.

Let’s say we’ve proposed a new analytics dashboard to a client, but after reviewing our design and data sources, the client says the solution doesn’t meet their needs and pushes back on several key features.

How would you approach the conversation to negotiate alternative solutions, and what steps would you take to ensure disagreements are handled constructively so we can still reach a successful outcome?

Let’s say you are given a model that predicts whether a piece of news is relevant or not when shared on Twitter.

How would you evaluate the model?

Assume we have some credit model, which has a calibrated (up to some error) score of how credit-worthy any individual person is.

For example, if the model’s estimate is 83% then we can assume the actual score is between 81% and 85%.

If we take 83% as a score cutoff and deem everyone above that score as credit-worthy, are we over-estimating or underestimating the actual population’s credit score?

You are tasked with flagging users who post fake comments on Amazon to then give to the modeling team for further analysis.  

<ol>
<li>How would you initially go about filtering users who post fake comments on Amazon?</li>

<li>Let’s say that the team needs more data points faster. How would you scale out the effort with minimal manual intervention?</li>
</ol>

How would you initially filter users who post fake comments on Amazon and scale the effort with minimal manual intervention

Flagging Users

Business Case

Let’s say you’re an ML engineer at Netflix. You have access to reviews of 10K movies. Each review contains multiple sentences along with a score ranging from 1 to 10.

How would you design an ML system to predict the movie score based on the review text?

Let’s say you work on Facebook Groups. 

How would you go about increasing the number of comments on each Group post?

How would you go about increasing the number of comments on each Group post

Increasing Group Comments

Product Sense & Metrics

How would you build a database for a consumer file storage company like Dropbox?

Note: The database must be able to store many different file type formats and keep updates and history of file changes.

Calculate Moving Average	SQL	Easy
Predict Customer Churn	Machine Learning	Medium
A/B Test Significance	Statistics	Medium
Optimize Query Performance	SQL	Hard
Feature Importance Analysis	Machine Learning	Medium
Clean Missing Data	Python	Easy
Neural Network Architecture	Deep Learning	Hard
Calculate Cohort Retention	SQL	Medium
Bayesian Probability	Statistics	Easy
Recommend Similar Products	Machine Learning	Hard

IHME Interview Questions

IHME Interview Guides

IHME Interview Questions

Challenge

Discussion & Interview Experiences

Discussion & Interview Experiences