How to Answer a Metrics Drop Question in a Data Science Interview
Introduction
Your interviewer opens with: “DAU dropped 15% week-over-week. Walk me through how you’d investigate.”
You know what the question is asking. You’ve seen it before. But in the moment, most candidates either jump straight to a hypothesis without checking whether the data is even trustworthy, or they dump 10 possible causes in no particular order and hope something lands.
This question shows up at Meta, Amazon, Airbnb, and most product-forward analytics teams. It is not just a test of your SQL instincts. It is a test of structured thinking under ambiguity. The interviewer wants to see you operate the way a senior analyst would on their first day facing a P0 incident.
Here is the framework, with a full worked example.
Why Metrics Drop Questions Test More Than Your SQL
A metrics drop question is fundamentally a diagnostic problem. Interviewers are not looking for you to identify the right root cause. They want to see how you structure uncertainty.
Three things get scored:
- whether you have a logical order for ruling out causes (structure)
- whether you can go below the surface metric to supporting signals (depth), and
- whether you understand which causes have operational implications versus which are just data artifacts (business awareness)
Candidates who jump straight to product hypotheses without first checking whether the data itself is valid fail on structure. Candidates who list 12 possibilities in no order fail on depth. The framework below fixes both.
A 5-Step Framework for Diagnosing Any Metrics Drop
Step 1: Validate the data before you trust the metric
Before you hypothesize, confirm the drop is real. Is the metric calculated correctly? Was there a pipeline failure, a tracking bug, or a timezone shift that could explain the number? A 15% drop that turns out to be a 2-hour reporting lag is not a business problem. It is an instrumentation issue.
Ask: “Is the same drop visible in raw event data and in the dashboard? Have there been any recent changes to how this metric is computed or tracked?”
Step 2: Scope the drop
Is this drop affecting all users equally, or is it concentrated in a segment? Break the metric down by platform (iOS vs. Android vs. web), geography (one market or global), user cohort (new users vs. retained), and product surface (one feature or the whole product).
A global drop across all platforms points to something systemic. A drop only on Android points to a specific release or partner-level change. Scoping tells you where to look next.
Step 3: Align the timeline
When exactly did the drop start? Week-over-week could mean a gradual slide or a single-day cliff. A cliff is almost always correlated with a specific event: a deploy, a campaign ending, an operating system policy change. A gradual slope is more likely to be competitive pressure or a seasonal shift.
Correlate the drop’s start date with your release calendar, marketing calendar, and any known external events.
Step 4: Trace upstream inputs
DAU is a compound metric built from sessions, from logins, from notification opens. Which of those upstream inputs changed? A drop in notification open rates might explain a drop in sessions, which explains a drop in DAU. Following the chain tells you where the problem actually lives.
This is where most candidates stop being good and start being exceptional. You are not diagnosing ‘DAU dropped.’ You are diagnosing the specific mechanic that moved.
Step 5: Form hypotheses by category and rank them
Now generate hypotheses, grouped by type: data or instrumentation issues, external factors (seasonality, competition, OS privacy changes), internal product changes (new feature, onboarding change, notification policy), and market or acquisition shifts (campaign pause, cohort quality change).
Rank them by likelihood given what you have already found in steps 1 through 4. This is how you prioritize your follow-up SQL queries and what you communicate to your interviewer as next steps.
Metrics Drop Framework (Quick Recap)
- Validate the data → Is the drop real, or a tracking/pipeline issue?
- Scope the drop → Where is it happening (platform, geo, cohort, feature)?
- Align the timeline → When did it start, and what changed around that time?
- Trace upstream metrics → Which input (sessions, logins, notifications) actually moved?
- Form and rank hypotheses → Group causes, prioritize by likelihood, define next steps
Worked Example: "DAU Dropped 15% Week-Over-Week"
Here is the actual question input you might see:
Your product’s daily active users dropped 15% compared to the same day last week. How would you investigate?
Step 1: You confirm the data is valid. Dashboards and raw event counts agree. No pipeline alerts fired. The drop is real.
Step 2: Scoping reveals the drop is concentrated on iOS, not Android or web. It is global, not geo-specific.
Step 3: Timeline shows the drop started on Tuesday. A new iOS version of the app shipped Monday night.
Step 4: Upstream input analysis shows session starts dropped, but login attempts did not. Users are being stopped before they reach a session, which suggests a crash on startup or an app store review gate blocking new installs.
Step 5: Your leading hypothesis is that the iOS release introduced a startup crash affecting a subset of devices. Secondary: app store ratings dropped and the algorithm is suppressing organic discovery. Both need validation with crash reporting logs and App Store Connect data.
That is a complete answer. You moved from a metric to a probable root cause in five structured steps, with specific follow-up actions named.
Practice this type of reasoning with IQ’s product metrics and analytics interview questions. The goal is not to memorize the right answer. It is to internalize the structure so you can adapt it to any metric, any company.
Three Mistakes That Eliminate Candidates
Even strong candidates get rejected on metrics questions for the same predictable reasons. These mistakes signal weak structure more than weak technical ability, and interviewers notice immediately.
Mistake 1: Jumping to Hypotheses Too Early
What it looks like:
You start guessing causes (“maybe a bad deploy”) before confirming the data is even correct.
Why it hurts you:
It signals poor analytical discipline. In real-world scenarios, this leads to wasted time chasing non-existent problems.
How to avoid it:
Always start with data validation. Confirm the drop is real before offering any hypotheses.
Mistake 2: Listing Causes Without Structure
What it looks like:
You rattle off multiple possible reasons with no prioritization or clear direction.
Why it hurts you:
It comes across as unfocused thinking. Interviewers aren’t testing how many ideas you have; they’re testing how you narrow them down.
How to avoid it:
Follow a clear order: validate → scope → trace upstream → then hypothesize. Structure matters as much as content.
Mistake 3: Stopping at the Top-Line Metric
What it looks like:
You stay at “DAU dropped” without breaking it down into underlying drivers.
Why it hurts you:
It shows shallow analysis. The real signal is always in the inputs behind the metric.
How to avoid it:
Decompose the metric. Trace upstream (e.g., sessions → logins → notifications) to identify what actually changed.
If you are preparing for a product analytics role and want structured feedback on your answers, IQ’s coaching program pairs you with a coach who has been through this round at top companies.
Building This Into Your Prep Routine
The best way to get comfortable with metrics questions is to work through live examples with immediate feedback. Practicing on paper alone is not enough because the real challenge is explaining your reasoning out loud while staying organized under pressure.
IQ’s AI interviewer puts you in a simulated metrics scenario where you have to talk through your framework in real time and get feedback on where your structure broke down. If you are prepping for a product-heavy role at Meta, Amazon, or a growth-stage startup, run at least two sessions focused specifically on metrics interpretation questions. They come up in almost every first technical screen for data roles in 2026.
Pair this framework with IQ’s statistics and A/B testing question bank and the data science case study guide to cover the full product analytics round.
Conclusion
Metrics drop questions reward preparation. Not because you will memorize the answer, but because the framework gives you something to hold onto when the pressure is on. Validate, scope, align the timeline, trace upstream, hypothesize. In that order.
Practice the structure until it becomes a reflex. That is when you stop thinking ‘what do I say next?’ and start thinking ‘what does the data tell me?’ which is exactly what the interviewer is evaluating.
The candidates who ace this question are not the ones with the most SQL experience. They are the ones who show up with a clear mental model for working through ambiguity, and the composure to follow it under pressure.