Let’s say we want to build a new delivery time estimate model for consumers ordering food delivery.

How would you determine if the new model predicts delivery times better than the old model?

When rolling out a new delivery time estimate model in production, what is an important consideration?

When rolling out a new delivery time estimate model in production, what is an important consideration?

Delivery Estimate Model I

The new model should be given to all users at once to test its robustness.

The new model should only be tested in one geography to limit its influence.

The new model should be tested on the busiest day to measure its maximum capacity.

The new model's variant test group should consist of a distribution of users that will be consistent as it rolls out to the larger population.

The new model should only be tested during non-peak hours to minimize potential disruptions.

What is one way to measure the performance of a new delivery time estimate model for food delivery?

What is one way to measure the performance of a new delivery time estimate model for food delivery?

Delivery Estimate Model II

Estimate the cost of implementing the new model.

Measure the speed of calculations of the new model.

Measure how well the predictions from the new model compare against the actual delivery time.

Compare the new model against the total revenue generated.

Compare the new model against the number of orders placed.

Delivery Estimate Model

Comments