Batch vs Mini-Batch vs Stochastic Gradient Descent

You’re working on an image recognition model at Walmart that powers a mobile feature allowing customers and associates to scan store shelves and instantly identify products, prices, and promotions. A single shelf image can contain dozens of visually similar grocery items, and accurate recognition is critical for inventory checks and in-store shopping assistance.

The training dataset includes tens of millions of labeled shelf images collected across thousands of Walmart stores with varying lighting conditions, camera quality, and layouts. The model is retrained daily to reflect packaging changes and newly introduced products, and the entire training pipeline must complete within a strict 2-minute SLA so the updated model can be deployed before stores open. Training runs on shared GPU infrastructure, making throughput, stability, and compute cost key constraints.

How would you decide between batch gradient descent, mini-batch gradient descent, and stochastic gradient descent in this setting? How would you investigate the trade-offs between convergence speed, training noise, GPU utilization, and generalization while ensuring the SLA is reliably met?