Ordinal Severity Metrics

You have a heavily imbalanced model that classifies chest X-rays into four ordered severity grades. How would you evaluate its predictions, what does mean absolute error add beyond accuracy, and what can a low average error hide about the rarest class?

Use grades 0 through 3 as an explicit practice coding assumption. Explain your reasoning with a small example; rank distance is not assumed to be a measure of clinical consequences.