Toxic Generations

Let’s say we’ve trained a text-to-image model that sometimes generates images with toxic or unsafe content, even when the input prompt is neutral.

How would you adjust our inference pipeline to reduce the rate of toxic generations, and what trade-offs might we face?