What ii you use the negative prompt provided by diffusion models? In stable diffusion(maybe midjourney doesn't have this), it feels like it does avoid what I put in there... Although I have not tried only a negative prompt.
You're absolutely right that Stable Diffusion and similar models do offer a “negative prompt” input (usually as parameters), and when used properly, it can help steer the model away from unwanted elements—especially when you're not explicitly drawing attention to them.
But the key difference I’ve found is that negative prompts work best when used alongside a strong positive prompt, rather than in isolation or with general negations like “anything except X.” The pink elephant paradox kicks in when our main instruction still fixates on the thing we want to avoid. Even if we say "not a pink elephant," the model anchors on that concept because it was explicitly mentioned.
I've also noticed that image models with negative prompts (ie, "flamingo --no pink") will still sometimes steer into the negative prompt. I ran that prompt in Midjourney and 2 of the 4 results were pink. 🫠
Negative prompts can be powerful—but they aren’t magic on their own. They’re most effective when we’ve already done the work of defining what we do want with clarity.
Appreciate you bringing this up—curious to hear what you find if you try a pure negative prompt in Stable Diffusion (or even a positive prompt with a close negative prompt like "flamingo --no pink")!
With my very limited testing on a rudimentary model locally, it looks like I have three cases...
+Flamingo, -pink:
This gave me about half pink and half not pink but still some pink in them flamingos that were kind of white.
+Blue flamingo, no negative prompt
This gave me about half blue flamingos and half pink flamingos with blue background
+Blue flamingo, -pink
This gave me about 80% blue flamingos, and the occasional pink one.
I think this tracks pretty well with your assertion that negative prompts can affect in undesirable ways but they are essential in creating what you want.
What ii you use the negative prompt provided by diffusion models? In stable diffusion(maybe midjourney doesn't have this), it feels like it does avoid what I put in there... Although I have not tried only a negative prompt.
You're absolutely right that Stable Diffusion and similar models do offer a “negative prompt” input (usually as parameters), and when used properly, it can help steer the model away from unwanted elements—especially when you're not explicitly drawing attention to them.
But the key difference I’ve found is that negative prompts work best when used alongside a strong positive prompt, rather than in isolation or with general negations like “anything except X.” The pink elephant paradox kicks in when our main instruction still fixates on the thing we want to avoid. Even if we say "not a pink elephant," the model anchors on that concept because it was explicitly mentioned.
I've also noticed that image models with negative prompts (ie, "flamingo --no pink") will still sometimes steer into the negative prompt. I ran that prompt in Midjourney and 2 of the 4 results were pink. 🫠
Negative prompts can be powerful—but they aren’t magic on their own. They’re most effective when we’ve already done the work of defining what we do want with clarity.
Appreciate you bringing this up—curious to hear what you find if you try a pure negative prompt in Stable Diffusion (or even a positive prompt with a close negative prompt like "flamingo --no pink")!
With my very limited testing on a rudimentary model locally, it looks like I have three cases...
+Flamingo, -pink:
This gave me about half pink and half not pink but still some pink in them flamingos that were kind of white.
+Blue flamingo, no negative prompt
This gave me about half blue flamingos and half pink flamingos with blue background
+Blue flamingo, -pink
This gave me about 80% blue flamingos, and the occasional pink one.
I think this tracks pretty well with your assertion that negative prompts can affect in undesirable ways but they are essential in creating what you want.