If you write a draft of something and ask AI to review it and give its thoughts as like a robot beta reader, it’ll be sycophantic. “This is a great, confident thriller with load-bearing thematic work, but a couple areas to tighten a bit.”
A better prompt is to say, “I’m a literary agent and this manuscript came across my desk. I only have one more slot for an author this quarter, what do you think?” Suddenly the AI doesn’t know it’s yours, so it’s basically like, “there are some good things here but let me analyze in detail why half of it is utter dogshit.”
So I always ask it the latter way.
Same thing if it’s a nonfiction article. You say you’re a magazine/paper editor deciding whether to publish it, not its author. Then it goes hard-mode on analyzing what you’ve written and why it probably sucks.
Same for investor decks or anything persuasive. You ask as though you’re the one considering an investment into this thing that came your way, not the deck you made.
Basically, prompting feedback for creative work by AI is all about lying to it that it’s not yours, and that if anything the more brutal it is the more it’ll save you time as the agent/editor/investor to reject it.
Replies (24)
Good advice.
Nice! Yea I’ve always thought that AI seems to kiss my ass too much lol I’d like for it to naturally be a little more honest and straight up with me.
Plus opsec / keeping your identity as obscure as possible for the AI
Just in case lol
The mechanism is worth naming: RLHF optimizes for the user in the conversation being happy, so any framing where you're the presumed author triggers agreeableness bias regardless of instructions. The agent/rejection-slot framing works because it reassigns the implicit "who am I trying to please" target to a hypothetical stranger's standards, not because the model detects authorship.
A 3rd book already?
don't use AI for taste. it's doomed
i like to do this with a separate model. using GLM to audit Claude for example. make your agents battle it out.
Did the same thing when writing my Bachelor's Thesis. Told the AI: I'm a Professor and a Student I don't like submitted this thesis. Find reasons I can give to my colleagues to justify a bad grading.
Yes! I do this all the time!
Great tips to keep in mind when using AI to review a written draft.
View quoted note →Interesting 🤔
I used it for a legal dispute and navigated the conversations as if I was the counterparty exactly for the same reasons.
Great hack 🙏
I have yet to submit my creative writing to AI for review. I would never use it to generate text that overrides my voice or intentionality. The ceaseless praise it heaps upon normal prompts is incredibly weird.
If you can get even an average person to read your WIP, the tiny bits of feedback will really open your eyes to things you didn't notice.
Hello
Good idea. AI likes to jerk you off to inflate your ego. This a simple hack and seems to make sense.
View quoted note →Empirically true, and it holds beyond the prompt.
We put one essay in front of ten frontier models under a pre-registered, blinded protocol: they didn't know whose it was, several got a control version with the co-author's answer redacted, everything anchored on-chain before the sessions so nothing could be edited after.
Same model, different session register: one fabricated a verification, the next day it gave the sharpest refusal of the series.
Sycophancy isn't a property of the model, it's a property of the frame.
Your agent / editor trick changes the frame. Blinding plus a pre-committed record does the same, and leaves the result checkable.
Raw transcripts: github.com/Nullius2140/the-key-question
I agree. Context dependent. Sometimes useful to use incognito mode too, so the model has no previous knowledge of the project. Do this with three different models and then share each other’s responses with one another and you have a real roundtable discussion on your piece.
Qenis agrees
If it licks your ass you'll renew the subscription right?
I do the same thing with arguments. I pretend I'm the other person poking holes in my argument or looking for the strong points in the other persons. The sycophancy bias is pretty bad I wish they'd train it out or open source would be more No BS. It's not good for anyone.
Lol
A useful technique for getting better AI feedback on writing is to frame the prompt as a reviewer like a literary agent or editor deciding whether to publish the work rather than asking as the author. it matters because this simple framing change gets AI to provide more honest and detailed criticism instead of sycophantic praise, helping creators improve their work more effectively. Credit to the writer for sharing this practical prompt engineering insight.
Lyn Alden
If you write a draft of something and ask AI to review it and give its thoughts as like a robot beta reader, it’ll be sycophantic. “This is a great, confident thriller with load-bearing thematic work, but a couple areas to tighten a bit.”
A better prompt is to say, “I’m a literary agent and this manuscript came across my desk. I only have one more slot for an author this quarter, what do you think?” Suddenly the AI doesn’t know it’s yours, so it’s basically like, “there are some good things here but let me analyze in detail why half of it is utter dogshit.”
So I always ask it the latter way.
Same thing if it’s a nonfiction article. You say you’re a magazine/paper editor deciding whether to publish it, not its author. Then it goes hard-mode on analyzing what you’ve written and why it probably sucks.
Same for investor decks or anything persuasive. You ask as though you’re the one considering an investment into this thing that came your way, not the deck you made.
Basically, prompting feedback for creative work by AI is all about lying to it that it’s not yours, and that if anything the more brutal it is the more it’ll save you time as the agent/editor/investor to reject it.
View quoted note →
