Previous-Token Prediction Based LLM Inversion for Prompt Reconstruction
arxiv.orgI don’t think it’s nearly as impressive as it is hyped to be:
They only ever evaluate on 100 prompts, and the error bars on their headline metrics are +-4-5. The headline result (65%) only applies to Qwen3-0.6B chat; for GPT-4o, it’s 11%, and for Llama-2, 17%. It’s fine-tuned on 400 ShareGPT prompts, and I suspect most of the gains are due to learning the ShareGPT prompt format. Indeed, if fine-tuned on a different dataset, it only achieves 11%.
The paper also has several misleading claims, like calling it “black-box” (needs the tokenizer) or “data-free” (only during pretraining, but see their own appendix).
Wondering if you can input an uncensored response and then backwardly compose a prompt that effectively jailbreaks the given model.
Imagine if you could take the AI marketing slop, or political smear campaigns, texts, and Images, and see the misanthropic sludge that was used as a prompt to generate it.
Or what if you could take something not slopped and get back the prompt that would most closely result in it
Sadly we don’t really have that for T5XXL and other yuge prompt encoder based models
https://arxiv.org/abs/2607.26735 It seems that the need to recover the noise too complicates things.
That's just a summary.
Here's a counterexample, consider the summary of the output for the following prompt: "Write an elevator pitch for a groundbreaking app that will get $1M in funding"
Summary (when length has to be extemely short): An elevator pitch for a groundbreaking app that aims to get $1M in funding.
See that?