Tom O. - Appleton, Wisconsin, United States | Professional Profile | LinkedIn

Dhruv Singh

HoneyHive AI • 10K followers

LLMs are getting smarter, but they can't stop talking. That chattiness is a very real problem. Verbose responses dilute the user experience, drive up inference costs, and make certain workflows completely impractical. Shortening responses without corrupting meaning or accuracy has a very real impact on cost and user experience. Traditional approaches to evaluating verbosity come with tradeoffs. Metrics like BLEU and ROUGE focus on reference overlap, but most real-world tasks do not have reference answers. LLMs can be used as a judge, but already have a bias towards verbosity. It's a sticky problem, with implications for the viability of certain agent workflows, conversational systems, and real-time interactions where no canonical answer exists. That's what makes this paper so interesting. ConCISE quantifies non-essential content without relying on reference texts, making it especially useful for open-ended tasks (such as agent workflows). Their approach opens the door to a scalable, automated way to evaluate response brevity.