Turn a Firehose of Feedback into a Backlog with Agents
516 Reddit comments, 27 Product Hunt threads, GitHub issues and Discord — launch week buries you in feedback that decays if unprocessed. The agent workflow that turned ours into a ranked backlog, with receipts.
Feedback is an asset with a half-life: unprocessed, it decays into a vibe. After launch week handed us 516 Reddit comments, a Product Hunt thread, and a swelling issue tracker, agents turned all of it into a ranked backlog where every card cites the real humans who asked. Two of the top three cards shipped in v0.4.4. The workflow, start to finish.
Launch-week feedback has a cruel shape: it arrives all at once, from five channels, in wildly mixed quality, precisely when you have the least attention to spare. Read it casually and you’ll remember the funniest insult and the nicest compliment, then confidently misbuild your roadmap around them both.
The fix isn’t discipline. It’s admitting this is a data-processing job and staffing it accordingly.
Pass one: account for everything
The first pass is deliberately dumb: every comment, from every channel, gets bucketed — bug report, feature ask, objection, praise, question — with the quote, the author, and the permalink preserved. The brief’s key sentence is account for every comment, because that’s the sentence a tired human violates. Ours came back with counts we still quote in arguments: utility skepticism, 12 distinct commenters; token-cost concern, 11.
The receipts matter more than the counts. “Users are worried about cost” starts a debate; eleven linked quotes end one.
Pass two: merge across channels
The pass most teams never do, because it requires holding everything at once. The same problem arrives dressed differently per platform: a Product Hunt commenter politely reports being unable to switch engines after setup; a Redditor says the onboarding dead-ended; a GitHub issue titles it precisely. One agent with all three corpora merges them into one backlog item with five receipts — and suddenly it’s not three small complaints, it’s one bug class blocking a fifth of new evaluators.
This pass produced our best roadmap insight of the year: our two most sophisticated users — on different platforms, in different words — asked for the same missing primitive, a visible “this decision needs your eyes” flag on agents. Read channel by channel, that’s two mild comments. Merged, it’s the clearest feature signal we’ve ever received.
Pass three: rank on evidence, file as work
The final pass turns merged items into kanban cards, ranked by things you can defend: how many distinct people, how blocked they were, whether they’re the kind of user you’re building for. The card’s description carries the quotes and links, so whoever picks it up — human or agent — starts from the actual reports rather than a paraphrase.
Then the backlog met reality: the top cards became v0.4.4’s headline fixes. And because every card carried its reporters, “tell the people who reported it” was a checklist item, not archaeology. One blocked user from launch week had said, verbatim, that he’d re-test when the fix shipped. The card knew his name. That follow-up is worth more than a week of marketing.
The compounding part
Do this once and it’s a great retro. The compounding version is a standing intake: the same bucketing agent runs on whatever accumulated overnight — new issues, Discord threads, stray mentions — and appends to the same backlog under the same merge rules, on a schedule. Feedback stops being an event and becomes a pipeline, and the backlog stays ranked by evidence instead of by whoever shouted most recently.
The uncomfortable, useful truth we took from ours: the community’s most upvoted praise and its sharpest criticism pointed at the same place — people love watching agents work and want more control over what they see. A backlog built on receipts forces you to hear both halves. That tension is now the roadmap’s spine, and we can cite our sources.
FAQ
How do agents turn raw comments into a backlog?
Three passes: bucket every comment (bug, feature ask, objection, praise) with quotes and links preserved; merge duplicates across channels so five phrasings of one problem become one item with five receipts; then rank by evidence — how many distinct people, how serious, how blocked. The output is kanban cards whose descriptions cite the actual users.
What's the advantage over just reading the comments yourself?
Coverage and honesty. A human skims, remembers the loudest five comments, and calls it the community's opinion. An agent instructed to account for every comment can't skip the boring middle — and the resulting backlog item carries links to every person who reported it, which also gives you a list of exactly who to tell when it ships.
Did this actually change what Munder Difflin shipped?
Directly. The launch-week backlog put the Windows messaging failure and onboarding dead-ends at the top on evidence volume, and both headline v0.4.4. The 'this decision needs your eyes' flag — asked for independently by two users on two different channels — is on the roadmap because cross-channel merging revealed it was one request, not two.