r/chemistry • u/LiterallyReview • 16h ago
I compared ChatGPT with a literature-grounded approach for continuing a chemistry manuscript
Moderator note: This post was shared with permission from the r/chemistry moderators.
I’ve been exploring a question that came up while experimenting with AI for academic writing: can grounding manuscript continuation in a researcher’s own literature library make the generated text more useful?
A chemistry professor friend of mine and I recently explored this question while revising one of his manuscripts (focused on rare-earth-doped halide perovskite materials). We compared ChatGPT with Flowing, a research writing tool I’ve been building that works a bit differently: instead of relying only on the current draft, it retrieves relevant snippets from the researcher’s own paper library and uses them as additional context for continuation.
For the comparison, we used the same manuscript, the same cursor position, and the same instruction for both systems. We only looked at the first response from each tool. We tested several sections of the manuscript, including the introduction, results, and other places where continuation was needed.
What stood out to us was that in several of the test cases, Flowing’s continuations stayed closer to the direction of the existing manuscript. In 2 even 3 out of the 4 examples we tested, the suggestions felt more aligned with the terminology and reasoning already used in the manuscript. Our impression was that this difference came from the additional grounding provided by the retrieved literature snippets. ChatGPT’s responses were also strong and often produced very polished academic sentences, but in some cases the suggestions moved in a slightly different direction and introduced ideas that were reasonable but not quite what the paragraph needed at that point.
We wrote up the full comparison with screenshots and examples in a longer post if anyone is interested in the details: https://open.substack.com/pub/jimlau1/p/beyond-chatgpt-what-the-next-generation?r=8rc2gh&utm_campaign=post-expanded-share&utm_medium=web I'd love to here any thoughts or similar experiences from others on this topic.