As UKRI explores using LLMs to review grants, it's a good time to revisit Bryan Wilder's excellent blog post.
There are a lot of naive reasons to oppose AI review ("you'll never automate human intuition!"). But there are also good reasons, including the *load-bearing role of human disagreement.*
Should LLMs be used to review papers? AAAI is piloting LLM-generated reviews this year. I wrote a blog post arguing that using LLMs as reviewers can have bad downstream consequences for science by centralizing judgments about what constitutes good research.
bryanwilder.github.io/files/llmrev...
