Research assessment: a round-up for early-career researchers

"Steve Goodman had just completed a three-page referee's letter for an academic colleague who was going for promotion in a law department, when he received a bundle of papers that surprised him. The package contained the recommendation letters of other referees. "They were eight to ten pages long," he says, adding that he had never seen letters like these before."

"Goodman, an epidemiologist at Stanford University in California, describes what he read as having been written by people who had digested the colleague's books, engaged with their arguments and produced a commentary similar to something you'd publish. His letter, by contrast, took half a day to write. His legal academic colleagues told him it was standard for law researchers to set aside a week to write referee letters, and that they received credit towards their own career development for doing so."

"The idea that you would do a deep dive into someone's scholarship to write a recommendation letter is "inconceivable in biomedicine", he says, where it is standard to focus on a researcher's publications and assess the importance and value of their contributions. Goodman did not know the scholar in question, but was asked to write the letter because their work bridged law and biomedicine."

An epidemiologist completed a brief three-page referee letter and then discovered other referees' letters of eight to ten pages that deeply engaged with the candidate's books and arguments. Law academics commonly spend a week on referee letters and receive career credit for the task, while biomedical refereeing typically focuses on publications and takes far less time. Research-assessment practices vary widely both within institutions and between countries. Some nations run large, costly nationwide assessment programmes that measure departments and individual staff for comparison. Those exercises consume millions of dollars and countless hours. Governments support them to ensure accountability in funding distribution.

Read at Nature