Google's recent directives instructing AI evaluators to engage with prompts outside their expertise raise concerns about the accuracy of evaluations, undermining quality control.
Under new guidelines, contractors are now asked to rate prompts they do not fully understand, which could compromise the accuracy of the evaluations.
One contractor expressed concern, stating, 'I thought the point of skipping was to increase accuracy by giving it to someone better?'—highlighting the potential pitfalls of the new system.
Previously, the guidance encouraged skipping prompts outside one's critical expertise, but the shift suggests a controversial move towards more generalized evaluations.
Collection
[
|
...
]