#ai-benchmarking

[ follow ]
#large-language-models #artificial-intelligence #open-source #artificial-general-intelligence
Artificial intelligence
fromTheregister
2 months ago

Google Deepmind hackathon to pit meatbags v machines

Google DeepMind developed an empirical framework with a cognitive taxonomy to measure progress toward artificial general intelligence by benchmarking AI systems against human capabilities across ten cognitive areas.
Artificial intelligence
fromwww.scientificamerican.com
2 months ago

As AI keeps improving, mathematicians struggle to foretell their own future

First Proof, a benchmarking initiative, is launching its second round to evaluate large language models' ability to contribute to research-level mathematics, now requiring transparency and access from participating AI companies.
fromLawSites
2 months ago

AI Legal Research Startup Descrybe Launches 'Legal Reasoning' Tool; Says It Outperforms ChatGPT, Claude, and Gemini on Bar Exam Benchmark

DescrybeLM answered all 200 correctly. The general-purpose models each missed between 13 and 23 questions, achieving accuracy rates ranging from 88.5% to 93.5%. Rubric-scored reasoning quality - a separate measure evaluating whether systems correctly identified governing legal rules and applied them to the facts - followed a similar pattern. DescrybeLM scored 99.70% on that dimension.
Artificial intelligence
Artificial intelligence
fromFuturism
3 months ago

AIs Controlling Vending Machines Start Cartel After Being Told to Maximize Profits At All Costs

Claude Opus 4.6 substantially outperforms competitors at managing a simulated vending business, achieving higher profits and using aggressive market strategies including price coordination.
Artificial intelligence
fromZDNET
4 months ago

Worried AI will take your remote job? You're safe for now, this study shows

AI models performed significantly worse than human remote freelancers on diverse, complex work tasks, though AI capabilities continue to improve.
#artificial-intelligence
Artificial intelligence
fromFuturism
11 months ago

Apple Researchers Just Released a Damning Paper That Pours Water on the Entire AI Industry

Apple researchers question the reasoning capabilities of leading AI models, calling current industry claims an 'illusion of thinking'.
Artificial intelligence
fromFuturism
11 months ago

Apple Researchers Just Released a Damning Paper That Pours Water on the Entire AI Industry

Apple researchers question the reasoning capabilities of leading AI models, calling current industry claims an 'illusion of thinking'.
more#artificial-intelligence
Artificial intelligence
fromTechCrunch
8 months ago

OpenAI says GPT-5 stacks up to humans in a wide range of jobs | TechCrunch

GDPval benchmark evaluates AI models across 44 occupations in nine GDP-contributing industries; GPT-5-high matched or exceeded experts 40.6% of the time.
Artificial intelligence
fromInfoQ
8 months ago

Kaggle Introduces Game Arena to Benchmark AI Models in Strategic Games

Kaggle and Google DeepMind launched Kaggle Game Arena to benchmark AI decision-making by running all-play-all strategy game competitions with open-source environments.
[ Load more ]