Feed Inazuma Eleven

Claude Opus 5.5 scored 97.2/100 on VC analyst research. Then 13 models tried to pick which startups would grow. Every one lost to a one-line rule: sort by past hiring growth 🤯 Research is cheap now. Judgment isn't. (Mento VC's SAB benchmark) #AIEvals #LLMs #VentureCapital

Gemini 4 Carbon dropped 2 weeks after Argon and is already #1 on the Artificial Analysis Intelligence Index (59). But look closer: accuracy up 48% to 58%, hallucination rate up 4% to 11%. Smarter and more confidently wrong? #Gemini4 #AIEvals #LLM

Researchers planted major errors in clinical trial reports. Human reviewers caught 27%. All 15 LLMs tested beat that, at 53% to 96%. Even small models doubled the human rate. Still think human-only peer review is safe for trials? #AI #LLMs #AIEvals #PeerReview

Same model, 23 points apart?! On the new Artificial Analysis Intelligence Index v5, Claude Opus 5.5 scores 35 at low effort and 58 at max, for 10x the cost per task ($0.98 vs $10.04). GPT-6.1 Sol max lands within 4 points at ~1/8 the price. #AIEvals #LLM #AgenticAI

Our take: the near-term win is making uncertainty inspectable, not an approval badge. Full analysis: t.co/uj8rnbn3qn What would you need to see in the first report? @ChrisPainterYup @Miles_Brundage #AISafety #AIEvals

Treat AI evaluations like a test suite: include failed tools, wrong permissions, ambiguous requests and recovery cases. t.co/JjUKEoqxsw #AIEvals

#AI can sound right. But who's checking if it actually is? Had a fantastic #QFireside chat with @MillanKaul on #AIEvals, trust, and why human thinking matters more than ever in #QA. 🎥 Here's a quick 2-minute recap! #Testing @ACCELQ #QCommunity t.co/5U2JXBNtns

More sessions like this are lined up for QCon London 2027, April 13-16. Find out more about QCon London 2027 here: t.co/bC8QULTGw6 #QConLondon #AIEvals #RAG

Anthropic gave six METR staff employee-like access on Oct 15, 2026. METR publishes its own analysis. Anthropic can't veto it, only redact categories like confidential IP. Access and veto power differ. Ask which a vendor means #JEBREX #AIEvals #Anthropic #AISafety #AIRegulation

More sessions like this are lined up for QCon London 2027, April 13-16. Find out more about QCon London 2027 here: t.co/t9Km6gDGUp #QConLondon #AIEvals #AIEngineering

Accuracy went DOWN and it still took #1 🤯 Liquid AI's d1-ultra tops the Artificial Analysis Decision Index at 65 (Jev 1.14: 57). Accuracy fell 41% to 32% vs d1, but when it can't get it right it says "I don't know" 85% of the time. d1? 2%. Fair trade? #AIAgents #LLM #AIEvals

Your #AI app sounds confident. But is it right? A medical diagnostics app told someone with diabetes symptoms to drink more water. Confident. Convincing. Yet completely wrong. Here's how I run AI evals with #Jev by @TypeSafeAI to catch this 👇 #AIEvals #AITesting #Testing t.co/bINBgFcaGL

Ragas. DeepEval. PyRIT. FMEval. Or bring your own. Chris Paton explains the one idea the whole Gen AI Evaluation Toolkit on AWS is built around. t.co/PoLF1L2ez8 #GenAI #AIEvals #OpenSource t.co/D4mVnjtHpH

Google's ADK used to pass an agent eval that ran zero cases. 2.10 now errors. Pipelines will go red on upgrade. Don't pin back: that red is information. #AgenticAI #AIEvals t.co/hHMP5FCTY6

PRDs need a few quality checks next to requirements. Not after the fact. If it is not on the quality contract at PRD time, it will not be on the gate at ship time. t.co/Mh8JIhYUcK @FutureAGI_ #AIevals #ProductManagement #FutureAGI #LLMEval

Fake users after you name real failure modes = leverage. Fake users that invent a fantasy product = noise. Simulate edge variants of blockers you already saw in traces. Not 1,000 random users first. t.co/FHo2PsGbtk @FutureAGI_ #AIevals #ProductManagement #FutureAGI #AIAgents

Optimization without a quality suite: Making the wrong thing cheaper and faster. Wait time down. Policy misses up on journey X. Nobody noticed until support did. Quality checks are the scoreboard for every improve step. t.co/GRU5ad5gCz @FutureAGI_ #AIevals #ProductManagement #FutureAGI #AIAgents

A sticky-note pile of "good examples" is not a launch gate. You need four kinds of cases: happy path, edge cases, attacks, and real failures from production. Ship coverage, not hope. t.co/V7rxosmEjc @FutureAGI_ #AIevals #ProductManagement #AIProduct t.co/CSlw2Ks4gv

BIP note: we stopped saying "the model is good." We say "0 blocker failures on journeys A/B/C this suite. 1 major with exception logged." Language change. Same team. Better ship calls. #BuildInPublic #AIevals #ProductManagement #FutureAGI

Eval theater autopsy: Golden prompts at launch. Green check. Then silence for 11 weeks. Production invented new failures. The suite never learned them. Continuous evals or decoration. Pick one. Part 1: t.co/EGVZvVmPQT #AIevals #BuildInPublic #ProductManagement #FutureAGI