Context: Back in ChinAI #224, we highlighted the SuperCLUE benchmark, released in May, which aimed to test large language models from Chinese and international labs along three main dimensions: 1) foundational capabilities such as dialogue and coding; 2) specialized and academic capabilities like physics knowledge; and 3) capabilities in Chinese-language particularities such as knowledge of classical Chinese literature and Chinese idioms. Last week, the SuperCLUE team released its July rankings (link to original Chinese), updated with 3700 confidential test questions and 20 total participating models. — Read More