Baidu reports Ernie 3.5 outperforms ChatGPT 4.0 — at least in Chinese language

Baidu Inc. recently announced that its chatbot, Ernie 3.5, has outperformed OpenAI's ChatGPT in various key measures. However, the context of these claims invites a more nuanced interpretation.

Ernie 3.5, Baidu's latest foundational model, reportedly outperformed OpenAI's ChatGPT built on GPT-3.5 in terms of general abilities, and even claimed superiority over the more advanced GPT-4 in several Chinese-language capabilities. These assertions are based on tests reported by China Science Daily using datasets from AGIEval and C-Eval, benchmarks designed for evaluating AI models. However, it's worth noting that these findings have not been corroborated by independent sources, highlighting the need for further verification in the global AI community.

Just three months after the beta release of Ernie Bot, built on Ernie 3.0, Baidu purports that Ernie 3.5 has achieved significant improvements in efficacy, functionality, and performance. These improvements include a two-fold increase in training throughput and a seventeen-fold increase in inference throughput compared to Ernie 3.0. If these claims hold true, they could potentially accelerate Baidu's model iteration upgrades, decrease training and usage costs, and improve user experience.

One of the unique aspects of Ernie 3.5 is its plugin feature. The 'Baidu Search' built-in plugin is said to enable Ernie Bot to generate real-time and precise information. Another feature, the ChatFile plugin, allegedly facilitates long text summaries and Q&A capabilities. Baidu has also expressed intentions to expand Ernie Bot by adding more plugins from its own repertoire as well as from third-party sources.

Baidu's announcement about Ernie 3.5's performance relative to OpenAI's ChatGPT is intriguing, and if accurate, could suggest important advancements in AI technology. However, the nature of these claims and the absence of independent verification calls for a careful and critical analysis of these developments. Future discussions in the AI community will be key in assessing the true impact of Ernie 3.5's reported advancements.