Google claims Gemini 2.5 Pro preview beats DeepSeek R1 and Grok 3 Beta in coding performance

Date:

Share post:

Join the event trusted by enterprise leaders for nearly two decades. VB Transform brings together the people building real enterprise AI strategy.ย Learn more


Google has released an updated preview ofโ€‹โ€‹ Gemini 2.5 Pro, its โ€œmost intelligentโ€ model, first announced in March and upgraded in May, as a preview, intending to release the same model to general availability in a couple of weeks.ย 

Enterprises can test building new applications or replace earlier versions with an updated version of the โ€œI/O editionโ€ of Gemini 2.5 Pro that, according to a blog post by Google, is more creative in its responses and outperforms other models in coding and reasoning.ย 

During its annual I/O developer conference in May, Google announced that it updated Gemini 2.5 Pro to be better than its earlier iteration, which it quietly released. Google DeepMind CEO Demis Hassabis said the I/O edition is the companyโ€™s best coding model yet.ย 

But this new preview, called Gemini 2.5 Pro Preview 06-05 Thinking, is even better than the I/O edition. The stable version Google plans to release publicly is โ€œready for enterprise-scale capabilities.โ€

The I/O edition, or gemini-2.5-pro-preview-05-06, was first made available to developers and enterprises in May through Google AI Studio and Vertex AI. Gemini 2.5 Pro Preview 06-05 Thinking can be accessed via the same platforms.ย 

Performance metrics

This new version of Gemini 2.5 Pro performs even better than the first release.ย 

Google said the new version of Gemini 2.5 Pro improved by 24 points in LMArena and by 35 points in WebDevArena, where it currently tops the leaderboard. The companyโ€™s benchmark tests showed that the model outscored competitors like OpenAIโ€™s o3, o3-mini, and o4-mini, Anthropicโ€™s Claude 4 Opus, Grok 3 Beta from xAI and DeepSeek R1.ย 

โ€œWeโ€™ve also addressed feedback from our previous 2.5 Pro releases, improving its style and structure โ€” it can be more creative with better-formatted responses,โ€ Google said in the blog post.ย 

What enterprises can expect

Googleโ€™s continuous improvement of Gemini 2.5 Pro might be confusing for many, but Google previously framed these as a response to community feedback. Pricing for the new version is $1.25 per million tokens without caching for inputs and $10 for the output price.ย 

When the very first version of Gemini 2.5 Pro launched in March, VentureBeatโ€™s Matt Marshall called it โ€œthe smartest model youโ€™re not using.โ€ Since then, Google has integrated the model into many of its new applications and services, including โ€œDeep Think,โ€ where Gemini considers multiple hypotheses before responding.ย 

The release of Gemini 2.5 Pro, and its two upgraded versions, revived Googleโ€™s place in the large language model space after competitors like DeepSeek and OpenAI diverted the industryโ€™s attention to their reasoning models.ย 

In just a few hours of announcing the updated Gemini 2.5 Pro, developers have already begun playing around with it. While many found the update to live up to Googleโ€™s promise of being faster, the jury is still out if this latest Gemini 2.5 Pro does actually perform better.ย 


Source link
spot_img

Related articles

GOLD BLADE remote DLL sideloading attack deploys RedLoader โ€“ Sophos News

Sophos analysts are investigating a new infection chain for the GOLD BLADE cybercriminal groupโ€™s custom RedLoader malware, which...

Dell Excludes Intel Then Karma Bites

Sometimes when you take the money, it backfires on you, just ask Dell. SemiAccurate has heard about the...

Off-Broadway Tips for Designing Accessible Events

What can meeting professionals learn from a scrappy off-Broadway theater known for the longest-running crime play and a...

An Holistic Framework for Shared Design Leadership โ€“ A List Apart

Picture this: Youโ€™re in a meeting room at your tech company, and two people are having what looks...