You heard Gemini 4 was out, opened the Gemini app or Google AI Studio, and found nothing in the model list. I'm sure plenty of people were confused.
Google did unveil Gemini 4 on September 30, but it hasn't opened it to regular users yet.
So let's check, one by one, when you can use it, whether it really beats Astra, and how people are reacting.
✦ Three questions, three short answers
The details come right after
First, the exact name. The model that was unveiled is Gemini 4 Argon.
Google introduced it as its top model, built for software development, professional work like law and finance, and cybersecurity defense.
✦ Who can use Gemini 4 right now?
The rollout order Google described
| Now | Fairwind Program members and Google's own employees |
| Next | Paid API customers and Google AI Ultra subscribers (no date) |
| Later | Developers, businesses, and regular consumers (no date) |
Right now the model is open to only two groups, and only in a limited way. One is the Fairwind Program, made up of cybersecurity defenders Google invited, which some reports put at more than 650 organizations.
The other is Google's own employees. There was also a report that participants and internal teams get it with the cyber safeguards removed.
Why lock it down? Google cited the risk of misuse that comes with high performance.
Koray Kavukcuoglu, who leads Google DeepMind, explained that releasing capabilities at this level safely takes a phased approach. Google also says it's taking part in a voluntary pre-release access process run by the US government.
With a string of AI agents slipping out of control this year, Google seems to be playing it safe.
So when does it open up? There's no official date yet.
Google's order is this: first paid API customers and Google AI Ultra subscribers, then developers and businesses, then regular consumers.
As for timing, all it says is "as soon as possible."
For reference, on September 24 Kavukcuoglu said Gemini 4 could launch "much earlier than the end of the year."
He said the model was in post-training at the time, so it may arrive well before year-end, but that doesn't mean it opens in a few days.
The model card, the safety report, isn't out yet either. There's also speculation it may arrive under a different name at general release, but that's unconfirmed.
✦ Price per 1 million tokens
Input / output, in US dollars
| Gemini 4 intro price $2 / $10 | Gemini 4 after intro $4 / $20 | GPT-6 Astra $10 / $50 |
Pricing is public. At first it's $2 per million input tokens and $10 per million output tokens. After the intro discount ends, it rises to $4 and $20.
By comparison, GPT-6 Astra is $10 and $50, so Gemini 4 is far cheaper. GPT-6 Astra's context window (how much it can take in at once) is 1.05 million tokens, but the maximum output of a single response is 128,000 tokens.
✦ Where Argon wins and where it loses
Google's own benchmark scores (%), Argon vs. Astra
| Argon ahead Legal agent 19.6 vs. 5.4 Software development 77.9 vs. 74.1 Video understanding 91.7 vs. 87.5 | Astra ahead FrontierSWE v2 55.0 vs. 65.5 Terminal-Bench Science 57.6 vs. 68.1 |
Now the part everyone's curious about: performance. The claim that it beat Astra is only half true.
By Google's own benchmarks, Argon ranks first or ties for first in 13 of the 18 it disclosed. In the legal-work agent test it scored 19.6% against Astra's 5.4%, a big gap.
On software development it was 77.9% vs. 74.1%, and on video understanding 91.7% vs. 87.5%.
There are big losses too. On FrontierSWE v2 Argon scored 55.0% to Astra's 65.5%, and on Terminal-Bench Science 57.6% to 68.1%, trailing by 10.5 points in both.
It also fell 9 points behind Claude Opus 5.5 on Terminal-Bench 4.0. So it didn't dominate across the board.
✦ Independent scores, and what a task really costs
Artificial Analysis Intelligence Index
| Model | Score | Cost / task |
|---|---|---|
| Claude Opus 5.5 | ~57.6 | $5.98 |
| GPT-6 Astra | ~52.7 | $3.26 |
| Gemini 4 Argon | ~52.6 | $1.99* |
Don't just trust the tests Google picked, so let's check an independent evaluation too. On the Intelligence Index from Artificial Analysis, which compares AI models, Argon scores about 52.6 and Astra about 52.7.
That's basically a tie. Claude Opus 5.5 is higher at about 57.6.
Here, cost per task means the token price multiplied by the tokens a model actually used. Opus has a lower per-token price, but it uses lots of tokens at max effort, so it came out highest at $5.98.
On the other hand, cost per task is $1.99 for Argon and $3.26 for Astra, so for now Argon is far cheaper.
But that's at the discounted price. Once standard pricing kicks in, it rises to about $3.98 and could end up costing more than Astra.
An interesting one is defense against prompt attacks. In a test where hidden instructions in outside documents try to trick the AI, the attack success rate was 0.7% for Argon, 1.0% for Claude Opus 5.5 and 8.5% for Astra.
On this one metric, Google is clearly ahead.
✦ The road to Gemini 4
In chronological order
The community reaction is fun too. It started on September 17.
A model called Gemini 3.8 Flash showed up on LMArena, the AI model arena site. Despite the name, it produced polished SVG art and 3D output, and people found that strange.
On Reddit, a comparison image set the cat SVG it drew beside Astra's, and people guessed Gemini 4 Pro was being tested in disguise.
A few days later, leaked numbers claiming to be this model's scores spread. Things like 88% on a software development test and 86.8% on a computer-use test.
But the official software development score was 77.9%. The testing conditions may have differed, but that isn't confirmed.
Rumors also said it accepted 10 million input tokens, yet the official announcement doesn't state an input limit.
✦ What Bloomberg reported, and how Google replied
The October 1 report in two cards
Screen design is weak
It's less impressive in real work than on benchmarks
DeepMind's chief: Google will always be at the frontier
The biggest talking point was Bloomberg's October 1 report. It said some Google employees doubt that "Gemini 4 scores well on benchmarks but isn't that impressive in actual work."
There were complaints that its coding results are uneven and its screen design is weak, and some employees reportedly felt Anthropic's and OpenAI's models are improving faster.
Google pushed back, saying it's inaccurate to say Gemini 4 underperforms in coding, and the DeepMind chief said he's certain Google will always be at the frontier. On the day of the report, Alphabet's stock rally lost a lot of momentum.
Some background is worth knowing. Google's previous top model was Gemini 3.1 Pro, released on February 19. The 3.5 Pro it said would come in June never arrived, and only cheaper Flash models came out in between.
Arriving after eight months, it's no wonder hopes and doubts are both pouring in. Headlines have converged on "great, but you can't use it yet," and many developers say it's frustrating to wait on a scoreboard alone.
✦ Where to look for the public release
So you don't miss the news
| Google AI Ultra subscriber | Watch for an announcement in the Gemini app |
| Developer | Check the model list in AI Studio |
| Everyone else | Wait for the general release news |
So what should we do now? Benchmarks are tests chosen by the company that made the model, so waiting for hands-on results is the right call.
If you use Google AI Ultra, watch for announcements in the Gemini app; if you're a developer, check the model list in AI Studio.
By the way, the similarly named Gemma 4 is a different, open model from Gemini 4, and it already came out on April 2.
To sum up: Gemini 4 is unveiled, but there's no firm date for general release.
By Google's numbers it beats Astra in several areas, but independent tests put it level, and in some areas it falls well behind.
Real-world performance will only be clear once we can use it after the general release.
Well then, shall we begin today's adventure?
Related posts:
Meta's Muse AI Free Guide: Enter an Invite Code, Both Get 1 Billion Tokens
No comments:
Post a Comment