Google Shakes Up the Market: The Revolution That Will Cut Your AI Costs by 80%
Google Cloud's new Model as a Service (MaaS) approach changes the rules of the game. Here is what you need to know about the new Gemini models, the support for open models, and practical ways to save on inference costs.
Not long ago, running large language models (LLMs) was reserved for companies with huge budgets or startups that had raised millions. The cost of "inference" (actually running the model) was the main barrier to moving from an impressive pilot to a product that runs at scale. Google Cloud's latest announcement about the new AI services in Vertex AI marks the end of the expensive era and the beginning of real democratization in this field.
Gemini 1.5 Flash: Fast, Smart, and Significantly Cheaper
The biggest news is the Gemini 1.5 Flash model. It is a model that has been optimized (Distillation) from the more powerful Pro model, to deliver high speed and low cost without sacrificing the quality of results on common tasks.
Google announced a drastic price reduction of about 80% for this model, which makes it one of the most cost-effective models on the market. As of today, the inference cost of Gemini 1.5 Flash is significantly lower than competitors like GPT-4o mini, which makes it possible to run text-heavy apps, long-document analysis, and data classification at costs of just a few cents per million tokens.
Vertex AI Model Garden: Full Freedom of Choice
Google understands that Gemini is not the only solution. Through the Model Garden inside the Vertex AI platform, users can now also run open models such as Meta's Llama 3, Mistral, and even Google's own open models, Gemma.
The advantage here is twofold:
- •Privacy and security: The model runs inside your VPC in Google Cloud, so the data never leaves the secure environment.
- •Infrastructure optimization: You can choose the type of GPU (like NVIDIA L4 or H100) that the open model will run on, and balance performance against price exactly according to your needs.
The Tools That Will Actually Save You Money
Beyond the cheaper models, Google introduced technological mechanisms that allow smarter financial management:
- •Provisioned Throughput: For organizations that need stability, Google now lets you "reserve" capacity in advance at a fixed price. This solves the Rate Limits problem and ensures that costs are fully predictable at the end of the month.
- •Batch API: If your task doesn't have to run in real time (for example, analyzing thousands of emails overnight), you can use the Batch API and get a discount of up to 50% compared to a regular call to the model.
- •Context Caching: A technological innovation that lets you "keep in memory" long contexts (like a whole book or a huge source code base). If you ask the model repeated questions about the same long text, you won't have to pay again and again for reading the original text.
Summary and Conclusions
Google Cloud's move is a clear signal to anyone involved in development: the focus is shifting from pure technical capability to economic efficiency. The combination of fast models like Gemini Flash with the flexibility of the Model Garden lets every developer, even without deep infrastructure knowledge, embed artificial intelligence in their products.
My recommendation? Don't settle for one model. Use Vertex AI to run A/B Testing between Gemini and open models, and use the Batch API for every task that doesn't require an immediate answer. The cumulative savings can reach tens of thousands of dollars a year.
Relevant links for further reading: