Tencent's Hy4: The model that sees your screen and takes control of your keyboard
The race for leading language models never stops for a moment, and this time Tencent is setting a new bar with the launch of the full version of Hy4 [1, 2]. If the Preview version launched last August laid the foundations, the current version turns the model into a real working tool, with computer vision and computer operation capabilities that change the rules of the game [3, 5, 10].
What's new in Hy4?
The most significant improvement in Hy4 is its ability to "see" [1, 8]. Unlike previous models that required a detailed textual description of the interface, here you can simply upload a screenshot of software, a website, or a complex graph, and the model will analyze the information and execute actions accordingly [1, 3, 10].
Beyond vision, Tencent has implemented Computer Use capabilities – the model is now capable of operating graphical user interfaces (GUI) and command-line interfaces (CLI) [1, 5, 10]. This means it doesn't just write code, but can perform long-term tasks within professional software like Blender or CAD, and manage workflows that span multiple applications simultaneously [1, 2, 10].
Why does this matter to you?
Tencent has aimed Hy4 at "real productivity" tasks [1, 7]. Here are the areas where the model shows impressive performance:
- •Software and Game Development: The model shows a significant improvement in GameDev and game logic capabilities, with deep integration for engines like Unity and Unreal Engine [2, 3, 10].
- •Office Work and Finance: The ability to work in real-time on Word, Excel, and PPT files makes it an efficient personal assistant [1, 3]. Additionally, the model has been optimized for financial analysis, with a dramatic decrease in the Hallucination rate from 15.76% in the Preview version to just 5.45% in the current version [1, 3].
- •Scientific Research: The model combines complex reasoning and scientific programming, which allows it to perform end-to-end research tasks [1, 2].
Data and Numbers
This is a massive model with 770 billion parameters [5, 7, 10]. In internal tests conducted by Tencent, the model showed significant improvements in various quality metrics, including Instruction Following capabilities and accuracy in performing complex tasks [1, 3]. It is important to note that although Tencent positions it in the first rank of open models, the company is transparent about the gaps that still exist compared to leading models in the global market, which indicates a sober and focused approach [5, 8].
Summary and Conclusions
The launch of Hy4 is a significant step for developers and professionals who are looking for AI-based automation that doesn't stop at text. The model's ability to understand visual interfaces and perform active actions on the computer is exactly the direction the industry is heading – from a passive Chatbot to an active Agent.
The model is now available through Tencent Cloud TokenHub, OpenRouter, and internal Tencent products like WorkBuddy and CodeBuddy [3, 5, 10]. If you are working on complex projects that require a combination of vision, programming, and task management, it is definitely worth examining the capabilities of Hy4.
Want to stay updated on everything new in the world of AI and Cloud? Follow me on Telegram or on WhatsApp.