Researchers at the Institute of Digital Games - University of Malta, Nemanja Rasajski and Chintan Trivedi, won the NVIDIA Best Paper Award at the 2024 European Conference on Computer Vision where their innovative approach addressed the one of the big challenges of AI - namely having AI models understand gameplay across multiple games.
They presented BehAVE, an AI tool developed together with Prof. Antonios Liapis, Prof. Georgios N. Yannakakis (Institute of Digital Games), and Dr Kostas Makantasis (Faculty of ICT), that can describe in text format what is happening on the screen based on video output only. And while state-of-the-art video encoders were able to do this, BeHAVE is groundbreaking because it can transfer what it learns from one game to apply it to another, even across genres. It is able to do this regardless of different visual styles and without access to the game engine.
BehAVE opens up new possibilities for applications such as behaviour recognition, AI-driven gameplay analysis, and video understanding because it allows for data collection without requiring access to a video game studio’s game engine which is often highly restricted. Not only does it offer a practical solution to the data collection challenge in video games, it also steps into the domain of general artificial intelligence.
“Arguably one of the main goals of artificial intelligence (AI) research is to produce machines that can solve multiple problems, not just one. In many cases AI does better than humans in specific tasks, such as playing chess or recognising images, but it lacks generalizability . For example, a chess-playing AI would fail miserably at another game, so developing an AI that is able to work across different games shows significant progress on existing technology,” explains Prof. Yannakakis.
Tested on 25 first-person shooter games such as Fortnite, PUBG, and Valorant, the framework demonstrated remarkable zero-shot transfer to unseen games, even when trained on an entirely different game like Minecraft genres.
The team presented their findings for the first time at the European Conference on Computer Vision (ECCV), the biennial premier research conference in Computer Vision and Machine Learning, managed by the European Computer Vision Association (ECVA). The prize for the Best Paper Award was a NVIDIA worth €2000. The research was supported by Project OPtiMaL and funded by Xjenza Malta through the SINO-MALTA Fund 2022.
The team is available for interviews and further explanations and demonstrations. Please contact Prof. Georgios Yannakakis via email for further information.
Read the full award-winning paper: “BehAVE – Behaviour Alignment of Video Game Encodings.”
Learn more about the BehAVE and the OptiMaL project online.
About the Institute of Digital Games:
The Institute of Digital Games is a world-class research and education centre at the University of Malta, focusing on game design, game analysis, and game AI. The Institute is ranked in the top 10 institutions active in technical games research and are regarded as some of the leading figures within their respective international research fields with over 22,000 citations and the h-index of 170 collectively across all staff members. The Institute does research with global industry leaders such as Ubisoft and runs multi-million euro EU projects with IBM and Zaha Hadid Architects.
