PRISMA-V

PRimate-inspired Integrated Spatiotemporal Memory Active Vision

01/12/2025 to 30/11/2028

Current AI systems for computer vision have achieved remarkable performance but face a critical limitation: They require enormous computational resources and energy consumption, which makes them impractical for autonomous agents and edge devices. In contrast, the primate visual system processes complex visual information with remarkable efficiency, accuracy and robustness while consuming fewer resources. Additionally, today's computer vision models are typically built from the perspective of static image/video processing rather than being designed from the ground up for embodied, active agents that move and explore their environment. This fundamental disconnect limits their applicability in real-world scenarios where agents must actively gather visual information. In contrast, the primate visual system dynamically coordinates with motor functions to interact with the environment. The goal of this proposal is to develop a neurobiologically inspired computer vision architecture that incorporates fundamental principles of primate vision to substantially improve the resource efficiency of current computer vision systems. Our proposal has three aims: (1) Build and optimize a network architecture inspired by the primate visual system. (2) Identify normative principles that govern the representations in different model components. (3) Quantitatively evaluate our model on task performance and neural + behavioral alignment. (4) Validate and finetune our model on task performance and neural + behavioral alignment. Our goal is to achieve competitive performance using a brain-inspired approach while maintaining strong representational similarity to biology, demonstrating how neuroscientific insights can advance AI while simultaneously deepening our understanding of biological vision.