Researchers From Uc Berkeley And Google Introduce An Ai Framework That Formulates Visual Question Answering As Modular Code Generation

CodeVQA is a recently proposed approach for Visual Question Answering (VQA) that formulates VQA as a program synthesis problem and utilizes code-writing language models which take questions as input and generate code as output. This framework’s main goal is to create Python programs that can call pre-trained visual models and combine their outputs to provide answers. CodeVQA uses primitive visual APIs wrapped around Visual Language Models to extract specific visual information from the image. This approach has the potential to improve the accuracy of VQA systems and reduce the need for large datasets.