Palm-e: An Embodied Multimodal Language Model

Google researchers Danny Driess and Pete Florence have developed PaLM-E, a new generalist robotics model that is capable of transferring knowledge from varied visual and language domains to a robotics system. This model has been trained on tasks such as object manipulation, navigation, and language understanding, and is the first of its kind to overcome the issue of lack of datasets on a scale comparable to large text corpora or image datasets.