Guest
In this week’s SlatorPod, we are joined by Graham Neubig, Associate Professor of Computer Science at Carnegie Mellon University, to discuss his research on multilingual natural language processing (NLP) and machine translation (MT). Graham discusses the research at Neulab, where they focus on various areas of NLP, including incorporating broad knowledge bases into NLP models and code generation. Graham expands on his Zeno GPT-MT Report comparing large language models (LLMs) with special-purpose machine translation models like Google Translate, Microsoft Translate, and DeepL. He revealed that GPT-4 was competitive from English to other languages, but struggled with very long sentences. When it comes to cost comparison, Graham highlights that GPT-3.5 Turbo (the model behind the free version of ChatGPT) is significantly cheaper than Google Translate and Microsoft Translator, but GPT-4 (available via OpenAI’s subscription) is more expensive. Graham predicts that companies will likely move towards using general-purpose LLMs and fine-tuning them for specific tasks like translation. The discussion also covers the recent flurry of speech-to-speech machine translation system releases. Graham talks about his startup, Inspired Cognition, which aims to provide tools for building and improving AI systems, particularly in text and code generation. Graham concludes the pod with advice for new graduates in the NLP field and his plans for Zeno and the Zeno report. Zeno Report: Chapter Markers: 00:00:00 Intro and Agenda 00:01:05 Professional Background and Interest in Language 00:03:56 Research at NeuLab 00:05:05 Impact of ChatGPT on NLP 00:07:20 Context in Machine Translation and LLMs 00:12:43 How GPT Handles Machine Translation 00:19:13 GPT Cost Comparison for Machine Translation 00:23:07 How LLMs Will Evolve 00:24:57 Why so Many Speech Translation Releases? 00:29:45 LLMs and Low-Resource Languages 00:32:17 Launching Inspired Cognition 00:35:29 Advice to Graduate Students 00:37:15 Plans for 2023 and Beyond
