Multimodal Neurons in Artificial Neural Networks

We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually.

Better Language Models and Their Implications

We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization.