Meta is giving away some of the family jewels: That’s the gist of an announcement from the company formerly known as Facebook this week. In a blog post on the Meta AI site, the company’s researchers announced that they’ve created a massive and powerful language AI system and are making it available free to all researchers in the artificial-intelligence community. Meta describes the move as an effort to democratize access to a powerful kind of AI—but some argue that not very many researchers will actually benefit from this largesse. And even as these models become more accessible to researchers, many questions remain about the path to commercial use.

Large language models are one of the hottest things in AI right now. Models like OpenAI’s GPT-3 can generate remarkably fluid and coherent text in just about any format or style: They can write convincing news articles, legal summaries, poems, and advertising copy, or hold up their end of conversation as customer-service chatbots or video-game characters. GPT-3, which broke the mold with its 175 billion parameters, is available to academic and commercial entities only via OpenAI’s application and vetting process.

Meta’s Open Pretrained Transformer (known as OPT-175B) matches GPT-3 with 175 billion parameters of its own. Meta is offering the research community not only the model itself, but also its codebase and extensive notes and logbooks about the training process. The model was trained on 800 gigabytes of data from five publicly available data sets, which are described in the “data card” that accompanies a technical paper posted by the Meta researchers to the ArXiv online preprint server. Read More