Will Language Models Be Free?

Will Language Models Be Free?

The goal of the EleutherAI research team is to create a clone of GPT-3 that will be available to everyone for free!

EleutherAI is a loose group of independent scientists developing GPT-Neo, an open, freely usable version of the OpenAI language model. The model could be ready as early as August, team member Connor Leahy told The Batch.

How it works: The aim is to match the speed and performance of the full version of GPT-3 with 175 billion parameters, with particular attention to eliminating social biases. The team has successfully completed a version with 1 billion parameters, and architectural experiments are currently underway. • CoreWeave is a cloud computing provider, offering the project free access to infrastructure. Eventually, they plan to host instances for paying customers. • The training corpus contains 825 GB of text. In addition to established text datasets, it includes IRC chat logs, YouTube subtitles, and summaries from the PubMed medical research archive. • The team has experimented with word pairing and assessed data on gender, religion, and racial bias using sentiment analysis. Examples exhibiting unacceptably high levels of bias have been removed.

Sources: https://www.eleuther.ai/projects/gpt-neo/?utm_campaign=The%20Batch&utm_medium=email&_hsmi=109400193&_hsenc=p2ANqtz-97S8QHC4I_gD5FOS4x1Kbemxu1-QOgYDK7ISGHv5FI0DCD8PUTp7MassDJNXr1aO6dHfbkD3TRsccIpLzA-5BMG0ZkAw&utm_content=109401005&utm_source=hs_email

https://pile.eleuther.ai/paper.pdf?utm_campaign=The%20Batch&utm_medium=email&_hsmi=109400193&_hsenc=p2ANqtz-8GfD5dSxg11QbzBCl0w-ZoJo35KrnM-mJNriVL8xDJVeaZWWK5d0XxNuIzaKuReGaqTJN9rzDFEYNuA66cntMoIbycqA&utm_content=109401005&utm_source=hs_email

Originally published on Facebook — link to post