Will Language Models Be Free?
The goal of the EleutherAI research team is to create a clone of GPT-3 that will be available to everyone for free! EleutherAI is a loose group of independent scientists developing GPT-Neo,…
The goal of the EleutherAI research team is to create a clone of GPT-3 that will be available to everyone for free!
EleutherAI is a loose group of independent scientists developing GPT-Neo, an open, freely usable version of OpenAI's language model. The model could be ready as early as August, team member Connor Leahy told The Batch.
How it works: The aim is to match the speed and performance of the full version of GPT-3, which has 175 billion parameters, with a particular focus on eliminating social biases. The team has successfully completed a version with 1 billion parameters and is currently conducting architectural experiments.
• CoreWeave is a cloud computing provider that is giving the project free access to infrastructure. Ultimately, it plans to host instances for paying customers.
• The training corpus contains 825 GB of text. In addition to established text datasets, it includes IRC chat logs, YouTube subtitles, and summaries from the PubMed medical research archive.
• The team has experimented with word pairing and assessed data on gender, religion, and racial bias through sentiment analysis. Examples exhibiting unacceptably high levels of bias have been removed.
Původní zdroj: wordpress