Great news! OpenAI has finally released its largest XLGPT-2 model with 1.5 billion…

Great news! OpenAI has finally released its largest XLGPT-2 model with 1.5 billion parameters and 48 layers. I like that people find the outputs of GPT-2 to be very convincing. The largest model has a 'credibility score' of 6.91 out of 10. That's just a little more than the outputs from the 774M model (6.72) and significantly more than the medium model 355M (6.07). The difference between the large and super large model is therefore relatively small (goodness, what a sentence:)). This was probably the last proverbial straw that led OpenAI to release the XL model.

From a pragmatic standpoint, I should add that the 774M model could no longer be trained/tuned on free Colab GPU.

Source: https://openai.com/blog/gpt-2-1-5b-release/?fbclid=IwAR1TjKlPbUMx8Q6kJKZuGCTsB5USgNVQtsdTJa9qRE7qLozLlp-f0STLsaU
Github: https://github.com/openai/gpt-2-output-dataset
Social impact: https://d4mucfpksywv.cloudfront.net/papers/GPT_2_Report.pdf
Paper: https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

Originally published on Facebook — link to post