Scientists have created a language neural network that approaches human quality. It is said to be so good that its authors are afraid to publish it!

Experts from OpenAI have proudly announced another technological success – the language model GPT-2. In the article's headline, they claim that its authors are afraid to publish it. Instead of releasing the full version, the scientists at OpenAI have only made a significantly trimmed-down version of the model available on GitHub, which is intended to demonstrate its basic functionality. The reason? The model reportedly achieves excellent results by today's standards and easily outperforms other models. Consequently, the authors were concerned about potential misuse.

I was surprised to find that a person can simply write a headline and GPT-2 will generate an entire article on the given topic! The original article includes a delightful example of a generated piece about how scientists discovered unicorns. 🙂

This model is also said to achieve very good results in question-answering tasks.

The decision not to publish the model has caused quite a stir among experts. On one hand, they argue that there has been no significant algorithmic breakthrough, merely an increase in data processing. It can be easily calculated that if you're in a hurry, you could compute a similar model at home using 80 graphics cards in about a month, costing you around 1 million crowns. Alternatively, if you want to "save money," it would take about 10 months on 8 graphics cards, costing approximately 450,000 CZK (AWS). 🙂

Do you think the threat of generating fake news using this algorithm is real, and did the authors do the right thing by only publishing the smallest model?

********** sources and links **********

Czech article (including the author's not-so-successful attempts with his own model): https://www.zive.cz/…/vedci-vytv…/sc-3-a-197236/default.aspx

Original article: https://blog.openai.com/better-language-models/

Github: https://github.com/openai/gpt-2

Discussion on the non-publication of larger models: https://www.fast.ai/2019/02/15/openai-gp2/

SQuAD: http://blog.alphai.cz/stanford-question-answering-dataset-…/
Competition from Google BERT: http://blog.alphai.cz/spolecnost-google-tento-tyden-zverej…/