How to access GPT-3, BERT or alike?

I am interested in accessing NLP models mentioned in scientific papers, to replicate some results and experiment.

But I only see waiting lists https://openai.com/blog/openai-api/ and licenses granted in large commercial deals https://www.theverge.com/2020/9/22/21451283/microsoft-openai-gpt-3-exclusive-license-ai-language-research .

How can a researcher not affiliated to a university or (large) tech company obtain access so to replicate experiments of scientific papers ?

Which alternatives would you suggest to leverage on pre-trained data sets ?

Topic pretraining openai-gpt nlp

Category Data Science

1
answered at

OpenAI has not released the weights of GPT-3, so you have to access it through their API. However, all other popular models have been released and are easily accessible. This includes GPT-2, BERT, RoBERTa, Electra, etc.

The easiest way to access them all in a unified way is by means of the Transformers Python library by Huggingface. This library supports using the models from either Tensorflow or Pytorch, so it is very flexible.

The library has a repository with all the mentioned models, which are downloaded automatically the first time you use them from the code. This repository is called the "model hub", and you can browse its contents here.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.