About Natural Question (NQ) benchmark in NLP
I recently learned that there is a benchmark called NQ.
https://ai.google.com/research/NaturalQuestions/visualization
Unlike other QA benchmarks which relevant document is povided with query, it has to find information from millions of corpus by itself.
For example, if question is when are hops added to the brewing process? other QA benchmark also provide only 1 document about brewing. While NQ provide whole wikipedia text and model has to find most relevant document and answer.
When I tried all the example in the NQ with google search engine it gave me the answer every time. Then
- How does google search engine is managing those questions so well(Since NLP such as BERT is pretty computationally expensive I think it is not likely google is running whole model for every search)
- If google is doing it in other ways than neural network.(Or mixing it with other method) Do we need to bother with method such as REALM(https://arxiv.org/pdf/2002.08909.pdf) which seems to be computationally heavy?
Topic bert search-engine nlp
Category Data Science