How to create a language translator from scratch?

I want to create a translator which can translate English, Korean and Tamil sentences into English sentence, I tried googletrans but is there any way to create something better than that using DL and NLP techniques?

Topic nlg rnn deep-learning machine-translation nlp

Category Data Science

1
answered at

Google translate itself uses Deep learning to translate sentences which can be seen here.

You can translate sentences across languages for which you need two things :

  1. A large dataset which has pairs of translations ( like English-French ). You can find such a dataset from here.
  2. A sequence-to-sequence RNN model. They have Encoder-Decoder architecture which encodes the source sentence into a thought vector and then decode it to form the translation. This image may be helpful. enter image description here The picture shows the Neural Machine Translation adopted by Google.

Also, TensorFlow and Keras help in creating such deep learning models. You can refer to this blog from the author of Keras. These models ( seq2seq ) have gained popularity in text summarization, NMT, and other sequence-to-sequence tasks.

If you wish to create a full-scale production ready language translator then you will need a GPU as these models are heavy to train as well as a very large corpus of text.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.