NLP - Retrieval-based model

My goal is to predict the most appropriate answer from an utterance, in a group of 21 potential answers. (I'm not sure the "question" is called utterance though. ) Example: Utterance: How are you today? Answers: Answer1, 2, ..., 21. I have a training file with this format: Utterance: Answers: Good answer, wrong answer1, wrong answer2,..., wrong answer20. My problem For the first time, we have to make a prediction from a group of possible answers, and, thus, this is …
Topic: chatbot nlp
Category: Data Science

NLP based age calculation

I am working on a chatbot with open domain questions using google API & Wikipedia API. Its working few cases like who, what, when kind of questions. when I ask for Age of someone, its collecting only content about that person. As a chatbot, it should respond with the number(age). is there any model or I need to go with rule based approach.? Give your valuable suggestions.
Category: Data Science

Stuck on building a customer support chatbot from scratch using reddit dataset

I've a trained model who can mimic day to day conversation occurring on reddit. But, here my problem is that I want it to reply to a specific use cases based on the vocabulary it had learned. Summary:- I am building a college chatbot, and there are many use cases. I've trained a model on reddit dataset, and now I've a model who can mimic reddit conversation. But, I want it to map it's vocabulary to one of my use …
Category: Data Science

Where to start with ChatBots?

I want to start my journey into ChatBots and how I can create them. I have read some articles regarding the type of chatbots. Basically there are 2 types of chatbots, one is a rule based and the other is a NLP/ML based. I am more interested in the latter and want some kind of started guide. I have read that these kind of chatbots usually use some kind of attention models (as they give state of the art results) …
Category: Data Science

chat bot nfL6.json training data

I am experimenting with chatterbot and the nfL6.json training data set. The data looks like this below, should I be using an nbestanswers or answer to train the chat bot? { "main_category": "Food & Drink", "question": "For those of you who have been to Mexico, how amazing is the Joya?", "nbestanswers": [ "I lived for 12 years in Mexico. Do not know a town by Joya.. Looked in my directory for city codes and did not find it either. Only …
Category: Data Science

Chatbot NLP model for inference on 11 GB GPU?

I’ve just found the amazing Huggingface library. It is an awesome piece of work. I would like to train a chatbot on some existing dataset or several datasets (e.g. the Pile). For training (or fine-tuning) the model I have no GPU memory limitations (48 GB GPU is available). For inference, I only have a GPU with 11 GB available. Inference should be feasible in real-time (i.e. below around 3 seconds) and the model should be adjustable, i.e. the source code …
Category: Data Science

How to frame queries and answers from customer Agent utterances using Deep Learning SOTA

I am working with smart-reply use case for Async chat customer and agent chat transcripts. The one chat bubble looks like this: From one chat bubble how can I identify intent queries/Question by Customers and Responses from Agents. In between chat there is lot of unnecessary conversation, so how can I remove such things. Final Data-set: cust_query Agent_response Pending Payment You've reached PayPal messaging! Hi, Person This is Person from Google and I'm happy to assist you today. I see …
Category: Data Science

NLP behind Google Assistant, Amazon Echo and Apple Siri

How do these assisant based models analyse text and convert them to commands, I mean how do they understand the intent, property and value. I just want to know what are the models used and also I am trying to make a assistant bot, does anyone know a better model that can identify the command from the text given?
Category: Data Science

Automating 3D Modeling using NLP

I would like to know if there is any way we can automate 3D modeling processes. Like if I give the model a text input such as "create a sphere and give it a red color" and the we need to get the model. To be precise, I would like to create a bot that can perform actions in a software such as blender, like I tell the bot what I would like to do and then it does it. …
Category: Data Science

Measuring precision and recall

Trying to improve my chat App: Using previous (pre-processed) chat interactions from my domain, I have built a tool that offers the user 5 possible utterances to a given chat context, for example: Raw: "Hi John." Context: hi [[USER_NAME]] Utterances : [Hi ,Hello , How are you, Hi there, Hello again] Of Course the results are not always relevant, for example: Raw: "Hi John. How are you? I am fine, are you in the office?" Context: hi [[USER_NAME]] how are …
Category: Data Science

How to take the keywords from the given dataset to train GPT-2 based chatbot?

I am working with a dataset that contains Questions on various Events conducted by a college and the corresponding answers for the queries. I am using this dataset to train a GPT-2 355M model to create a chatbot where users can get their queries answered. But i am not getting good results and i feel that's because the questions in the dataset are in the " -Query " format. For example, Ques: "Cicada3302 - Do I need to have any …
Category: Data Science

How to build a symptom checker and medical diagnose chat bot

I want to build a chatbot which can diagnose an illness depending upon the symptoms which are given to it. I want to show you an example scenario of how it should work; Application - A , User - U A) Hello, What is your symptom? U) Headache A) How long do you have this symptom? ( System will list possible choices to the user) U) 3 Months A) Do you also have photo-affinity? ( System will provide simple "Yes-No-I …
Category: Data Science

Can AI (NLP) convert user questions (text) into database SQL queries?

I have been reading about NLP but got confused and not able to figure out - if it is feasible for NLP to convert questions in natural language to transform into SQL queries (so that it can execute on concerned database and fetch the output). Ex. If the user raises a question to the AI engine (application) - How many new customers have registered on my website this month? The AI should parse this using NLP techniques and convert it …
Category: Data Science

Autonomous learning - chatbots

My chatbots need to be trained when we get new data or feedbacks from users. Can someone provides ways how these chatbots can learn on themselves and become intelligent day by day? Some of techniques may be : Reinforcement learning - though I could not find a successful implementation and have asked a separate question. Any successful implementation reference would be helpful. Building a pipeline with a user feedback - this has 2 drawbacks. One user or admin has to …
Category: Data Science

chatbot encoder/decoder: why do we need to use chatbot answer as the decoder inputs?

I am looking into the chatbot tutorial at: https://medium.com/predict/creating-a-chatbot-from-scratch-using-keras-and-tensorflow-59e8fc76be79 It uses sequence to sequence model with encoder/decoder to solve the problem: encoder_inputs = tf.keras.layers.Input(shape=( None , )) encoder_embedding = tf.keras.layers.Embedding( VOCAB_SIZE, 200 , mask_zero=True ) (encoder_inputs) encoder_outputs , state_h , state_c = tf.keras.layers.LSTM( 200 , return_state=True )( encoder_embedding ) encoder_states = [ state_h , state_c ] decoder_inputs = tf.keras.layers.Input(shape=( None , )) decoder_embedding = tf.keras.layers.Embedding( VOCAB_SIZE, 200 , mask_zero=True) (decoder_inputs) decoder_lstm = tf.keras.layers.LSTM( 200 , return_state=True , return_sequences=True ) decoder_outputs …
Category: Data Science

Detect named entities inside words

Some languages have word endings with their nouns (like Finnish, e.g. "in Berlin" -> "Berliinissä"). I have tried to annotate the characters in the training data as entities, but then I run the model, it doesn't detect the characters inside the word. When those characters are a separate word, only then they're detected. I am unable to think of an implementation to effectively detect named entities within a word. Any suggestions would be helpful.
Category: Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.