Train a model when input can contain a smaller options output with the correct output

Question

Train a model when input can contain a smaller options output with the correct output

AJ AJ

2022年3月19日 18:16

I have service order lines to charge customers, each line needs to be set to an actual product.

If the customer had only one product, so all lines are set to that product.

But, if the there are many products, currently a trained employee does the matching, each line to one product.

e.g.

enforcement fees ----------- backup YYY transmitter

text message fees ---------- main XXX transmitter installation

installation fee ----------- main XXX transmitter installation

I can train all the kinds of lines against all products.

To improve prediction I will add to the input the products the customer has.

In traditional ML my soft max will have all the hundreds products and I'll need to take the probability of my customers product and adjust their prediction.

e.g.

input: [ enforcement fees, backup YYY transmitter, main XXX transmitter installation]

output: [..,..,.., backup YYY transmitter 0.02, main XXX transmitter installation 0.01,..]

Then my adjustment will be [0.67,0.33].

Is there a better way to mark inputs as the optional output, or eliminate all other predicts during training?

Thank you!

Topic softmax neural-network

Category Data Science

AJ AJ · Accepted Answer · 2022年3月19日 18:16

So at last I did a multicategory DNN. I added all options of predictions as inputs with prefix of 'xy', and the optional answers with higher values.

A few snippets of code:

# optionial correct answers are brought seperated by ','

d_dummy = data["stuffedX"].str.get_dummies(sep=',').add_prefix('xy_') c = pd.concat([data, d_dummy], axis=1) df = data[data.columns[~data.columns.isin([...original_columns...])]] df2 = data[data.columns[data.columns.isin([...original_columns...])]]

a , b = df.iloc[:,1:], df['xProductNumber'] rename_dict = dict(zip(a.columns,a.columns.str.split("_").str[1])) a = a.rename(columns=rename_dict) #renamed columns of a as per b

#Now compare tp create a mask and assign 11

m = pd.DataFrame([a.columns]*len(a),columns=a.columns) == b.to_numpy()[:,None] df = df.assign(**a.mask(m,11).set_axis([*rename_dict.keys()],axis=1))

Train a model when input can contain a smaller options output with the correct output

About