How to implement predict_proba() with spark.ml logistic regression
In sklearn
it is possible to use the predict_proba()
function to obtain the probability matrix.
Since in spark.ml
that function is not implemented, how can I implement it?
The model.transform
(dataset) function indeed generates a field called probability. Apparently, it is not the same probability matrix since it is different from the one produced by sklearn
. Thanks for your help.
Topic multitask-learning pyspark logistic-regression
Category Data Science