Capping labels negatively impacts business metric

I have this deep neural network model with an integer label to predict. The label is heavily skewed so we cap the labels at some value (let's say 90 %ile).

Now when we build and run the model, it performs well in general. But in online experiment shows degradation in business metrics for a fraction of users that have high value labels.

If we don't cap the label, the business metrics gets skewed for users with low number of activities.

What are my best options to deal with such issue? Adding a new feature? Multi tower learning? Any idea can be super helpful.

Thanks.