Should non-stationary time series be differenced when fit through neural networks?

I am fitting a recurrent neural network (RNN) on some non-stationary time series data. I know that, in the case of linear models, it is common practice to difference the series in order to make them stationary and remove trends.

However, is this also the case when fitting neural networks? I have found several tutorials and examples where stationarity is not even checked and others which recommend first differencing if the data is non-stationary. What is the best way to go, theory-wise?

Moreover, do we need to standardize/scaling a differenced dataset prior to fitting neural networks? Differencing is already a data transformation which reduces the range of values, therefore I am not quite sure whether scaling further is appropriate.

I have done a few experiments and it seems like the routine differencing -- scaling -- invert scaling -- invert differencing yields very poor predictions compared to using unscaled data.