How can i adapt accuracy metric for multiclass classification?

I have a problem which is multiclass e.g. That is 4 classes. I would like a custom metric to assess the model where only if class 3 is predicted as class 2 and class 2 is predicted as class 3 (i.e. those in the middle) then it is penalized less.

How can i do this by adapting the sklearn accuracy_score metric of similar?

e.g. comparing:

predicted_labels = [1,3,0,0,2..]
actual = [0,0,2,1,3,3...]