NLP - support comments analysis

I am new to NLP and looking for some direction since after all my reading I haven't found a definite approach and the subject matter is vast. The project is to focus on specific fields of support comments using NLP and Python. The goal is that from the comments I want to verify that the comment is in fact a well made comment for that field. Some requirements is the context of entered text is relevant to the field it has been entered in, it is informative based on the specific field, it is not just a few unhelpful words entered, it can detect that words/sentences have similar semantic meaning (e.g problem is, issue encountered etc).

I first looked into TF-IDF but it doesn't consider context. I have been reading into other deep learning models such as BERT as it includes context. I have read though that BERT was designed more for sentence prediction and missing words rather than for comparing semantic meaning of multiple sentences. USE would be better for comparing if sentences are similar although all the comments will have a general theme and I doubt they will be similar enough to design a system around. Topic modelling sounded like what i'm looking for but I think it's more for general topics you find in newspapers and books rather than comment sections. For the text summarisation option maybe there is not enough text in each comment to give a helpful summary. I think I would prefer unlabelled data if possible so as to minimise manual input but if it's neccessary then we could evaluate it. Is there something in fine tuning a pretrained model such as BERT to analyse all the existing comments for a specific field and somehow extract from that if a new comment is informative and relevant by comparing it to the existing ones?

Example field would be Problem description. I am looking for direction on how to classify the content of that field as relevant and informative.

Any direction is appreciated or if you know of existing related material, papers, videos or tutorials.

Thanks