Choosing a right algorithm for template-based text generation

I am doing a text generation project -- the task is to basically represent the statistical data in a readable way.

The way I decided to go about this is template-based: each data type has a template for how sentence should be formed and what synonyms can be used.

I'm torn about whether some kind of ML techinque can bolster this template-based approach. Text should be unique -- so I need an algorithm that optimises for uniqueness.

Now, there are API solutions that can give me uniqueness score at the end of the text (and even in the middle of it) -- so my first instinct was to try out reinforcement learning with sparse rewards. Templates can be represented as a tree, which the algorithm traverses, getting rewards in the end and during its trip. Input is its current options on where to go and the output is its decision on where to go.

The problem with this approach is that after it succeeded in generating an unique text, it can't generate the same one (I mean it can, but the score would be 0), which might prove difficult for the model to learn. Also many articles on the web indicate that RL is really, really hard to tune properly.

I am now in the pre-research process, so any feedback on how I should approach this task is appreciated. Maybe there is no need for ML at all?

What do you think? My instinct tells me that such problems should have established solutions and I'm just searching the wrong way.

Thanks!