Can pre-trained transformers (I.e., BERT) handle numerical/spatial data

I’m curious to know if pre-trained transformers could handle search queries that include numerical data or make references to spatial relationships.

Take an example dataset of a list of restaurants, each with a distance relative to the city centre. Would transformers be able to handle:

”Which restaurant is closest to city centre?”

”Which restaurant is 2km from the city centre?”

Curious to know if anyone has any opinions on this or has seen any articles/examples covering this aspect of searching.

Thanks!