how to avoid timeouts with remote API requests?

I'm developing a plugin, WP SoundSystem, that makes Wordpress rest requests to my remote API api.spiff-radio.org.

That API is scraping content on the web.
Which means that it might be slow to get the data. (EDIT:that data is just plain text tracklists)

So

1. WP SoundSystem plugin requests data on api.spiff-radio.org...
2. ...api.spiff-radio.org scrapes the web...
3. ...api.spiff-radio.org returns a response...
4. ...WP SoundSystem plugin gets the response

WHICH might be quite slow. I already set a 20 seconds timeout (default is 5) when querying the API with wp_remote_request. But still, I often get a timeout error.

Since I don't know if the API request will require 5 or 150 seconds, is there a way to disable timeouts for this particular request ? Or should I just set a very large timeout ?

How would you handle this ?

Thanks.