JS / jQuery in Elementor pages vs JS file

I have a lot of JavaScript and jQuery functions that are all working fine, but it's quite fragmented and hard to maintain as some are in Elementor HTML widgets, others in specific Pho functions annd some in an enqueued JS file.

Ideally I'd like to move everything into the single JS file so it's easier to maintain, but would there be any performance implications? For example some scripts only need to run on specific pages but I assume they would load everywhere in a file. Would this have any realistic performance implications or am I being overly obsessive...?