Launch Media Library for specific MU site

I have a form on my front end that I'd like to call a media library modal, but I want the library accessed to be a specific MU subsite.

I trued to switch_to_blog around the wp_enqueue_media function, but it is not pulling the media from the targeted site.

Is there a way to achieve this? I don't see a setting the jquery call to initiate the modal but maybe I missed it?

switch_to_blog( 2 );
  wp_enqueue_media();
restore_current_blog();