Popular posts with Varnish ESI

I am trying to get Varnish ESI to work with displaying the most viewed posts on a new blog I am working on. Here is my current script;

In theme file:

esi:remove
?php get_template_part( 'partials/homepage/most-popular-loop' ); ?
/esi:remove
!--esi esi:include src="/lib/plugins/esihandler.php"/ --

To call the popular posts:

function wpb_set_post_views($postID) {
    $count_key = 'wpb_post_views_count';
    $count = get_post_meta($postID, $count_key, true);
    if($count==''){
        $count = 0;
        delete_post_meta($postID, $count_key);
        add_post_meta($postID, $count_key, '0');
    }else{
        $count++;
        update_post_meta($postID, $count_key, $count);
    }
}
//To keep the count accurate, lets get rid of prefetching
remove_action( 'wp_head', 'adjacent_posts_rel_link_wp_head', 10, 0);

function wpb_track_post_views ($post_id) {
    if ( !is_single() ) return;
    if ( empty ( $post_id) ) {
        global $post;
        $post_id = $post-ID;    
    }
    wpb_set_post_views($post_id);
}
add_action( 'wp_head', 'wpb_track_post_views');

function wpb_get_post_views($postID){
    $count_key = 'wpb_post_views_count';
    $count = get_post_meta($postID, $count_key, true);
    if($count==''){
        delete_post_meta($postID, $count_key);
        add_post_meta($postID, $count_key, '0');
        return "0 View";
    }
    return $count.' Views';
}

I have got ESI to work properly by not caching the loop displaying the popular posts (tested by displaying a timestamp inside the loop and outside). However, I don't fully understand whether it will work properly if the actual single post pages are fully cached with Varnish. I guess it's pulling the post view count from the database and displaying it somewhere. But if the single post page is cached in Varnish, wouldn't the count stay the same?

I wonder if someone is able to help me out a little here. It seems to work properly if I pass Varnish for this particular site so no caching is active. Just I'm not sure if it's going to work with the full vcl running on the site.

Thanks in advance, not sure if this should really go into the WP stack exchange or more server side so please let me know.