Why is WP template_include overwritting all templates rather than specified page?

I have created a custom plugin for WP that has an action that includes a template via template_include as shown here:

public function __construct( $wp_custom_plugin, $version ) {

    $this-wp_custom_plugin = $wp_custom_plugin;
    $this-version = $version;

    add_action('template_include', array( $this, 'add_my_template' ));    

}

Here is the function that references the template:

public function add_my_template($template){
  if ( is_page( 'my-unique-page' ) ) {
      $template = plugin_dir_path( __FILE__ ) . 'page-my-unique-page.php' ;
      return $template;
    }
  }
}

Okay, so my expectation here is that i would only see this template on my-site.com/my-unique-page. In practice I am not seeing this restriction, but rather the template is applied site wide. Any suggestions on how I can modify to limit the display to the single page? Thanks.