Putting content into header.php without using wp_head

@bryan-willis has a great answer that allows you to individually remove things from wp_head. This can be a better approach in most cases, but if you don't really need all the extra stuff that come with wp_head(), you may decide to go without it.

However, if you remove wp_head entirely from your header or template, you won't be able to use hooks like wp_enqueue_scripts, which makes things easy when you want to load your own CSS and JS files in your template. That's why if you keep wp_head in your template but remove everything from it except the ones you need, your template will output a clean head while allowing you to benefit from wp_head().

This way, you won't have to identify and remove every specific thing that WordPress or other themes or plugins load on your template.

Remove all functions from wp_head hook, and maybe from wp_footer hook except the ones you need.

function unhook_wp_head_footer() {

    if ( /* some condition */ ) {

        global $wp_filter; // it contains all hooks

        foreach ( $wp_filter['wp_head']->callbacks as $priority => $wp_head_hooks ) {
            if ( is_array( $wp_head_hooks ) ){
                foreach ( $wp_head_hooks as $idx => $wp_head_hook ) {
                    if ( $wp_head_hook['function'] !== 'wp_enqueue_scripts' // keep functionality of wp_enqueue_scripts
                    && $wp_head_hook['function'] !== 'wp_print_head_scripts' // to allow wp_enqueue_scripts load js inside the head element
                    && $wp_head_hook['function'] !== 'wp_print_styles' ) { // to allow wp_enqueue_scripts load css inside the head element
                        remove_action( 'wp_head', $wp_head_hook['function'], $priority );
                    }
                }
            }
        }

        foreach ( $wp_filter['wp_footer']->callbacks as $priority => $wp_footer_hooks ) {
            if ( is_array( $wp_footer_hooks ) ) {
                foreach ( $wp_footer_hooks as $idx => $wp_footer_hook ) {
                    if ( $wp_footer_hook['function'] !== 'wp_print_footer_scripts' ) { // to allow wp_enqueue_scripts load scripts in the footer
                        remove_action( 'wp_footer', $wp_footer_hook['function'], $priority );
                    }
                }
            }
        }
    }
}
add_action( 'wp', 'unhook_wp_head_footer' );

In case of keeping wp_enqueue_scripts, dequeue all CSS and JS that you don't need.

// dequeue all assets if not from your plugin
public function dequeue_assets() {

    if ( /* some condition */ ) {

        global $wp_styles, $wp_scripts;

        $prefix = 'plugin-handle'; // your plugin name used as prefix in $handle of wp_enqueue_script or wp_enqueue_style function calls

        foreach ( $wp_styles->queue as $handle ) {
            if ( strpos( $handle, $prefix ) !== 0 ) {
                wp_deregister_style( $handle );
                wp_dequeue_style( $handle );
            }
        }

        foreach ( $wp_scripts->queue as $handle ) {
            if ( strpos( $handle, $prefix ) !== 0 ) {
                wp_deregister_script( $handle );
                wp_dequeue_script( $handle );
            }
        }
    }
}
add_action( 'wp_enqueue_scripts', 'dequeue_assets', 9999 );

Removing wp_head will have some negative effects as it won't allow you to use virtually 90% of the plugins on wordpress.org. Also some of that code you are seeing only shows up when you are logged in (unless your theme is specifically adding it). For example, the dashicons, open sans, and admin-bar stylesheets and inline styles at the bottom are all there so you can use the admin bar on the frontend.

Here are some options.

1. Custom header - This is a safe alternative if you don't want wp_head in some of your pages, but still want to be able to use it and other wordpress plugins on other templates.

1. Create a file header-custom.php in your theme folder that doesn't include wp_head
2. In your custom page template call that header using <?php get_header( 'custom' ); ?>
3. If you wan't to be able to hook into this still with through your plugin you can use a custom action instead of wp_head. Basically all you have to do is add <?php do_action('custom_head'); ?> and then in your plugin hook into that the same way you would wp_head.

Example

function my_plugin_custom_head_action() {
    // do stuff here
}
add_action('custom_head', 'my_plugin_custom_head_action');

There's also a plugin that makes this process of using custom hooks/actions very easy.

2. Remove stuff you don't want from wp_head

// Remove emojis
function disable_emojis_wp_head() {
    remove_action( 'wp_head', 'print_emoji_detection_script', 7 );
    remove_action( 'admin_print_scripts', 'print_emoji_detection_script' );
    remove_action( 'wp_print_styles', 'print_emoji_styles' );
    remove_action( 'admin_print_styles', 'print_emoji_styles' );    
    remove_filter( 'the_content_feed', 'wp_staticize_emoji' );
    remove_filter( 'comment_text_rss', 'wp_staticize_emoji' );  
    remove_filter( 'wp_mail', 'wp_staticize_emoji_for_email' );
    add_filter( 'tiny_mce_plugins', 'disable_emojis_wp_tinymce' );
}
add_action( 'init', 'disable_emojis_wp_head' );

function disable_emojis_wp_tinymce( $plugins ) {
    if ( is_array( $plugins ) ) {
        return array_diff( $plugins, array( 'wpemoji' ) );
    } else {
        return array();
    }
}

// Remove other crap in your example
add_action( 'get_header', function() {
    remove_action('wp_head', 'rsd_link'); // Really Simple Discovery service endpoint, EditURI link
    remove_action('wp_head', 'wp_generator'); // XHTML generator that is generated on the wp_head hook, WP version
    remove_action('wp_head', 'feed_links', 2); // Display the links to the general feeds: Post and Comment Feed
    remove_action('wp_head', 'index_rel_link'); // index link
    remove_action('wp_head', 'wlwmanifest_link'); // Display the link to the Windows Live Writer manifest file.
    remove_action('wp_head', 'feed_links_extra', 3); // Display the links to the extra feeds such as category feeds
    remove_action('wp_head', 'start_post_rel_link', 10, 0); // start link
    remove_action('wp_head', 'parent_post_rel_link', 10, 0); // prev link
    remove_action('wp_head', 'adjacent_posts_rel_link', 10, 0); // relational links 4 the posts adjacent 2 the currentpost
    remove_action('template_redirect', 'wp_shortlink_header', 11);  
    remove_action('wp_head', 'adjacent_posts_rel_link_wp_head', 10, 0 );
}, 99);

// Remove adminbar inline css on frontend
function removeinline_adminbar_css_frontend() {
    if ( has_filter( 'wp_head', '_admin_bar_bump_cb' ) ){
        remove_filter( 'wp_head', '_admin_bar_bump_cb' );
    }
}
add_filter( 'wp_head', 'removeinline_adminbar_css_frontend', 1 );

And if you want to remove the rest api link I believe this will work:

remove_action( 'template_redirect', 'rest_output_link_header', 11, 0 );

See this WPSE post here for additional solutions removing rest api.