REST api returns 404 on some WordPress sites

My code:

?php

public function __construct()
{
    $valid_ips = array(
        '1.2.3.4',
    );
    // It seems the problem is here! Because it returns 404 which means the endpoint is not registered.
    if (in_array($_SERVER['REMOTE_ADDR'], $valid_ips)) {
        add_action('rest_api_init', array($this, 'register_my_endpoint'));
    }
}

public function register_my_endpoint()
{
    register_rest_route('items/v1', '/get-items', array(array(
        'methods' = 'GET, POST',
        'callback' = array($this, 'some_func'),
        'permission_callback' = '__return_true'
    )));
}

The problem is that it is working and registering the endpoint on some sites but not in the others. Am I doing something wrong here? I'm sure my IP that is requesting is correct as I have checked it with https://www.ipify.org/. Is this possible that the $_SERVER['REMOTE_ADDR'] has some weird value in some wordpress sites? Or can this be modified with the client that has used it with something like .htaccess and the like. By the way, I have also added IPV6 in the $valid_ips list but no good result.

How can I fix or at least diagnose it?