Check User Agent

$_SERVER['HTTP_USER_AGENT'] =
CCBot/2.0 (https://commoncrawl.org/faq/)