+- Webnetics UK Ltd. - Forums (https://www.webneticsuk.com/forum)
+-- Forum: News & Announcements (https://www.webneticsuk.com/forum/forumdisplay.php?fid=2)
+--- Forum: VWDesigns Blog (https://www.webneticsuk.com/forum/forumdisplay.php?fid=12)
+---- Forum: Programming (https://www.webneticsuk.com/forum/forumdisplay.php?fid=17)
+---- Thread: Robot Visit - Check when the search engines are visiting (/showthread.php?tid=88)
Robot Visit - Check when the search engines are visiting - webnetics - 27-04-08
Receive instant notification by email when a search engine spider has visited your website. This is a great way to monitor and manage your search engine submission plan.
• Are you getting listed on search engines?
• Do you want to know when your pages are getting indexed?
• Do you want to know what search engines are visiting and when?
• Are you an affiliate for many merchant programs and need to keep track of stats?
Then this is the script for you. Simply copy and paste a one line code into your website and as soon as a search engine spider visits your website you will receive an email notification containing the following information:
• Name of the Search Engine
• Name of the Spider
• Name of the page visited
• Exact day and time of visit
This gives you a great way to monitor if your website is being indexed by the major search engines. It also provides a great way of collecting stats of the habits of the search engine spiders. Giving you the opportunity to create search engine friendly pages which are correctly optimised for maximum search engine positioning.
Feature List
• Supports all the major search engines
• One step easy install
• Email notification of Spider visits.
• No database is required.
• Easy and simple to use.
• No special PHP knowledge required.
Currently supports the following search engines:
• Googlebot - [Google] - www.google.com
• Alta Vita - [Scooter] - www.altavista.com
• Teoma - [Ask Jeeves/Teoma] - www.teoma.com
• Lycos - [Lycos_Spider_(T-Rex)] - www.lycos.com
• Inktomi - [Slurp] - www.inktomi.com
• Mirago - [HenryTheMiragorobot] - www.mirago.com
• AlltheWeb - [FAST-WebCrawler] - www.alltheweb.com
PHP Code:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>RobotVisit v1.0</title>
<meta http-equiv = "Content-Type" content = "text/html; charset=iso-8859-1">
</head>
<body>
<?PHP
/*****************************************************
** Filename.....…...: robotvisit.php
** Package Name.....: Free Version
** Client ..........: N/A
** Specification No.: SVWD-0004
** Author..….....…..: Virtual Web Designs (Webnetics UK Ltd.)
** Contact.....….…..: support@vwdesigns.co.uk
** File Version.....: 1.0
** Date .......……...: 1/04/2008
** Change No.. .....: N/A
** Last Changed ....: N/A
*****************************************************/
############## START OF CONFIG SECTION #############
## Change the following to suit your needs:
$EmailAddress=""; ## NOTIFICATION EMAIL ADDRESS
$EmailAddress=preg_replace("/[\n\r]+/", "", "$EmailAddress");
############## END OF CONFIG SECTION #############
$footer="------------------------------------------------------------------------------
RobotVisit v1.0 - Virtual Web Designs
------------------------------------------------------------------------------";
$today=date("F j, Y, g:i a"); ## GET THE TIME OF THE ROBOT VISIT
$header=
"-----------------------------------------------------------------------------
SEARCH ENGINE ROBOT HAS VISITED YOUR WEBSITE
-----------------------------------------------------------------------------";
## GOOGLE SEARCH ENGINE
if (eregi("Googlebot", $HTTP_USER_AGENT))
{
global $header;
global $footer;
global $today;
if ($QUERY_STRING != "")
{
$GoogleLink="http://" . $SERVER_NAME . $PHP_SELF . '?' . $QUERY_STRING;
}
else
{
$GoogleLink="http://" . $SERVER_NAME . $PHP_SELF;
}
$GoogleEmail="
" . $header . "
Hi There,
The Search Engine robot Googlebot has been detected on your website.
Googlebot has visited http://$SERVER_NAME
Googlebot has crawled the page " . $GoogleLink . "
Time of visit - " . $today . "
Search Engine Link: http://www.google.com
" . $footer . "
";
@mail("$EmailAddress", "Google Robot Visit", $GoogleEmail, "From: <Google Robot>$EmailAddress");
}
## ALTAVISTA SEARCH ENGINE
if (eregi("Scooter", $HTTP_USER_AGENT))
{
global $header;
global $footer;
global $today;
if ($QUERY_STRING != "")
{
$AVLink="http://" . $SERVER_NAME . $PHP_SELF . '?' . $QUERY_STRING;
}
else
{
$AVLink="http://" . $SERVER_NAME . $PHP_SELF;
}
$AVEmail="
" . $header . "
Hi There,
The Search Engine robot Scooter has been detected on your website.
Scooter has visited http://$SERVER_NAME
Scooter has crawled the page " . $AVLink . "
Time of visit - " . $today . "
Search Engine Link: http://www.altavista.com
" . $footer . "
";
@mail("$EmailAddress", "AltaVista Robot Visit", $AVEmail, "From: <Alta Vista>$EmailAddress");
}
## ALL THE WEB SEARCH ENGINE
if (eregi("FAST-WebCrawler", $HTTP_USER_AGENT))
{
global $header;
global $footer;
global $today;
if ($QUERY_STRING != "")
{
$FASTLink="http://" . $SERVER_NAME . $PHP_SELF . '?' . $QUERY_STRING;
}
else
{
$FASTLink="http://" . $SERVER_NAME . $PHP_SELF;
}
$FASTEmail="
" . $header . "
Hi There,
The Search Engine robot FAST-WebCrawler has been detected on your website.
FAST-WebCrawler has visited http://$SERVER_NAME
FAST-WebCrawler has crawled the page " . $FASTLink . "
Time of visit - " . $today . "
Search Engine Link: http://www.alltheweb.com
" . $footer . "
";
@mail("$EmailAddress", "AllTheWeb Robot Visit", $FASTEmail, "From: <FAST Robot>$EmailAddress");
}
## ASK JEEVES - TEOMA
if (eregi("Ask Jeeves/Teoma", $HTTP_USER_AGENT))
{
global $header;
global $footer;
global $today;
if ($QUERY_STRING != "")
{
$ASKLink="http://" . $SERVER_NAME . $PHP_SELF . '?' . $QUERY_STRING;
}
else
{
$ASKLink="http://" . $SERVER_NAME . $PHP_SELF;
}
$ASKEmail="
" . $header . "
Hi There,
The Search Engine robot Ask Jeeves/Teoma has been detected on your website.
Ask Jeeves/Teoma has visited http://$SERVER_NAME
Ask Jeeves/Teoma has crawled the page " . $ASKLink . "
Time of visit - " . $today . "
Search Engine Link: http://www.ask.com & http://www.teoma.com
" . $footer . "
";
@mail("$EmailAddress", "Ask Jeeves-Teoma Robot Visit", $ASKEmail, "From: <ASK/Teoma Robot>$EmailAddress");
}
## INKTOMI DIRECTORY
if (eregi("Slurp", $HTTP_USER_AGENT))
{
global $header;
global $footer;
global $today;
if ($QUERY_STRING != "")
{
$SLURPLink="http://" . $SERVER_NAME . $PHP_SELF . '?' . $QUERY_STRING;
}
else
{
$SLURPLink="http://" . $SERVER_NAME . $PHP_SELF;
}
$SLURPEmail="
" . $header . "
Hi There,
The Search Engine robot Slurp has been detected on your website.
Slurp has visited http://$SERVER_NAME
Slurp has crawled the page " . $SLURPLink . "
Time of visit - " . $today . "
Search Engine Link: http://www.inktomi.com
" . $footer . "
";
@mail("$EmailAddress", "Inktomi Robot Visit", $SLURPEmail, "From: <Inktomi Robot>$EmailAddress");
}
## MIRAGO SEARCH ENGINE
if (eregi("HenryTheMiragorobot", $HTTP_USER_AGENT))
{
global $header;
global $footer;
global $today;
if ($QUERY_STRING != "")
{
$MiragoLink="http://" . $SERVER_NAME . $PHP_SELF . '?' . $QUERY_STRING;
}
else
{
$MiragoLink="http://" . $SERVER_NAME . $PHP_SELF;
}
$MiragoEmail="
" . $header . "
Hi There,
The Search Engine robot HenryTheMiragorobot has been detected on your website.
HenryTheMiragorobot has visited http://$SERVER_NAME
HenryTheMiragorobot has crawled the page " . $MiragoLink . "
Time of visit - " . $today . "
Search Engine Link: http://www.mirago.com
" . $footer . "
";
@mail("$EmailAddress", "Mirago Robot Visit", $MiragoEmail, "From: <Mirago Robot>$EmailAddress");
}
## LYCOS SEARCH ENGINE
if (eregi("Lycos_Spider_(T-Rex)", $HTTP_USER_AGENT))
{
global $header;
global $footer;
global $today;
if ($QUERY_STRING != "")
{
$LycosLink="http://" . $SERVER_NAME . $PHP_SELF . '?' . $QUERY_STRING;
}
else
{
$LycosLink="http://" . $SERVER_NAME . $PHP_SELF;
}
$LycosEmail="
" . $header . "
Hi There,
The Search Engine robot Lycos_Spider_(T-Rex) has been detected on your website.
Lycos_Spider_(T-Rex) has visited http://$SERVER_NAME
Lycos_Spider_(T-Rex) has crawled the page " . $LycosLink . "
Time of visit - " . $today . "
Search Engine Link: http://www.lycos.com
" . $footer . "
";
@mail("$EmailAddress", "Lycos Robot Visit", $LycosEmail, "From: <Lycos Robot>$EmailAddress");
}
?>
</body>
</html>
Installation
To install the PHP RobotVisit to your website, simply add the code below to your webpage and upload the robotvisit.php file in the same directory as the pages the code below has been pasted into. Please note that the page extension needs to be .php or PHP equivalent.
The nearer the top of the page the better as this will give a better indication of when you have been visited by one of the robots. The code can be added to as many or as few pages as you require. Ideally it should be inserted before the opening <html> tag. Paste the code below in as many pages on your website as required.
PHP Code:
<?PHP include "robotvisit.php" ?>Warning (!): [/php]If you have a large traffic website, you may be wise to use this script sparingly, because if you have a good listing on the search engines you could end up receiving quite a few emails! I did this on one of my Affiliate script websites and ended up with over a 1000 emails that day!. But then at least you know your website has a good listing on the search engines.