This is a discussion on mobots.txt within the Windows Web Servers forums, part of the Web Server and Related Forums category; We have noticed request for a page called /mobots.txt from http://www.mnnerprise.net/ IP address = 24.211.162....
|
|||||||
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
|
|||
|
We have noticed request for a page called /mobots.txt from
http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log files Has anyone else seen this? |
|
|||
|
"rachael traub" <rachiegirl@yahoo.com> wrote in message
news:8b2e411f.0408041244.3fb8bd64@posting.google.c om... > We have noticed request for a page called /mobots.txt from > http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log > files > > Has anyone else seen this? I wonder if they were after robots.txt and got it wrong? |
|
|||
|
i always get robots.txt in mine.... what is that?
"voigtstr" <vSoPiAgMtSsPtArM@REMOVECAPSbigpond.net.au> wrote in message news:idhQc.33835$K53.5349@news-server.bigpond.net.au... > "rachael traub" <rachiegirl@yahoo.com> wrote in message > news:8b2e411f.0408041244.3fb8bd64@posting.google.c om... > > We have noticed request for a page called /mobots.txt from > > http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log > > files > > > > Has anyone else seen this? > > I wonder if they were after robots.txt and got it wrong? > > |
|
|||
|
Sir_Civik wrote: > > i always get robots.txt in mine.... what is that? Read this; http://search.msn.com/webmasters/guidelines.aspx > "voigtstr" <vSoPiAgMtSsPtArM@REMOVECAPSbigpond.net.au> wrote in message > news:idhQc.33835$K53.5349@news-server.bigpond.net.au... > > "rachael traub" <rachiegirl@yahoo.com> wrote in message > > news:8b2e411f.0408041244.3fb8bd64@posting.google.c om... > > > We have noticed request for a page called /mobots.txt from > > > http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log > > > files > > > > > > Has anyone else seen this? > > > > I wonder if they were after robots.txt and got it wrong? GramPaHugs, Alex, -- No endorsement is implied or intended. ************************************************** ** * <a href="http://home.mn.rr.com/apbiii">AOL Click</a> * <a href="http://grampahugs.is-a-geek.org/">AOL Click</a> ************************************************** ** |
|
|||
|
[un-jeopardied]
On Thu, 5 Aug 2004 10:56:21 -0400, "Sir_Civik" <helpme@sircivik.ca> wrote: > >"voigtstr" <vSoPiAgMtSsPtArM@REMOVECAPSbigpond.net.au> wrote in message >news:idhQc.33835$K53.5349@news-server.bigpond.net.au... >> "rachael traub" <rachiegirl@yahoo.com> wrote in message >> news:8b2e411f.0408041244.3fb8bd64@posting.google.c om... >> > We have noticed request for a page called /mobots.txt from >> > http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log >> > files >> > >> > Has anyone else seen this? >> >> I wonder if they were after robots.txt and got it wrong? >> > >i always get robots.txt in mine.... what is that? > robots.txt is a hack solution for giving hints to search engine crawlers and other automated software. You can give guidelines to search crawlers to tell them which parts of the site should not be indexed if, for example, you have some duplicated content and you only want to include the canonical ("permalink") version in the index of Google and other search engines. Some sites also discourage search engines from roaming into tricky, dynamic areas application-type pages (rather than purely informational pages). The indexing robots request and parse this file before they begin "crawling" your site for pages and links. You can read more about the robots.txt file here: http://www.searchengineworld.com/rob...s_tutorial.htm Of course, anything you specify is only a guideline. Some robots won't request the file at all and others will ignore parts of it if it suits them. All the best, -Claire |