Bluehost.com Web Hosting $6.95

mobots.txt

This is a discussion on mobots.txt within the Windows Web Servers forums, part of the Web Server and Related Forums category; We have noticed request for a page called /mobots.txt from http://www.mnnerprise.net/ IP address = 24.211.162....


Go Back   Usenet Forums > Web Server and Related Forums > Windows Web Servers

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 08-04-2004
rachael traub
 
Posts: n/a
Default mobots.txt

We have noticed request for a page called /mobots.txt from
http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log
files

Has anyone else seen this?
  #2 (permalink)  
Old 08-05-2004
voigtstr
 
Posts: n/a
Default Re: mobots.txt

"rachael traub" <rachiegirl@yahoo.com> wrote in message
news:8b2e411f.0408041244.3fb8bd64@posting.google.c om...
> We have noticed request for a page called /mobots.txt from
> http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log
> files
>
> Has anyone else seen this?


I wonder if they were after robots.txt and got it wrong?


  #3 (permalink)  
Old 08-05-2004
Sir_Civik
 
Posts: n/a
Default Re: mobots.txt

i always get robots.txt in mine.... what is that?


"voigtstr" <vSoPiAgMtSsPtArM@REMOVECAPSbigpond.net.au> wrote in message
news:idhQc.33835$K53.5349@news-server.bigpond.net.au...
> "rachael traub" <rachiegirl@yahoo.com> wrote in message
> news:8b2e411f.0408041244.3fb8bd64@posting.google.c om...
> > We have noticed request for a page called /mobots.txt from
> > http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log
> > files
> >
> > Has anyone else seen this?

>
> I wonder if they were after robots.txt and got it wrong?
>
>



  #4 (permalink)  
Old 08-05-2004
Alex Barna
 
Posts: n/a
Default Re: mobots.txt



Sir_Civik wrote:
>
> i always get robots.txt in mine.... what is that?


Read this;
http://search.msn.com/webmasters/guidelines.aspx

> "voigtstr" <vSoPiAgMtSsPtArM@REMOVECAPSbigpond.net.au> wrote in message
> news:idhQc.33835$K53.5349@news-server.bigpond.net.au...
> > "rachael traub" <rachiegirl@yahoo.com> wrote in message
> > news:8b2e411f.0408041244.3fb8bd64@posting.google.c om...
> > > We have noticed request for a page called /mobots.txt from
> > > http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log
> > > files
> > >
> > > Has anyone else seen this?

> >
> > I wonder if they were after robots.txt and got it wrong?



GramPaHugs,
Alex,

--

No endorsement is implied or intended.

************************************************** **
* <a href="http://home.mn.rr.com/apbiii">AOL Click</a>
* <a href="http://grampahugs.is-a-geek.org/">AOL Click</a>
************************************************** **
  #5 (permalink)  
Old 08-05-2004
Claire Tucker
 
Posts: n/a
Default Re: mobots.txt

[un-jeopardied]
On Thu, 5 Aug 2004 10:56:21 -0400, "Sir_Civik" <helpme@sircivik.ca>
wrote:
>
>"voigtstr" <vSoPiAgMtSsPtArM@REMOVECAPSbigpond.net.au> wrote in message
>news:idhQc.33835$K53.5349@news-server.bigpond.net.au...
>> "rachael traub" <rachiegirl@yahoo.com> wrote in message
>> news:8b2e411f.0408041244.3fb8bd64@posting.google.c om...
>> > We have noticed request for a page called /mobots.txt from
>> > http://www.mnnerprise.net/ IP address = 24.211.162.104 in our web log
>> > files
>> >
>> > Has anyone else seen this?

>>
>> I wonder if they were after robots.txt and got it wrong?
>>

>
>i always get robots.txt in mine.... what is that?
>


robots.txt is a hack solution for giving hints to search engine
crawlers and other automated software. You can give guidelines to
search crawlers to tell them which parts of the site should not be
indexed if, for example, you have some duplicated content and you only
want to include the canonical ("permalink") version in the index of
Google and other search engines. Some sites also discourage search
engines from roaming into tricky, dynamic areas application-type pages
(rather than purely informational pages).

The indexing robots request and parse this file before they begin
"crawling" your site for pages and links.

You can read more about the robots.txt file here:
http://www.searchengineworld.com/rob...s_tutorial.htm

Of course, anything you specify is only a guideline. Some robots won't
request the file at all and others will ignore parts of it if it suits
them.

All the best,
-Claire
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are Off
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT +1. The time now is 10:24 PM.


Powered by vBulletin® Version 3.7.3
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO 3.0.0