The FreeBSD Diary

The FreeBSD Diary (TM)

Providing practical examples since 1998

If you buy from Amazon USA, please support us by using this link.
[ HOME | TOPICS | INDEX | WEB RESOURCES | BOOKS | CONTRIBUTE | SEARCH | FEEDBACK | FAQ | FORUMS ]
Scripts / handy tips
 New Topic  |  Go to Top  |  Go to Topic  |  Search  |  Log In   Previous Message  |  Next Message 
 Re: robots.txt syntax checker
Author: Dan Langille 
Date:   11-02-02 04:17

It's what a robot should read before it reads other stuff from your website. See http://www.robotstxt.org/wc/exclusion-admin.html.

 Reply To This Message  |  Forum List  |  Flat View   Newer Topic  |  Older Topic 

 Topics Author  Date
 robots.txt syntax checker   new
Dan Langille 06-02-02 03:06 
 Re: robots.txt syntax checker   new
new 10-02-02 22:28 
 Re: robots.txt syntax checker   new
Dan Langille 11-02-02 04:17 


 Forum List  |  Need a Login? Register Here 
 User Login
 User Name:
 Password:
 Remember my login:
   
 Forgot Your Password?
Enter your email address or user name below and a new password will be sent to the email address associated with your profile.
How to get the most out of the forum

phorum.org