Robots

F

Fred

If I add the following meta tag to my pages will it allow the robot to visit?

<meta name="robots" content="index,follow">
<meta name="revisit-after" content="7 days">

Thanks in anticipation of the usual helpful advice from this group. (e-mail address removed)
 
S

Steve Easton

Yes, but you should also add
a robots.txt file to the root of your
web.
Here's a "how to."
http://www.pageresource.com/zine/robotstxt.htm
--
using 2k PRO but....95isalive
This site is best viewed............
........................with a computer

If I add the following meta tag to my pages will it allow the robot to visit?

<meta name="robots" content="index,follow">
<meta name="revisit-after" content="7 days">

Thanks in anticipation of the usual helpful advice from this group. (e-mail address removed)
 
M

MD WebsUnlimited.com

Yes.

Take a look at http://www.searchenginewatch.com

You may also wish to take a look at Meta Tag Maker at http://www.websunlimited.com/order/Product/MTM2002/mtm2002_help_dir.htm
--
Mike -- FrontPage MVP '97 - '02
http://www.websunlimited.com
Our latest products "At Your Command" and IncludeASP
http://www.websunlimited.com/order/Product/AYC/ayc.htm
http://www.websunlimited.com/order/product/includeASP/includeASP.htm
------------------------------------------------------------------------------------------------
If you think I'm doing a good job, let MS know at (e-mail address removed)

If I add the following meta tag to my pages will it allow the robot to visit?

<meta name="robots" content="index,follow">
<meta name="revisit-after" content="7 days">

Thanks in anticipation of the usual helpful advice from this group. (e-mail address removed)
 
F

Fred

Sorry for being a bit thick but the site you gave started with "This is a useful file that keeps search engines from indexing pages you do not want spidered" Obviously I want the opposite to happen and would welcome advice on 'What is - and where is it located, the root of my web' (told you I was thick) and getting even thicker, what do I place in the robots.txt file?
Yes, but you should also add
a robots.txt file to the root of your
web.
Here's a "how to."
http://www.pageresource.com/zine/robotstxt.htm
--
using 2k PRO but....95isalive
This site is best viewed............
.......................with a computer

If I add the following meta tag to my pages will it allow the robot to visit?

<meta name="robots" content="index,follow">
<meta name="revisit-after" content="7 days">

Thanks in anticipation of the usual helpful advice from this group. (e-mail address removed)
 
T

Tom Pepper Willett

If you want them to index your site, you do not need a robots.txt file.
--
-----
Tom Pepper Willett
Microsoft MVP - FrontPage
http://www.microsoft.com/office/frontpage/prodinfo/default.mspx
To find the best Newsgroup for FrontPage support see:
http://www.net-sites.com/sitebuilder/newsgroups.asp
----
Sorry for being a bit thick but the site you gave started with "This is a
useful file that keeps search engines from indexing pages you do not want
spidered" Obviously I want the opposite to happen and would welcome advice
on 'What is - and where is it located, the root of my web' (told you I was
thick) and getting even thicker, what do I place in the robots.txt file?
Yes, but you should also add
a robots.txt file to the root of your
web.
Here's a "how to."
http://www.pageresource.com/zine/robotstxt.htm
--
using 2k PRO but....95isalive
This site is best viewed............
.......................with a computer

If I add the following meta tag to my pages will it allow the robot to
visit?

<meta name="robots" content="index,follow">
<meta name="revisit-after" content="7 days">

Thanks in anticipation of the usual helpful advice from this group.
(e-mail address removed)
 
S

Steve Easton

If your robots.txt file contains
nothing but:

User-agent:*

Then it tells the bots they have permission to
index the entire site.

If there's a section you don't want indexed,
then you add:

Disallow: /cgi-bin/
to keep them out of the cgi-bin,

Disallow: /images/
to keep them out of your images folder,
and so forth.

However the robots.txt file is open
"to the public" and can be read by anyone.

Example:
http://www.95isalive.com/robots.txt


--
using 2k PRO but....95isalive
This site is best viewed............
........................with a computer

Sorry for being a bit thick but the site you gave started with "This is a useful file that keeps search engines from indexing pages you do not want spidered" Obviously I want the opposite to happen and would welcome advice on 'What is - and where is it located, the root of my web' (told you I was thick) and getting even thicker, what do I place in the robots.txt file?
Yes, but you should also add
a robots.txt file to the root of your
web.
Here's a "how to."
http://www.pageresource.com/zine/robotstxt.htm
--
using 2k PRO but....95isalive
This site is best viewed............
.......................with a computer

If I add the following meta tag to my pages will it allow the robot to visit?

<meta name="robots" content="index,follow">
<meta name="revisit-after" content="7 days">

Thanks in anticipation of the usual helpful advice from this group. (e-mail address removed)
 
S

Stefan B Rusynko

Agree except for all the log file entries for "page not found" when they look for it
- IMHO best to include one even it is just and "empty" one w/ User-agent:*





| If you want them to index your site, you do not need a robots.txt file.
| --
| -----
| Tom Pepper Willett
| Microsoft MVP - FrontPage
| http://www.microsoft.com/office/frontpage/prodinfo/default.mspx
| To find the best Newsgroup for FrontPage support see:
| http://www.net-sites.com/sitebuilder/newsgroups.asp
| ----
| | Sorry for being a bit thick but the site you gave started with "This is a
| useful file that keeps search engines from indexing pages you do not want
| spidered" Obviously I want the opposite to happen and would welcome advice
| on 'What is - and where is it located, the root of my web' (told you I was
| thick) and getting even thicker, what do I place in the robots.txt file?
| | Yes, but you should also add
| a robots.txt file to the root of your
| web.
| Here's a "how to."
| http://www.pageresource.com/zine/robotstxt.htm
| --
| using 2k PRO but....95isalive
| This site is best viewed............
| .......................with a computer
|
| | If I add the following meta tag to my pages will it allow the robot to
| visit?
|
| <meta name="robots" content="index,follow">
| <meta name="revisit-after" content="7 days">
|
| Thanks in anticipation of the usual helpful advice from this group.
| (e-mail address removed)
|
|
|
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Meta Tags for Robots 1
FP and preventing robots 3
Robots 7
Analysis says I have no Meta Robots - Where/How? 1
Robots.txt 3
Search engine optimization 2
How to keep Google from listing site 3
meta tags 7

Top