Search engine spidering of pages with Frontpage "includes"

[Rick Baker]

This one has me puzzled and I was hoping that someone
here could help.

I have a large content-rich site (several hundred pages).
Most of my pages have several "common" areas. The one
that I am concerned about is the header section with my
site's banner and main navigation bar.

I have opted to implement this header area as a
Frontpage "include page" to save time when I need to make
changes (much better to change one page than hundreds). I
am using Frontpage 2000.

It is a high priority for me to have the keyword-rich
links in my main navigation bar recognized and followed
by the search engine spiders. My question is twofold:

1. Will the search engine spiders reognize the "included
content" as content on the page being indexed? For
example, will the spiders go to my home page and see the
header section as part of the home page or as part of a
separate page that is linked to the home page?

2. Secondly, if they do recognize this content, will they
be able to follow the links correctly? When I run a
search engine spider simulator, the "text" in the
included section seems to be recognized, but the "links"
appear strange. Instead
of "http://www.mysite/mypage.htm", the spider recognizes
the links as "http://..mypage.htm", "http://mypage.htm",
or some variation of that. This is despite the fact that
I have created the hyperlinks using the full URL (ie.
http://www.mysite/mypage.htm).

When I try to click on the indexed links, they do not
function (although they function correctly on my site)
and a "not found" error page results.

I obviously need to spiders to be able to follow these
links correctly in order to index my whole site. If
anyone knows how I can accomplish this (or knows if I
have already done it correctly) it would be greatly
appreciated.

Thanks in advance,

Rick Baker
mailto:[email protected]
http://www.littlepluto.com

 

[Thomas A. Rowe]

See inline below.

--

==============================================
Thomas A. Rowe (Microsoft MVP - FrontPage)
WEBMASTER Resources(tm)

FrontPage Resources, Forums, WebCircle,
MS KB Quick Links, etc.
==============================================

This one has me puzzled and I was hoping that someone
here could help.

I have a large content-rich site (several hundred pages).
Most of my pages have several "common" areas. The one
that I am concerned about is the header section with my
site's banner and main navigation bar.

I have opted to implement this header area as a
Frontpage "include page" to save time when I need to make
changes (much better to change one page than hundreds). I
am using Frontpage 2000.

It is a high priority for me to have the keyword-rich
links in my main navigation bar recognized and followed
by the search engine spiders. My question is twofold:

1. Will the search engine spiders reognize the "included
content" as content on the page being indexed? For
example, will the spiders go to my home page and see the
header section as part of the home page or as part of a
separate page that is linked to the home page?

Search engines will see the included content as part of each container page
that it indexes.

2. Secondly, if they do recognize this content, will they
be able to follow the links correctly? When I run a
search engine spider simulator, the "text" in the
included section seems to be recognized, but the "links"
appear strange. Instead
of "http://www.mysite/mypage.htm", the spider recognizes
the links as "http://..mypage.htm", "http://mypage.htm",
or some variation of that. This is despite the fact that
I have created the hyperlinks using the full URL (ie.
http://www.mysite/mypage.htm).

When using the FP includes to link to content that is within your web site
(or subweb), you should use relative hyperlinks so that FP can manage your
hyperlinks.

Whenever you use a absolute URL, be sure to always specific it the same way,
so that you are not causing the browser/server to create new sessions.

http://www.mydomain.com
http://mydomain.com
https://www.mydomain.com

The above create different browser/server sessions.

I wouldn't rely on a simulator, instead search for you content in the
various search engines and then check the links back to your site.