frame.pl and robots
Tue, 16 Mar 1999 21:15:20 +1100 (EST)
> Hi there,
> FYI: I reported last week that robots have a hard time indexing l2h
> docs with the current way frames are implemented. I assumed that the
> the bots need the links and the text to be indexed, to be present in
> the /noframes part of the document. In oder to test this, I placed
> some docs generated with NO_FRAMES. It worked indeed. At least
> AltaVista is now able to index such docs.
Uli, I'm happy to make whateve changes you think are desirable.
But please, would you tell me explicitly what HTML tags should go where ?
Currently there is a <META....> with an attribute value of 'norobots'
Should this be removed ?
... or retained in some files but no others ?
Is it <META .. > tags that robots search, or the <TITLE..> or other
things as well ?
All the best,