Lists Home |
Date Index |
> Also one thing I've never understood, most spiders I've ever seen depend
> such links are rather common, how do spiders deal with them? Just
A sitemap page usually satisfies providing a link that allows the spiders to
reach otherwise dynamically setup links.
It doesn't show the page-to-page linking structure but at least it gives them a
way to spider.