Hi,
In an effort to stop duplicate content I know you recommended only allowing /merchant/ to be indexed but on my particular site I would rather not be linking to the merchant directory as /brand/ will be my main navigation, so easily crawlable, is it a problem search engine wise to ban everything except /brand/ ? so:
User-agent: *
Disallow: /categories.php
Disallow: /merchants.php
Disallow: /reviews.php
Disallow: /category/
Disallow: /merchant/
Disallow: /review/
Disallow: /admin/
Disallow: /search.php
Disallow: /jump.phpHi David,
If I'm going down this route should I still register the normal sitemap.php (which seems to be based around merchants) or is there a brand sitemap or does it not matter?
Cheers,
Chris.
Hi Chris,
As you were with the sitemap - the "merchant" aspect of it is only used to create the "index", each individual merchant sitemap then maps the individual product pages...
Cheers,
David.
Hi,
That would be OK - on the proviso that every product on your site has a brand (which it sounds like it will if that is your primary navigation). The only reason I suggest /merchant/ as the permitted robots route is because it is guaranteed to link to all products - that's all!
Cheers,
David.