Hi,
We are using Price Tapestry within 2 websites.
We primarily use feeds from Affiliate Window and generate sitemaps using iGooMap for Apple. (We are unable to use sitemap.php as we need many other pages to get indexed as well as the ones from Price Tapestry).
We started out with around 10,000 products on each site and were able to generate sitemaps that listed every single product. After these sitemaps were uploaded, Google appeared to start indexing all the pages.
We have recently increased the product ranges on each site to more than 150,000 products and now are unable to generate sitemaps in the same way. (It simply takes too long for iGooMap to process the page for so many products).
My proposed solution was to restrict iGooMap from listing individual products, whilst allowing it to list merchants and brands. I have placed the restriction on iGooMap itself (rather than in robots.txt), in the hope that the search engines will still index the products individually.
However, it appears that Google is now no-longer indexing the individual product pages (and in fact we were in a better overall position with only 100,00 products on the site).
Any ideas or options would be much appreciated.
Kind regards,
Arya Marafie.
Hello,
For sites with tens or hundreds of thousands of products it's a good idea to have an additional site map generated by a third party sitemap generator. You can add the additional site map in your Google Webmaster Tools account.
I've done exactly that using XML Sitemap Generator.
If you can install PT then you'll find this software easy to install and use and the support is very good too. You can use a Cron job to automate site map generation on a regular basis.
Regards,
Ben
Hello Arya,
As far as I know, you can submit more than one sitemap URL per site to Google, so I would recommend keeping Price Tapestry's "sitemap.php" to cover your products, and then restrict your other sitemap program to the product pages and keep that for your other pages...
Cheers,
David.