You are here:  » To block or not to block /html/

Active Forum Topics


To block or not to block /html/

Submitted by IG on Tue, 2017-10-31 19:39 in

Hi David

In your post "Updated: Asynchronous Amazon and eBay API Modules" (https://www.pricetapestry.com/node/6197) you advice to "add a Disallow: entry for /html/ to your site's robots.txt file".

I had a peak on your demo site where /html/ isn't blocked and therefore I am wondering if it still makes sense to block /html/ or not?

Cheers, IG

Submitted by support on Wed, 2017-11-01 06:48

Hi,

Yes I think it should be - thanks for pointing that out have added to demo site robots.txt...

Cheers,
David.
--
PriceTapestry.com