jump.php recording inordinately large number of clicks

It could be search engine crawlers following the jump script - so it might be an idea to exclude it with robots.txt:

If you want to reset the click to zero there's a script in the following thread:

http://www.pricetapestry.com/node/141

Cheers,
David.

Hi David,

Thanks for the quick reply.

Where do I put the robots.txt file ?

Is what you put above the only text which it needs to contain?

Are there any downsides to doing the above?

(if not, it might be a good idea to include a robots.txt with pricetapestry)

The file above would need to go in the top level directory of your website, so that it can be requested as:

If you have installed Price Tapestry into a sub-directory, robots.txt still needs to go in the top level, but should contain the following (if Price Tapestry is installed a directory called "shopping"):

Cheers,
David.

Does this stop the search engines seeing the affiliate links at all? I have already done this, thinking it might prevent being doomed as "just another affiliate site" by the search engines!!

Hi David,

Thanks for your great help as usual. I know you probably have dealt with this all before and likely know exactly what is happening.

I added a robots.txt in the root of the site under public_html. Price tapestry is in the root also.

I took the following robots.txt from this forum - it's the one you use:

User-agent: Mediapartners-Google*
Disallow:

User-agent: ia_archiver
Disallow: /

User-agent: *
Disallow: /categories.php
Disallow: /brands.php
Disallow: /reviews.php
Disallow: /category/
Disallow: /brand/
Disallow: /review/
Disallow: /admin/
Disallow: /search.php
Disallow: /jump.php

Yet, I can still see clicks increasing literally every few seconds, even with the above robots.txt.

Does the robots.txt take time to take effect, ie. if the robot is already in the site does it complete the crawling of all pages without checking for a robots.txt and only look for the robots.txt on the next site entry? (that would explain why it seems to still be crawling my site like crazy even after I've added a robots.txt)?

Or, does the robot look for the robots.txt on each page access (in which case I don't understand why clicks are still increasing)?

Hi,

Yes - it may take some time before the search engines look for robots.txt again, although I think most do at least once a day during a heavy crawl (which is what it sounds like you're getting at the moment!)

Cheers,
David.

If I want google to still crawl and register my sites, but for jump.php to not count robots is there a way to do this?

Thanks,
Wayman Luy

Hi Waymen,

Preventing jump.php in robots.txt won't stop search engines indexing your site - but should stop them recording clicks so it should all be fine...

Cheers,
David.