You are here:  » crawl errors


crawl errors

Submitted by rolli1 on Thu, 2015-08-13 10:02 in

Hi David,

I get a lot of 404 Errors in webmastertools sitemap.php

The url which isnot found Comes from
http://www.example.com/product/DKNY-JEANS-Tops-Item-37329795.html
and should be
http://www.example.com/products/DKNY-JEANS-Tops-Item-37329795.html

Where is the code for changing product into products?

Regards

Roland

Submitted by support on Thu, 2015-08-13 10:30

Hi Roland,

I think you're running a distribution from before there was a centralised function for creating the product HREF so in this case, edit sitemap.php directly, search for

product/

...and REPLACE with:

products/

...as required.

Cheers,
David.
--
PriceTapestry.com

Submitted by rolli1 on Fri, 2015-08-14 08:41

Hi David,
I already have edited sitemap.php before. My Research shows that this error comes from a very old datafeed which I have deleted since long time. Don't know why this appears now again:

http://www.example.com/sitemap.php?filename=24285_257896_mp.txt&start=50000

How can I delete this sitemap, which I think does only exist in the database? I think it was not deleted correctly.

Regards

Roland

Submitted by support on Fri, 2015-08-14 09:13

Hello Roland,

Please could you email me a link to the actual sitemap page that is generating the old URLs and I'll check that out right away for you...

If you could also attach to the email from this installation:

sitemap.php
includes/tapestry.php

Thanks!
David.
--
PriceTapestry.com

Submitted by rolli1 on Wed, 2015-08-19 16:05

Hi David,
the link is

{link saved}

but the file 24285_257896_mp.txt as well as the feed 24285_257896_mp.txt does not exist. I dleted it about 6 months ago. The error also appeard only 4 weeks ago and not before.

Regards

Roland

Submitted by support on Thu, 2015-08-20 07:44

Hi Roland,

It sounds like a request for a removed per-feed sitemap, so what would actually be worth doing is modifying sitemap.php to return 404 (Not Found) in this instance. To do this, look for the following code at line 4:

  header("Content-Type: text/xml");

...and REPLACE with:

  if (isset($_GET["filename"]))
  {
    if (!file_exists("feeds/".$_GET["filename"]))
    {
      header("HTTP/1.0 404 Not Found");
      exit();
    }
  }
  header("Content-Type: text/xml");

Cheers,
David.
--
PriceTapestry.com

Submitted by rolli1 on Tue, 2015-08-25 12:43

Thanks for that....I will watch this.

Best regards

Roland