You are here:  » File does not exist but PT fetches content of error page


File does not exist but PT fetches content of error page

Submitted by CashNexus on Sun, 2020-01-26 14:55 in

Hello David,
making some tests I suddenly met a strange issue.
I've changed the name of datafeed from
http://example.com/test/Everton.xml.gz
to
http://example.com/test/1Everton.xml.gz
So Apache says "File Not Found" as it should be.

I was expecting Status = "ERROR - TRANSFER FAILED" - but PT fetch anyway says Status = OK and downloads the content of error page.
Is that a correct behaviour ? Could you cross-check this issue ?
Or we need to add to /includes/automation.php a new "status" for 404 errors ?

Best regards,
Serge

Submitted by support on Mon, 2020-01-27 08:59

Hello Serge,

It should result in an error but it sounds like in this case the remote web server is returning the HTML of a 404 error page but it is not actually returning status code 404 and that would result in the download of the error page being considered a successful transfer.

Don't forget that you can use the "Abort if less than" setting of an Automation Tool job to specify a minimum file size to be considered a successful transfer...

Cheers,
David.
--
PriceTapestry.com

Submitted by CashNexus on Mon, 2020-01-27 18:36

Hello David, you're right, I can setup ABORT by a minsize.
At the same time let me note it's not an issue of 404 status return.
Under CentOS I test like
curl -s --head --request GET https://altertag.com/df_http_test/Everton_Direct-_Everton_USA.xml.gz
and you get status 200 OK.
But change the request to another filename, for example
curl -s --head --request GET https://altertag.com/df_http_test/Everton_Direct-_Everton_UK.xml.gz
- and you will get clear 404 status code.
...not a huge problem, of course, but PT should read such a 404 response and return fetch with an error "FILE NOT EXISTS"...IMHO...
Best regards,
Serge

Submitted by support on Tue, 2020-01-28 10:00

Hello Serge,

Thanks, yes the PHP default is to return the response regardless of status code however there is an option to return an error for response codes >= 400. To enable this, edit includes/automation.php and look for the following code at line 149:

    curl_setopt($ch,CURLOPT_HEADER,0);

...and REPLACE with:

    curl_setopt($ch,CURLOPT_HEADER,0);
    curl_setopt($ch,CURLOPT_FAILONERROR,1);

Cheers,
David.
--
PriceTapestry.com

Submitted by CashNexus on Tue, 2020-01-28 11:48

Hello David,
YES ! Your solution solved the problem, so now we have correct logic PT answer "ERROR - TRANSFER FAILED" if a file does not exist or renamed.
Thank you and have a nice day !
Serge