You are here:  » File does not exist but PT fetches content of error page

Active Forum Topics


File does not exist but PT fetches content of error page

Submitted by CashNexus on Sun, 2020-01-26 14:55 in

Hello David,
making some tests I suddenly met a strange issue.
I've changed the name of datafeed from
http://example.com/test/Everton.xml.gz
to
http://example.com/test/1Everton.xml.gz
So Apache says "File Not Found" as it should be.

I was expecting Status = "ERROR - TRANSFER FAILED" - but PT fetch anyway says Status = OK and downloads the content of error page.
Is that a correct behaviour ? Could you cross-check this issue ?
Or we need to add to /includes/automation.php a new "status" for 404 errors ?

Best regards,
Serge

Submitted by support on Mon, 2020-01-27 08:59

Hello Serge,

It should result in an error but it sounds like in this case the remote web server is returning the HTML of a 404 error page but it is not actually returning status code 404 and that would result in the download of the error page being considered a successful transfer.

Don't forget that you can use the "Abort if less than" setting of an Automation Tool job to specify a minimum file size to be considered a successful transfer...

Cheers,
David.
--
PriceTapestry.com

Submitted by CashNexus on Mon, 2020-01-27 18:36

Hello David, you're right, I can setup ABORT by a minsize.
At the same time let me note it's not an issue of 404 status return.
Under CentOS I test like
curl -s --head --request GET https://altertag.com/df_http_test/Everton_Direct-_Everton_USA.xml.gz
and you get status 200 OK.
But change the request to another filename, for example
curl -s --head --request GET https://altertag.com/df_http_test/Everton_Direct-_Everton_UK.xml.gz
- and you will get clear 404 status code.
...not a huge problem, of course, but PT should read such a 404 response and return fetch with an error "FILE NOT EXISTS"...IMHO...
Best regards,
Serge

Submitted by support on Tue, 2020-01-28 10:00

Hello Serge,

Thanks, yes the PHP default is to return the response regardless of status code however there is an option to return an error for response codes >= 400. To enable this, edit includes/automation.php and look for the following code at line 149:

    curl_setopt($ch,CURLOPT_HEADER,0);

...and REPLACE with:

    curl_setopt($ch,CURLOPT_HEADER,0);
    curl_setopt($ch,CURLOPT_FAILONERROR,1);

Cheers,
David.
--
PriceTapestry.com

Submitted by CashNexus on Tue, 2020-01-28 11:48

Hello David,
YES ! Your solution solved the problem, so now we have correct logic PT answer "ERROR - TRANSFER FAILED" if a file does not exist or renamed.
Thank you and have a nice day !
Serge