You are here:  » fetch.php and fetchall.php mod breaking fetch after large import of LinkShare datafeeds.


fetch.php and fetchall.php mod breaking fetch after large import of LinkShare datafeeds.

Submitted by BobL on Thu, 2013-05-30 13:18 in

Hi David,
I hope all is well.
I am having a problem with the LinkShare feeds that I'm not sure if the Dedicated server is timing out or what is happening here.

This is one of the feeds that break on LinkShare feeds.
{code saved}

The 1st feed does import, but the fields in the automation tool never update the time and status.
And none of the rest ever get fetched.
Any ideas on this one?

Thanks for the fantastic support that you have given us all already.
Bob L.

Submitted by support on Thu, 2013-05-30 13:40

Hello Bob,

A couple of users have reported problems fetching LinkShare feeds using "php" as the handler however they seem to work fine using CURL (I suspect this is something to do with a PASV mode requirement by their FTP server). To switch, edit your config.advanced.php and change line 43 as follows:

  $config_automationHandler = "curl";

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by BobL on Thu, 2013-05-30 14:04

Bob L.

Well I was going to ask how to enable php handlers on our dedicated server ... as it isn't enabled.
We are using curl to do the fetch.
What we are seeing is the temp file being created and the cron finishes before, and email received before the 1st temp file has been renamed to the file name.
Resolving {link saved}
Connecting to {link saved}|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: `/dev/null'

0K 9.52M=0s

2013-05-30 07:41:10 (9.52 MB/s) - `/dev/null' saved [43]

Submitted by support on Thu, 2013-05-30 15:21

Hi Bob,

Can you confirm the full CRON command you have scheduled to run?

It does sound like an execution time limit issue. Normally, there are separate php.ini files for the Apache module, and the command line version of PHP, but looking at your email I think you may be scheduling an HTTP request to fetch[all].php when you could be invoking it by the command line - let me know what you're using and if possible I'll suggest an alternative;

If you login to your server and become root, you should find the 2 php.ini files as

/etc/php5/apache2/php.ini (Apache module)
/etc/php5/cli/php.ini (command line)

To enable "php" as the automation handler, check in each of the above files for any current setting of "allow_url_fopen". If it is set to "Off", change the line as follows, or add the following line if the directive doesn't exist:

allow_url_fopen = On

For the time limit, the directive to look for is max_execution_time. You might want to leave this limited for the Apache module (it's typically set to "30" (seconds) in most distribution's standard PHP installation), but if you find this directive in the command line version of php.ini it should be safe to disable by changing the value to "0".

Always make a backup copy first before making changes to any configuration files!! e.g.

cp php.ini php.ini.ok

Cheers,
David.
--
PriceTapestry.com

Submitted by BobL on Thu, 2013-05-30 16:22

Bob L.
Hi David thanks for the quick response...Appreciate it.

This is my cron line that I'm using for the fetchall.php mod that you emailed me.

/usr/bin/wget -O /dev/null "{link saved}"

Submitted by support on Thu, 2013-05-30 16:27

Thanks Bob,

I'll follow up by email with the command line version to try!

Cheers,
David.
--
PriceTapestry.com

Submitted by BobL on Fri, 2013-05-31 11:08

Bob L.

David.....:)

That last mod on the includes/database.php

Proves you are the "Man with the Midas touch".

Worked like a awesome "Charm";

Many thanks.