You are here:  » Cron one feed at a time until 100% then shift to the next


Cron one feed at a time until 100% then shift to the next

Submitted by mikecdprice on Wed, 2013-08-14 13:43 in

Hello David,

Is there a way to do the cron automation that fetches the remote feed/file one at a time until 100% then on to the next. I am noticing at certain times of the day they will work fine and other times it will error out. I think some take longer than expected and it errors out. They change on which one errors sometimes while fetching.

This is something I have been noticing.

Thank you,
Michael

Submitted by support on Wed, 2013-08-14 14:21

Hi Michael,

cron.php does fetch sequentially - so it will wait for a job to finish before continuing. Something to bear in mind is that (unfortunately) some networks feed generating URLs are quite slow to response due (I presume) to work going on server side to generate the feed, so adding a timeout might help but it would need to be quite generous.

To do this, I think you are already but if not change to using CURL as the automation handler by changing line 43 in config.advanced.php as follows:

  $config_automationHandler = "curl";

Then in includes/automation.php look for the following code at line 124:

  curl_setopt($ch,CURLOPT_FOLLOWLOCATION,TRUE);

...and REPLACE with:

  curl_setopt($ch,CURLOPT_FOLLOWLOCATION,TRUE);
  curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,60);

That will set a 60 second timeout on each job. Now that _may_ mean that sometimes a particular feed isn't updated - not really a lot that can be done about that if a remote server is not responding at the time of your CRON job, but if that appears to be a problem let me know and the stepped method may help (where you call cron.php multiple times daily).

Cheers,
David.
--
PriceTapestry.com

Submitted by mikecdprice on Wed, 2013-08-14 15:01

Hello David,

Perfect. Thank you

Michael