Is there a possibility to only update the changed feeds when fetching all the feeds? Some sites don't update their feed daily so it's kind of overkill to update and import those feeds every day.
There's no straight forward way to achieve this I'm afraid - the best solution would be if merchants / networks were to support the "If-Modified-Since" header in the HTTP request for their feeds.
Otherwise, it would be no problem to add a new field to the job configuration page to only run that job either on a particular day of the week or every n days so if you'd like to give that a go, let me know and I'll work out the mods however unless you actually know when a feed is going to be updated there is the possibility that you could end up updating completely out of sync - e.g. feed updates every Sunday but you only update every Saturday - making your data potentially a week out of date.
Bear in mind that when cron runs, feeds are imported into a temporary table so there is no downtime of your site at all, so if the process is happening in reasonable time I would suggest simply updating all feeds daily regardless of whether or not they have actually changed...
Hello Kees,
There's no straight forward way to achieve this I'm afraid - the best solution would be if merchants / networks were to support the "If-Modified-Since" header in the HTTP request for their feeds.
Otherwise, it would be no problem to add a new field to the job configuration page to only run that job either on a particular day of the week or every n days so if you'd like to give that a go, let me know and I'll work out the mods however unless you actually know when a feed is going to be updated there is the possibility that you could end up updating completely out of sync - e.g. feed updates every Sunday but you only update every Saturday - making your data potentially a week out of date.
Bear in mind that when cron runs, feeds are imported into a temporary table so there is no downtime of your site at all, so if the process is happening in reasonable time I would suggest simply updating all feeds daily regardless of whether or not they have actually changed...
Cheers,
David.
--
PriceTapestry.com