You are here:  » Automate slow import from URL feed


Automate slow import from URL feed

Submitted by suffolk66 on Sat, 2012-01-28 08:58 in

My feed comes from a URL and is automated to load once a day. THe feed has approx 20,000 products and so what's the best way of updating it (presumably, using Slow Import)?

Submitted by support on Sat, 2012-01-28 17:38

Hi there,

If you haven't done so already, set your feed up in the Automation Tool to be fetched and download to your /feeds/ folder. Next, all you need to do is set-up a CRON job to run the cron.php automation script on the schedule required, so if your feed is being updated daily, you could set it up to run daily.

The Support Info page of the /admin/ section of your Price Tapestry installation automatically works out what CRON job command options are available. Use Option 1 in preference to Option 2, and the command displayed (which invokes cron.php) is the exact command you need to enter into the CRON section of your hosting control panel.

That should be all you need to do - cron.php fetches all feeds configured in the Automation Tool, and then imports all feeds.

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by mally on Mon, 2012-01-30 18:42

Hi David, I'm using the above method, however one of the files has about 50,000 items held within.

When I check the cron print is only loads upto ...[13100/49024]

Its loading 100 at a time looking at the list.

Do I need to increase the 100 at a time (where would I change this)

Thanks Mally

Submitted by support on Mon, 2012-01-30 19:42

Hi Mally,

Import progress is only reported every 100 products so assuming that wasn't the last report in the file that would indicate a transient feed generation error as I know you've been using the script for several years now. The first thing I would anticipate is that the next scheduled CRON job would report something like

[49000/13123]

...i.e. back to normal (the first number is the current progress, the second number is the last number imported for that feed).

If that's not the case, if you could email me a link to the site and a copy of the CRON command / if applicable fetch.sh script being invoked and I'll check it out further...

Cheers,
David.
--
PriceTapestry.com