slow import tool via cron or command line
Hi
The slow import tool helps keep my server load down, is there any way to get the similar affect via command line/cron so I do not have to have a browser window open.
thanks
brent
Is there a way to get this slow import tool implemented on the previous version of PT with a cron? Also regarding http://www.pricetapestry.com/node/2554, does the temporary table work well with 250K plus products. I am trying to see what works better with cron jobs not to overload server resources with my old PT sites.
Hi,
Yes - I have a version of the slow import tool for use with the original version of Price Tapestry. As above, if you want to email me one of your fetch scripts I'll add the required code (it requires a wget call rather than executing the command via PHP).
The temporary table mod will also work with this version if your site is exclusively CSV feeds, as the slow import tool is not compatible with XML. If that's OK, if you could also send me your includes/admin.php i'll add that at the same time for you...
Cheers,
David.
Hi David,
I emailed you for your mod that allowed me to run the slow import by cron. It worked for all my feeds except one that had 263,000 products (efaucets). With efaucets, it seemed to import the products (although I'm not entirely sure) but it didn't show any imported products in the admin page.
The reason I'm not sure is becausse of this:
Before the import, I would have 463,000 products. after the import, it would show about 200,000 products, but, the database actually showed 463,000 products, i.e. the 263,000 may or may not have been imported b/c it already existed there, but, the total products would get reduced in the admin page.
Do you know what the problem might be?
BTW, slow import run manually does work.
Thanks.
Hi,
That sounds like the redirect limit was reached when wget is making the repeated calls to fetch feeds_import_slow.php. I would suggest increasing the value of $config_slowImportBlock in config.advanced.php, which will result in less repeated requests. It can probably be increased from 100 significantly; try it out using the web based slow import on a smaller feed, with perhaps a value of 500 or maybe even 1000 - (so use a feed with a least more products than the test value...)
Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
I cannot seem to find my registration code to download the latest version of the script which includes the slow import tool. Also will this enable me to import feeds from via CRON JOB in a php script as I no longer has access to shell.
Thanks,
Simon
Hi Simon,
I have a version of the Slow Import Tool re-worked for the original distribution that I will send you now, plus I'll retrieve your registrations details and forward those also...
Cheers,
David.
--
PriceTapestry.com
Hello Brent,
Sure - if you could email me one of your existing fetch / automation scripts I'll add the code that will enable slow import to be run via cron...
Cheers,
David.