My new installation is struggling to load large csv files - anything over 300 or so products is unreliable, and anything over 1000 usually times outbefore all of the products have loaded - is there a way to increase the timeout period for loading files?
Hi
i just signed up to your code, i'm trying it out on site: http://www.buy24-7.co.uk
uploaded a couple of feeds , then i was uploading one and this came up;
Warning: set_time_limit() has been disabled for security reasons in /home/fhlinux199/b/buy24-7.co.uk/user/htdocs/admin/feeds_import.php on line 8
Fatal error: Maximum execution time of 30 seconds exceeded in /home/fhlinux199/b/buy24-7.co.uk/user/htdocs/includes/admin.php on line 134
what are the best ways to fix these issues
kind regards
philip stone
ps. I must say your code is excellent for the price of it, i have been trying another code i got on theselect.com, and believe me yours is much easier to use! and less time consuming!
Hi Phil,
This is the same issue that other users are having where thier web hosting company has limited the amount of time that a PHP script can run.
The import script does set the maximum, but your host does not allow it. Have you been able to try the command line script (/scripts/import.php ) as per the post above?
I'm going to work on a work-around for users on restricted PHP installations today. In the mean time, you might be able to email your host and ask for the restriction to be turned off on your account....
My server doesn't have shell access so i'm struggling a little - I'm trying putting it through a cron job which I can turn on and off when I need to but that doesn't seem to want to work yet. Are there any other ways of executing import.php?
David,
sorry to post in such an old thread - but this was the only related one I could find!
did you ever implement the magicparser_parse() offset ? or else is there any other way of getting round the problem of only needing part of a very large feed. play.com xml feed has over 90,000 records for example - this takes a few hours to process on my system!!
thanks,
Steve.
Hi Steve,
I'm afraid this isn't turning out to be as straight forward as I'd hoped due to the way that PHP's internal XML parser operates. What i'm looking at doing is releasing a slow import tool for CSV files only at first, and then hoping to extend this to XML in due course...
Cheers,
David.
Hiya,
I've been looking at this issue with a number of users, and it turns out that it is quite common for webhosts to fix the maximum execution time for a PHP script.
The import script already has set_time_limit(0) at the top; so a timeout should indicate this being the problem. I am thinking of ways to get around this, which will probably involve splitting the import into chunks of, say, 100 products.
Of those with the web based timout problem, most are finding that the command line script works OK (/scripts/import.php [docs]) - it also produces a nice output to show the import progress (which also helps keep your terminal connection aline).
Let me know how you get on with the import script, and i'll get thinking about the phased import. I need to write a mod to Magic Parser in order to do this; because it will take almost as long for Magic Parser to "seek" to, say, record 5000 as it does to import the products. I can see what to do; basically, I need to record an offset into the file at the point at which you tell Magic Parser to stop; and then add a new parameter to the MagicParser_parse() function that can give an offset into the file from where to begin parsing.
I think it can be done :)