Hello David,
I dont have problems to import small feeds (.csv).
But If I want to import large csv files over 300 MB, after 10 min. the WinXP Download-Window opens and want to Download a file called feeds_import.php.
Do you know this problem?
David is spot on this one. I had the same issue with a 25MB feed.
Hello David,
You sent me an alternative version of Magic Parserthis version last week but I got errors. Sure, I try to import via the command line but I get errors which I posted here:
http://www.pricetapestry.com/node/630
I dont think that is a time error, my php.ini has:
post_max_size = 2000M
upload_max_filesize = 2000M
max_execution_time = 6000000 ; Maximum execution time of each script, in seconds
max_input_time = 120000000 ; Maximum amount of time each script may spend parsing request data
memory_limit = 2000M
I can import the complete database of wikipedia which is 7,8 GB via command line. But I can't work with pricetapestry via command line.
I had send you the wikimedia for 4 days, you can remeber?
Maybe you can help?
Thanks!
Regards,
Hani
Hi Hani,
I had a look at the wikipedia import; and I think it actually imports by calling a routines running on the site via HTTP - it doesn't use MySQL via the command line at all - it only looks like it does. Your CLI version of PHP would need to be recompiled with MySQL support to make this work.
The best thing to do would be to contact your hosting support desk and explain that you need to use PHP with MySQL from the command line; and that MySQL is not compiled in. They should be able to help; otherwise i'm not sure what to suggest. Let me know if they are not able to help and i'll have a think about possible alternatives, but at this time i'm not sure what to suggest. It is certainly not practical to import 300MB via the browser as you are likely to reach timeouts not only within your own system but also in any proxy server between you and your sute - such as a transparent proxy being run by your ISP...
Let me know if your hosting company is able to help and we'll take it from there....
Cheers,
David.
Dear David,
you wrote:"Your CLI version of PHP would need to be recompiled with MySQL support to make this work".
How can I test it, it is compiled or not?
It is registered in phpinfo.php?
<?php
echo phpinfo();
?>
I ask my serverhouser last week, but he said, that is compiled.
Regards,
Hani
Hello hani,
We discovered that MySQL was not compiled into your CLI version of Apache using the version test script in the following thread:
http://www.pricetapestry.com/node/630
It is worth sending that script to your host, and also make sure that they understand that it is the command line version of PHP, not the Apache Module.
Cheers,
David.
Hi hansi,
300MB is enormous and will almost certainly reach a browser timeout, let alone any server side (PHP or Apache) timeout. A feed of this size should really be imported via the command line - you will need to login to your server / hosting account to do this - full details here:
http://www.pricetapestry.com/node/582
I will also email you an alternative version of Magic Parser that is faster with CSV files (but less robust) which may help, but I still doubt that you would be able to import a 300MB CSV file through the browser - but it might be worth trying first...
Cheers,
David.