My hosting provider has canceled all Cron jobs due to: shared hosting server heavy overload. CPU was used for 100%
I am running three scripts, once a day with a few hours interval.
The importall.php
<?php
header("Content-Type: text/plain");
passthru("php ../scripts/import.php @MODIFIED");
?>I do not do CRON jobs but I submit the script link in my browser, ending in @MODIFIED
So, is the best way to avoid overusage of CPU by using the same script link but @ALL? Or would using the Slow Import work better instead?
Hello Bat,
@ALL will always be much lower processor load as there is no need to DELETE per feed (the products table is emptied using TRUNCATE at the start) but this only applies to the regular import script. It is not by default enabled in slow import as that only works for CSV feeds so if you have XML feeds registered also they would not be imported.
However, if you do ONLY have CSV feeds on your site, then it is easy to modify admin/feeds_import_slow.php to TRUNCATE the products table when using @ALL. To do this, first look for the following code at line 394:
print "<p><a href='feeds_import_slow.php?filename=@ALL&startTime=".$now."'>mport All</a></p>";...and REPLACE with:
print "<p><a href='feeds_import_slow.php?filename=@ALL&truncate=1&startTime=".$now."'>mport All</a></p>";Finally, look for the following code beginning at line 165:
if ($_GET["filename"]=="@ALL")
{...and REPLACE with;
if ($_GET["filename"]=="@ALL")
{
if (isset($_GET["truncate"]))
{
$sql = "TRUNCATE `".$config_databaseTablePrefix."products`";
database_queryModify($sql,$result);
}Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hello Al,
The first tip for reducing load during import, if at all possible depending on your setup, is to use import.php @ALL instead of @MODIFIED. This is because deleting thousands of rows from an already very large is a very slow process.
However, I appreciate that it might not be desirable to have your site down for as long as it takes to perform the import, so there is a very nice modification described in the following thread that imports to a temporary table, and once completed switches it over to the live table. This will mean that you can then use @ALL without interruption to your site:
http://www.pricetapestry.com/node/3355
In addition, several users have found that inserting a 1 second sleep() every 100 products or so during the import also make a significant difference...
http://www.pricetapestry.com/node/1216
If you're not sure about any of the above modifications, email me the files across and I'll patch them for you...
Cheers,
David.