Hi David,
Sorry for so many questions but this one might be pretty easy lol.
I work with 100's of merchants so who have very large data feeds up in the 8GB to 10GB range.
I can see this causing me issues with my server that actually runs my websites by using up all my storage space.
Are there any options available where I could have the data feed files downloaded automatically elsewhere therefore not using up my main server storage space?
Best Regards
David
Hi David.
I ended in this thread looking for a solution to reduce the local disk usage during import. Sometimes the server crashes as there is not free space as all the feeds are piled up.
But before implement this.
What are the pros or cons of using this method rather than the default one via cron?
Thank you in advance.
Hi,
There's no downside to this approach at all vs the default - once feeds are imported their only purpose is to drive what is displayed on the /admin/ home page (Register, De-Register etc.) and that can be done by way of the empty placeholder files created by this mod...
Cheers,
David.
--
PriceTapestry.com
Hi David,
Feeds only need to be present whilst being imported, however a file of the registered feed filename should always exist in the /feeds/ directory as if it were not preset the de-register function would be displayed on the /admin/ home page and could be invoked accidentally.
So with that in mind, it is a straight forward modification to scripts/cron.php to change the process so that feeds are fetched and imported individually, and following the import the file is replaced with an empty file of zero bytes so that you never have feeds consuming space in your hosting account (other than a single feed as it is being imported). To try this, edit scripts/cron.php and look for the following code beginning at line 61:
$sql = "SELECT * FROM `".$config_databaseTablePrefix."jobs` ORDER BY filename";if (database_querySelect($sql,$jobs))
{
foreach($jobs as $job)
{
fetch();
}
}
...and either comment out or DELETE that section.
Then look for the following code beginning at (now) line 81:
if (file_exists($config_feedDirectory.$feed["filename"])){
import();
}
...and REPLACE with:
$sql = "SELECT * FROM `".$config_databaseTablePrefix."jobs` WHERE filename='".database_safe($feed["filename"])."'";if (database_querySelect($sql,$jobs))
{
$job = $jobs[0];
fetch();
}
if (file_exists($config_feedDirectory.$feed["filename"]))
{
import();
}
$fp = fopen($config_feedDirectory.$feed["filename"],"w");
fclose($fp);
Hope this helps!
Cheers,
David.
--
PriceTapestry.com