Hi David,
Couple of month's ago i moved to a vps server at eukhost (RAM = 400mb). Support is great and fast.
Allthough i am having issues on the import of products into my database. I have about 774.000 products now from 328 merchants on one particuler domain, on which i make the cronjob run on a weekly basis to update the database. Somehow this takes almost 1,5 days, and the control panel to plesk gets very slow. Yesterday all my sites were offline and control panels were inaccessible. This happened a couple of times already. I reckon it whas the import which overloaded the server.
Do you have any reccomendation on how to solve this?
Mike
also couldnt u run a cron job each night and only import 100,000 items each night which should only take an hour or 2
I used to have the exact same setup. Unfortunatly I gave up in the end. SQL would crash every few days no mater how I did the import. Tried running it with 2 databases alternating between imports. Tried importing a few feeds per day etc etc. If you continue with a VPS can I suggest you invest in some website monitoring - I use serviceuptime - about $5 a month.
Switched back to a reseler hosting account. I bought a PC and installed linux with assosiated Apache, PHP and MySQL. I now download, import and upload the database all from the local PC. It actually works out cheaper to do it that way but a bit of manual work required.
Could you elaborate on the costs of running your server from home?
Hi,
It's an interesting option that has described for operating a very large database without the need for high end hosting accounts - bear in mind that it's not actually serving your website from home, the principle is to use the local computer to run the import process, and then upload the completed database (I presume) to your main web server.
Essentially you could install Linux on any old PC - it's great for putting older machines into use again, so to that extent, it should cost much at all; but if you wanted to buy a PC specifically to run linux and setup your own server I would look for one advertised specifically with Linux installed - Dell offer Linux as an option on many of their computers.
Bear in mind that it is not a straight forward option; and would require knowledge of MySQL datatbase files, and process start / stop etc. so that the database can be uploaded to your VPS correctly - if you are having import problems it might worth looking into them with the usual remedies in the first instance... lots of information in this thread:
http://www.pricetapestry.com/node/582
...and don't forget of course that the updated distribution (which you may not be running having been a user for a few months now) includes the "Slow Import Tool" which enables you to import very large CSV feeds without any timeout problems at all. I have a version of this tool that is compatible with the previous version of Price Tapestry so drop me an email if you'd like to give that a go...
Cheers,
David.
Hello Mike,
With that many products / merchants, if you are not using @ALL it is almost certainly the DELETE part of the import of each merchant that is causing the slow down (I assume that you are not using import.php @ALL). If this is the case, I would recommend the modification described in the following thread which will import to a temporary table and then swap it over the live products table after import:
http://www.pricetapestry.com/node/2554#comment-10312
Cheers,
David.