You are here:  » Automatic download of updated feeds


Automatic download of updated feeds

Submitted by PriceSpin on Wed, 2006-03-29 17:19 in

Can anyone tell me how I can add a feature that once I have registered a feed and imported it for the first time, that I can also give PriceTapestry a URL to download the feed say every 5 days automatically and import it when done?

Cheers
Robert

Submitted by support on Wed, 2006-03-29 17:23

Hi Robert,

I'm going to write an article on this as it's something most users are going to want to do. Quick questions;

1) Do you have SSH or Telnet access to your server?

2) If you do, do you have permission to use the wget command?

Submitted by PriceSpin on Wed, 2006-03-29 17:25

I have root ssh on my server and I do have wget as well...

Cheers David

Submitted by support on Wed, 2006-03-29 17:28

Excellent - and presumably then you have the ability to setup CRON jobs...?

It basically boils down to scheduling a CRON job to use wget to download the feeds; and then calling the import.php automation script which is part of the Price Tapestry distribution:

http://www.pricetapestry.com/node/7

I'll put some more comprehensive notes together on this.

Submitted by PriceSpin on Wed, 2006-03-29 17:35

Thanks David, once you have the guide together Ill implement it straight away...

Submitted by PriceSpin on Fri, 2006-04-14 22:25

any updates on the guide for this?

Submitted by support on Sat, 2006-04-15 06:35

Sorry about the delay in writing this; i'll try and have a go this weekend.

Submitted by philstone on Sat, 2006-04-15 07:54

I have a cron job set up but it doesn't seem to run properly

it is :

cd /home/buy247/public_html/scripts; /usr/local/bin/php -f /home/buy247/public_html/scripts/import.php aw1.php

the code in aw1.php is

<?php
function download($file_source, $file_target) {
       $rh = fopen($file_source, 'rb');
       $wh = fopen($file_target, 'wb');
       if ($rh===false || $wh===false) {
// error reading or opening file
           return true;
       }
       while (!feof($rh)) {
           if (fwrite($wh, fread($rh, 1024)) === FALSE) {
                   // 'Download error: Cannot write to file ('.$file_target.')';
                   return true;
               }
       }
       fclose($rh);
       fclose($wh);
       // No error
       return false;
   }
//AW feeds
download('http://products.affilite.com/xmlproductoutput.php?mid=***&user=******&password=******&nozip=1','feed1.xml');
download('http://products.affiliate.com/xmlproductoutput.php?mid=***&user=******&password=******&nozip=1','feed2.xml');
?>

does anyone see why this wouldn't work properly

also, can a script like this be made for tradedoubler that imports the files - gunzipping them as they are imported to my webspce?

Regards

Phil

Submitted by support on Sat, 2006-04-15 08:09

Hi Phil,

I'll cover unzip and gunzipping in the tutorial.

With regards to your script; I notice that you've cron'ed the import.php command, but with the parameter being the name "aw1.php". The parameter to import.php is either a filename, or @ALL or @MODIFIED.

Assuming that your aw1.php script is in the scripts directory, my guess from looking at what you have above is that the cron statement should be (with newlines at each semi-colon for clarity):

cd /home/buy247/public_html/scripts;
/usr/local/bin/php -f /home/buy247/public_html/scripts/aw1.php;
/usr/local/bin/php -f /home/buy247/public_html/scripts/import.php @MODIFIED;

...so the process is 1) Change directory to "scripts", 2) Run your aw1.php script to download the files, and finally 3) run import.php @MODIFIED to import any updated feeds.

Submitted by philstone on Sat, 2006-04-15 09:07

thanks Dave

Working 100% now, thanks AGAIN!

Submitted by support on Sat, 2006-04-15 10:47

Hi,

First draft of a tutorial now posted here:
http://www.pricetapestry.com/node/198

Submitted by PriceSpin on Sat, 2006-04-15 15:12

Thanks david, looks pretty straight forward... gonna give it ago today.

Robert