You are here:  » Importing pseudo-XML (php) feeds


Importing pseudo-XML (php) feeds

Submitted by madstock on Wed, 2006-09-13 15:54 in

By slightly butchering the original sitemap.php file, I am able to generate a file named "feed.php", that gives our indexed product name, and URL :

<?php
  require("includes/common.php");
  require("includes/database.php");
  header("Content-Type: text/xml");
  print "<?xml version='1.0' encoding='UTF-8'?>";
    print "<products>";
    $sql = "SELECT name FROM `".$config_databaseTablePrefix."products` ORDER by search_name LIMIT 50000";
    if (database_querySelect($sql,$rows))
    {
      $sitemapBaseHREF = "".$config_baseHREF;
      foreach($rows as $row)
      {
        print "<product>";
        {
          $sitemapHREF = "products.php?q=".urlencode($row["name"]);
        }
        print "<productname>".$row["name"]."</productname>";
        print "<deeplink>".$sitemapBaseHREF.$sitemapHREF."</deeplink>";
        print "<price>0.0</price>";
        print "</product>";
      }
    }
    print "</products>";
?>

This is fine and dandy, displays the results as required, and if I need to download to my local PC, it shows itself to be an XML file, which is great.

If I wish to automate download of this file across to another version of Price Tapestry ( a sort of "main index" that will index the indices, if you will), using the following code, saved as downloadfeeds.php:

<?php
function download($file_source, $file_target) {
       $rh = fopen($file_source, 'rb');
       $wh = fopen($file_target, 'wb');
       if ($rh===false || $wh===false) {
// error reading or opening file
           return true;
       }
       while (!feof($rh)) {
           if (fwrite($wh, fread($rh, 1024)) === FALSE) {
                   // 'Download error: Cannot write to file ('.$file_target.')';
                   return true;
               }
       }
       fclose($rh);
       fclose($wh);
       // No error
       return false;
   }
//Feeds
download('url/to/feed.php','feed.xml');
download('url/to/next/feed.php','feed2.xml');
?>

I run into trouble, as I am given the error message:

Warning: fopen(url/to/feed.php): failed to open stream: Connection refused in /home/sites/sitename/public_html/subdirectory/feeds/downloadfeeds.php on line 3

I would imagine that this would be because the "feed.xml" (i.e. feed.php) is created on-the-fly, and as such is not a file full of lovely links. (I have tried using full URLs and relative links, changing the name from feed.php to feed.xml etc.).

I have therefore 2 questions to put before your collective brilliance:

1. How can I automatically grab this stuff to stick across to another folder, and
2. Where I am currently using:

print "<price>0.0</price>";

in the original script, I would love to be able to display the price, too - I have tried all manner of variations on

        print "<price>".$row["price"]."</price>";

to no avail.

Would anyone please be able to help?

Cheers!

Submitted by madstock on Wed, 2006-09-13 16:13

Addendum: To save the "download", if I change the paths to common.php and database.php to their full path, I am able to physically place the feed.php file in the /feeds/ folder.

However, as it is not an XML file, it is not recognised by PT - do I need to get the original file to churn out a physical XML file in some way?

Submitted by support on Wed, 2006-09-13 16:37

Hi,

Regarding this error:

Warning: fopen(url/to/feed.php): failed to open stream: Connection refused in /home/sites/sitename/public_html/subdirectory/feeds/downloadfeeds.php on line 3

...I don't think that's anything you're doing wrong - that's down to the configuration of your server for some reason refusing to serve HTTP to destination server.

Are the two instances on the same server? It could be a local firewalling issue in that Apache is configured not to accept connections from localhost/127.0.0.1 for example.

However, the first test I would do would be try and fopen() some other URL that you know has no problem, for example one of the BBC News RSS feeds:

http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml

If you can fopen() that, then I think it's a firewall problem on your source server. If you cannot fopen() the BBC link, then you want to be looking at the PHP configuration on the destination server, although it does sound like URL wrappers are enabled.

Cheers,
David.

Submitted by madstock on Wed, 2006-09-13 17:21

Thanks for the swift response, as always!

I have swapped the fopen urls back to relative paths as opposed to URLs, and the errors magically disappeared - however I end up with an 823 byte file as it "downloads" the actual script as opposed to the output of the script.

Would there be any way in php of doing the following two steps:

1. Running the script
2. Saving the output of the script to another file (in the /feeds/ folder if possible)?

Sorry if this is way beyond the scope of product support.

EDIT: I forgot to mention that the script itself works fine when being accessed through a browser.

Submitted by support on Wed, 2006-09-13 19:06

If you are able to run the exec() function you should be able to do something like this:

<?php
  exec
("php -q /path/to/script.php > /path/to/feeds/output.xml");
?>

The -q will stop PHP generating the HTTP headers, so just the raw output should be generated, which is then piped in a destination file using the > character.

Hope this helps!
David.

Submitted by madstock on Wed, 2006-09-13 19:30

Thanks David, that is fantastic!

The exec function is disabled on the server (along with passthru and any sort of alternative), however I can run that from the command line, and it works a treat. (It will be used as part of the ongoing CRONs anyway).

(In a Columbo style) Just one more thing - would it be possible to include the price in the original feed.php file?

I am currently defining product name, deeplink and a price as follows:

print "<productname>".$row["name"]."</productname>";
print "<deeplink>".$sitemapBaseHREF.$sitemapHREF."</deeplink>";
print "<price>0.0</price>";

But would ideally like to include the correct value for "price".

Thanks again for everything that you have done,

Duncan

Submitted by support on Thu, 2006-09-14 08:54

Hi Duncan,

Price is in the products table, so you should be able to do it like this - minor modification to your original code:

<?php
  require("includes/common.php");
  require("includes/database.php");
  header("Content-Type: text/xml");
  print "<?xml version='1.0' encoding='UTF-8'?>";
    print "<products>";
    $sql = "SELECT name,price FROM `".$config_databaseTablePrefix."products` ORDER by search_name LIMIT 50000";
    if (database_querySelect($sql,$rows))
    {
      $sitemapBaseHREF = "".$config_baseHREF;
      foreach($rows as $row)
      {
        print "<product>";
        {
          $sitemapHREF = "products.php?q=".urlencode($row["name"]);
        }
        print "<productname>".$row["name"]."</productname>";
        print "<deeplink>".$sitemapBaseHREF.$sitemapHREF."</deeplink>";
        print "<price>".$row["price"]."</price>";
        print "</product>";
      }
    }
    print "</products>";
?>

Should do the trick...
Cheers,
David.

Submitted by madstock on Thu, 2006-09-14 09:20

Thanks again, works a treat!