Hi David,
First off: Great script, also this is my first post,
sorry if this is posted else where, but the forum isnt exactly great for searching.
Ive been trying to generate a quick and easy way to update all my feeds, using just php,
the setup i have at the moment is a wordpress site with the PT inside a sub folder, (/compare/)
Just now I have the script grabing the feed (.zip)
extracting that file on the server then moving it to my feeds folder, (.csv)
Im running into trouble when trying to register and import the feed.
Each time the script is run the csv file is visible in the admin area, it says its registered,
but if you manualy click on import it dosnt work, im guessing its down to when its being registered,
PHP EXAMPLE
<?php
// this was copied from one post,
admin_register("datafeed.csv","csv|124|1|0","TEST_FEED","PRODUCTNAME","DESCRIPTION","IMAGEURL","DEEPLINK","PRICE", "","","","");
?>
<?php
// changed to this to match what i though would be the csv contents?
admin_register("datafeed.csv", "csv|124|1|0","TEST_FEED","merchant_name", "PRODUCTNAME","DESCRIPTION","aw_image_url","aw_deep_link","PRICE", "","","","");
?>
The script file is located in the root of the PT install,
Inside admin home i can see the feed, if I click register it presents me with the csv|124|1|0 autodetect format (Which ive copied to the php code above), clicking next I have to match up the merchant name, image, and deep link, im sure this is were its all going wrong?
I've included the full source code for this test page below.
Im hoping to tap into the wordpress cron feature and run this on a daily/weekly basis?
COMPLETE AUTOMATION CODE
{code saved}
thanks for your reply David,
Ok here is were im up to...
Ive moved my test file into /scripts/test.php
fixed my urls using the ../
my test page will ouput the following
File Download: DONE...
File Saved: ../feeds/datafeed_89725.csv
File Unzip: DONE...
Remove Temp File: ../temp/datafeed.zip DONE...
Register Feed: DONE...
Import Feed: DONE...
Import Reviews: DONE...within the admin area i can see the feed listed
and all the links are clickable, ( Register | Filters | Import )
on clicking register the feed format page will show that Im using the csv|124|1|0 format.
clicking next shows that some fields are not being matched...
Merchant Name, Image URL, Buy URL,
is it the php code that should change to reflect the currently used csv file?
<?php
admin_register("datafeed_89725.csv", "csv|124|1|0","TEST_FEED","PRODUCTNAME","DESCRIPTION","IMAGEURL","DEEPLINK","PRICE", "","","","");
?>
<?php
admin_import("datafeed.csv");
?>
Hi,
Could you post the URL to your test feed for me (i'll remove the link
before publishing your post) and I'll download the feed to my test
server and check the registration fields for you...
Cheers,
David.
--
PriceTapestry.com
Hi,
Thanks for that. Note that quoted text is not auto-detected if the
header row itself is not quoted, so the correct Format String is
actually:
csv|124|1|34(quoted text, header row, pipe separated)
With that in place, the field mapping should be as follows
Merchant Name: merchant_name
Product Name: product_name
Description: description
Image URL: merchant_image_url
Buy URL: aw_deep_link
Price: search_price
Category: merchant_category
Brand: brand_name
..now my apologies, I'd overlooked that the code you have found
from the forum refers to an earlier version of Price Tapestry
(something else I shall be looking into soon with the
forthcoming release), making the call to admin_register()
require you to populate a $registerFields[] array.
This would make your automation script registration code as follows:
<?php
$registerFields = array();
$registerFields["name"] = "product_name";
$registerFields["description"] = "description";
$registerFields["image_url"] = "merchant_image_url";
$registerFields["buy_url"] = "aw_deep_link";
$registerFields["price"] = "search_price";
$registerFields["category"] = "merchant_category";
$registerFields["brand"] = "brand_name";
admin_register("datafeed_89725.csv","csv|124|1|34","","merchant_name",$registerFields,"","");
?>
Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David
again thanks for the reply..
I've replaced the code with the snippet above, still no luck,
the csv is still marked as registered, and all links are clickable,
is it the formating of the CSV file, would it be beneficial to perhaps change the formating which is output from AW?
currently like you said its in PIPE format?
Im not sure if it helps but the PT version im using is:
Revision 11/09A Copyright (c) 2005-2010 IAAI Software, All Rights Reserved
PHP Version 5.2.9
MySQL Version 5.0.45-log
Install Path /var/www/vhosts/example.com/httpdocs/compare/
regards
M
Hi,
When I tested your feed on my test server I noticed that the unzipped
file contains Unix permissions, and is set to Owner access only.
Before the script could access the feed, I had to make it "world"
readable, so that may be the problem you are experiencing.
To see if this is the case, look at the unzipped file in the remote
window of your FTP program and look for "Permissions", or maybe
"Properties..." and then "Permissions".
Check that the file has read access to all users - Owner/Group/Word.
What I suspect may be happening is that the file is being unzipped
with owner access, which could mean that PHP is subsequently unable
to access the unzipped file.
I notice that your script is unzipping using PHP's internal zip
functions instead of shelling out to unzip (by which means it is
possible to unzip to stdout and subsequently pipe to a file that
would be guaranteed to be readable) so this shouldn't be a problem,
but could you perhaps check that this is working correctly and the
permissions are as expected.
If still not sure; could you post the URL to your site and the filename
as unzipped in your /feeds/ directory and I'll take a look at that
directly... (i'll remove your URLs of course before posting as
always)
Cheers,
David.
--
PriceTapestry.com
Hi David,
Fantastic, that did the job..
the feeds are now downloading, unzipping, registering and importing all just with a run of the php file.
require("../includes/common.php");
require("../includes/admin.php");
require("../includes/filter.php");
require("../includes/MagicParser.php");
$registerFields = array();
$registerFields["name"] = "product_name";
$registerFields["description"] = "description";
$registerFields["image_url"] = "aw_image_url";
$registerFields["buy_url"] = "aw_deep_link";
$registerFields["price"] = "search_price";
$registerFields["category"] = "merchant_category";
$registerFields["brand"] = "brand_name";
admin_register("datafeed_89725.csv","csv|124|1|34","","merchant_name",$registerFields,"","");
echo "<br />Register Feed: DONE...";
admin_import("datafeed_89725.csv");
echo "<br />Import Feed: DONE...";
admin_importReviews();
echo "<br />Import Reviews: DONE...";ok, yea Im a muppet, I've been working with two different test pages.
and yea i was still updating and uploading to the root of PT,
then i noticed I was running the same script inside the /scripts/ folder :|
so added the new code and away it went.
thanks a lot...!
now all I have to do is...
hook into wordpress and use the cron feature to run it for me :)
Hi David,
I am having a similar problem. I'm using a php script to download the zip and unzip it but for some reason the import command doesn't import the script. Am I doing something wrong or is there another way i can import the csv like calling the import script itself the with the csv in the url?
<?php
#!/usr/bin/php -q
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/MagicParser.php");
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/common.php");
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/admin.php");
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/filter.php");
$workingDir = "{code saved}"; // must be writable by PHP
$url = "{code saved}"; // compressed document to fetch
// fetch document to working directory
$filename = $workingDir."car.csv";
if (!copy($url,$filename))
{
print "File download failed - check permissions!";
}
// shell out to unzip command
$cmd = "/usr/bin/unzip -p ".$filename." > ".$filename.".unzipped";
exec($cmd);
unlink($filename);
rename($filename.".unzipped",$filename);
// temp.xml should now be unzipped, and can be parsed as normal, so take
// replace the remainder with your existing code - feed should be in $filename
admin_import($filename);
echo "<br />Import Feed: DONE...";
admin_importReviews();
echo "<br />Import Reviews: DONE...";
function myRecordHandler($record)
{
// process $record
}
MagicParser_parse($filename,"myRecordHandler","csv|124|1|34");
?>
Hi,
First of all, you can remove this section from the end as it's not
required...
function myRecordHandler($record)
{
// process $record
}
MagicParser_parse($filename,"myRecordHandler","csv|124|1|34");What looks like the problem is that the admin_import() function
accepts the filename without any directory prefix, so in this case
simply "car.csv"...
The way you could modify this with least code changes would be to
replace this line:
admin_import($filename);...with:
$filename = str_replace($workingDir,"",$filename);
admin_import($filename);Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
That worked in so much that within the admin it says the feed is imported but it says it has 0 products. Do I need to register the feed and fields each time in this script like so?
$registerFields = array();
$registerFields["name"] = "product_name";
$registerFields["description"] = "description";
$registerFields["image_url"] = "aw_image_url";
$registerFields["buy_url"] = "aw_deep_link";
$registerFields["price"] = "search_price";
$registerFields["category"] = "merchant_category";
$registerFields["brand"] = "brand_name";
admin_register("datafeed_89725.csv","csv|124|1|34","","merchant_name",$registerFields,"","");
Is it possible the feed isn't finished downloading before it is imported. Should I run a secondary script that does the importing like an hour later?
Thanks again, I'm nearly there with this.
Hi,
You shouldn't need to do that - I normally recommend that using PHP style automation such as this that you manually register feeds, and make sure that they import correctly via /admin/ before leaving the PHP script to do the download / import.
However, it just occurred to me that the includes are probably not including all the required files. In place of:
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/MagicParser.php");
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/common.php");
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/admin.php");
require("/hsphere/local/home/zcom/site.co.uk/shopping/includes/filter.php");...instead use:
$installDir = "/hsphere/local/home/zcom/site.co.uk/shopping/";
require($installDir."config.php");
require($installDir."config.advanced.php");
require($installDir."includes/admin.php");
require($installDir."includes/tapestry.php");
require($installDir."includes/database.php");
require($installDir."includes/admin.php");
require($installDir."includes/filter.php");
require($installDir."includes/MagicParser.php");Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi David,
I've added the extra includes but it's still showing 0 products. Here is my script now. The csv is readable by all.
<?php
#!/usr/bin/php -q
$installDir = "/hsphere/local/home/huntzcom/site.co.uk/shopping/";
require($installDir."config.php");
require($installDir."config.advanced.php");
require($installDir."includes/tapestry.php");
require($installDir."includes/database.php");
require($installDir."includes/admin.php");
require($installDir."includes/filter.php");
require($installDir."includes/MagicParser.php");
$workingDir = "/hsphere/local/home/huntzcom/site.co.uk/shopping/feeds/"; // must be writable by PHP
$url = "{link saved}"; // compressed document to fetch
// fetch document to working directory
$filename = $workingDir."car.csv";
//if (!copy($url,$filename))
//{
// print "File download failed - check permissions!";
//}
// shell out to unzip command
//$cmd = "/usr/bin/unzip -p ".$filename." > ".$filename.".unzipped";
//exec($cmd);
//unlink($filename);
//rename($filename.".unzipped",$filename);
// temp.xml should now be unzipped, and can be parsed as normal, so take
// replace the remainder with your existing code - feed should be in $filename
$filename = str_replace($workingDir,"",$filename);
admin_import($filename);
echo "<br />Import Feed: DONE...";
admin_importReviews();
echo "<br />Import Reviews: DONE...";
?>
Hi,
Could you confirm that when you import manually from /admin/ it does import correctly?
Cheers,
David.
--
PriceTapestry.com
Hi,
The script worked fine on my test server (all I changed was $installDir, $workingDir and $filename), but it just occurred to me that I have come across occasions where PHP is not configured to internally change directory to location of the script; so that might be the problem.
Where you have:
admin_import($filename);REPLACE that with:
chdir($workingDir);
admin_import($filename);Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Hi there,
Welcome to the forum and thanks for your comments!
I am aware of the search difficulties now that the forum is becoming so
large and will be looking to address this sortly.
Regarding the register / import not working; the admin functions are
designed to work from the level below the Price Tapestry installation
folder (e.g. /scripts/); in the first instance, please try moving your
script to that level (either /scripts/ or /admin/) and then in place of
this code:
require("includes/common.php");require("includes/admin.php");
require("includes/filter.php");
require("includes/MagicParser.php");
use:
require("../includes/common.php");require("../includes/admin.php");
require("../includes/filter.php");
require("../includes/MagicParser.php");
That should be all it is; but if you still find that although the
test feed is showing as registered (i.e. Import link is clickable)
within /admin/ let me know and I'll look into it further...
Cheers,
David.
--
PriceTapestry.com