Hi David
I import every 3 days and I am getting lots of 404's when products are no longer in the feeds, however, they sometimes reappear in later feeds. I at present delete all products before reimporting them, I know that I could use modified, would this keep the product in the database if it was not in the latest feed. Also could the price be removed and changed to something like "check availability"
I found a mod on 'node 4608', would this work and if so would I have to use modified instead of all in the imports.
Brent
Hi David,
I have one merchant in particular who's feed keeps dropping, i.e. one day it's full, the next day the products disappear.
How would I go about always keeping a product in a database and therefore keeping the product page - even if the product is no longer for sale? Maybe replacing the price with a custom message or after a certain period, removing the price/merchant listing all together?
Additionally, is a mod available that would point the user in the direction of a newer product? Say for example a user lands on the iPhone 4S product page, a message at the top of the product page could point the user to the new iPhone 5 page?
Thanks in advance.
Hi Marcus,
It's not totally straight forward to keep expired products in the database since the script is designed to tightly map the product database with your feeds - which is how everything is kept up to date - but the modification described in this thread would go a long way towards that goal. With `timestamp` and `deleteme` fields on the database, if you simply removed the following lines from the modification:
$sql = "DELETE FROM `".$config_databaseTablePrefix."products` WHERE deleteme=`1`";
database_queryModify($sql,$insertId);...deleted products would then stay in the database, and you could use the `deleteme` flag on the product page (html/product.php) and in the price comparison table (html/prices.php) to make sure that you don't display the product link etc. e.g. in place of the following code in html/prices.php:
<td align='center'><nobr><a href='<?php print tapestry_buyURL($product); ?>' <?php print javascript_statusBar("go to ".$product["merchant"]); ?>><?php print translate("Visit Store"); ?></a></nobr></td>...you might use:
<td align='center'>
<?php if($product["deleteme"]): ?>
Product Not Currently Available
<?php else: ?>
<nobr><a href='<?php print tapestry_buyURL($product); ?>' <?php print javascript_statusBar("go to ".$product["merchant"]); ?>><?php print translate("Visit Store"); ?></a></nobr>
<?php endif; ?>
</td>Cheers,
David.
--
PriceTapestry.com
Hi David,
Thanks for your help with this - that mod seems to work well except for 2 potential issues;
1) I had a set of feeds that completely dropped of the merchant network (I wasn't aware) and when importing them, the file in /feeds/ just contained the server error message. This resulted in all the merchant products disappearing from the site (despite the above mod). Do you think this could be down to the fact that the "feed" file changed format? Or something else? Either way, how can I stop this and similar issues from resulting in all products being removed from the respective merchant?
2) How does this work with mapped products? For example, if I map a product after it's been removed from a merchant feed? - I'm currently having issues mapping products, for example;
This product ({link saved}) is mapped to the following page {link saved} - but is still appearing on it's own.
Additionally, another set of products that I have mapped to the above URL are not doing so; {link saved}
Many thanks in advance for your help.
Marc.
Hi Marc,
I spotted a logic error in the code constructed the update / insert SQL, which I have now corrected in /node/3291 it's the following replacement which goes within the admin__importRecordHandler function in includes/admin.php:
global $now;
if (!$now)
{
$now = time();
}
$sql .= ",deleteme='0'";
$sql2 = "SELECT id FROM `".$config_databaseTablePrefix."products`
WHERE original_name='".database_safe($importRecord["original_name"])."' AND merchant='".database_safe($importRecord["merchant"])."'
LIMIT 1";
if (database_querySelect($sql2,$result))
{
$search = "INSERT INTO `".$config_databaseTablePrefix."products`";
$replace = "UPDATE `".$config_databaseTablePrefix."products`";
$sql = str_replace($search,$replace,$sql);
$sql .= " WHERE original_name='".database_safe($importRecord["original_name"])."' AND merchant='".database_safe($importRecord["merchant"])."'";
}
else
{
$sql .= ",timestamp='".$now."'";
}
if (database_queryModify($sql,$insertId))
{
$admin_importProductCount++;
}Unfortunately this will require a "reset" of the products table so that the correct code can function from a fresh, full import. To do this, create a file truncate.php containing the following code:
<?php
require("includes/common.php");
$sql = "TRUNCATE `".$config_databaseTablePrefix."products`";
database_queryModify($sql,$result);
print "Done.";
?>
...upload and browse to once from the top level Price Tapestry installation folder, and then delete the file.
If either of the issues persists following a subsequent import after the clean import let me know and I'll investigate further for you...
Cheers,
David.
--
PriceTapestry.com
Thanks David.
That seemed to fix both the issues I had, however I have discovered another (overlooked) one. Using a Filter to Drop a particular product no longer works if it's already been imported to the site.
Any idea on how to fix this?
Thanks for your help.
Hi Marcus,
Ah yes - with no delete taking place a subsequent filter triggering the drop record flag would have no effect so the record needs to be "physically" deleted at that point.
In includes/admin.php, first look for the following code at line 231:
if ($filter_dropRecordFlag) return;...and REPLACE with;
/* if ($filter_dropRecordFlag) return; */(or you could just delete the line!) then look for the following code at line 342:
if (!$importRecord["merchant"]) return;...and REPLACE with:
if (!$importRecord["merchant"]) return;
if ($filter_dropRecordFlag)
{
$sql = "DELETE FROM `".$config_databaseTablePrefix."products` WHERE merchant='".database_safe($importRecord["merchant"])."' AND original_name='".database_safe($importRecord["original_name"])."'";
database_queryModify($sql,$result);
return;
}...should do the trick!
Cheers,
David.
--
PriceTapestry.com
Perfect, thanks David.
I also asked earlier in the thread about the possibility of pointing a user towards a new version of a product (if available);
"Is a mod available that would point the user in the direction of a newer product? Say for example a user lands on the iPhone 4S product page, a message at the top of the product page could point the user to the new iPhone 5 page?"
If no previous mod exists then no worries, I did search around but couldn't spot anything.
Hi Marcus,
That would be straight forward to add custom text file includes - let me you know if that would be suitable; otherwise related products would generally point to newer versions of the same product if available...
Cheers,
David.
--
PriceTapestry.com
Ah yes, good points.
Another issue I have come across is very very slow imports. I'm not sure if this is related, but it's started happening recently and a back and forth with my host can't find any issue with the server.
Is there anything that has changed that could be causing this? Previously a feed of 600k+ products would take 5 minutes to import, now it's taking hours to get a fraction imported.
Any ideas would be appreciated.
Marc.
Just to confirm, I have removed the mod on my test server and import speeds went back to normal.
Hi Marco,
Was it only after the final fix in this comment that the speed dropped or after the initial changes to update / delete from delete / insert?
Cheers,
David.
--
PriceTapestry.com
Hi Marcus,
Ah - the products table could use a merchant,original_name index which should make a significant difference with this mod in place.
To add the index, use the following dbmod.php script. Create the file, upload to the main Price Tapestry installation folder then browse to the file once. After "Done." is displayed, delete the file.
<?php
set_time_limit(0);
require("includes/common.php");
$sql = "CREATE INDEX merchant_original_name
ON `".$config_databaseTablePrefix."products` (merchant,original_name)";
database_queryModify($sql,$result);
print "Done.";
?>
Hope this helps!
Cheers,
David.
--
PriceTapestry.com
Thanks David.
Unfortunately I applied the mod and re-registered a feed, it is still as slow as previous using the slow import tool...
Hi Brent,
One simple thing to do is simply to remove the 404 (Not Found) header from products.php so the page remains valid and won't result in 404 being reported by GWT for example; users may still find what they are looking for by way of Related Products and the text can be changed easily. In that file, look for the following code at line 72:
header("HTTP/1.0 404 Not Found");$banner["h2"] .= "(".translate("product not found").")";
...and REPLACE with just
$banner["h2"] .= "(".translate("product not currently available").")";It would be possible to keep product records based on a slightly different version of the update / insert instead of delete / insert mod (essentially, setting price to 0.00 instead of deleting records where deleteme is still set to 1 after the import) and then making a number of changes to the display code to handle accordingly. There would however be a number of issues that would require consideration such as search results (unfortunately it's extremely inefficient to exclude results where price = 0.00) and also the possibility of image_url's being invalid when products are not currently stocked etc. but if you want to give something like that a go let me know and I'll point you in the right direction...
Cheers!
David.
--
PriceTapestry.com