You are here:  » Amounts of Results Returned from DB


Amounts of Results Returned from DB

Submitted by Ch3W on Sun, 2008-08-10 23:12 in

Hi there,

With my site, it is very frequent to have products with the same name that belong to different categories. I've noticed (and can totally understand) that the script does not return a product with the same name more than once when a user searches. Any way this can be lifted somehow?

As it stands, if a user searches for a product where the product name exists in more than one category, the product is only displayed for one category search and not all of the category searches that also have the same product name.

I hope I have explained this okay?

Thanks in advance.

Submitted by support on Mon, 2008-08-11 07:37

Hi,

If the product exists in different categories, it should appear in the search results for all categories that it is registered under. If you are finding this is not the case, if you could perhaps post or email to me a link to your site, and some example searches that highlight the problem I will have a look into it for you...

Cheers,
David.

Submitted by Ch3W on Mon, 2008-08-11 18:02

**** I'm not sure if I submitted this post already, apolgoies for possible re-post *******

Hi David,

I've spent the day drilling into this as much as I could...
It seems I have the following problem:

If a product/data feed contains more than one product with the same product name (different merchants & categories, just the same product name), it does not get imported into the database.

I have ran multiple tests to prove this on two hosts (dreamhost.com and 1&1 here in the UK), both with fresh installations and datafeed formats in both XML and CSV.

How I came to this conlcusion:
* I have a data feed that contains products with identicle names for multiple categories.
* I ensured that a particular product already existed on the datafeed before importing it into the database
* After importing, the product in mention did not get imported into the database
* On the datafeed, I noticed that the same product name existed for two different categories
* I manually edited the datafeed, changing the name of the product that was not being imported
* I then re-imported the datafeed, and the product had then been imported under its new name successfully

After that, I did the following:
* I created a mini-feed with only 10 products on, containing one product that contained the same name as another, but in different categories
* I repeated the tests above, and had exactly the same results.
* I then noticed that when two products with the same name are imported from a datafeed, the product that is listed first on the feed (from top to bottom), is the product that is imported successfully in the database.

Then:

* To ensure that there wasn't a fault with the product entry in the feed, I manually edited the datafeed, and gave an identical and unique name to two seperate products in different categories.
* I then imported the feed, and as suspected, only one of the products existed in the database.

Note that I tried importing products via both the GUI in /admin/ and the import.php via a CLI.

So to clarify, if more than one product in the datafeed contains the same name as another, for some reason it is not being imported into the database.
Also, if a product is named after another product that already exists in the database, the product with the duplicate name is removed during the import process.

I'm soooo sorry for the long explanation, I hope I've included enough information. Once again I thank you for your valued support..

Do you have a paypal donation page? If so, please either mail it to me or link here.

Cheers,
Matt

Submitted by support on Tue, 2008-08-12 08:41

Hello Matt,

My apologies; the behaviour you are noticing is not to do with search but to do with the "duplicate" filter that is applied during import. Any subsequent product with the same name as a product already imported for that merchant is dropped, regardless of category. This is because the main product comparison does not differentiate between categories.

If you wanted to experiment with allowing duplicate product names where they exist in different categories (so that the product shows up when doing a category search for each category in which the product exists), you could make the following change to includes/admin.php. Look for the following code beginning on line 248:

    /* create dupe_hash value */
    $dupe_key = $admin_importFeed["merchant"];
    // uncomment any additional fields that you wish to filter duplicates on (description not recommended)
    $dupe_key .= $record[$admin_importFeed["field_name"]];
    // $dupe_key .= $record[$admin_importFeed["field_description"]];
    // $dupe_key .= $record[$admin_importFeed["field_image_url"]];
    // $dupe_key .= $record[$admin_importFeed["field_buy_url"]];
    // $dupe_key .= $record[$admin_importFeed["field_price"]];
    $dupe_hash = md5($dupe_key);

...and change it as follows:

    /* create dupe_hash value */
    $dupe_key = $admin_importFeed["merchant"];
    // uncomment any additional fields that you wish to filter duplicates on (description not recommended)
    $dupe_key .= $record[$admin_importFeed["field_name"]];
    $dupe_key .= $record[$admin_importFeed["field_category"]];
    // $dupe_key .= $record[$admin_importFeed["field_description"]];
    // $dupe_key .= $record[$admin_importFeed["field_image_url"]];
    // $dupe_key .= $record[$admin_importFeed["field_buy_url"]];
    // $dupe_key .= $record[$admin_importFeed["field_price"]];
    $dupe_hash = md5($dupe_key);

This will permit the same product name in different categories to be imported. I notice that you have already made changes to products.php so that when searching a particular category clicking through a product page should only select the product from that category...

Hope this helps!

Cheers,
David.

Submitted by Ch3W on Tue, 2008-08-12 12:09

Okay great, thanks for that.

My only concern now is that if genuine duplicates exist in the data feed, that they will be imported into the database.

Do you plan to have the script check for duplicates using more than just one unique identifier (in this case the product name?). If the duplicate detection script checked the product name + the category (and maybe all the other fields for that product entry), then it would be a valid duplicate and could safely be removed.

Cheers,
Matt

Submitted by support on Tue, 2008-08-12 12:13

Hi Matt,

I wouldn't recommend adding the description to the dupe check for performance reasons, but it might be an option to add the buy URL - just uncomment the following line from the same block of code:

    // $dupe_key .= $record[$admin_importFeed["field_buy_url"]];

(remove the // from the start)

Check first of course that the products that differ only by category do have different buy URLs otherwise this will return to the previous scenario of only importing one of the products...!

Cheers,
David.

Submitted by Ch3W on Wed, 2008-08-13 23:35

Hi just popped by to say thanks for your help, works fine now :)

Submitted by Ch3W on Sun, 2014-11-16 19:06

Hi there,

Can you please tell me how this should look on the latest version of PT?

I found the dupe code on admin.php and changed it to the following, but it didn't work:

/* create dupe_hash value */
$dupe_key = $importRecord["merchant"];

// I commented out this: $dupe_key .= $searchName;
// And replaced it with this:

$dupe_key .= $importRecord["buy_url"];

$dupe_hash = md5($dupe_key);

thanks,
Matt

Submitted by support on Mon, 2014-11-17 15:40

Hello Matt,

Check out this recent comment - the best way to permit duplicates is simply to remove the unique key on the dupe_hash field. The same thread is current and may provide other useful info for working with duplicates...

Cheers,
David.
--
PriceTapestry.com