You are here:  » How to change feed import order v12/10B


How to change feed import order v12/10B

Submitted by Convergence on Tue, 2015-06-23 17:02 in

Greetings,

First, a question:

If a merchant has multiple feeds, say one sale feed and one regular feed and the sale feed has SOME same products with lower prices than the regular feed: If the sale feed gets imported before the regular feed, which product price is shown; the first imported feed or the second?

That being said: We'd like to be able to change the import order of feeds as products that are price mapped from different merchants (and have the same price) we would like to have our preferred merchants show first in the list of mapped products. Imagine it could be like the filter sort order mod for v12/10B. This would allow us to show our preferred merchants (with the same price as non-preferred merchants) above the non-preferred merchants while allowing the lowest price merchants to still rank by lowest price.

Can this be done?

Thanks!

Thanks!

Submitted by support on Wed, 2015-06-24 07:42

Hi,

Regarding duplicate products from sale / non-sale feeds, it would always be the version imported first that would "win", since the second occurrence of the same product would be dropped due to the de-duplication process.

Importing manually this is no problem of course - simply import the sale feed before the normal feed. To control the import sequence for your automation process, if using scripts/cron.php look for the following code at line 81:

  $sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds`";

...and REPLACE with:

  $sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds` ORDER BY filename";

(if using import.php @ALL, the same code is line 60)

With this in place, simply ensure that the feed you wish to import first is alpha-numerically more significant, so the easiest way would be to rename with a leading number indicating the required order of import, e.g.

1-Merchant-Sale-Feed.csv
2-Merchant-Normal-Feed.csv

So in this case, 1-Merchant-Sale-Feed.csv would always be imported before 2-Merchant-Normal-Feed.csv due to the ORDER BY clause added to the SQL.

Regarding preferred merchants featuring higher in the price comparison table alongside others with the same price, it's straight forward to sort the results by a merchant priority value where the price is the same. To do this, simply add the following code to the top of html/prices.php:

<?php
  $merchantPriority
["Merchant A"] = 1;
  
$merchantPriority["Merchant B"] = 2;
  
$merchantPriority["Merchant C"] = 3;
  function 
cmpMerchantPriority($a,$b)
  {
    global 
$merchantPriority;
    if (
$a["price"] == $b["price"])
    {
      
$a = (isset($merchantPriority[$a["merchant"]])?$merchantPriority[$a["merchant"]]:0);
      
$b = (isset($merchantPriority[$b["merchant"]])?$merchantPriority[$b["merchant"]]:0);
      if (
$a == $b)
      {
        return 
0;
      }
      return (
$a $b) ? : -1;
    }
    return (
$a["price"] < $b["price"]) ? -1;
  }
  
usort($prices["products"],"cmpMerchantPriority");
?>

Edit / add entries to the $merchantPriority array as required, and the higher the value, the higher the priority, so in the above case, Merchant C would be listed above Merchant B for the same price result...

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by stevebi on Thu, 2015-12-24 07:10

Hello David,

And how about first merchant position at searchresults.php?

Cheers

Steve

Submitted by support on Thu, 2015-12-24 09:12

Hi Steve,

As search results are generated by a summary query the same code doesn't really apply - the order of results is determined by the sort in effect (defaulting to relevance). Bear in mind that the cheapest price will always be displayed by normal search results, and then by clicking through to the product page those results of the same price will appear in priority order as per the above...

Cheers,
David.
--
PriceTapestry.com

Submitted by Convergence on Wed, 2016-05-04 15:36

Hi David,

Modifying/editing /html/prices.php when there are a dozen plus merchants who are regularly being added and deleted is a tad cumbersome.

That being said, have some more questions:

1) How does /html/prices.php determine the display order if 7 merchants all have the same price?
2) Is there a way to display same price merchants by import order? Keeping in mind we still want the lowest price shown first (of course) - just by a preferred merchant.

Thanks, again!

Submitted by support on Wed, 2016-05-04 16:24

Hi,

Where products are the same price, the order is entirely down to MySQL internals so essentially is undefined. However, it is no problem to re-order by import order. To give this a go, add the following PHP section to the very top of html/prices.php:

<?php
  
function prices_cmp($a$b)
  {
    global 
$imported;
    if (
$a["price"] == $b["price"])
    {
      if (
$imported[$a["filename"]] == $imported[$b["filename"]])
      {
          return 
0;
      }
      return (
$imported[$a["filename"]] < $imported[$b["filename"]]) ? -1;
    }
    return (
$a["price"] < $b["price"]) ? -1;
  }
  
$ins = array();
  foreach(
$prices["products"] as $p)
  {
    
$ins[] = "'".database_safe($p["filename"])."'";
  }
  
$in implode(",",$ins);
  
$sql "SELECT filename,imported FROM `".$config_databaseTablePrefix."feeds` WHERE filename IN (".$in.")";
  
database_querySelect($sql,$rows);
  
$imported = array();
  foreach(
$rows as $row)
  {
    
$imported[$row["filename"]] = $row["imported"];
  }
  
usort($prices["products"],"prices_cmp");
?>

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by Convergence on Sun, 2016-05-08 15:24

Hi David,

Editing the above scripts/cron.php and import.php didn't work.

I suspect it's because we use import_slow.php for our CRON.

This is our heavily modded import_slow.php:

{code saved}

Would we just need to edit line 215?

Thanks!

Submitted by support on Mon, 2016-05-09 07:29

Hi,

Import order mod can be applied at line 231 of the code you posted;

$sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds` WHERE format LIKE 'csv%' AND imported < '".$startTime."' LIMIT 1";

...REPLACE with:

$sql = "SELECT * FROM `".$config_databaseTablePrefix."feeds` WHERE format LIKE 'csv%' AND imported < '".$startTime."' ORDER BY filename LIMIT 1";

Cheers,
David.
--
PriceTapestry.com

Submitted by Convergence on Mon, 2016-05-09 12:31

Hi David,

Thank you so much.

I don't know if pasting my import_slow.php in the forum changed the line numbers (removing blank rows, or?) but, my line 231 does not match what you said we should edit.

I show the above code on line 211 and 326.

Should we edit the first occurrence or the second?

Thanks!

Submitted by support on Mon, 2016-05-09 13:21

Hi,

Ah yes the line numbers changed as a result of the formatting, sorry about that, and in fact, _both_ instances of that line (I had missed the first one) should be updated to the replacement including the ORDER BY clause...

Cheers,
David.
--
PriceTapestry.com

Submitted by Convergence on Mon, 2016-05-09 15:20

Hi David,

PERFECT!

Thank you so much!

Submitted by Convergence on Tue, 2016-05-10 03:04

Hi David,

Found another spot that still displays the merchants in non-alpha-numeric import order.

Right under the product description. Believe it's /html/product.php - Our /html/product.php is VERY heavily modded.

Suggestions on how to sort this?

Thanks!

Submitted by support on Tue, 2016-05-10 08:29

Hi,

Rather than perform the sort twice, instead move the sort by imported code suggested above from html/prices.php and instead position it at the very top of html/product.php.

Then in the last line of the mod where you have:

  usort($prices["products"],"prices_cmp");

...REPLACE with:

  usort($prices["products"],"prices_cmp");
  $product["products"] = $prices["products"];

Cheers,
David.
--
PriceTapestry.com

Submitted by Convergence on Mon, 2016-05-16 16:52

Hi David,

Worked PERFECTLY - Thank you!