You are here:  » Custom filter to query database


Custom filter to query database

Submitted by sirmanu on Tue, 2018-10-30 10:33 in

Hi David. As you probably know, I am using intensively the uidmap script.
However, I have a merchant which doesn't provide the feed with the GTIN/EAN field. Instead, they send me a CSV which maps productid of that merchant to the GTIN. I have already a column productid in pt_products table.

I thought about import that CSV manually into a table, and then, with a custom filter, check if the GTIN exists for the productid. If it is, fill this value.

How can I achieve this? Until now, I didn't have any filter which query database. I don't know if it is possible.

Thanks.

Submitted by support on Tue, 2018-10-30 10:58

Hi,

How many products are in the CSV? It's almost certainly easiest to do this in memory. Can you post an example of a couple of lines from the CSV that you need to reference?

Thanks,
David.
--
PriceTapestry.com

Submitted by sirmanu on Tue, 2018-10-30 11:55

Sure. These are some lines of the CSV:

id;ns1:gtin
459144;3420XX1750243
459145;3420581XX0014
520958;64318XX14363
536675;34205XX750229
536676;3420XX1750038
536677;3420XX1750021
556086;505575XX13267
676494;
676495;
556087;505XX54413670
556088;50557544XX687
536678;34205817XX0267
536696;643187XX4349
503821;871668XX73959
536826;871668XX73966
520959;64XX87010808
520960;643XX7008164

There are in total almost 50.000 lines.
Notice that sometimes the GTIN is empty.

Submitted by support on Tue, 2018-10-30 12:49

Hi,

Have a go with the following - add to includes/filter.php to add support for a new "Product ID to GTIN" filter;

  /*************************************************/
  /* Product ID to GTIN */
  /*************************************************/
  $filter_names["id2gtin"] = "Product ID to GTIN";
  function filter_id2gtinConfigure($filter_data)
  {
    print "<p>There are no additional configuration parameters for this filter.</p>";
  }
  function filter_id2gtinValidate($filter_data)
  {
  }
  function filter_id2gtinExec($filter_data,$text)
  {
    global $config_feedDirectory;
    global $filter_id2gtinMap;
    global $filter_record;
    if (!is_array($filter_id2gtinMap))
    {
      $filename = $config_feedDirectory."id2gtin.csv";
      $fp = fopen($filename,"r") or die("Could not open ".$filename);
      while (($data = fgetcsv($fp,0,";")) !== FALSE)
      {
        if ($data[1])
        {
          $filter_id2gtinMap[$data[0]] = $data[1];
        }
      }
      fclose($fp);
    }
    if (isset($filter_id2gtinMap[$filter_record["productid"]]))
    {
      return $filter_id2gtinMap[$filter_record["productid"]];
    }
    return $text;
  }

Upload your CSV containing the mappings to the /feeds/ folder of the installation as id2gtin.csv. Then add a new instance of the filter to the GTIN/EAN for the feed. The filter will look at `productid` in the product record being imported (from the global $filter_record) to match and return the associated GTIN if set...

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by sirmanu on Tue, 2018-10-30 13:19

Excellent! Just a little clarification. Can I configure the name of the CSV when setting up up the filter? This is because maybe in a future, I would need to add this filter to another merchant.
Also If I apply that change I need something like:

$filter_id2gtinMap[$importRecord["merchant"]][$filter_record["productid"]];

isn't it?

Submitted by support on Tue, 2018-10-30 13:32

Sure - the below will add a configuration parameter for the CSV filename;

  /*************************************************/
  /* Product ID to GTIN */
  /*************************************************/
  $filter_names["id2gtin"] = "Product ID to GTIN";
  function filter_id2gtinConfigure($filter_data)
  {
    widget_textBox("CSV Filename","filename",TRUE,$filter_data["filename"],"",3);
  }
  function filter_id2gtinValidate($filter_data)
  {
    if (!$filter_data["filename"])
    {
      widget_errorSet("filename","required field");
    }
  }
  function filter_id2gtinExec($filter_data,$text)
  {
    global $config_feedDirectory;
    global $filter_id2gtinMap;
    global $filter_record;
    if (!is_array($filter_id2gtinMap))
    {
      $filename = $config_feedDirectory.$filter_data["filename"];
      $fp = fopen($filename,"r") or die("Could not open ".$filename);
      while (($data = fgetcsv($fp,0,";")) !== FALSE)
      {
        if ($data[1])
        {
          $filter_id2gtinMap[$data[0]] = $data[1];
        }
      }
      fclose($fp);
    }
    if (isset($filter_id2gtinMap[$filter_record["productid"]]))
    {
      return $filter_id2gtinMap[$filter_record["productid"]];
    }
    return $text;
  }

No other mods required, but it would need to be added on a per feed (where the feed is for a single merchant) basis...

Cheers,
David.
--
PriceTapestry.com

Submitted by sirmanu on Tue, 2018-10-30 13:52

But if $filter_id2gtinMap is a global variable that is not unset or resetted in any site, how is going to affect this in the case the the productid is the same for both merchants (with different gtin)? That is why I thought about specify the merchant into this array. Maybe I am wrong and in some place this variable is being unset??

Submitted by support on Tue, 2018-10-30 14:24

Ah yes, sorry about that - the following will create a separate array for each filename (as an element of $filter_id2gtinMap keyed by the filename)...

  /*************************************************/
  /* Product ID to GTIN */
  /*************************************************/
  $filter_names["id2gtin"] = "Product ID to GTIN";
  function filter_id2gtinConfigure($filter_data)
  {
    widget_textBox("CSV Filename","filename",TRUE,$filter_data["filename"],"",3);
  }
  function filter_id2gtinValidate($filter_data)
  {
    if (!$filter_data["filename"])
    {
      widget_errorSet("filename","required field");
    }
  }
  function filter_id2gtinExec($filter_data,$text)
  {
    global $config_feedDirectory;
    global $filter_id2gtinMap;
    global $filter_record;
    if (!is_array($filter_id2gtinMap[$filter_data["filename"]]))
    {
      $filename = $config_feedDirectory.$filter_data["filename"];
      $fp = fopen($filename,"r") or die("Could not open ".$filename);
      while (($data = fgetcsv($fp,0,";")) !== FALSE)
      {
        if ($data[1])
        {
          $filter_id2gtinMap[$filter_data["filename"]][$data[0]] = $data[1];
        }
      }
      fclose($fp);
    }
    if (isset($filter_id2gtinMap[$filter_data["filename"]][$filter_record["productid"]]))
    {
      return $filter_id2gtinMap[$filter_data["filename"]][$filter_record["productid"]];
    }
    return $text;
  }

Cheers,
David.
--
PriceTapestry.com