You are here:  » Include filter when inporting feeds


Include filter when inporting feeds

Submitted by Stephen Provis on Wed, 2007-10-24 17:18 in

I've been using Pricetapestry for a short while and am very impressed with it. I am a bit stuck though..... Basically I am creating a very specific comparison site and want to only include certain types of products. Example: I have a site which is designed mainly for HD (High Def) products and would like to have a filter which only imports products with the words 'HD', 'High Def' or 'High Definition'.

Is there a way of only importing items from the feeds that conform to a set criteria?

Many thanks

Steve

Submitted by support on Wed, 2007-10-24 17:34

Hi Steve,

This has come up before - but the thread became quite long so i'll copy the new filter you need here. Basically, this involves adding a new filter to your site, called "Drop Record If Not (RegExp)". This filter applies a regular expression to the field value, and drops the record if there is no match. The code is as follows, add this to your includes/filter.php

  /*************************************************/
  /* dropRecordIfNotRegExp */
  /*************************************************/
  $filter_names["dropRecordIfNotRegExp"] = "Drop Record If Not RegExp";
  function filter_dropRecordIfNotRegExpConfigure($filter_data)
  {
    print "Drop record if field does not match regular expression:<br />";
    print "<input type='text' size='40' name='text' value='".widget_safe($filter_data["text"])."' />";
    widget_errorGet("text");
  }
  function filter_dropRecordIfNotRegExpValidate($filter_data)
  {
  }
  function filter_dropRecordIfNotRegExpExec($filter_data,$text)
  {
    global $filter_dropRecordFlag;
    if($filter_dropRecordFlag)
    {
      return $text;
    }
    else
    {
      $filter_dropRecordFlag = !ereg($filter_data["text"],$text);
    }
    return $text;
  }

Having added the filter, you can now add this against the product name field, and enter a regular expression similar to the following:

(HD|High Def)

You can add as many terms as you want in between the brackets (separated by the pipe character), which will mean the record is only imported if the field contains those terms...

Hope this helps!
Cheers,
David.

Submitted by Stephen Provis on Wed, 2007-10-24 17:45

Hi Chris,

Sounds perfect! I will give this a try and post back if I have any problems.
Thanks for the very quick responce!!

Steve

Submitted by knight01 on Thu, 2007-11-01 04:01

David,
How can this be applied to two different fields? I need to check against the ProductName and the LongDesc.

I believe if I use the droprecordifnot searching for, Keyword|Key word|Other, in the ProductName, then when it does the droprecordifnot search on the LongDesc it will only search those records that were not dropped during the ProductName search. this would cause it to miss records I want.

I presume the solution is based somewhere in the

global $filter_dropRecordFlag;
    if($filter_dropRecordFlag)
    {
      return $text;
    }

It seems like I need to create a flag to add the record rather than drop it and search for non-flagged records with each progressive search.

Thanks for any advice in pointing me in the right direction...

Perfume & Cologne Comparison Site

Submitted by support on Thu, 2007-11-01 08:44

Hi,

Because you don't really know what order the filters will be applied, the only way to do this is to code them together in one single filter.

Luckily, the script copies the entire record into the global space in the variable $filter_record, so you can do all of this in one filter. Instead of adding another filter against the description field, instead just use one copy of the filter against the product name field, and then modify the filter exec function as follows:

  function filter_dropRecordIfNotRegExpExec($filter_data,$text)
  {
    global $filter_dropRecordFlag;
    global $filter_record;
    if($filter_dropRecordFlag)
    {
      return $text;
    }
    else
    {
      $test = implode(" ",$filter_record);
      $filter_dropRecordFlag = !ereg($filter_data["text"],$test);
    }
    return $text;
  }

This version will test against all fields. If you want to restrict it to product name and description fields then it will be a bit more complicated as you have to make changes elsewhere to find out what field names the product name and description fields have; but seeing as the only other fields are price / image URL and buy URL this should work fine as those fields are unlikely to contain the keywords you are looking for...

Cheers,
David.

Submitted by marco@flapper on Sun, 2008-01-20 18:54

I Have implemented this filter, but is does not seem to work. I think I have 15 notebooks in my feed, but the result is only 2 products. How can I debug this to see what goes wrong?

Submitted by support on Mon, 2008-01-21 12:32

Hello Marco,

Can you post a couple of example product names that you are trying to drop; and then the value that you are entering in the filter configuration; for example:

(Word1|Word2)

The brackets are important here as they are part of the regular expression...

Cheers,
David.

Submitted by marco@flapper on Fri, 2008-01-25 09:23

Hi David,

I want to check for name and description. This is an example:

<productID>89efd0082bdc3cd249bc2f50bcc14e06c7f41803</productID>
  <name>Compaq Presario A900ED</name>
  <price currency="EUR">699.00</price>
  <productURL>http://www.dynabyte.nl/tradetracker_redirect.php?tt=760_0_16905_&r=https%3A%2F%2Fwww.dynabyte.nl%2Farticle.php%3Ftc%3Dtradetr%26xi%3D23467</productURL>
  <imageURL>https://www.dynabyte.nl/image.php?xi=23467</imageURL>
- <description>
- <![CDATA[ 17" Notebook
  ]]>
  </description>
- <categories>
  <category name="Hardware" path="Hardware" />
  </categories>
- <additional>
  <field name="merk" value="COMPAQ" />
  <field name="type" value="A900" />
  <field name="EAN" value="883585775873" />
  <field name="levertijd" value="1-2 dagen" />
  </additional>
  </product>

Submitted by support on Mon, 2008-01-28 09:39

Hello Marco,

Can you confirm that the filter itself is working, by operating it just against the Description field; with the filter value:

Notebook

This should import only products with Notebook in the description field. We can then modify the filter to work against more than one field...

Cheers,
David.

Submitted by marco@flapper on Tue, 2008-02-12 20:30

Hi David,

Yes this seems to work!

Marco.