You are here:  » Exact match and negative keywords via search form / input field


Exact match and negative keywords via search form / input field

Submitted by bird on Thu, 2014-10-09 12:20 in

Hi David,

I'm currently using the search form to build some internal links. The problem is, that the current search query sometimes returns results that are not as accurate as they should be.

E.g. the term "iPhone 6 16GB" returns "Apple iPhone 5c 16 GB ..." as first result.
I found this node http://www.pricetapestry.com/node/3883 that explains how the complete query can be modified but this causes problems with other search terms in my case.

Wouldn't it be great, if the search term could be modified via the search form / input field like Google does it?

Keywords1 Keyword2 -> Fuzzy logic (the current search query)
"Keyword1 Keyword2" -> Exact match for the terms but other words in the title are allowed %
Keyword1 -Keyword2 -> Only returns results that do not include Keyword2

This would be very comfortable in my opinion. What do you think?

Best regards
Bernhard

Submitted by support on Thu, 2014-10-09 12:52

Hello Bernhard,

One option would be to utilise the full capabilities of MySQL's FULLTEXT indexing, which supports operators exactly as you suggest. Full details of the formatting can be found at:

http://dev.mysql.com/doc/refman/5.5/en/fulltext-boolean.html

However, since the $q parameter is sanitised, plus hyphen is used in place of spaces, I think it would be best to add a new operator and search handler for a pure FULLTEXT search with all characters permitted.

To give this a go, edit search.php and look for the following code at line 6:

  $q = (isset($_GET["q"])?tapestry_normalise($_GET["q"],":\."):"");

...and REPLACE with:

  if (substr($_GET["q"],0,2)=="ft")
  {
    $q = $_GET["q"];
  }
  else
  {
    $q = (isset($_GET["q"])?tapestry_normalise($_GET["q"],":\."):"");
  }

Next, look for the following code around line 194:

      case "bw":

...and REPLACE with:

      case "ft":
        $where = "MATCH name AGAINST ('".database_safe($parts[1])."' IN BOOLEAN MODE)";
        $sql = "SELECT SQL_CALC_FOUND_ROWS id,COUNT(id) AS numMerchants,MIN(price) as minPrice,MATCH name AGAINST ('".database_safe($parts[1])."' IN BOOLEAN MODE) AS relevance FROM `".$config_databaseTablePrefix."products` WHERE ".$where.$priceWhere." GROUP BY search_name";
        $orderBySelection = $orderByFullText;
        break;
      case "bw":

To use, submit your query with the FULLTEXT boolean mode expression following ft: for example:

ft:+Keyword1 -Keyword2

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by bird on Thu, 2014-10-09 14:02

Hi David,

thanks for your fast reply!
I think there is a bracket missing $q = isset($_GET["q"]);

Unfortunately the code doesn't work so far. When I enter ft:+iPhone or ft:iPhone the query in the input field is replaced with a "1" and the search term is set to "1", returning only results for "1".

Best regards
Bernhard

Submitted by support on Thu, 2014-10-09 14:26

Hi Bernhard,

Corrected above - superfluous "isset" also, and a slight modification required to the SQL - working nicely now!

Cheers,
David.
--
PriceTapestry.com

Submitted by bird on Thu, 2014-10-09 14:45

Hi David,

thank you very much for your support!

It works now but it doesn't really solve the problem ;) The first result for "ft:+iPhone 6" is still "5 W USB Power Adapter für iPod/iPhone, Handy-Stromversorgung" which doesn't correctly match the term "iPhone 6". Am I doing something wrong?

Submitted by bird on Thu, 2014-10-09 14:53

Just for the record - negative search terms work fine! :)

Submitted by bird on Thu, 2014-10-09 14:59

Sorry, that I put this in different comments.

The results for ft:"iPhone 6" look good but the search term is rewritten to ft:\ and the meta title retuns ft:\"iPhone 6\". I guess there is a Problem with the " in the search query?

Submitted by support on Thu, 2014-10-09 15:03

Hi,

That sounds like Magic Quotes are enabled on your PHP installation - to fix, where you have this line in the first section of modified code:

     $q = $_GET["q"];

...REPLACE with:

     $q = stripslashes($_GET["q"]);

Cheers,
David.
--
PriceTapestry.com

Submitted by bird on Thu, 2014-10-09 15:30

Hi David,

thank you! I disabled magic quotes and alternatively tried the new code and both ways work fine! There are only 2 minor "visual" problems left:

1. After entering ft:"iPhone 6" the term in the input field is replaced with ft: only. Sorting works but I guess it might be irritating.

2. The title retuns "ft:iPhone 6" but this can pobably be fixed in the search.php / $header["title"] = $q;

Submitted by support on Fri, 2014-10-10 08:09

Hello Bernhard,

Re 1, that sounds like your search form is being populated with $parts[0]. If you could check your html/searchform.php and locate the value attribute of the input box (name='q') and see if you currently have:

$parts[0]

...in which case, REPLACE with:

$q

...and you'll see the full query value by default.

Re 2, <title> is being set as expected - but could be modified conditionally if $parts[0] is "ft", for example, in the case of these queries if you would prefer the title to be just the actual FULLTEXT query part then where you have:

  $header["title"] = $q;

...REPLACE with:

  if ($parts[0]=="ft")
  {
    $header["title"] = $parts[1];
  }
  else
  {
    $header["title"] = $q;
  }

Cheers,
David.
--
PriceTapestry.com

Submitted by TWDesigns on Mon, 2016-04-04 05:40

Greetings!

I have ran in to the same issue. I implemented the above code which seems to work for what I need but search results return show ft:search terms. Example: "ft:Long Dresses" instead of "Long Dresses"

Did I miss something somewhere or am I doing something wrong.

I have a few custom menu items that I wanted to return products with all matching keywords. The above appears to work but the ft showing up seems to be my only issue.

Thanks!

Submitted by support on Mon, 2016-04-04 08:51

Hi,

The above would only affect the page title - to change what is displayed in the banner ("Search results for ...") edit search.php and look for the following code at line 409:

      $banner["h2"] = translate("Product search results for")." <strong>".htmlspecialchars($q,ENT_QUOTES,$config_charset)."</strong>&nbsp;";

...and REPLACE with:

      $title_q = (isset($parts[1])?$parts[1]:$q);
      $banner["h2"] = translate("Product search results for")." <strong>".htmlspecialchars($title_q,ENT_QUOTES,$config_charset)."</strong>&nbsp;";

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by TWDesigns on Mon, 2016-04-04 14:34

Perfect as always! Thanks again that did the trick.

Submitted by sirmanu on Tue, 2016-11-08 18:16

Hi!
How can I modify the script in order to have exact matching with ft:"whatever" when priceasc or pricedesc is selected?

Submitted by support on Tue, 2016-11-08 18:43

Hi,

Firstly, a correction to the above, the second replacement in the original reply to this thread (to add support for the ft: operator) includes this line:

         $orderBySelection = $orderByDefault;

...but should be:

         $orderBySelection = $orderByFullText;

Corrected above.

With that in place, for an exact match regardless of sort, use the "+" operator in front of keywords that must exist. In other words, were you to be using:

ft:keyword1 keyword2

...by default, the top results would contain _both_ keyword1 and keyword2, but this is only a result of the relevancy logic. When sorting by price, it may be the case that only one of the keywords is present in the top results. However, using:

ft:+keyword1 +keyword2

...then all results will contain keyword1 _and_ keyword2, regardless of sort order...

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by sirmanu on Tue, 2016-11-08 19:06

Sorry David, I didn't explain well. Usually, with large databases, sorting by priceAsc or desc is useless because the first products listed are going to be with a low relevance.
My idea was to make this exact match invisible to the user. So form example, if the user looks for Calvin Klein Trousers for lower price, internally, we are making:

ft: +Calvin +Klein +Trousers but ordered, by price.

Submitted by support on Tue, 2016-11-08 20:01

Ah - in that case, the thing to do is to make the default application of FULLTEXT search to be BOOLEAN MODE and all keywords required. To do this, edit search.php and look for the following code beginning at line 282:

          $where = "MATCH ".$matchFields." AGAINST ('".database_safe($parts[0])."')";
          $sql = "SELECT SQL_CALC_FOUND_ROWS id,COUNT(id) AS numMerchants,MIN(price) as minPrice, MATCH ".$matchFields." AGAINST ('".database_safe($parts[0])."') AS relevance FROM `".$config_databaseTablePrefix."products` WHERE ".$where.$priceWhere." GROUP BY search_name";

...and REPLACE with:

          $against = "+".str_replace(" "," +",$parts[0]);
          $where = "MATCH ".$matchFields." AGAINST ('".database_safe($against)."' IN BOOLEAN MODE)";
          $sql = "SELECT SQL_CALC_FOUND_ROWS id,COUNT(id) AS numMerchants,MIN(price) as minPrice, MATCH ".$matchFields." AGAINST ('".$against."' IN BOOLEAN MODE) AS relevance FROM `".$config_databaseTablePrefix."products` WHERE ".$where.$priceWhere." GROUP BY search_name";

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by sirmanu on Wed, 2016-11-09 22:33

Oks. Excellent!
As I only wanted for priceasc or pricedesc, I added the conditional

if ($sort=="priceAsc" || $sort=="priceDesc"){
//above code
}

Also I had to comment these lines for plural

$words = explode(" ",$parts[0]);
$newWords = array();
foreach($words as $word)
{
  if (substr($word,-1)=="s")
  {
    $newWords[] = substr($word,0,-1);
  }
}
$allWords = array_merge($words,$newWords);
$parts[0] = implode(" ",$allWords);

If not, some results did not show.