You are here:  » Drop products without images or broken image links


Drop products without images or broken image links

Submitted by IG on Mon, 2015-03-30 12:56 in

Hi David

A new day - a new question.

How can I drop all products of a feed without images. If possible, I would like to drop not only products where the image url is missing, but also products with an image url, but no image under that url. Is this possible?

Kind regards,
Ivo

Submitted by support on Mon, 2015-03-30 14:24

Hello Ivo,

A record can by dropped if a given field is empty using a Drop Record RegExp filter, with the following expression:

^$

(in a RegExp, ^ is the beginning of line anchor, and $ is the end of line anchor, so the above matches an empty field)

Testing images for validity would be straight but there are two quite significant caveats which you might want to consider first; the main one being the time taken to import since every image URL will have to be requested to check for validity. The second issue is that many commercial web servers are configured to throttle or even block IP addresses making "unusual" request patterns so this could result in this method incorrectly assuming that an image is invalid on this basis.

If you do want to give it a go, the following new code for an "Image Check" filter could be used to do this - add to includes/filter and then attach a new Image Check filter to the Image URL field. This will also perform the empty check, so if using this version there would be no need for a Drop Record RegExp filter also. Note that this requires PHP's GD library, and also allow_url_fopen enabled...

  /*************************************************/
  /* imageCheck */
  /*************************************************/
  $filter_names["imageCheck"] = "Image Check";
  function filter_imageCheckConfigure($filter_data)
  {
    print "<p>There are no additional configuration parameters for this filter.</p>";
  }
  function filter_imageCheckValidate($filter_data)
  {
  }
  function filter_imageCheckExec($filter_data,$text)
  {
    global $filter_dropRecordFlag;
    if($filter_dropRecordFlag)
    {
      return $text;
    }
    if (!$text)
    {
      $filter_dropRecordFlag = TRUE;
    }
    if (!imagecreatefromstring(file_get_contents($text)))
    {
      $filter_dropRecordFlag = TRUE;
    }
    return $text;
  }

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by wesse249 on Fri, 2015-05-08 19:48

Hello,

This solution works well. But when you set it up on every i costs a lot of load time.

Is there maybe another solution?

Thanks

Submitted by support on Sat, 2015-05-09 08:45

Hello JR,

The process of fetching each image would be necessary to achieve the objective but if you wanted to be able to add / test new feeds without the delay the filter could be modified easily to apply only to a full import - i.e. one initiated by CRON, by checking if $admin_importALL is set. If you'd like to give this a go, use the following alternative version of the filter code:

  /*************************************************/
  /* imageCheck */
  /*************************************************/
  $filter_names["imageCheck"] = "Image Check";
  function filter_imageCheckConfigure($filter_data)
  {
    print "<p>There are no additional configuration parameters for this filter.</p>";
  }
  function filter_imageCheckValidate($filter_data)
  {
  }
  function filter_imageCheckExec($filter_data,$text)
  {
    global $filter_dropRecordFlag;
    global $admin_importAll;
    if (!isset($admin_importAll))
    {
      return $text;
    }
    if($filter_dropRecordFlag)
    {
      return $text;
    }
    if (!$text)
    {
      $filter_dropRecordFlag = TRUE;
    }
    if (!@imagecreatefromstring(file_get_contents($text)))
    {
      $filter_dropRecordFlag = TRUE;
    }
    return $text;
  }

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by TWDesigns on Mon, 2015-09-14 00:58

Hey David,

I think this will do what I need. I would like to remove products with missing images as I import them. I added the first script to my filters and on slow import this is the error I receive.

Warning: imagecreatefromstring() [function.imagecreatefromstring]: Data is not in a recognized format in /includes/filter.php on line 376

What did I do wrong!

Submitted by support on Mon, 2015-09-14 08:21

Hi,

Ah - if warnings are enabled, even though the functionality is correct with imagecreatefromstring() returning FALSE if unable to create an image which is the intention, then it can be suppressed by prefixing the call with @ as follows:

     if (!@imagecreatefromstring(file_get_contents($text)))

(included above also)

Cheers,
David.
--
PriceTapestry.com

Submitted by TWDesigns on Thu, 2015-09-17 01:58

Worked!

Thank you.