You are here:  » Feed imports less than half


Feed imports less than half

Submitted by 2fer on Sun, 2016-01-03 15:56 in

Hi again David and a happy 2016 to you!
I'm back with a feed import problem. Before using a feed, I view it and look for any obvious issues that may need a filter, and this feed looks fine except for some weird period strings in the descriptions. They may be intended as ellipses, or introduced in some other way. They are not uniform and some of the descriptions contain only a few "... . ." and no text. This may or may not be the reason a feed with 8000+ products is importing only 2100+ products (?) so I'm here for direction. If they were uniform then a drop record filter would be simple, but when I tried that it imported only 900+ products.

If I copy them from various entries I get the assortment shown here:

. . . . .
. . . . . .
. . . . . . . .
. . . . . . . .
. .
. .
. . . . . . . .
.. . . . .
. . . . . .

Note that there are spaces after the last '.' in some lines and some strings are in the middle of the description. I don't see another issue in the datafeed that might prevent its import. Everything else looks just perfect in the rest of the fields.

I should also mention that when I try to use the Slow Import with this feed, it processes for quite a while, shows no progress, then reloads a copy of the slow import page heading below the original and then stops. It is configured to import 1500 records between rests.

What I want to do is to drop the record if there are only '.' as a description, and to replace the strings of spaces and periods within descriptions with a single '. ' sequence. (period and space) Any suggestions?

Thank you for any ideas here.
2fer

Submitted by support on Mon, 2016-01-04 08:58

Hello 2fer,

You could use a Drop Record If Not RegExp filter to drop records that do not contain at least one letter or number which should do the trick - use the following expression on the configuration page:

/[a-z0-9]/i

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by 2fer on Tue, 2016-01-05 03:19

Thanks David,
That will help, but there are just a few products with the '. . . ' (only) description problem. Would those few problems keep the rest of the normal looking lines from being processed? I will try the feed checker utility.

The more prevalent problem is the actual text descriptions with a variety of '. . . . .' strings in the middle of the description. I'm guessing there's no filter for those. It just seems a shame to process 8684 lines to end up with 2140 products. I will contact the merchant and ask what they can do.

I appreciate the help, as always!
2fer

Submitted by support on Tue, 2016-01-05 09:11

Hi 2fer,

As the description field is optional anyway it won't be responsible for the drop in records imported from the expected number in the feed - the only reasons for a record to be dropped are invalid or missing price, product name or buy URL, of if the product is a duplicate for that merchant.

Duplication is the most common reason for records to be dropped, a good example might be where a clothing products feed has generic product names e.g. "Jumper", with differentiation e.g. colour / size in other fields. Where this is the case those fields can be combined with the name field using Text Before / Text After filters as required. The Parse Analysis feed utility will tell you how many duplicates are in the feed, plus the number of invalid records which should help...

One final thing that can occur would be if the auto-detected format string was not quite right, for example, if only some fields are quoted, and none in the first 2 lines quoted text would not be picked up by the auto-detection. To check if the format is correct use the navigation buttons to page through the first few records in the same data on Feed Registration Step 2 - if you notice " characters at the front of any of the values that would indicated that quoted text has not been auto-detected, in which case click Back in your browser and then manually selected the correct format from the drop-down box on Feed Registration Step 1...

Hope this helps!

Cheers,
David.
--
PriceTapestry.com