You are here:  » Characters issue in Category section


Characters issue in Category section

Submitted by stevebi on Wed, 2014-03-19 10:41 in

Hi,

As I have inserted a trial feed, I noticed that at /categories.php appeared wrong characters like "amp" and "�"

How can I solve this issue?

Regards

Steve

Submitted by support on Wed, 2014-03-19 11:18

Hi Steve, and welcome to the forum!

When categories and brands are imported, their values are "normalised" - i.e.made safe for use in URLs which involves stripping certain characters such as "&" and ";". What's happened in the case of "amp", is that your feed contains the HTML entity for "&" which is &. The "&" and ";" are stripped, leaving "amp". Strictly speaking feeds shouldn't contain HTML markup because they are not HTML, but many do so the best work around in this case would be to go to Global Filters and add a new Search and Replace filter to the Category field. On the configuration page for the filter, enter as follows:

Search:

" amp "

(without the quotes - but note the leading and trailing spaces)

Replace:

" and "

On the next import, "amp" will appear correctly as "and".

Regarding "�", this is down to a character encoding mis-match between the site configuration and feed data. The default encoding is utf-8 but your feed is most likely iso-8859-1 encoded so to change your site's character set to iso-8859-1 edit config.php and change line 4 as follows:

  $config_charset = "iso-8859-1";

Now, you may find that whilst this resolves the problem for this particular feed (if you are only working with one feed so far), if you come to add additional feeds that are in the far more common utf-8 encoding then it would make sense to have your site in utf-8 as per the default, and then treat this feed separately with "UTF8 Encode" (ISO-8859-1 to UTF-8 conversion) filters on the Product Name, Category and Brand fields for this feed. Let me know of course if you're not sure at all.

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by stevebi on Thu, 2014-03-20 08:16

Hi David,

And thank you for your reply.

1) Is there a way to set character encoding UTF-8 to a specific feed? since I do not want to change the current encoding of the rest site which is UTF-8

2) Also at the following link {link saved} i am facing some issues
a) how can I remove the html markup?
b) how can I change the price format to 6,42 € and also add the euro sign

3) How can I add some fields in Field Mapping?

Regards

Steve

Submitted by support on Thu, 2014-03-20 09:57

Hi Steve,

> 1) Is there a way to set character encoding UTF-8 to a specific feed?
> since I do not want to change the current encoding of the rest
> site which is UTF-8

Sure. Leave your $config_charset as "utf-8", and then for any feed that is _not_ utf-8 encoding, from the /admin/ home page click "Filters" alongside the feed, and then add a "UTF-8 Encode" filter to the Name, Description, Category and Brand fields, so you'll add 4 instances of the filter altogether, one for each field. For more info on the various filters available check out this page.


> 2) Also at the following link {link saved} i am facing some issues
> a) how can I remove the html markup?

From the /admin/ home page, click Filters alongside the feed and add a new Strip HTML filter to the Description field - that should do the trick!


> b) how can I change the price format to 6,42 € and also add the euro sign

I am going to include currency separator and symbol before / after the price options as template configuration settings in the new theme, however for the current theme it's just a case of performing a Search and Replace operation using your text editor as follows:

html/featured.php and html/searchresults.php

Search:

$config_currencyHTML.$product["minPrice"]

Replace:

str_replace(".",",",$product["minPrice"].$config_currencyHTML

html/searchresults.php and html/prices.php

Search:

$config_currencyHTML.$product["price"]

Replace:

str_replace(".",",",$product["price"]).$config_currencyHTML

html/product.php

Search:

$config_currencyHTML.$mainProduct["price"]

Replace:

str_replace(".",",",$mainProduct["price"]).$config_currencyHTML


> 3) How can I add some fields in Field Mapping?

There's a step-by-step guide to adding custom fields on this page...

Hope this helps!

Cheers,
David.
--
PriceTapestry.com

Submitted by 2fer on Wed, 2014-05-07 07:08

I've run into a similar problem - the only category I am importing has an & in the middle. It is not URLencoded, shows in Registration as & with a space on either side. I have tried every form I can imagine to Search and Replace the character & with and but since I can't actually see the category I don't know what it has been converted to. I have tried:
Blue & Red > Blue and Red
Blue&Red > Blue and Red
Blue amp Red > Blue and Red
Blue &amp Red > Blue and Red
BlueRed > Blue and Red
Blue Red > Blue and Red
Blue&Red > Blue and Red
and still, no Category shows up after importing. It shows 37 products which is what I expect, but they can only be found by searching for specific product names or Brands, not in Categories. I know there must be a way - help?
(BTW, this is a setup feed converted to UTF8, but I recently found out that many of the feeds I import are in WIN-1252, not ISO-8859-1 or UTF8):(

Submitted by support on Wed, 2014-05-07 08:15

Hi 2fer,

There are several ways an & can be encoded - on its own it should have been removed by normalisation, but that should also have happened if entity encoded as & leaving you with "amp".

Perhaps the best thing to do is to remove normalisation on the category field by looking for the following code at line 191 in includes/admin.php:

  $importRecord["category"] = tapestry_normalise($importRecord["category"]);

...and temporarily comment out this line, e.g.

  // $importRecord["category"] = tapestry_normalise($importRecord["category"]);

Re-import, and then from the Category A-Z you can use your web browsers View > Source menu to see exactly how the category field is formed, and what to use in Search and Replace / Explode filters as required.

Cheers,
David.
--
PriceTapestry.com

Submitted by 2fer on Wed, 2014-05-07 19:02

Hi David,
Thanks for the quick help, but nothing is showing for categories. I edited the includes/admin.php and uploaded it. I use a color editor for php and the commented line is uncolored (not active). I tried importing the feed and nothing changed so I deleted all filters that I had in place attempting to "Search and Replace" for the & character and left only the "Drop Record if Not" and imported again. No change. I re-Registered it and imported again, still nothing resembling the category is showing up in the shop. Each time I import it, it shows the correct time as it should to indicate that it was imported. I even checked in a different browser in case refreshing wasn't showing me the changes. Any ideas?

The script is using that category text to "Drop Record if Not", but only the first word in the phrase, that part is working perfectly.

I will need to put back the original includes/admin.php before tonight because it might affect two other feed imports on cron.
Thank you,
2fer

Submitted by 2fer on Thu, 2014-05-08 03:13

Hi David,
I thought I should update this. I gave up on the setup feed and imported a feed untouched, the way it will be imported. I get the same problem in that it does import the products, but there is no corresponding category showing up and filters don't do anything.
Thanks,
2fer

Submitted by support on Thu, 2014-05-08 08:48

Hello 2fer,

That actually sounds like the category field isn't being imported at all (for this feed) so I wonder if it's a registration issue... On Feed Registration Step 2, check that the field being mapped to "Category" is showing data in the Sample Data displayed below the form. If it's an XML feed, be aware of a nested category field for example CATEGORY/NAME - in which case the auto-detection will have picked up on the [always empty] parent element "CATEGORY".

If that looks OK, you can check the categories being imported using the Feed Inspector tool if you've not already given that a go. If that does show categories, then it sounds like a front-end problem rather than a registration / import problem. In which case, if you could let me know the installation URL (I'll remove before publishing your reply) I'll take a look for you. Alternatively, no categories are showing after import, if you could let me know the installation URL and feed filename, I'll fetch the feed from your /feeds/ folder to my test server and check it out for you...

Cheers,
David.
--
PriceTapestry.com

Submitted by 2fer on Fri, 2014-05-09 00:08

Not a part of this question, just a comment..
Oh, Wow! You have been busy David! Thank you, the Feed Inspector looks like it will be a great help for sorting through some aggravating feeds. I found a lot of other tools while I was getting that, all look useful.

For this issue I need to do some backtracking because it seems to me I have replaced the includes/admin.php file more than once so it is not standard. I do believe that categories are being imported, as the file shows no issues in pre-upload inspections and because that category is being used to determine what lines are being imported. I can find the products looking by Brand, but something is happening to the format so it does not show up in Category.

Unfortunately I need to be out today until this afternoon, so I won't have any real information to work with until too late today. I do appreciate your help and advice, this particular shop has another problematic feed that has very seldom finished importing so I want to use some of the new Tools I just downloaded to see if they can shed any light before taking up your time today.
Thank you so much,
2fer