Special characters in feed

Submitted by didkan on Mon, 2010-05-17 10:20 in Price Tapestry

I've just installed PriceTapestry and registered my first feed. However it seems any field in my CSV feed file that contains special characters (such as the Swedish characters å,ä,ö) is not being imported. Is this a known issue or could it be my PHP configuration?

To explain: Any fields containing special characters is not visible in the sample data on feeds_register_step2.php.
The same page also gives this notification:

Notice: Undefined index: merchant in E:\htdocs\test.piilolinssit.fi\admin\feeds_register_step2.php on line 133

Maybe it has to do with the same problem?

Regards
Anders

Hello Anders, The special

Submitted by support on Mon, 2010-05-17 13:40

Hello Anders,

The special characters should not cause any problems with any feed type (CSV or XML), so I'm not quite sure what's going on there.

More on that in a moment, in the meantime, the undefined index warning shouldn't be related, and I will resolve that in the distribution - in the mean time, to suppress warnings, you can add the following code on the very first line of includes/common.php:

error_reporting(E_ERROR);

Regarding the feed not importing special characters, that shouldn't be a problem even if $config_charset is incorrect in config.php. Could you perhaps email me a link to your installation, any password required for /admin/ and the filename of the feed causing the problem and I'll check it out for you...

Cheers,
David.

Hi David, thanks for your

Submitted by didkan on Tue, 2010-05-18 07:09

Hi David, thanks for your prompt reply.

After posting to you I actually got it working (the special character problem) by converting to feed file to utf-8 (previously Latin-1) and that sorted it out. That was of course a temporary fix just to prove PriceTapestry is able to handle special characters (which I never doubted... ;). The next problem occured when I wanted to make the "fix" permanent by using the "encode utf-8" filter (I presume that is what filters are for). When trying to add a filter I got a whole of errors. I eventually tracked it down to the this cause:
The filter table in the db has a field

data (blob, NOT NULL)

and when trying to add the utf-8 filter the script is not passing any value for the data field which threw a db error regarding "no default value for 'data' exists" (or something like that).
To fix it I just removed the 'NOT NULL' attribute from the data field. Will this cause me any other problems with other filters in the future do you think?

This turned out to be more of a bug-report than what the initial question hinted. If you want me to post it in a new thread or somewhere else, please tell me.

Regards
Anders

Hi Anders, Removing NOT NULL

Submitted by support on Tue, 2010-05-18 08:17

Hi Anders,

Removing NOT NULL is fine - null or empty string (which it would be without NOT NULL) will both work OK within the script when returned as a value from mysql_fetch_assco()...

Cheers,
David.