importing 0xe3809c character, aka wave dash

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

importing 0xe3809c character, aka wave dash

Jorg Heymans
Hi,

I am having problems importing sql files containing WAVE DASH or MINUS
character :

WARNING:  ignoring unconvertible UTF-8 character 0xe3809c
WARNING:  ignoring unconvertible UTF-8 character 0xe28892

The data contains japanese characters and is imported correctly apart
from these two. It seems that postgres is filtering out these characters
during import, resulting in incorrect data.


I have isolated a small (<1kb) testcase for the wavedash,
www.domek.be/testwavedash.sql.

Note that the sql files are produced using postgis' shp2pgsql tool from
ESRI shape files.

The database is setup using EUC_JP encoding.

Regards
Jorg

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend
Reply | Threaded
Open this post in threaded view
|

Re: importing 0xe3809c character, aka wave dash

Jorg Heymans
FWIW, I edited the mappings under
src/backend/utils/mb/Unicode, added the character mappings and rebuilt
postgres to make the characters imported correctly.

Learning more about the problem, it seemed that there is no 100%
standard for mapping certain characters and everyone sort of does them
how they see fit.

Regards
Jorg

Jorg Heymans wrote:

> Hi,
>
> I am having problems importing sql files containing WAVE DASH or MINUS
> character :
>
> WARNING:  ignoring unconvertible UTF-8 character 0xe3809c
> WARNING:  ignoring unconvertible UTF-8 character 0xe28892
>
> The data contains japanese characters and is imported correctly apart
> from these two. It seems that postgres is filtering out these characters
> during import, resulting in incorrect data.
>
>
> I have isolated a small (<1kb) testcase for the wavedash,
> www.domek.be/testwavedash.sql.
>
> Note that the sql files are produced using postgis' shp2pgsql tool from
> ESRI shape files.
>
> The database is setup using EUC_JP encoding.
>
> Regards
> Jorg
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
       subscribe-nomail command to [hidden email] so that your
       message can get through to the mailing list cleanly