Problems using UTF8 encoding

[okbob]

Problems using UTF8 encoding

[okbob]

2017-09-15 23:42 GMT+02:00 Albert Cervera i Areny <[email protected]> :

It seems that pspg is not capable of printing some special characters such as "á" which is printed as "M-CM-!". The symbol is properly shown using "less". LANG is set to ca_ES.utf8 and psql encoding to utf8.

please, can me send a UTF code of this char? can me send test case? Or original output from psql psql \o somefile select * from tab; \o and send me somefile

…

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#20>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAhXcyqYRfxGwMMCpWCS_m-GL2HTb394ks5siu9ggaJpZM4PZi4I> .

[albertca]

Test case output:

x.txt

and dump:

dump.txt

[okbob]

I don't see any issue on these data in my Gnome Terminal - can me send a screenshot? less doesn't use ncurses, so lot of things can be different.

[ajouanjean]

Same problem here

❯ echo $LANG                                                                                                                                                
fr_FR.UTF-8

❯ echo $LC_CTYPE                                                                                                                                            fr_FR.UTF-8

❯ cat .psqlrc #pager is pspg 
[...]
\setenv PAGER 'pspg -s 0'
\set border 2
[...]


❯ psql -d postgres                                                                                                                                          
[local] dtm@postgres=#
[local] dtm@postgres=# SELECT 'un bel été' from generate_series(1, 100);

The output gives this:

And if I try with less as the pager:

[local] dtm@postgres=# \setenv PAGER 'less'
[local] dtm@postgres=# SELECT 'un bel été' from generate_series(1, 100);

I have this:

First I thought it was because I used PuttY (Kitty here) on windows, but same apply to a terminal window in Ubuntu 16.04 (although it is within a VirtualBox machine).

Hope this clarifies.

[okbob]

It is work on my Linux - Fedora 25. Please, can you show log of compilation. Maybe there are some issue with ncurses. "less" doesn't use ncurses, so there should not be this problem.

[okbob]

2017-09-18 10:20 GMT+02:00 ant1jcv <[email protected]>:

Same problem here ❯ echo $LANG fr_FR.UTF-8 ❯ echo $LC_CTYPE fr_FR.UTF-8 ❯ cat .psqlrc #pager is pspg [...] \setenv PAGER 'pspg -s 0' \set border 2 [...] ❯ psql -d postgres [local] ***@***.***=# [local] ***@***.***=# SELECT 'un bel été' from generate_series(1, 100); The output gives this: [image: kitty_portable_2017-09-18_16-15-21] <https://user-images.githubusercontent.com/26791663/30533637-f40ebbfc-9c8c-11e7-8a71-7db064a6df90.png> And if I try with less as the pager: [local] ***@***.***=# \setenv PAGER 'less' [local] ***@***.***=# SELECT 'un bel été' from generate_series(1, 100); I have this: [image: kitty_portable_2017-09-18_16-17-10] <https://user-images.githubusercontent.com/26791663/30533640-f8aa8038-9c8c-11e7-951f-9983d2e091f2.png> First I thought it was because I used PuttY (Kitty here) on windows, but same apply to a terminal window in Ubuntu 16.04 (although it is within a VirtualBox machine). Hope this clarifies.

Maybe it is Ubuntu issue: Pistos/diakonos#71

…

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAhXc3SLDOVN8ctUt7BEoY0HTQPQanO3ks5sjifJgaJpZM4PZi4I> .

[ajouanjean]

Maybe it is a ncurses vs. ncursesw issue:

• ruby/curses#6: says that ncursesw should be used for UTF8 chars ("wide" chars)
• openwrt/packages#526

Sorry, I am no expert at all at C programming (nor at SQL actually), so it might not be a good lead... just some googling around.

Anyway hope this helps!

[okbob]

2017-09-19 3:40 GMT+02:00 ant1jcv <[email protected]>:

Maybe it is a ncurses vs. ncursesw issue: ruby/curses#6 <ruby/curses#6>: says that ncursesw should be used for UTF8 chars ("wide" chars) and openwrt/packages#526 <openwrt/packages#526> Sorry, I am no expert at all at C programming (nor at SQL actually), so it might not be a good lead... just some googling around.

When you use last version and generate Makefile, then ncursesw should be used. But maybe there is some wrong in Ubuntu environment

…

Anyway hope this helps! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAhXczF13UXdIz_djt8m70rw9NhfCnRdks5sjxuNgaJpZM4PZi4I> .

[ajouanjean]

Ok it is working now.

What I did:

• Install latest dev lib of ncursesw: sudo apt-get install libncursesw5-dev
• Update the repo git pull
• Configure

❯ ./configure                                                                                                                                               datamart@occitane-datamart-pink
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for ncursesw via pkg-config... yes
checking for working ncursesw/curses.h... yes
checking for working ncursesw.h... no
checking for working ncurses.h... yes
configure: creating ./config.status
config.status: creating config.make

• Make

❯ make                                                                                                                                                      datamart@occitane-datamart-pink
gcc src/pspg.c -o pspg -g -O2    -lncursesw -ltinfo

• Make install

❯ _ make install                                                                                                                                            datamart@occitane-datamart-pink
tools/install.sh bin pspg "/usr/local/bin"

And now it is OK.
@albertca may be that could help you too.

[albertca]

2017-09-19 5:42 GMT+02:00 ant1jcv <[email protected]>:

Ok it is working now. What I did: - Install latest dev lib of ncursesw: sudo apt-get install libncursesw5-dev - Update the repo git pull - Configure ❯ ./configure ***@***.*** checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for gcc... (cached) gcc checking whether we are using the GNU C compiler... (cached) yes checking whether gcc accepts -g... (cached) yes checking for gcc option to accept ISO C89... (cached) none needed checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for ncursesw via pkg-config... yes checking for working ncursesw/curses.h... yes checking for working ncursesw.h... no checking for working ncurses.h... yes configure: creating ./config.status config.status: creating config.make - Make ❯ make ***@***.*** gcc src/pspg.c -o pspg -g -O2 -lncursesw -ltinfo - Make install ❯ _ make install ***@***.*** tools/install.sh bin pspg "/usr/local/bin" And now it is OK. @albertca <https://github.com/albertca> may be that could help you too.

Yes. It worked correctly. It seems that ./configure already showed me a warning that UTF would not work and I had to completely ignored that. Sorry for the noise! —

…

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFljioPP7ePNfaOxz6KhjrtgEKRTe-8ks5sjzg0gaJpZM4PZi4I> .

[okbob]

2017-09-19 12:42 GMT+02:00 Albert Cervera i Areny <[email protected]> :

2017-09-19 5:42 GMT+02:00 ant1jcv ***@***.***>: > Ok it is working now. > > What I did: > > - Install latest dev lib of ncursesw: sudo apt-get install > libncursesw5-dev > - Update the repo git pull > - Configure > > ❯ ./configure ***@***.*** > checking for gcc... gcc > checking whether the C compiler works... yes > checking for C compiler default output file name... a.out > checking for suffix of executables... > checking whether we are cross compiling... no > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether gcc accepts -g... yes > checking for gcc option to accept ISO C89... none needed > checking how to run the C preprocessor... gcc -E > checking for grep that handles long lines and -e... /bin/grep > checking for egrep... /bin/grep -E > checking for ANSI C header files... yes > checking for gcc... (cached) gcc > checking whether we are using the GNU C compiler... (cached) yes > checking whether gcc accepts -g... (cached) yes > checking for gcc option to accept ISO C89... (cached) none needed > checking for pkg-config... /usr/bin/pkg-config > checking pkg-config is at least version 0.9.0... yes > checking for ncursesw via pkg-config... yes > checking for working ncursesw/curses.h... yes > checking for working ncursesw.h... no > checking for working ncurses.h... yes > configure: creating ./config.status > config.status: creating config.make > > > - Make > > ❯ make ***@***.*** > gcc src/pspg.c -o pspg -g -O2 -lncursesw -ltinfo > > > - Make install > > ❯ _ make install ***@***.*** > tools/install.sh bin pspg "/usr/local/bin" > > And now it is OK. > @albertca <https://github.com/albertca> may be that could help you too. > Yes. It worked correctly. It seems that ./configure already showed me a warning that UTF would not work and I had to completely ignored that. Sorry for the noise!

No problem - I am happy so this issue is fixed

…

— > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#20 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/ AAFljioPP7ePNfaOxz6KhjrtgEKRTe-8ks5sjzg0gaJpZM4PZi4I> > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAhXcyjQ0IfouTUp6bxsi_x8pzAqOGgJks5sj5p_gaJpZM4PZi4I> .