zosopentools/libiconvport

libiconv

GNU libiconv provides an implementation of the iconv() function and the iconv program for character set conversion. For use on systems which don't have one, or whose implementation cannot convert from/to Unicode.

libiconvport's Issues

iconv issue converting 1047 to 819

Commands used to convert the ebcdic file to ascii using iconv and and /bin/iconv
touch
chtag -tc ISO8859-1
iconv -f IBM-1047 -t ISO8859-1 $1.ebcdic >

t IBM-1047 T=on -rw-r----- 1 @02858 @ISSTC00 221 Feb 23 16:46 TBASVC_02132024.csv_GG
t ISO8859-1 T=on -rw-r----- 1 @02858 @ISSTC00 221 Feb 23 16:49 TBASVC_02132024.csv_GG.zopen_iconv
t ISO8859-1 T=on -rw-r----- 1 @02858 @ISSTC00 221 Feb 23 16:52 TBASVC_02132024.csv_GG.iconv

1047 file
bash-5.2$ cat TBASVC_02132024.csv_GG
"DATE","TRANNAME","CLIENTID","TBAIND","TBAS01","cputm","ttrans","cpuptrn","db2reqct","mqreqct",DB2/tran,SUs/tran,TotalCost
"12FEB24","ABCD",00000,999,"SERVICEABC",12445,428684,0.0290,375813738,249414,876.67,209.61,373.35

819 file using iconv
bash-5.2$ cat TBASVC_02132024.csv_GG.zopen_iconv
▒▒▒ႌ▒▒▒++▒(ႌ▒▒<▒▒+▒▒▒▒▒▒▒+▒▒▒▒됑▒▒▒▒▒▒▒_▒▒▒▒▒▒/>˂▒▒▒▒▒▒▒▒>▒▒▒▒▒▒▒▒▒Ȃ▒▒_▒▒▒▒▒Ȃ▒▒▒▒▒/>▒▒▒▒▒▒/>▒▒?▒/%▒?▒Ȏ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒䂌▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒16:56:52 RC=(0) [SYSA] bash-5.2$

819 file using /bin/iconv
bash-5.2$ cat TBASVC_02132024.csv_GG.iconv
"DATE","TRANNAME","CLIENTID","TBAIND","TBAS01","cputm","ttrans","cpuptrn","db2reqct","mqreqct",DB2/tran,SUs/tran,TotalCost
"12FEB24","ABCD",00000,999,"SERVICEABC",12445,428684,0.0290,375813738,249414,876.67,209.61,373.35

Remove - and validate - environment variables being set in shell for libiconv

In buildenv we should only need to set the environment variables on in the binary (e.g. keep zopen_append_to_zoslib_env and remove zopen_append_to_env)

installed code has library dependency to hardcoded zoslib directory

This 'works ok' if you build locally, but if you try to download the libiconv pax file from github and use it, it has the fully qualified path in the dependency

@IgorTodorovskiIBM fyi ... we need a more 'robust' way to 'clean up' paths

iconv man page heading

Reporting what looks to be a surprise.

ICONV(1) Linux Programmer's Manual

iconv fails for conversion _from_ non ASCII when run under _bash_

works fine when run under the standard shell.
@IgorTodorovskiIBM not sure if this is a bash bug, an iconv bug, a zoslib bug, or some combination of all of these.

get check-ebcdic test running after merge

it is now failing but hand-testing is all good

man page says Linux

Update stable release to include changes from DEV release

The stable release does not include the patches that went into the DEV release - such as the new line fix

tests fail for the library code

best guess would be that it's not getting the header files and iconv library in the dev build (but it's a guess)

Issue using the zoau e2a and a2e commands with libiconv (using altbin)

Running e2a and a2e using the default from and to parameters...
looks like files are being tagged without conversion.

[SYSA] bash-5.2$ which iconv
/hewitt/zopentools/zopen_repo/usr/local/altbin/iconv

[SYSA] bash-5.2$ iconv --version
iconv (GNU libiconv 1.17)

[SYSA] bash-5.2$ which e2a
/usr/lpp/IBM/zoau/v1r3/bin/e2a

[SYSA] bash-5.2$ which a2e
/usr/lpp/IBM/zoau/v1r3/bin/a2e

Tag output stream from iconv __toCcsid()

          Should we consider using https://www.ibm.com/docs/en/zos/2.2.0?topic=lf-toccsid-convert-codeset-name-coded-character-set-id ?

Originally posted by @IgorTodorovskiIBM in #5 (comment)

Allow for "/ZOS_UNIX" in iconv_open to be activated via environment variable

In a similar way to the iconv command which respects the ICONV_EBCDIC_ZOS_UNIX environment variable

Invalid iconv output using echo?

What am I missing?
thanks

bash-5.2$ echo "A B C" | iconv -f ISO8859-1 -t IBM-1047
▒@▒@▒RC=(0) [SYSA] bash-5.2$ echo "A B C" | iconv -f ISO8859-1 -t IBM-1047 | od -hc
0000000 c140 c240 c315
301 @ 302 @ 303 025
0000006

--

bash-5.2$ echo "A B C" | od -hc
0000000 4120 4220 430a
A B C \n
0000006

bash-5.2$ echo "A B C" | iconv
A B C
bash-5.2$ echo "A B C" | iconv -f ISO8859-1
A B C
bash-5.2$ echo "A B C" | iconv -f ISO8859-1 -t ISO8859-1
A B C
bash-5.2$ echo "A B C" | iconv -f ISO8859-1 -t ISO8859-1 | od -hc
0000000 4120 4220 430a
A B C \n
0000006

Validate EBCDIC Latin-1 code pages (at a minimum) for accuracy

1047 had (at least) one bug in it where 0x0a -> 0x25 instead of 0x15 (this is already patched locally)

generated file should be properly tagged with the right ccsid

If I iconv from ISO8859-1 to IBM-1047, it should tag the output stream as 1047 ccsid.
Same for all other CCSIDs supported by z/OS.

Question: what about CCSID's that iconv knows about but z/OS doesn't? Do those exist or is the ccsid field just a number that isn't validated?

This has been crudely done for just IBM-1047 but needs to be expanded.