ibm/python-itoolkit

itoolkit is a Python interface to the XMLSERVICE toolkit for the IBM i platform.

Python XMLSERVICE Toolkit

itoolkit is a Python interface to the XMLSERVICE toolkit for the IBM i platform.

from itoolkit import *
from itoolkit.transport import DatabaseTransport
import pyodbc

conn = pyodbc.connect("DSN=*LOCAL")
itransport = DatabaseTransport(conn)
itool = iToolKit()

itool.add(iCmd5250('wrkactjob', 'WRKACTJOB'))
itool.call(itransport)
wrkactjob = itool.dict_out('wrkactjob')

print(wrkactjob)

For more, check out the samples.

Feature Support

• Call ILE programs & service programs
• Call CL Commands
• Call PASE shell commands

Documentation

The docs can be found at http://python-itoolkit.readthedocs.io/en/latest

Installation

You can install itoolkit simply using pip:

python -m pip install itoolkit

Tests

To test the installed itoolkit

python -m pytest

To test the local code:

PYTHONPATH=src python -m pytest

Contributing

Please read the contribution guidelines.

Releasing a New Version

Run the following commands

# checkout and pull the latest code from master
git checkout master
git pull

# bump to a release version (a tag and commit are made)
bumpversion release

# build the new distribution and upload to PyPI
poetry publish --build

# bump to the new dev version (a commit is made)
bumpversion --no-tag patch

# push the new tag and commits
git push origin master --tags

License

MIT - See LICENSE

python-itoolkit's Issues

SSH transport (new enhancement proposal)

With the release of the new itoolkit-utils package from IBM, it is now feasible to enable SSH as a transport mechanism for the various language toolkits, including Python.

The RPM, when installed, will put a binary, /QOpenSys/pkgs/bin/xmlservice-cgi that can be called through an SSH channel.

This enhancement is very important because it will enable a Python application from any platform (at least any platform where Paramiko works) to talk to IBM i and call RPG, Db2, etc. This can be done without the need to set up an Apache server on IBM i or an ODBC driver on the client. In that sense, it will be simpler than existing transports.

Improve irow handling for 5250 output parameters

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

Retrieving output parameters is somewhat painful:

#!python

itool.add(iCmd('rtvjoba', 'RTVJOBA USRLIBL(?) SYSLIBL(?) CCSID(?N) OUTQ(?)'))
itool.call(itransport)
rtvjoba = itool.dict_out('rtvjoba')
if 'success' in rtvjoba:
    print(rtvjoba['row'][0]['USRLIBL'])
    print(rtvjoba['row'][1]['SYSLIBL'])
    print(rtvjoba['row'][2]['CCSID'])
    print(rtvjoba['row'][3]['OUTQ'])

you can fix it...

#!python

itool = iToolKit(irow=0)
itool.add(iCmd('rtvjoba', 'RTVJOBA USRLIBL(?) SYSLIBL(?) CCSID(?N) OUTQ(?)'))
itool.call(itransport)
rtvjoba = itool.dict_out('rtvjoba')
if 'success' in rtvjoba:
    print(rtvjoba['USRLIBL'])
    print(rtvjoba['SYSLIBL'])
    print(rtvjoba['CCSID'])
    print(rtvjoba['OUTQ'])

... but because irow is specified on the toolkit object instead of the iCmd object, it will break anything which actually returns rows. Should make it available on iCmd object (and make irow=0 the default for output parameters in the iCmd object).

QMHSNDM parameters

Original report by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I am trying to figure out what I am doing wrong with the following call to QMHSNDM. I believe I have the parameter list correctly defined.

To recreate the issue, first create the message queue CRTMSGQ MSGQ(AARONLIB/KAFKA) SIZE(100 100 *NOMAX) TEXT('Kafka') AUT(*CHANGE)

ipgm.py

from itoolkit import *
from itoolkit.lib.ilibcall import *
itransport = iLibCall()

#itool = iToolKit(iparm=0, iret=0, ids=1, irow=0)
itool = iToolKit()

itool.add(
 iPgm('qmhsndm','QMHSNDM',{'error':'on'})
 .addParm(iData('msgId','7a',''))
 .addParm(iData('qualMsgF','20a',''))
 .addParm(iData('msgTxt','2a','Hi'))
 .addParm(iData('msgTxtLen','10i0','2'))
 .addParm(iData('msgType','10a','*INFO'))
 .addParm(iData('msgQueues','20a','KAFKA     AARONLIB  '))
 .addParm(iData('nbrQueues','10i0','1'))
 .addParm(iData('rplyQueue','20a',''))
 .addParm(iData('msgKey','4a',''))
 .addParm(
  iDS('errorCode')
   .addData(iData('bytesProv','10i0',''))
   .addData(iData('bytesAvail','10i0',''))
  )
)

itool.trace_open()
itool.call(itransport)
itool.trace_close()

output = itool.dict_out('sndpgmmsg')
print(output)

Here is the output I am getting.

$ python3 ipgm.py
***********************
control Tue Oct 10 15:54:34 2017
 ctl (*here *cdata) ipc (*na) ebcdic_ccsid (0) pase_ccsid (0)
input Tue Oct 10 15:54:34 2017
<?xml version='1.0'?>
<xmlservice><pgm error="on" name="QMHSNDM" var="qmhsndm"><parm io="both" var="p1"><data type="7a" var="msgId"/></parm><parm io="both" var="p2"><data type="20a" var="qualMsgF"/></parm><parm io="both" var="p3"><data type="2a" var="msgTxt"><![CDATA[Hi]]></data></parm><parm io="both" var="p4"><data type="10i0" var="msgTxtLen"><![CDATA[2]]></data></parm><parm io="both" var="p5"><data type="10a" var="msgType"><![CDATA[*INFO]]></data></parm><parm io="both" var="p6"><data type="20a" var="msgQueues"><![CDATA[KAFKA     AARONLIB  ]]></data></parm><parm io="both" var="p7"><data type="10i0" var="nbrQueues"><![CDATA[1]]></data></parm><parm io="both" var="p8"><data type="20a" var="rplyQueue"/></parm><parm io="both" var="p9"><data type="4a" var="msgKey"/></parm><parm io="both" var="p10"><ds var="errorCode"><data type="10i0" var="bytesProv"/><data type="10i0" var="bytesAvail"/></ds></parm></pgm></xmlservice>

output Tue Oct 10 15:54:34 2017
<?xml version='1.0'?>
<xmlservice><pgm error="on" name="QMHSNDM" var="qmhsndm">
<parm io="both" var="p1">
<data type="7a" var="msgId"><![CDATA[]]></data>
</parm>
<parm io="both" var="p2">
<data type="20a" var="qualMsgF"><![CDATA[]]></data>
</parm>
<parm io="both" var="p3">
<data type="2a" var="msgTxt"><![CDATA[Hi]]></data>
</parm>
<parm io="both" var="p4">
<data type="10i0" var="msgTxtLen"><![CDATA[2]]></data>
</parm>
<parm io="both" var="p5">
<data type="10a" var="msgType"><![CDATA[*INFO]]></data>
</parm>
<parm io="both" var="p6">
<data type="20a" var="msgQueues"><![CDATA[KAFKA     AARONLIB]]></data>
</parm>
<parm io="both" var="p7">
<data type="10i0" var="nbrQueues"><![CDATA[1]]></data>
</parm>
<parm io="both" var="p8">
<data type="20a" var="rplyQueue"><![CDATA[]]></data>
</parm>
<parm io="both" var="p9">
<data type="4a" var="msgKey"><![CDATA[
parse (fail) Tue Oct 10 15:54:34 2017
3c3f786d6c2076657273696f6e3d2731 <?xml version='1
2e30273f3e0a3c786d6c736572766963 .0'?>.<xmlservic
653e3c70676d206572726f723d226f6e e><pgm error="on
22206e616d653d22514d48534e444d22 " name="QMHSNDM"
207661723d22716d68736e646d223e0a  var="qmhsndm">.
3c7061726d20696f3d22626f74682220 <parm io="both"
7661723d227031223e0a3c6461746120 var="p1">.<data
747970653d22376122207661723d226d type="7a" var="m
73674964223e3c215b43444154415b5d sgId"><![CDATA[]
5d3e3c2f646174613e0a3c2f7061726d ]></data>.</parm
3e0a3c7061726d20696f3d22626f7468 >.<parm io="both
22207661723d227032223e0a3c646174 " var="p2">.<dat
6120747970653d223230612220766172 a type="20a" var
3d227175616c4d736746223e3c215b43 ="qualMsgF"><![C
444154415b5d5d3e3c2f646174613e0a DATA[]]></data>.
3c2f7061726d3e0a3c7061726d20696f </parm>.<parm io
3d22626f746822207661723d22703322 ="both" var="p3"
3e0a3c6461746120747970653d223261 >.<data type="2a
22207661723d226d7367547874223e3c " var="msgTxt"><
215b43444154415b48695d5d3e3c2f64 ![CDATA[Hi]]></d
6174613e0a3c2f7061726d3e0a3c7061 ata>.</parm>.<pa
726d20696f3d22626f74682220766172 rm io="both" var
3d227034223e0a3c6461746120747970 ="p4">.<data typ
653d223130693022207661723d226d73 e="10i0" var="ms
675478744c656e223e3c215b43444154 gTxtLen"><![CDAT
415b325d5d3e3c2f646174613e0a3c2f A[2]]></data>.</
7061726d3e0a3c7061726d20696f3d22 parm>.<parm io="
626f746822207661723d227035223e0a both" var="p5">.
3c6461746120747970653d2231306122 <data type="10a"
207661723d226d736754797065223e3c  var="msgType"><
215b43444154415b2a494e464f5d5d3e ![CDATA[*INFO]]>
3c2f646174613e0a3c2f7061726d3e0a </data>.</parm>.
3c7061726d20696f3d22626f74682220 <parm io="both"
7661723d227036223e0a3c6461746120 var="p6">.<data
747970653d2232306122207661723d22 type="20a" var="
6d7367517565756573223e3c215b4344 msgQueues"><![CD
4154415b4b41464b4120202020204141 ATA[KAFKA     AA
524f4e4c49425d5d3e3c2f646174613e RONLIB]]></data>
0a3c2f7061726d3e0a3c7061726d2069 .</parm>.<parm i
6f3d22626f746822207661723d227037 o="both" var="p7
223e0a3c6461746120747970653d2231 ">.<data type="1
30693022207661723d226e6272517565 0i0" var="nbrQue
756573223e3c215b43444154415b315d ues"><![CDATA[1]
5d3e3c2f646174613e0a3c2f7061726d ]></data>.</parm
3e0a3c7061726d20696f3d22626f7468 >.<parm io="both
22207661723d227038223e0a3c646174 " var="p8">.<dat
6120747970653d223230612220766172 a type="20a" var
3d2272706c795175657565223e3c215b ="rplyQueue"><![
43444154415b5d5d3e3c2f646174613e CDATA[]]></data>
0a3c2f7061726d3e0a3c7061726d2069 .</parm>.<parm i
6f3d22626f746822207661723d227039 o="both" var="p9
223e0a3c6461746120747970653d2234 ">.<data type="4
6122207661723d226d73674b6579223e a" var="msgKey">
3c215b43444154415b <![CDATA[
parse step: 2 (1-ok, 2-*BADPARSE, 3-*NOPARSE)
{'error': {'error1': ' ?xml version=\'1.0\'? xmlservice pgm error="on" name="QMHSNDM" var="qmhsndm" parm io="both" var="p1" data type="7a" var="msgId" ![CDATA[]] /data /parm parm io="both" var="p2" data type="20a" var="qualMsgF" ![CDATA[]] /data /parm parm io="both" var="p3" data type="2a" var="msgTxt" ![CDATA[Hi]] /data /parm parm io="both" var="p4" data type="10i0" var="msgTxtLen" ![CDATA[2]] /data /parm parm io="both" var="p5" data type="10a" var="msgType" ![CDATA[*INFO]] /data /parm parm io="both" var="p6" data type="20a" var="msgQueues" ![CDATA[KAFKA AARONLIB]] /data /parm parm io="both" var="p7" data type="10i0" var="nbrQueues" ![CDATA[1]] /data /parm parm io="both" var="p8" data type="20a" var="rplyQueue" ![CDATA[]] /data /parm parm io="both" var="p9" data type="4a" var="msgKey" ![CDATA[', 'sndpgmmsg': {...}, 'error': '*BADPARSE'}}

Make PyPI distribution installable via pip on other platforms

In issue #26 you mentioned that you have a version of the code which can be built on other platforms (presumably skipping the compilation required for iLibCall). Does that refer to what's proposed in #16?

Regardless, are you ready to release that yet? Or, if it was already slated to be included in 1.6.0, then I can report that it's not working for me on Windows. ;)

You also gave a workaround in #26 that consists of invoking pip with a bunch of optional parameters. Would it be worth documenting that in a more prominent place, until the simpler pip invocation is working?

QMHSNDM parameters

Original report by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I am trying to figure out what I am doing wrong with the following call to QMHSNDM. I believe I have the parameter list correctly defined.

To recreate the issue, first create the message queue CRTMSGQ MSGQ(AARONLIB/KAFKA) SIZE(100 100 *NOMAX) TEXT('Kafka') AUT(*CHANGE)

ipgm.py

from itoolkit import *
from itoolkit.lib.ilibcall import *
itransport = iLibCall()

#itool = iToolKit(iparm=0, iret=0, ids=1, irow=0)
itool = iToolKit()

itool.add(
 iPgm('qmhsndm','QMHSNDM',{'error':'on'})
 .addParm(iData('msgId','7a',''))
 .addParm(iData('qualMsgF','20a',''))
 .addParm(iData('msgTxt','2a','Hi'))
 .addParm(iData('msgTxtLen','10i0','2'))
 .addParm(iData('msgType','10a','*INFO'))
 .addParm(iData('msgQueues','20a','KAFKA     AARONLIB  '))
 .addParm(iData('nbrQueues','10i0','1'))
 .addParm(iData('rplyQueue','20a',''))
 .addParm(iData('msgKey','4a',''))
 .addParm(
  iDS('errorCode')
   .addData(iData('bytesProv','10i0',''))
   .addData(iData('bytesAvail','10i0',''))
  )
)

itool.trace_open()
itool.call(itransport)
itool.trace_close()

output = itool.dict_out('sndpgmmsg')
print(output)

Here is the output I am getting.

$ python3 ipgm.py
***********************
control Tue Oct 10 15:54:34 2017
 ctl (*here *cdata) ipc (*na) ebcdic_ccsid (0) pase_ccsid (0)
input Tue Oct 10 15:54:34 2017
<?xml version='1.0'?>
<xmlservice><pgm error="on" name="QMHSNDM" var="qmhsndm"><parm io="both" var="p1"><data type="7a" var="msgId"/></parm><parm io="both" var="p2"><data type="20a" var="qualMsgF"/></parm><parm io="both" var="p3"><data type="2a" var="msgTxt"><![CDATA[Hi]]></data></parm><parm io="both" var="p4"><data type="10i0" var="msgTxtLen"><![CDATA[2]]></data></parm><parm io="both" var="p5"><data type="10a" var="msgType"><![CDATA[*INFO]]></data></parm><parm io="both" var="p6"><data type="20a" var="msgQueues"><![CDATA[KAFKA     AARONLIB  ]]></data></parm><parm io="both" var="p7"><data type="10i0" var="nbrQueues"><![CDATA[1]]></data></parm><parm io="both" var="p8"><data type="20a" var="rplyQueue"/></parm><parm io="both" var="p9"><data type="4a" var="msgKey"/></parm><parm io="both" var="p10"><ds var="errorCode"><data type="10i0" var="bytesProv"/><data type="10i0" var="bytesAvail"/></ds></parm></pgm></xmlservice>

output Tue Oct 10 15:54:34 2017
<?xml version='1.0'?>
<xmlservice><pgm error="on" name="QMHSNDM" var="qmhsndm">
<parm io="both" var="p1">
<data type="7a" var="msgId"><![CDATA[]]></data>
</parm>
<parm io="both" var="p2">
<data type="20a" var="qualMsgF"><![CDATA[]]></data>
</parm>
<parm io="both" var="p3">
<data type="2a" var="msgTxt"><![CDATA[Hi]]></data>
</parm>
<parm io="both" var="p4">
<data type="10i0" var="msgTxtLen"><![CDATA[2]]></data>
</parm>
<parm io="both" var="p5">
<data type="10a" var="msgType"><![CDATA[*INFO]]></data>
</parm>
<parm io="both" var="p6">
<data type="20a" var="msgQueues"><![CDATA[KAFKA     AARONLIB]]></data>
</parm>
<parm io="both" var="p7">
<data type="10i0" var="nbrQueues"><![CDATA[1]]></data>
</parm>
<parm io="both" var="p8">
<data type="20a" var="rplyQueue"><![CDATA[]]></data>
</parm>
<parm io="both" var="p9">
<data type="4a" var="msgKey"><![CDATA[
parse (fail) Tue Oct 10 15:54:34 2017
3c3f786d6c2076657273696f6e3d2731 <?xml version='1
2e30273f3e0a3c786d6c736572766963 .0'?>.<xmlservic
653e3c70676d206572726f723d226f6e e><pgm error="on
22206e616d653d22514d48534e444d22 " name="QMHSNDM"
207661723d22716d68736e646d223e0a  var="qmhsndm">.
3c7061726d20696f3d22626f74682220 <parm io="both"
7661723d227031223e0a3c6461746120 var="p1">.<data
747970653d22376122207661723d226d type="7a" var="m
73674964223e3c215b43444154415b5d sgId"><![CDATA[]
5d3e3c2f646174613e0a3c2f7061726d ]></data>.</parm
3e0a3c7061726d20696f3d22626f7468 >.<parm io="both
22207661723d227032223e0a3c646174 " var="p2">.<dat
6120747970653d223230612220766172 a type="20a" var
3d227175616c4d736746223e3c215b43 ="qualMsgF"><![C
444154415b5d5d3e3c2f646174613e0a DATA[]]></data>.
3c2f7061726d3e0a3c7061726d20696f </parm>.<parm io
3d22626f746822207661723d22703322 ="both" var="p3"
3e0a3c6461746120747970653d223261 >.<data type="2a
22207661723d226d7367547874223e3c " var="msgTxt"><
215b43444154415b48695d5d3e3c2f64 ![CDATA[Hi]]></d
6174613e0a3c2f7061726d3e0a3c7061 ata>.</parm>.<pa
726d20696f3d22626f74682220766172 rm io="both" var
3d227034223e0a3c6461746120747970 ="p4">.<data typ
653d223130693022207661723d226d73 e="10i0" var="ms
675478744c656e223e3c215b43444154 gTxtLen"><![CDAT
415b325d5d3e3c2f646174613e0a3c2f A[2]]></data>.</
7061726d3e0a3c7061726d20696f3d22 parm>.<parm io="
626f746822207661723d227035223e0a both" var="p5">.
3c6461746120747970653d2231306122 <data type="10a"
207661723d226d736754797065223e3c  var="msgType"><
215b43444154415b2a494e464f5d5d3e ![CDATA[*INFO]]>
3c2f646174613e0a3c2f7061726d3e0a </data>.</parm>.
3c7061726d20696f3d22626f74682220 <parm io="both"
7661723d227036223e0a3c6461746120 var="p6">.<data
747970653d2232306122207661723d22 type="20a" var="
6d7367517565756573223e3c215b4344 msgQueues"><![CD
4154415b4b41464b4120202020204141 ATA[KAFKA     AA
524f4e4c49425d5d3e3c2f646174613e RONLIB]]></data>
0a3c2f7061726d3e0a3c7061726d2069 .</parm>.<parm i
6f3d22626f746822207661723d227037 o="both" var="p7
223e0a3c6461746120747970653d2231 ">.<data type="1
30693022207661723d226e6272517565 0i0" var="nbrQue
756573223e3c215b43444154415b315d ues"><![CDATA[1]
5d3e3c2f646174613e0a3c2f7061726d ]></data>.</parm
3e0a3c7061726d20696f3d22626f7468 >.<parm io="both
22207661723d227038223e0a3c646174 " var="p8">.<dat
6120747970653d223230612220766172 a type="20a" var
3d2272706c795175657565223e3c215b ="rplyQueue"><![
43444154415b5d5d3e3c2f646174613e CDATA[]]></data>
0a3c2f7061726d3e0a3c7061726d2069 .</parm>.<parm i
6f3d22626f746822207661723d227039 o="both" var="p9
223e0a3c6461746120747970653d2234 ">.<data type="4
6122207661723d226d73674b6579223e a" var="msgKey">
3c215b43444154415b <![CDATA[
parse step: 2 (1-ok, 2-*BADPARSE, 3-*NOPARSE)
{'error': {'error1': ' ?xml version=\'1.0\'? xmlservice pgm error="on" name="QMHSNDM" var="qmhsndm" parm io="both" var="p1" data type="7a" var="msgId" ![CDATA[]] /data /parm parm io="both" var="p2" data type="20a" var="qualMsgF" ![CDATA[]] /data /parm parm io="both" var="p3" data type="2a" var="msgTxt" ![CDATA[Hi]] /data /parm parm io="both" var="p4" data type="10i0" var="msgTxtLen" ![CDATA[2]] /data /parm parm io="both" var="p5" data type="10a" var="msgType" ![CDATA[*INFO]] /data /parm parm io="both" var="p6" data type="20a" var="msgQueues" ![CDATA[KAFKA AARONLIB]] /data /parm parm io="both" var="p7" data type="10i0" var="nbrQueues" ![CDATA[1]] /data /parm parm io="both" var="p8" data type="20a" var="rplyQueue" ![CDATA[]] /data /parm parm io="both" var="p9" data type="4a" var="msgKey" ![CDATA[', 'sndpgmmsg': {...}, 'error': '*BADPARSE'}}

Clean up Sphinx documentation build errors

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

docstrings should be converted to restructuredtext or at least escape certain special values in rst as we currently get a bunch of warnings building documentation:

/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iToolKit.trace_open:1: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iToolKit.trace_open:3: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iPgm:1: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iPgm:5: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iPgm:5: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iPgm:7: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iPgm:22: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iPgm.add:10: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iSrvPgm:1: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iSrvPgm:5: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iSrvPgm:5: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iSrvPgm:7: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iSrvPgm.add:10: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iCmd:1: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iCmd:1: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iCmd:10: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iCmd.add:10: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iCmd5250:1: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iCmd5250:1: WARNING: Inline emphasis start-string without end-string.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iCmd5250.add:10: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iSh.add:10: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iDS.add:10: WARNING: Unexpected indentation.
/home/kadler/projects/python-itoolkit/src/itoolkit/itoolkit.py:docstring of itoolkit.iData.add:10: WARNING: Unexpected indentation.

Drop Python 2.7 Support

Since Python 2.7 went end of life as of Jan 2020, we should drop support at some point in the future. This would allow some simplification of the code and allow us to use inline typing. I'm not sure how many users there are of the Python 2 version.

iData should allow non-str values

The way the XML is generated, an exception will be generated unless all the values are strings. This should be fixed so that you can pass in a number. eg.

iPgm('zzcall', 'ZZCALL').addParm(iData('var', '7p4', '32.1234'))

should be able to be specified as

iPgm('zzcall', 'ZZCALL').addParm(iData('var', '7p4', 32.1234))

Migrate iLibCall to ctypes

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

Currently iLibCall is C code and can only be compiled on IBM i. This makes building a pain, since we can only put a dist on PyPI and not a wheel. If we moved iLibCall to ctypes, it would not require any compilation at all, so we could build a wheel, which wouldn't require a build during install either.

Transports need a way to indicate failure

What is the use case for this enhancement? ie. What problem are you trying to solve?

Currently, transports do not have a defined way to indicate that they failed to call XMLSERVICE. These types of failures would include, eg.

• HttpTransport failing to connect to the remote endpoint
• Missing stored procedures for DatabaseTransport
• Missing xmlservice-cli utility for the SshTransport
• Error loading XMLSERVICE in DirectTransport
• ...

Describe the solution you'd like

I think we should have a defined Exception class for transport errors defined in itoolkit.transport. If a transport receives an error while attempting to call XMLSERVICE, it could throw this error. If it has already caught an exception, the caught exception could be added as the cause of the thrown transport exception if applications want more details. Transports could subclass the main exception if they need to, but I think that may be unnecessary.

# itoolkit.transport.exception.py
import exceptions

class TransportError(exceptions.StandardError):
    pass

# test.py

from itoolkit.transport import DirectTransport, TransportError
from itoolkit import *

tk = iToolKit()
transport = DirectTransport()

try:
    tk.call(transport)
except TransportError as e:
    print(e)

_direct.py needs more test coverage

Currently _direct.py has 8% test coverage as we completely bypass this module when testing direct.py.

We'll need to mock ctypes a bit to make it work, but it shouldn't be too bad.

How to access objects that are in Auxiliary Storage Pool (ASP) ?

I'm not a RPG developer but we are trying automate some using ansible on IBM I and we came across an issue to related Auxiliary Storage Pool. We have our libraries or programs in ASP and we are trying to access them via itoolkit. Is it possible to access via itoolkit ?

If it's not right place to ask this question, please close the issue.

On Japanese or French systems run command via DatabaseTransport fails with ibm_db_dbi::Error

• itoolkit version: python3-itoolkit-1.6.1-1.ppc64
• Python version: python3-3.6.10-2.ppc64
• OS Name and Version: 7.2 Japanese, 7.3 French
• IBM i version: 7.2 Japanese, 7.3 French
• XMLSERVICE version: XML Toolkit 1.9.9.3

Describe the bug

Code

conn = dbi.connect()
itool = iToolKit()
itransport = DatabaseTransport(conn)
itool.add(iCmd('command1', command, {'error': 'on'}))
itool.call(itransport)
command_output = itool.dict_out('command1')

Error
For example, calling the above code to run CL command "DLTLIB LIB(NOTEXISTS)", then an error is reported as below. The error doesn't appear on English system.

Traceback (most recent call last):
File "/tmp/ansible_ibmi_object_save_payload_bwwr4eug/ansible_ibmi_object_save_payload.zip/ansible/module_utils/ibmi/ibmi_util.py", line 173, in itoolkit_run_command
File "/QOpensys/pkgs/lib/python3.6/site-packages/itoolkit/itoolkit.py", line 1155, in call
xml_out = itrans.call(self)
File "/QOpensys/pkgs/lib/python3.6/site-packages/itoolkit/transport/database.py", line 66, in call
return "".join(row[0] for row in cursor).rstrip('\0')
File "/QOpensys/pkgs/lib/python3.6/site-packages/itoolkit/transport/database.py", line 66, in
return "".join(row[0] for row in cursor).rstrip('\0')
File "/QOpensys/pkgs/lib/python3.6/site-packages/ibm_db_dbi.py", line 1128, in next
row = self.fetchone()
File "/QOpensys/pkgs/lib/python3.6/site-packages/ibm_db_dbi.py", line 1492, in fetchone
row_list = self._fetch_helper(1)
File "/QOpensys/pkgs/lib/python3.6/site-packages/ibm_db_dbi.py", line 1476, in _fetch_helper
raise self.messages[-1]
ibm_db_dbi.Error: ibm_db_dbi::error: SystemError(' returned a result with an error set',)

Have no idea why the can't be displayed on the git issue. So put a snapshot here.

New feature request: ODBC connection support with itoolkit_lite

Original report by Jesse G (Bitbucket: ThePrez, GitHub: ThePrez).

---

I would like the following code to work:

#!python
import pyodbc
from itoolkit import *
from itoolkit.db2.idb2call import *

conn = pyodbc.connect('DRIVER={IBM i Access ODBC Driver};SYSTEM=system.mydomain.com;UID=MYUSER;PASSWORD=PASSWORD')
itransport = iDB2Call(conn)

However, the idb2call import fails with:

ModuleNotFoundError: No module named 'ibm_db'

Which makes sense. We'd depend on ibm_db is there to support calls where the toolkit fetches the connection for you, like so:

#!python
itransport = iDB2Call(user,password)

As a test, I installed ibm_db (not easy) and used this variation of iDB2Call

#!python
itransport = iDB2Call(conn)

Resulting error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\Python36\lib\site-packages\itoolkit\itoolkit.py", line 1098, in call
    xml_out = itrans.call(self)
  File "C:\Python\Python36\lib\site-packages\itoolkit\db2\idb2call.py", line 131, in call
    stmt = ibm_db.prepare(conn, sql)
Exception: Supplied connection object Parameter is invalid

Inspecting the code, it looks like we are using the ibm_db support for prepared statements. Since we have a connection object already, I suspect everything can be accomplished with the PEP0249 spec in such a way that any database connection would work.

Of course, it would impact existing design a bit. My personal preference would be to always require the user to fetch a PEP0249 connection object as step 1.

If we had this feature, we could remotely connect to IBM i via ODBC without requiring the extra steps of XMLService setup. It would still remain generic enough for local DB2 calls to work or ibm_db calls from off-box, including IBM BlueMix. I am unconvinced that performance would improve.

iCmd, iSh, iPgm, ... should take kwargs instead of options dictionary

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

compare:

#!python

iPgm('foo', 'FOO', {'lib': 'MYLIB', 'function': 'bar' }) # call function 'bar' in MYLIB/FOO

to:

#!python

iPgm('foo', 'FOO', lib='MYLIB', function='bar')

https://pythontips.com/2013/08/04/args-and-kwargs-in-python-explained/ (this works similarly in Ruby as well, just search for kwargs)

Migrate CI and publishing to GitHub Actions

GitHub Actions will make it easier for us to publish releases to PyPI automatically.

State full XMLSERVICE calls

Reading up online how to make use of state full XMLSERVICE calls for faster response and simulate an open recurring job (as if I was using the 5250 terminal window), this is the route I would like to go. Especially since I would be able to gather data based on an "elapsed time" as well. For example, I could run "wrkdsksts" and get read/writes/reqs about disks since the time the job was first run, or maybe "dspsyssts" and get CPU over a period of time. Without this, I would only get data at the time the job was executed and it would be a snapshot at that time, not over a period of time.

My question is, using ipc and ctl parameters during the transport definition of idb2call, is this possible? Does the call to the stored procedure end the job or will it leave it open to re-use if I were to call it again?

I'm trying to follow the examples here (It's PHP, but at least gives me an idea of what's capable):
http://yips.idevcloud.com/wiki/index.php/XMLService/XMLSERVICEConnect
http://yips.idevcloud.com/wiki/index.php/XMLService/XMLSERVICEConfig

Thanks!

Drop Python 3.4 Support

Like #56, but for Python 3.4. Python 3.4 went EOL over a year ago and Python 3.4 in OPS went EOL at the same time or at least by the end of 2019. I think we should time this with 2.0 as well, considering the usage of OPS and the fact that we still need to support 2.7.

iCmd5250 doesn't work properly for CL commands with parameters

@kadler after installing SI70667 PTF we are getting following error

sh: syntax error at line 1 : `(' unexpected
<?xml version="1.0" ?><xmlservice><sh error="on" var="aspgroup_cmd">
</sh>
</xmlservice>

Originally posted by @ranga543 in #48 (comment)

Python 3 and iToolkit error

Original report by Siegfried (Bitbucket: zipperlemeran, GitHub: Unknown).

---

Hello, I tried to execute the following program from the IBM supplied examples in /QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/itoolkit/sample:

python3 icmd5250_dspsyssts.py

and got the following error:
Traceback (most recent call last):
File "icmd5250_dspsyssts.py", line 16, in
itool.call(config.itransport)
File "/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/itoolkit/itoolkit.py", line 1098, in call
xml_out = itrans.call(self)
File "/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/itoolkit/lib/ilibcall.py", line 88, in call
return itoolkit.itoollib.xmlservice(itool.xml_in(),self.ctl,self.ipc,self.ebcdic_ccsid,self.pase_ccsid)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 802: ordinal not in range(128)

Attached you can find information about the environment variables the the Python modules installed.

Thank you
Siegfried

Project needs pull request and issue templates

I propose we use something similar to IBM/nodejs-itoolkit#92

Cannot escape question mark in CL command (was: Weird behavior trying to escape single-quotes)

Sometimes, when trying to issue a CL command with embedded single-quotes, it doesn't work. But sometimes it does. It's probably clearest to illustrate with code:

#! /QOpenSys/pkgs/bin/python3

from itoolkit import iToolKit, iCmd
from itoolkit.db2.idb2call import iDB2Call as DatabaseTransport
# The above line is deprecated and will eventually be:
# from itoolkit.transport import DatabaseTransport

import ibm_db_dbi as db2

conn = db2.connect()
trans = DatabaseTransport(conn)

itool = iToolKit(irow=0)

# No apostrophes - OK
itool.add(iCmd('cmd', "SNDMSG MSG('Hello, everyone!') TOUSR(jyeung)"))
itool.call(trans)
out = itool.dict_out('cmd')
print(out)

itool.clear()

# One apostrophe - OK
itool.add(iCmd('cmd', "SNDMSG MSG('Howdy, y''all!') TOUSR(jyeung)"))
itool.call(trans)
out = itool.dict_out('cmd')
print(out)

itool.clear()

# Two apostrophes - fail
itool.add(iCmd('cmd', "SNDMSG MSG('Howdy, y''all! How ''bout another aposterfee?') TOUSR(jyeung)"))
itool.call(trans)
out = itool.dict_out('cmd')
print(out)

itool.clear()

# Two apostrophes - OK!?
itool.add(iCmd('cmd', "SNDMSG MSG('First: ''; second: ''.') TOUSR(jyeung)"))
itool.call(trans)
out = itool.dict_out('cmd')
print(out)

Three of those four work fine. The third and longest one produces an error ('CPF0006', as reported in the 'error1' output dictionary).

All four work from a genuine CL command line.

PyPI page is outdated

Not sure this is the right place to report, but the PyPI page for itoolkit has some outdated links.

In particular, the project homepage in the left navbar, the XMLSERVICE project link, samples, contribution guidelines, and license.

I think it is definitely good for the project to continue to be listed on PyPI. As such, it's worth keeping that page up to date.

That use itoolkit to run one sql query returns an BADPARSE error.

Recreated steps:

1. Use iSqlQuery to run a sql "SELECT DATA FROM qusrsys.QATOCTCPIP WHERE KEYWORD='RMTNAMESV';" Below is the code.

   itool = iToolKit()

   itool.add(iSqlQuery('query', sql, {'error': 'on'}))
   itool.add(iSqlFetch('fetch'))
   itool.add(iSqlFree('free'))

   itool.call(itransport)

   command_output = itool.dict_out('fetch')

2. However, seems there is a BADPARSE error.

The output:
{'error': {u'error1': u' ?xml version='1.0'? xmlservice sql var="query" query error="on" var="query" conn='conn1' stmt='stmt1' success ![CDATA[+++ success SELECT DATA FROM qusrsys.QATOCTCPIP WHERE KEYWORD='RMTNAMESV';]] /success /query /sql sql var="fetch" fetch block="all" error="fast" var="fetch" stmt='stmt1' row data desc='DATA' ![CDATA[9.0.128.50 9.5.178.0 U ', 'fetch': {...}, u'error': u'*BADPARSE'}}"

Migrate setup to poetry

We're using poetry for other projects since it seems the best modern solution for dependency management for Python. We should use it here, too.

Enforce PEP-8 style guide

Code should be cleaned up to use PEP-8 guidelines.

Move documentation to repo

Original report by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Currently the documentation for the python-itoolkit is located here. It should be moved to this repo so it is 1) more tightly coupled with the actual code, and 2) change tracking is in place.

Do IBM'ers agree? @rangercairns @ThePrez @kadler

Note, this is what we're doing for the Node.js db connector documentation.

I will seek out community volunteers to accomplish task if IBM'ers give thumbs up.

Failing Database transport tests

https://github.com/IBM/python-itoolkit/runs/128588156

Improve subpackage handling

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

This is annoying:

#!python

from itoolkit.lib.ilibcall import *
from itoolkit.db2.idb2call import *
from itoolkit.rest.irestcall import *

Better would be

#!python

from itoolkit import iLibCall
from itoolkit import iDb2Call
from itoolkit import iRestCall

and let itoolkit package handle attempting to load the correct subpackage above and throw exception if can't be loaded.

Add type declarations

https://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html
https://mypy.readthedocs.io/en/latest/existing_code.html

This would allow us to have better docs using https://pypi.org/project/sphinx-autodoc-typehints/

Ideally, we would use inline type declarations, but they can't be used if the code needs to be Python 2 compatible. For Python 2 compatibility, you must use comments, but this requires Python 3.8 or typed-ast be installed in order for sphinx-autodoc-typehints to be able to read them.

Cannot install itoolkit due to iconv.h

Hi,

I've tried installing itoolkit but every time it tries to install it says it cannot build it because it is missing iconv.h
I've installed all sorts of libraries from Visual Studio hoping that the internationalization converter is in it and nothing. I've even gone to lengths of building my own libiconv found here and still nothing:
https://www.codeproject.com/Articles/302012/%2FArticles%2F302012%2FHow-to-Build-libiconv-with-Microsoft-Visual-Studio

Just simply running the "pip install itoolkit" gives me this error:

running build_ext
    building 'itoolkit/itoollib' extension
    creating build\temp.win-amd64-3.6
    creating build\temp.win-amd64-3.6\Release
    creating build\temp.win-amd64-3.6\Release\itoolkit
    creating build\temp.win-amd64-3.6\Release\itoolkit\lib
    C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ic:\users\firstname.lastname\appdata\local\programs\python\python36\include -Ic:\users\firstname.lastname\appdata\local\programs\python\python36\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt" -IC:\PROGRA~1\IBM\SQLLIB\INCLUDE -IC:\PROGRA~1\IBM\SQLLIB\LIB "-IC:\Program Files\IBM\MQ\tools\c\include" "-IC:\Program Files\IBM\MQ\tools\cplus\include" /Tcitoolkit/lib/itoollib.c /Fobuild\temp.win-amd64-3.6\Release\itoolkit/lib/itoollib.obj
    itoollib.c
    itoolkit/lib/itoollib.c(16): fatal error C1083: Cannot open include file: 'iconv.h': No such file or directory
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2

Am I missing some other dependency to be able to install itoolkit? Has anyone run into this issue before?

Thanks,

Deegz

Provide a close() API to close database connections

What is the use case for this enhancement? ie. What problem are you trying to solve?

There will be many QSQSRVR jobs on IBM i systems if these database connections are not closed in time.

Describe the solution you'd like

Provide a close() API to close database connection, and update the examples if needed.

iLibCall fails on 64-bt Python versions

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

All calls to XMLSERVICE through iLibCall fail with *NODATA. Other transports are unaffected.

Example:

#!python
from itoolkit import *
from itoolkit.lib.ilibcall import *

itransport = iLibCall()
itool = iToolKit()

itool.add(iCmd5250('wrkactjob', 'WRKACTJOB'))
print(itool.xml_in())
itool.call(itransport)
wrkactjob = itool.dict_out('wrkactjob')
print(wrkactjob)

Output:

#!text
<?xml version='1.0'?>
<xmlservice><sh error="fast" var="wrkactjob"><![CDATA[/QOpenSys/usr/bin/system WRKACTJOB]]></sh></xmlservice>

{'error': {'error': '*NODATA', 'wrkactjob': {...`

Passing variables using addParm

Hi,

is there a way to pass variables with addParm of ipgm call instead of quoted values. I checked the documentation, but the examples show using quoted strings for passing values.

itool.add(
  iPgm('mykey','MYPGM')
    .addParm(iData('parm1','10p0','3458589'))
    .addParm(iData('parm2','5p0','1253'))
)

For example in the above code I would like to use parm1 & parm2 values to be based on variables defined in my python script dynamically for each call instead of quoted values 345859 and 1253 from above.

I am using XMLSERVICE db2 call for above and running python 3.6.8 in virtual environment on V7R3 machine.
Appreciate your suggestions/help.

Thanks,
Arun

Transport calling mechanism seems inverted

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

How you call itoolkit transports seems backwards to me:

#!python

itransport = iLibCall()
itool = iToolKit()
itool.add(...)
itool.call(itransport)

would make more sense to me as:

#!python

itransport = iLibCall()
itool = iToolKit()
itool.add(...)
itransport.call(itool)

Migrate relevant documentation from YiPs

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

Migrate any useful documentation from the YiPs page in to the docs/ directory.

Need to know or learn reStructuredText.

I would guess about the only thing not already here is the bit about QTEMP.

Allow iPgm.addParm to pass options to iParm

Right now, you have to import itoolkit.itoolkit.iParm separately and jump through some hoops to pass a parameter by value to an ILE procedure, because addParm doesn't give you access to any iParm options.

I've submitted pull request #28 to address this. (More detailed text is attached to the pull request.)

iSrvPgm lib option doesn't work

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

#!python

from itoolkit import *
itool = iToolKit()
itool.add(iSrvPgm('test','TEST','test', iopt={'lib': 'MYLIB'}))
print(itool.xml_in())

gives

#!xml

<?xml version='1.0'?>
<xmlservice><pgm error="fast" func="test" name="TEST" var="test"/></xmlservice>

The problem is here:

#!python

    def __init__(self, ikey, iname, ifunc, iopt={}):
        # parent class
        iPgm.__init__(self, ikey, iname, {'func': ifunc})

iopt is completely ignored

ODBC transport doesn't work

Original report by John Yeung (Bitbucket: jky, GitHub: jky).

---

There are a couple of small fixes needed to bring it up to minimal functionality:

1. On the cursor execute (currently line 111 in iodbccall.py), ipc and clt are undefined. They need to be self.ipc and self.ctl, respectively. (Note the flipping of letters in the latter.)

2. The "rows" being returned are true database rows (as you would expect from PEP 249) so they cannot be directly appended to the XML output string. You have to get the first (and only) element, which is a string, and strip off the nulls (currently line 116):

   xml_out += row[0].rstrip('\x00')

When I made those changes, the simple iCmd example (with RTVJOBA) worked.

Is there a problem with the Japanese environment in iCmd5250 xml output?

• itoolkit version: 1.6.1
• Python version: 3.6.11
• OS Name and Version: IBM i
• IBM i version: V7R3M0
• XMLSERVICE version: 2.0.2-dev
• ** DSPSYSVAL SYSVAL(QCCSID)** : 5035

Describe the bug

Is there a possibility that XML will be corrupted if Japanese is included in the result of iCmd5250?

The following will result in an error
Does the output contain Japanese?

from itoolkit import *
from itoolkit.transport import DatabaseTransport
import ibm_db_dbi

itool.add(iCmd5250('dspsyssts', 'DSPSYSSTS', {'error': 'on'}))
itool.call(itransport)
dspsyssts = itool.dict_out('dspsyssts')
if 'error' in dspsyssts:
    print(itool.xml_out())
    exit()
else:
    print(dspsyssts['dspsyssts'])

result

<?xml version="1.0" ?><xmlservice>
<error>*BADPARSE</error>
<error><![CDATA[ ?xml version='1.0'? xmlservice sh error="on" var="dspsyssts" ![CDATA[ 1 5770SS1 V7R3M0 160422 SPIKE73 20/10/16 08:54:49 JST CPU % . . . . . . . . . . . . . . : .9 ASP . . . . . . . . . . . . . : 95.45 G CPU % . . . . . . . : .1 ASP % . . . . . . . . . : 45.4769 . . . . . . . . . . . . . . . : 00:00:01 . . . . . . . . . . . . : 95.45 G . . . . . . . . . . : 1381 . . . . . . . . . . . . . : 4885 M % . . . . . . . . . . . . : .007 . . . . . . . . . . . . : 4928 M % . . . . . . . . . . . . : .010 s mw MAX -------DB------- ----- DB----- ACT- WAIT- ACT- b M b M ACT s s WAIT INEL INEL mw e ] 1 679.20 342.25 +++++ .0 .0 .0 .0 54.9 .0 .0 *MACHINE *FIXED 2 4447.21 6.00 121 .0 .0 .0 .0 4344.6 .0 .0 *BASE *FIXED 3 575.97 .00 144 .0 .0 .0 .0 .0 .0 .0 *INTERACT *FIXED 4 57.59 .00 5 .0 .0 .0 .0 .0 .0 .0 *SPOOL *FIXED * * * * * * * * * *]] /sh /xmlservice ]]></error>
</xmlservice>

The following will succeed
Doesn't Japanese appear in the output?

from itoolkit import *
from itoolkit.transport import DatabaseTransport
import ibm_db_dbi

conn = ibm_db_dbi.connect()
itransport = DatabaseTransport(conn)
itool = iToolKit()

itool = iToolKit(iparm=0, iret=0, ids=1, irow=0)
itool.add(iCmd('rtvjoba', 'RTVJOBA USRLIBL(?) SYSLIBL(?) CCSID(?N) OUTQ(?)'))
itool.call(itransport)

rtvjoba = itool.dict_out('rtvjoba')
print (rtvjoba)
if 'error' in rtvjoba:
    print (rtvjoba['error'])
    exit()
else:
    print('USRLIBL = ' + rtvjoba['USRLIBL'])
    print('SYSLIBL = ' + rtvjoba['SYSLIBL'])
    print('CCSID   = ' + rtvjoba['CCSID'])
    print('OUTQ    = ' + rtvjoba['OUTQ'])

result

{'success': '+++ success RTVJOBA USRLIBL(?) SYSLIBL(?) CCSID(?N) OUTQ(?)', 'USRLIBL': 'QGPL       QTEMP', 'SYSLIBL': 'QSYS       QSYS2      QHLPSYS    QUSRSYS', 'CCSID': '5035', 'OUTQ': '*DEV'}
USRLIBL = QGPL       QTEMP
SYSLIBL = QSYS       QSYS2      QHLPSYS    QUSRSYS
CCSID   = 5035
OUTQ    = *DEV

iCmd5250 not working with DirectTransport

@kadler I tried with iCmd5250 but it seems to be throwing some kind of error
test.py

from itoolkit import *
from itoolkit.transport import DirectTransport

itransport = DirectTransport()
itool = iToolKit()
itool.add(iCmd5250('aspgroup_cmd', 'SETASPGRP ASPGRP(TEST)'))
itool.call(itransport)
cmd_out = itool.dict_out('aspgroup_cmd')

if 'error' in cmd_out:
  print ('test')
  print (cmd_out['error'])
  exit()

{'aspgroup_cmd': {'error': {...}}, u'error1': u" ?xml version='1.0'? xmlservice /sh /xmlservice ", u'error': u'*BADPARSE'}

Originally posted by @ranga543 in #47 (comment)

Method to specify keys seems backwards

Original report by Kevin Adler (Bitbucket: kadler, GitHub: kadler).

---

The way to specify keys seems backwards:

#!python

itool.add(
  iPgm('mykey','MYPGM')
    .addParm(iData('inchara','1a','a'))
    .addParm(
      iDS('INDS1')
        .addData(iData('dscharb','1a','b'))
        .addData(iData('desdec','12p2','3.33'))
  )
)

makes more sense to me as:

#!python

itool.add('mykey', iPgm('MYPGM')
    .addParm('inchara', iData('1a','a'))
    .addParm('INDS1', iDS()
        .addData('dscharb', iData('1a','b'))
        .addData('desdec', iData('12p2','3.33'))
  )
)

Need to document how to use ODBC as a transport

Inconsistency with the iData class.

Original report by Anonymous.

---

As seen in the documentation for iData : http://yips.idevcloud.com/wiki/index.php/XMLSERVICE/Python#iData

The arguments at the top are listed as ikey, iparm, itype, ival, and iopt:

      ikey  (str): XML <data ... var="ikey"> for parsing output.
      iparm (obj): dom for parameter or return or ds.
      itype (obj): data type [see XMLSERVICE types, '3i0', ...].
      ival  (obj): data type value.
      iopt (dict): option - dictionay of options (below)

However the source shows that the the class doesn't use iparm in the init function:

#!python

def __init__(self, ikey, itype, ival, iopt={}):

We're having some issues with getting data back from srvpgms and we think this information might help.

Thanks

calling rpg return success but no rpg executed in IBM i

Hello Kevin
this is my code :

from itoolkit import *
from itoolkit.db2.idb2call import *
itransport = iDB2Call()
itool = iToolKit()

itool.add(iCmd('addlib1', 'addlible SICGA00F55'))
itool.add(iCmd('addlib2', 'addlible $$CLI00F55'))
itool.add(iCmd('addlib3', 'addlible $$CLI00O'))
itool.add(iCmd('callpgm', "CALL PGM($$CLI00O/OC005) PARM(X'001F' X'0056217F' ' ' )"))
itool.call(itransport)
data = itool.dict_out()
print(data)

this is the result :

4.3:zeta@~/pyWork> python3 rpg1.py
{'addlib1': {'success': '+++ success addlible SICGA00F55'}, 'addlib2': {'success': '+++ success addlible $$CLI00F55'}, 'addlib3': {'success': '+++ success addlible $$CLI00O'}, 'callpgm': {'success': "+++ success CALL PGM($$CLI00O/OC005) PARM(X'001F' X'0056217F' ' ' )"}}

all i see is success , however the call CALL PGM($$CLI00O/OC005) PARM(X'001F' X'0056217F' ' ' ) does not call the programm OC005.
How can i debug it ?

Also i tried this approach to call my program :

itool.add(
iPgm('my_key','OC005')
.addParm(iData('gcnusr','1','2p0'))
.addParm(iData('gcnuec','56217','6p0'))
.addParm(iData('$status','1a',' '))
)

but always returns error!!!

4.3:zeta@~/pyWork> python3 rpg1.py
{'addlib1': {'success': '+++ success addlible SICGA00F55'}, 'addlib2': {'success': '+++ success addlible $$CLI00F55'}, 'addlib3': {'success': '+++ success addlible $$CLI00O'}, 'my_key': {'error': '*** error OC005', 'version': 'XML Toolkit 1.9.9.3', 'xmlhint': 'p(44) 2p0 ', 'xmlhint1': 'p(69) <data type="1" var="gcnusr"', 'xmlhint2': 'p(69) 2p0', 'xmlhint3': 'p(69) 0000000000000000', 'xmlhint4': 'p(44) <parm io="both" var="p1"', 'xmlhint5': 'p(44) 2p0 ', 'xmlhint6': '<parm io="both"', 'xmlhint7': 'p(44) <parm io="both" var="p1"', 'xmlhint8': 'p(44) 2p0 ', 'jobipc': '*na', 'jobipcskey': 'FFFFFFFF', 'jobname': 'QSQSRVR', 'jobuser': 'QUSER', 'jobnbr': '167411', 'curuser': 'RAMOS', 'ccsid': '1140', 'dftccsid': '1140', 'paseccsid': '819', 'syslibl': 'QCIRES QSYS QSYS2 QHLPSYS QUSRSYS UTILIT', 'usrlibl': '$$CLI00O $$CLI00F55 SICGA00F55 QGPL QTEMP QFNTCPL300 QFNTCPL QFNT09'}}

any help ?

Regards

Deprecate the trace functions for standard logging

Python has a built in logging infrastructure. We could leverage that instead of writing our own (not very well implemented) tracing facility. A good doc page on how to set things up and investigate errors using logging (including adding a filter to just get the itoolkit calls) would be even better.

Undefined name: 'data' in ./src/itoolkit/transport/direct.py

flake8 testing of https://github.com/IBM/python-itoolkit on Python 3.7.1

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./src/itoolkit/transport/direct.py:83:20: F821 undefined name 'data'
            return data.decode('utf-8')
                   ^
./src/itoolkit/transport/direct.py:85:20: F821 undefined name 'data'
            return data
                   ^
2     F821 undefined name 'data'
2

E901,E999,F821,F822,F823 are the "showstopper" flake8 issues that can halt the runtime with a SyntaxError, NameError, etc. Most other flake8 issues are merely "style violations" -- useful for readability but they do not effect runtime safety.

• F821: undefined name name
• F822: undefined name name in __all__
• F823: local variable name referenced before assignment
• E901: SyntaxError or IndentationError
• E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

SQL commands in SQLRPGLE programs causing ibm_db.connect (from ibm_db library) to fail for new connections

This is a strange issue that I've come across:

I'm calling SQLRPGLE programs using the iToolkit library, for some reason when I call a program that has SQL statements like select, insert, etc. the connection is somehow being locked and other users intermittently cannot connect and the ibm_db.connect method (from ibm_db library) fails and returns None. I'm not sure if anyone had a similar issue and I realize the issue comes from calling a different library but the stack trace fails in this method, any help would be greatly appreciated!

Thanks in advance

Deprectated transports do not work

iRestCall, iDB2Call, and iLibCall do not work, bu t always return 'error': '*NODATA'.

The problem is caused by refactoring of the code as sub-classes of HttpTransport, DatabaseTransport, and DirectTransport and not returning a value from the sub-classed call methods, causing None to be returned instead.

Run cl command to create a library in an IASP via DirectTransport on a French system failed

• itoolkit version: python3-itoolkit-1.6.1-1.ppc64
• Python version: python3-3.6.10-2.ppc64
• OS Name and Version: IBM i 7.3 French
• IBM i version: IBMi 7.3 French
• XMLSERVICE version: XML Toolkit 1.9.9.3

Describe the bug

On IBM i French system run a CL command which is to create a library in an iASP fails. The issue doesn't appear on an English system.

command = 'CRTLIB LIB(TRYIASP1) ASP(*ASPDEV) ASPDEV(IASP1)'
itransport = DirectTransport()
itool.add(iCmd(‘command’, command, {‘error’: ‘on’}))
itool.call(itransport)

"stderr": "iToolKit result dict does not have key 'joblog', the output is {'error': {'error': '*BADPARSE', 'error1': ' ?xml version='1.0'? xmlservice cmd error="on" exec="cmd" var="command" success |¢CDATA¢+++ success QSYS/SETASPGRP ASPGRP(IASP1)!! /success /cmd cmd error="fast" exec="cmd" var="chgjob" success |¢CDATA¢+++ success CHGJOB CCSID(37)!! /success /cmd cmd error="on" exec="cmd" var="command" success |¢CDATA¢+++ success CRTLIB LIB(TRYIASP1) ASP(*ASPDEV) ASPDEV(IASP1)!! /success /cmd /xmlservice ', 'command': {...}}}",