ibm/xmlservice

XML-based interface for accessing IBM i resources

XMLSERVICE

XMLSERVICE is a set of procedures written in ILE RPG that allow you to interact with IBM i resources such as programs and commands using a plain XML protocol. XMLSERVICE can be called directly or via high-level language toolkit.

Example

TODO ...

Documentation

Documentation is at https://xmlservice.readthedocs.io/

Building from Source

Build requirements

Building requires Python 3 and GNU make. These can be installed with yum: yum install python3 make-gnu

You will also need the ILE RPG compiler installed (5770-WDS option 31) along with the following PTFs:

• 7.3: SI62605
• 7.2: SI62604
• 7.1: SI62580

Building

PATH=/QOpenSys/pkgs/bin:$PATH

git clone https://github.com/IBM/xmlservice.git

cd xmlservice

python3 ./configure

make

Customizing the Build

You can customize the build by passing options to configure:

• --library: set the build library
• --debug: set the debug level (DBGVIEW CL parameter)

Running the Tests

TODO ...

Language Toolkits

• .NET
• Node.js
• PHP
• Python
• Ruby
• Swift

Contributing

See CONTRIBUTING.md

Releasing a New Version

This project uses bumpversion to manage its version numbers. To update official releases, run the following commands:

# checkout and pull the latest code from master
git checkout master
git pull

# bump to a release version (a tag and commit are made)
bumpversion release

# bump to the new dev version (a commit is made)
bumpversion --no-tag patch

# push the new tag and commits
git push origin master --tags

License

BSD

xmlservice's Issues

Qp0sDisableSignals ???

What's the point of mapping Qp0sDisableSignals to Qp0sEnableSignals ?

      *++++++++++++++++++++++++++++++++++++++++++++++++++++++++
      * Qp0sDisableSignals(): Disable signals
      *++++++++++++++++++++++++++++++++++++++++++++++++++++++++
     D Qp0sDisableSignals...
     D                 PR            10I 0 extproc('Qp0sEnableSignals')

Convert tests to not require external toolkits

There are various tests already, but they are written using the PHP or Python toolkits. These should be converted to not require these toolkits and call XMLSERVICE directly.

Steps involved:

• Build XMLSERVICE test runner
• Convert PHP tests to raw XMLSERVICE tests
• Convert Python tests to raw XMLSERVICE tests

I imagine the test framework being being basically a set of input XML files and corresponding expected output XML files, along with some sort of test runner.

Drop calling /usr/lib/start32 to bring PASE up

For a mixed requests in xmlserver input with requests that call both PASE(/usr/lib/start32) and JVM(QSYS/QSQSTPJC) in a job, SQL query failed later. As start32 is not recommended to be called with security feature enabled anymore, thinking of only use JVM to wake PASE up. Main changes are in plugpase.rpgle.
Currently in parm ctl, *JAVA/*SQLJAVA is used to start JVM.

1. <sh> triggers start32. *java triggers JVM. FAILED on db2_query().

call qxmlserv.iPlug32k('*na',
'*here*sqljava',
'<?xml version=''1.0''?>
<myscript>

<sh> ps -ef </sh>

<sql>
  <options options=''opt1'' autocommit=''off'' commit=''none''/>
  <connect conn=''local'' options=''opt1''/>
  <query>
    SELECT * FROM QUSRSYS.QATOCHOST
  </query>
  <fetch block=''all'' desc=''on''/>
</sql>

</myscript>
',
'?');

2. WORKS. After *java start JVM to have PASE up as well, won't trigger calling start32.

call qxmlserv.iPlug32k('*na',
'*here*sqljava',
'<?xml version=''1.0''?>
<myscript>

<sql>
  <options options=''opt1'' autocommit=''off'' commit=''none''/>
  <connect conn=''local'' options=''opt1''/>
  <query>
    SELECT * FROM QUSRSYS.QATOCHOST
  </query>
  <fetch block=''all'' desc=''on''/>
</sql>

<sh> ps -ef </sh>

</myscript>
',
'?');

spawn call overwrites instead of ORs flags

See:

xmlservice/src/plugipc.rpgle

Line 1488 in 20800d6

inherit.flags = SPAWN_SETJOBNAMEARGV_NP;

It overwrites a mutually exclusive job name flag, but it's also getting rid of the multithreaded job flag. Is this intentional? If so, the previous two lines should be dropped or this turned into an |=?

Version Identifiable?

Original report by Brian Jerome (Bitbucket: bjerome, GitHub: brianmjerome).

---

Is is possible to determine the version installed? I'd like to check for updates to the version periodically.

If not, can this be added somehow?

Would it be possible to install the XMLSERVICE through YUM like newer versions of Node?

Migrate PHP tests to PHP toolkit repo

While we're converting the existing PHP tests to not use the PHP toolkit in #20, they may be useful to https://github.com/zendtech/IbmiToolkit. We should submit them over there.

XMLSERVICE jobs run in QBATCH

Original report by Don (Bitbucket: msddcb, GitHub: Unknown).

---

We have Zend server running on a Linux box using ODBC connection to IBMi
V7R3.

Currently all the XMLSERVICE jobs are running in QBATCH and submitted via
the QS36EVOKE job queue.

I presume that this is based on the job description xmlservice/ZSVR_JOBD
that specifies that job queue.

While I expect if I change the job queue entry on this job description that
the XMLSERVICE jobs will then use that job queue and I intend on either
creating a new sub-system description or use the QUSRWRK sub-system.

As this is a production environment I am hoping someone can confirm a
change to the job queue on the job description will not cause any
unexpected results.

Thanks to any responses

Invoke XMLSERVICE from PASE shell

Original report by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I am in a situation where it would be handy to invoke XMLSERVICE from a shell script. Being able to do so will allow me to eliminate the need to install Ruby/Node.js/Python iToolKits.

Here's what I have so far:

$ alias db2='function _db2(){ echo "Running $1"; system -i "call QSYS/QZDFMDB2 parm('\''$1'\'')"; };_db2'
$ db2 "CALL QXMLSERV.iPLUG4K('''', ''*immed'', ''<?xml version="1.0"?>'', ?)"
Running CALL QXMLSERV.iPLUG4K('''', ''*immed'', ''<?xml version=1.0?>'', ?)

 **** CLI ERROR *****
         SQLSTATE: 07001
NATIVE ERROR CODE: -313
Number of host variables not valid.

This error makes sense, but I am not finding a way to specify a host variable from a shell to obtain the output of the last parameter. I believe this might be remedied by wrapping iPLUG4K with another RPG program that writes to standard out, but I wanted to first check if anyone has gotten it to work without that.

Stored procedures truncate result

Originally I wrote the itoolkit for Python equivalent to use system API QESRSRVA to retrieve service attributes. I debugged to the point the the stored procedure is called and just directly called it.

There seems to be an issue where the maximum records returned from the iPLUG stored procedures is 4 of 3000 length each.

I have tested this with ODBC through itoolkit and JDBC through Run SQL Scripts utility. Drivers have been updated to current versions from ACS packages. I've tried both stateless and stateful (IPC) methods. User profile is setup with CCSID 37.

Here is the call:

call QXMLSERV.iPLUGR512K('*na', '*here *cdata', '<?xml version="1.0"?><xmlservice><pgm error="on" name="QESRSRVA" var="retrieveServiceAttrs"><parm io="both" var="p1"><ds len="receriverVarLen" var="receiverVar"><data type="10i0" var="numServiceAttrRetrieved"/><data dim="14" type="10i0" var="offsetsToServiceAttrTempls"/><ds var="serviceAttrTemplates"><ds var="key1template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="1a" var="attr"/></ds></ds><ds var="key2template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="1a" var="attr"/></ds></ds><ds var="key3template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="1a" var="nameFormat"/><data type="17a" var="serviceProviderName"/></ds></ds><ds var="key4template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="10a" var="typeOfPTFInstall"/></ds></ds><ds var="key5template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data enddo="numOfEntries" type="10i0" var="numOfEntries"/><data dim="50" dou="numOfEntries" type="10a" var="userList"/></ds></ds><ds var="key6template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="1a" var="attr"/></ds></ds><ds var="key7template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="1a" var="attr"/></ds></ds><ds var="key8template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="10i0" var="attr"/></ds></ds><ds var="key9template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="10a" var="ecsMsgQueue"/><data type="10a" var="library"/></ds></ds><ds var="key10template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="30a" var="attr"/></ds></ds><ds var="key11template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="30a" var="attr"/></ds></ds><ds var="key12template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="30a" var="attr"/></ds></ds><ds var="key13template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data type="1a" var="attr"/></ds></ds><ds var="key14template"><data type="10i0" var="serviceAttrKey"/><data type="1a" var="dataTypeOfServiceAttr"/><data type="1a" var="statusOfServiceAttr"/><data type="2a" var="reserved"/><data type="10i0" var="lengthOfServiceAttr"/><ds var="serviceAttr"><data enddo="numOfEntries2" type="10i0" var="numOfEntries"/><data dim="20" dou="numOfEntries2" type="10a" var="ptfGroupList"/></ds></ds></ds></ds></parm><parm io="both" var="p2"><data setlen="receriverVarLen" type="10i0" var="lenOfReceiverVar"/></parm><parm io="both" var="p3"><data type="10i0" var="numOfServiceAttrKeys"><![CDATA[14]]></data></parm><parm io="both" var="p4"><ds var="serviceAttrKeys"><data type="10i0" var="key1"><![CDATA[1]]></data><data type="10i0" var="key2"><![CDATA[2]]></data><data type="10i0" var="key3"><![CDATA[3]]></data><data type="10i0" var="key4"><![CDATA[4]]></data><data type="10i0" var="key5"><![CDATA[5]]></data><data type="10i0" var="key6"><![CDATA[6]]></data><data type="10i0" var="key7"><![CDATA[7]]></data><data type="10i0" var="key8"><![CDATA[8]]></data><data type="10i0" var="key9"><![CDATA[9]]></data><data type="10i0" var="key10"><![CDATA[10]]></data><data type="10i0" var="key11"><![CDATA[11]]></data><data type="10i0" var="key12"><![CDATA[12]]></data><data type="10i0" var="key13"><![CDATA[13]]></data><data type="10i0" var="key14"><![CDATA[14]]></data></ds></parm><parm io="both" var="p5"><ds len="errorCodeLen" var="errorCode"><data type="10i0" var="bytesProvided"/><data type="10i0" var="bytesAvailable"/><data setlen="errorCodeLen" type="7a" var="exceptionID"/><data type="1a" var="reserved"/></ds></parm></pgm></xmlservice>');

For reference, here is the original itoolkit build:

            itool = iToolKit()
            itool.add(
                iPgm("retrieveServiceAttrs", "QESRSRVA", {"error": "on"})
                .addParm(
                    iDS("receiverVar", {"len": "receriverVarLen"}) # output
                    .addData(iData("numServiceAttrRetrieved", "10i0"))
                    .addData(
                        iData("offsetsToServiceAttrTempls", "10i0", iopt={"dim": "14"})
                    )
                    .addData(
                        iDS("serviceAttrTemplates")
                        .addData(
                            iDS("key1template") # Automatic Problem Analysis
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "1a"))
                            )
                        )
                        .addData(
                            iDS("key2template") # Automatic Problem Reporting
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "1a"))
                            )
                        )
                        .addData(
                            iDS("key3template") # Service Provider to Report Problem
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("nameFormat", "1a"))
                                .addData(iData("serviceProviderName", "17a"))
                            )
                        )
                        .addData(
                            iDS("key4template") # PTF Install Type
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("typeOfPTFInstall", "10a"))
                            )
                        )
                        .addData(
                            iDS("key5template") # Critical Message Recipients
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("numOfEntries", "10i0", iopt={"enddo": "numOfEntries"}))
                                .addData(iData("userList", "10a", iopt={"dim": "50", "dou": "numOfEntries"}))
                            )
                        )
                        .addData(
                            iDS("key6template") # Send Data Packets
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "1a"))
                            )
                        )
                        .addData(
                            iDS("key7template") # Copy PTFs
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "1a"))
                            )
                        )
                        .addData(
                            iDS("key8template") # PTF group levels
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "10i0"))
                            )
                        )
                        .addData(
                            iDS("key9template") # ECS message queue
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("ecsMsgQueue", "10a"))
                                .addData(iData("library", "10a"))
                            )
                        )
                        .addData(
                            iDS("key10template") # System-Disabled Reporting Connection Number
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "30a"))
                            )
                        )
                        .addData(
                            iDS("key11template") # System-Disabled Call-Back Connection Number
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "30a"))
                            )
                        )
                        .addData(
                            iDS("key12template") # Service Provider Connection Number
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "30a"))
                            )
                        )
                        .addData(
                            iDS("key13template") # Order PTFs automatically
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("attr", "1a"))
                            )
                        )
                        .addData(
                            iDS("key14template") # Order PTF groups
                            .addData(iData("serviceAttrKey", "10i0"))
                            .addData(iData("dataTypeOfServiceAttr", "1a"))
                            .addData(iData("statusOfServiceAttr", "1a"))
                            .addData(iData("reserved", "2a"))
                            .addData(iData("lengthOfServiceAttr", "10i0"))
                            .addData(
                                iDS("serviceAttr")
                                .addData(iData("numOfEntries", "10i0", iopt={"enddo": "numOfEntries2"}))
                                .addData(iData("ptfGroupList", "10a", iopt={"dim": "20", "dou": "numOfEntries2"}))
                            )
                        )
                    )
                )
                .addParm(iData("lenOfReceiverVar", "10i0", iopt={"setlen": "receriverVarLen"}))  # input
                .addParm(iData("numOfServiceAttrKeys", "10i0", "14"))  # input
                .addParm(
                    iDS("serviceAttrKeys") # input
                    .addData(iData("key1", "10i0", "1"))
                    .addData(iData("key2", "10i0", "2"))
                    .addData(iData("key3", "10i0", "3"))
                    .addData(iData("key4", "10i0", "4"))
                    .addData(iData("key5", "10i0", "5"))
                    .addData(iData("key6", "10i0", "6"))
                    .addData(iData("key7", "10i0", "7"))
                    .addData(iData("key8", "10i0", "8"))
                    .addData(iData("key9", "10i0", "9"))
                    .addData(iData("key10", "10i0", "10"))
                    .addData(iData("key11", "10i0", "11"))
                    .addData(iData("key12", "10i0", "12"))
                    .addData(iData("key13", "10i0", "13"))
                    .addData(iData("key14", "10i0", "14"))
                )
                .addParm(
                    iDS("errorCode", {"len": "errorCodeLen"}) # input/output
                    .addData(iData("bytesProvided", "10i0")) # input
                    .addData(iData("bytesAvailable", "10i0")) # output
                    .addData(iData("exceptionID", "7a", iopt={"setlen": "errorCodeLen"})) # output
                    .addData(iData("reserved", "1a")) # output
                    #.addData(iData("exceptionData", "char(*)")) # output: TODO
                )
            )

Edit: It seems that the parameters do not align correctly so I will try to remap lengths appropriately (though RPG style from C style conversions seem to be correct after multiple takes). However, I believe there may still be an indirect bug. Just taking a wild guess but there may be some kind of overflow issue where more checking of sizes needs to happen. I may take a crack at debugging the RPG at some point.

Output Not Escaped

Original report by Danny Roessner (Bitbucket: droessner, GitHub: danny-hcs).

---

I have this test RPG program:

#!rpg
**free

dcl-pi *N;
  output char(10);
end-pi;

output = '&<>"''';

return;         

I use XMLSERVICE to call the program:

#!SQL

call QXMLSERV.iPLUG4K(
    'NA',
    '*here',
    '<pgm name="ESCAPETEST" lib="DROESSNER" error="on"><parm io="out"><data name="OUTPUT" type="10a"></data></parm></pgm>',
    'NULL'
);

The output is:

#!XML
<pgm name="ESCAPETEST" lib="DROESSNER" error="on">
  <parm io="out">
    <data name="OUTPUT" type="10a">&<>"'</data>
  </parm>
  <success><![CDATA[+++ success DROESSNER ESCAPETEST ]]></success>
</pgm>

None of the special XML characters are being escaped.

DATA tags in output have wrong converted chars after migrated to new build environment

In new compile environment, RPG source files stay in ccsid 819. CRTRPGMOD from IFS sources files could not specify conversion to job ccsid. The best way may be change the xmlservice code to be independent with source file ccsid. Use hexidecimal to initialize the Unicode constants for DATA tag.

Bashing Packed Value

Original report by Anonymous.

---

I am having trouble with something in xmlservice. I am trying to pass a 20p0 field back in a webservice via xmlservice, and it bashing my value... the simple answer might just be that packed decimal doesn't support a 20p0 definition... but my rpg programs are working fine with that. In the RPG program you see it is 12345678901234567890 (bb). In the result you can see that I am getting a different value.

This is the RPG program.(TESTPGM)

H AlwNull(*UsrCtl)

D p1 S 20p 0
C *Entry PLIST
C PARM p1
/free
p1 = 12345678901234567890;
*inlr = *on;
return;
/end-free

Here is the XMLService script... simple test...

Secure call XMLSERVICE please

User:
Password:
XML Input:
<textarea readonly name="xmlin" rows="20" cols="100"> </textarea>

My response is the following

**12345678901234567000**

Migrate documentation from Yips Wiki to docs

The raw wiki content was migrated under b18c314 and the text version under 140f1ce. The docs there need to be converted to understandable documentation and converted to restructured text under the docs directory.

NOTE: Each text file contains the URL to the page it was retrieved from to make it easy to look at the rendered HTML, as it's sometimes useful to see the documentation visually.

The basic structure I have in mind is this:

• An intro page with the basics
• A page to document the list of actions/tags available with links to each action's documentation
• A page/section for each action's documentation
• A page describing all the ways you can call XMLSERVICE: CGI, database stored procedures, etc
• A page describing the control options and what they do
• A page describing what IPC is and why to use it (should be noted that it's quite advanced topic)
• A page on error handling (hah!)
• A page on pitfalls (numerous)

Using qsh

Original report by Nathanael BONNET (Bitbucket: nbonnet_gaia, GitHub: nathanaelGaia).

---

Hi,

With version 1.9.8, I tried to use qsh.

It works for a simple commande like ls, but I got an error with /QIBM/ProdData/OS/WebServices/V1/server/bin/listWebServices.sh

#!rpgle
**free
ctl-opt actgrp(*new) ;

dcl-s myXmlIn  char(4096) inz ;
dcl-s myXmlOut char(4096) inz ;

// Doesn't work
myXmlIn =
   '<?xml version="1.0"?><script><qsh rows="on">' +
     '/QIBM/ProdData/OS/WebServices/V1/server/bin/' +
       'listWebServices.sh -server ''ws_test''' +
   '</qsh></script>' ;
exec sql call XMLSERVICE/iPLUG4K('*na','*here',:myXmlIn,:myXmlOut);

// Works fine
myXmlIn =
   '<?xml version="1.0"?><script><qsh rows="on">' +
     'ls' +
   '</qsh></script>' ;
exec sql call XMLSERVICE/iPLUG4K('*na','*here',:myXmlIn,:myXmlOut);

return; 

myXmlOut contains :

#!xml
<?xml version="1.0"?>
<script>
	<qsh rows="on">
		<error>*** error
			/QIBM/ProdData/OS/WebServices/V1/server/bin/listWebServices.sh
			-server 'ws_test'</error>
		<error>202</error>
		<version>XML Toolkit 1.9.8</version>
		<error>
			<status>202</status>
			<errnoxml>1100014</errnoxml>
			<xmlerrmsg><![CDATA[XML run pgm excp]]></xmlerrmsg>
			<xmlhint><![CDATA[xml pase exec]]></xmlhint>
		</error>
		<error>
			<status>202</status>
			<errnoxml>1100014</errnoxml>
			<xmlerrmsg><![CDATA[XML run pgm excp]]></xmlerrmsg>
			<xmlhint><![CDATA[xml pase exec]]></xmlhint>
		</error>
		<jobinfo>
			<jobipc></jobipc>
			<jobipcskey>FFFFFFFF</jobipcskey>
			<jobname>QPADEV000C</jobname>
			<jobuser>QSECOFR</jobuser>
			<jobnbr>158541</jobnbr>
			<jobsts>*ACTIVE</jobsts>
			<curuser>QSECOFR</curuser>
			<ccsid>1147</ccsid>
			<dftccsid>1147</dftccsid>
			<paseccsid>0</paseccsid>
			<langid>FRA</langid>
			<cntryid>FR</cntryid>
			<sbsname>QINTER</sbsname>
			<sbslib>QSYS</sbslib>
			<curlib>NB</curlib>
			<syslibl>QSYS QSYS2 QHLPSYS QUSRSYS</syslibl>
			<usrlibl>OUTILS QGPL QTEMP</usrlibl>
			<jobcpffind>see log scan, not error list</jobcpffind>
		</jobinfo>
	</qsh>
</script>

I tried with using CDATA, several values for IPC and Ctl, qualifing by /usr/bin/sh.

Thanks
Nathanaël

Passing variable by value

Original report by Sebastian Misiewicz (Bitbucket: SMisiewicz, GitHub: sebastian-misiewicz).

---

Hi
I'm trying to call service program function (MQOPEN) WebSphere MQ API on iSeries.
The MQOPEN procedure is defined as

#!rpg
D* MQOPEN Function -- Open Object
D******************************************************

DMQOPEN           PR                  EXTPROC('MQOPEN')
D* Connection handle
D HCONN                         10I 0 VALUE
D* Object descriptor
D OBJDSC                       468A
D* Options that control the action of MQOPEN
D OPTS                          10I 0 VALUE
D* Object handle
D HOBJ                          10I 0
D* Completion code
D CMPCOD                        10I 0
......

As you can see 1st and 3rd parameters are passed by VALUE.

But the doc says that passing by value is untested and should not be used
(I'm using XML -> DB2 connection)
I could create an ILE RPG which accepts parameters by reference then call appropriate
api with variable passed by value. But I'm wondering if it's possible to fix it ?
I was trying to debug plugxml.rpgle and looks like list of parameters pointers were corrupted

#!rpg

       // by value argument?
       if node.xmlPrmRet = XML_IS_PARM
       and node.xmlBy = XML_BY_VAL
       and node.xmlCallAs = XML_FUNC_SRVPGM;
         pCopy = pArgv;  <------------ ?!?!?!?!!
       endif;                        

Regards
Sebastian

XMLSTOREDP service program not exporting RUNASCII

With #33, XMLSTOREDP is only exporting functions defined in the export source. In #27, I said we needed to only export the iPlug* functions, since the XMLSTOREDP service program is ostensibly there to support the stored procedures.

I forgot about RUNASCII and other functions that were used by both the Python toolkit iLibCall and the xmlservice-cli utility. Ideally these utilities would use RUNASCII from a dedicated service program instead of XMLSTOREDP, but we don't have one of those and they are already using it so we need to keep that behavior.

Fix copyright headers on source files

https://github.com/IBM/repo-template#license

Should be:

      * Copyright contributors to the XMLSERVICE project 
      * SPDX-License-Identifier: BSD-3-Clause

Use `make` to build/install

Original report by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Are there plans to have this be built from the IFS using something like make?

Would love to have it as simple as:

$ git clone ssh://[email protected]/inext/xmlservice-rpg.git
$ cd xmlservice-rpg
$ make

Build test utilities in to separate library

The test utilities (eg. ZZCALL) are used by the various language toolkits for their integration tests, but these utilities are not XMLSERVICE provided by IBM in QXMLSERV. This makes it hard to test the toolkits without building XMLSERVICE manually. If we separate the test utilities in to a separate library, it ensures that we can run the tests without depending on XMLSERVICE from the XMLSERVICE library.

Rewrite stored procedure as an SQL UDF

A UDF does not require an SQL pre-compiler or embedded SQL.

Issue with array of DS

Original report by Guido Sangiovanni (Bitbucket: gsangiov, GitHub: sangio90).

---

Hi and thank you for the great work you made,
I'm trying to send an XML to a RPG program that expects 2 parameters:

• 1 Input DS
• 1 Output Array of DS

#!xml

<pgm name='programname' lib='libraryname'>
        <parm>
            <ds var='input'>
                <data var='inputfieldone' type='3p0'>100</data>
            </ds>
        </parm>
        <parm>
                <ds var='output' dim='999'>
                    <data var='outputfieldone' type='13A'></data>
                    <data var='outputfieldtwo' type='15p3'>0</data>
                </ds>
        </parm>
</pgm>

if I set dim='1' or i don't set it at all, I flawlessly get the first result of the program (that is expected to return 6 rows), whenever I set anything different from '1' as dim I get an error on the packed output parameter.

The parameters are OK otherwise it wouldn't even work with dim='1' so I'm a bit lost.
Is there anything I'm missing?

Thanks

Convert to free format

This will make it much easier to contribute.

How to fix IPLUG15M in QXMLSERV type *N not found. error

IPLUG15M in QXMLSERV type *N not found.

[feature] job name will always be XMLSERVICE

xmlservice/src/plugipc.rpgle

Line 1496 in 20800d6

arg1 = %trim(mydir) + %trim(mypgm) + x'00';

Because dir+pgm will always expand to /libname.LIB/XMLSERVICE.PGM, and SPAWN_SETJOBNAMEARGV_NP gets set, this will always make a job of the name XMLSERVICE when spawning. A user asks if this could be customizeable. My guess is to set argv0 to whatever, but still pass what is currently argv0 as the first parameter to spawn.

Properly escape content in built CDATA blobs

xmlservice can include characters invalid in XML. For example, EBCDIC substitution characters (3F), such as stored in a database or if the new encoding can't represent the character, are turned into ASCII ones (1A), which are not allowed characters in XML (all ASCII control characters except newline are forbidden), even in a CDATA block. Some ideas:

• If we're outputting to Unicode, put a Unicode replacement character (U+FFFD) in instead. I believe XML allows these. Unicode output isn't guaranteed though.
• Silently drop the characters or replace with i.e. a space. Would require a scan of the buffer before appending to build the CDATA block. Might be surprising for users.
• Don't use CDATA, always use entities to escape XML special or disallowed characters. Complicates building the string.

Basically, the clients should always get back a valid XML blob that doesn't need special handling before parsing.

Related to zendtech/IbmiToolkit#178

Add releases with save files

This will make it easier to download and test new versions of XMLSERVICE. Ideally, this would be done by a CI.

Is there any example to show how to call stored procedures?

Tried many times to call stored procedures on IBM i.

I can send SQL statements to XMLSERVICE and it's working perfectly. Does anyone have a working example to show what format of HTTP request should I send to IBM i ? (ctl, ipc, xmlin...)

Call to service program functions don't have %PARMS() set properly - ie with *NOPASS parameters and <pgm mode="ile">

It doesn't seem like service program functions are getting the %PARMS set properly when called. You can see this with the XMLSERVICE(ZZNOPASS) function for example: (I ran this in ACS Run SQL Scripts). I've tried on both my work system and PUB400 with the same result in both places.

call qxmlserv.iplug4k('*NA', '*HERE',
   '<?xml version="1.0" ?>'
|| '<script><pgm name="ZZSRV" lib="XMLSERVICE" func="ZZNOPASS">'
|| '<parm><data type="10a" var="myName">Ranger</data></parm>'
|| '<parm><data type="10a" var="yourName">Buddy</data></parm>'
|| '<return><data type="50a" var="blah" varying="on">asdf</data></return>'
|| '</pgm></script>'
, '');

Here's the output I receive back:

Output Parameter #4 (C0) = <?xml version="1.0" ?><script><pgm name="ZZSRV" lib="XMLSERVICE" func="ZZNOPASS">
<parm><data type="10a" var="myName">Ranger</data></parm>
<parm><data type="10a" var="yourName">Buddy</data></parm>
<return><data type="50a" var="blah" varying="on">my name is -1 Ranger NOPASS</data></return>
<success><![CDATA[+++ success XMLSERVICE ZZSRV ZZNOPASS ]]></success>
</pgm></script>

I've looked at the source file PLUGILE ileSRV() and don't see anything obvious, but I'm not sure how the %PARMS works behind the scenes. Should it be doing something extra before calling the service program sub-procedure to tell the OS how many parameters are being passed? Also I suspect the code where it iterates through argv to set argc might not work properly for *OMIT parameters with a *NULL if there are subsequent parameters filled in, but I haven't been able to test that yet.

Itoolkit's iPgm - Library setup

iToolkit's iPgm is taking Library List from User's JOBD library list.
Any way to give the library list in run time.?

I tried submitting the job with a JOBD which has a set of libraries but no use. When I console the result after run( ). I see below XML.

\n \n XML Toolkit 1.9.9\n \n 1100016\n \n \n \n \n 10000 05\n \n \n \n \n 1100016\n \n \n \n \n *NA\n FFFFFFFF\n< jobname>QSQSRVR \n QUSER\n 266967\n *ACTIVE\n USER01\n 37\n 37\n 819\n ENU\n US\n QSYSWRK\n QSYS\n \n QSYS QSYS2 QHLPSYS QUSRSYS\n QTEMP LIBRARY1 LIBRARY2\n see log scan, not error list \n \n \nundefined

Node.js and field names as input to ipgm.js and xmlservice

Original report by Magne Kofoed (Bitbucket: magnekofoed, GitHub: magnekofoed).

---

Is it possible to put field names into ipgm.js (and xmlservice) and get the field names in the response? Both single variable names and field names in datastructures.
I want to have the field names in the response from xmlservice in order to create a better json response.

XMLSERVICE enable JSON protocol

Original report by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Many script languages support JSON parsers. A nice feature of XMLSERVICE would be to accept, parse, execute, and return JSON, much same as current XML support. Perhaps a hybrid of XML and JSON, if pure JSON not possible.

Add support for Free Format parameter declaration

Hi!

In IBM/nodejs-itoolkit#75 I suggested to add support for free format parameter syntax ("char(10)" instead of "10A").
@ThePrez suggested to open an issue here.

This may be something related to #24 - but as far as I've seen, #24 is more about changing this packages' source to total free syntax.

Add exports file for XMLSTOREDP

XMLSTOREDP currently exports a ton of functions, most of which should be exported from the main XMLSERVICE service program (#26) instead. Basically just the iPlug* entries should be exported from XMLSTOREDP for use by the stored procedures (https://github.com/IBM/xmlservice/blob/master/src/xmlstoredp.sql.in).

• Add exports file
• Update Makefile.in to use it

xmlcgi only supports password length less/equal to 10

How to use pure xml to call PGM/CMDwith Chinese Chars (937)?

My web page will input Chinese chars like '測試'

My nodejs will send xml str to XmlService with Post method and the form body like below:
<?xml version="1.0"?> <xmlservice> <cmd>CHGLIBL LIBL(QGPL QTEMP)</cmd> <pgm name='PGMNAME' lib=''> <parm io='both'><data type='32a'>63jo3fzykm0wo0s408w04kw44</data></parm> <parm io='both'><data type='10a'>USR</data></parm> <parm io='both'><data type='10a'>Test</data></parm> <parm io='both'><data type='10a'></data></parm> <parm io='both'><data type='50a'>測試</data></parm> <parm io='out'><data type='1024a' var='$out' ></data></parm> </pgm> </xmlservice>

The cmd string:
<?xml version="1.0"?> <xmlservice> <cmd>CHGLIBL LIBL(LIBNAME QGPL QTEMP)</cmd> <cmd exec='rexx'>CMD1 SSNID(7kr4yyvy0dk4o8k884k8wkccg) USR(USE) PAE(''A004'')TSKTXT(''測試'') RTNMSG(?) </cmd> </xmlservice>

PS: HTTPAdmin already config the CCSSID with 937/950

Could give us the example how to deal with them? thanks.

@jimoibm
@kadler

ftpcompile.sh missed SITE subcommand

Original report by Ji Mo (Bitbucket: JiMoJiMo, GitHub: Unknown).

---

When using ftpcompile.sh to upload source code to IBM i server, quote namefmt 1 received an error. ftpfew.sh has the same issue.

Thousands of XMLSERVICE jobs

Original report by Don (Bitbucket: msddcb, GitHub: Unknown).

---

I am trying to control the number of XMLSERVICE jobs running on IBMi at V7R3.

I have this issue both where Zend server is running on IBMi and where Zend server is on a remote Linux box and using ODBC to connect to IBMi.

Recently we had over 13000 XMLSERVICE jobs running. Ending the sub-system and restarting is the only way we are currently managing but I am hoping there is a setting that will allow the number of jobs to be controlled.

Thanks you for any suggestions

ftpcompile.sh missed from migration

@kadler I didn't find the ftpcompile.sh in GitHub which was in the first level directory in BitBucket. That script is used to push a build from linux to i and building PTF process also needs it.

Multiple result sets

Is xmlservice able to return multiple result sets from a stored procedure? If so, 1. how do we return more than one result set using xmlservice and 2. how do we access the result sets after the first one?

Thank you in advance!

Changing 'commit' option in XML SQL statement does NOT work

Hi All,

To set the transaction-isolation level to NO COMMIT in a XML SQL document, I created the following XML:

<?xml version='1.0'?>
<myscript>
<sql>
<connect options='opt1' />
<options options='opt1' autocommit='off' commit='none' />
<query>
<![CDATA[
... Do a UPDATE/DELETE query an unjournaled object/member
]]>
</query>
<rowcount stmt='stmt1' />
</sql>
</myscript>

I got the error: Member <...> not journaled to journal *N. It looked like the option commit='none' did not work.

After doing lots of trying, the following XML worked for me:

<?xml version='1.0'?>
<myscript>
<sql>
<options options='opt1' commit='none' />
<connect options='opt1' />
<query>
<![CDATA[
... Do a UPDATE/DELETE query an unjournaled object/member
]]>
</query>
<rowcount stmt='stmt1' />
</sql>
</myscript>

I'd say there is a bug in parsing options in XML document and the developer guide http://yips.idevcloud.com/wiki/index.php/XMLService/XMLSERVICEQuick is not right.

1. The must be declare before
2. Does the order of attributes in <options ...> take into account?

Link common modules in to shared service program instead of static linking

Currently XMLMAIN.PGM, XMLSERVICE.PGM, XMLCGI.PGM, and XMLSTOREDP.SRVPGM link in all the common modules. This is obvious duplication and could be reduced by building a new service program containing all these modules and binding to the service program from each of these objects.

Calling XMLSERVICE from a 64-bit PASE job fails

Original report by me.

---

When calling XMLSERVICE from within a 64-bit PASE job, the result is failure. The return coe is 0, but no output is returned. This is due to assumptions throughout the code that assume that a PASE pointer is 32-bits.

This causes the iLibCall transport to fail on 64-bit Python versions: IBM/python-itoolkit#17

PH¨P interface and call existant RPG

Original report by Anonymous.

---

Hi,
We have a RPG application with 5250 screens
I have no RPG talent but i want to develop an interface with PHP on a Wamp.
If i install XMLService, may i developp PHP interface and call RPG prg (existant) ? Without RPG modifications?

Is it easy like this?

Sorry for my bad english.

Jérôme

Calls to programs or service program functions with mode="opm" do not handle *OMIT parameters properly

In plugile.rpgle functions ilePGM() and ileSRV() they are walking forward through the argv pointers looking for a null pointer to count how many parameters there are. In the case of *OMIT you can have a spot in the middle where a parameter has a null pointer but then subsequent valid parameters. The code is skipping out and not recognizing those subsequent valid parameters. It needs to walk backwards instead.

I will add more comments later today,

Passsing UCS2 data type between CLient and Server - SQLRPGLE program

Original report by Anonymous.

---

Hello,

Working on XMLSERVICE 1.9.2 Toolkit.

Looking to exchange UCS2 data type between client (SQLRPGLE) and server(SQLRPGLE) it does not work with the following XML definition in the client program:

<data type='100A' var='wrkUcs2Data' *after(13488/13488) >x

Based on the documentation available on website: (http://youngiprofessionals.com/wiki/index.php/XMLService/DataTypes), don't see any data type definition for UCS2 variables.

Any example would be of help.

email address: [email protected]

Regards.
Chaks.

Output contains \n characters

Original report by Danny Roessner (Bitbucket: droessner, GitHub: danny-hcs).

---

When calling QXMLSERV.iPLUG512K(?, ?, ?, ?) using Run SQL Scripts (I used https://bitbucket.org/litmis/nodejs-itoolkit/src/master/test/rpg/zzcall.rpgle to test with) the output has newline characters after every and tag. Can these unnecessary characters be removed?

Node.js reading 'stdout' of CGI pgm (QtmhWrStout)

Original report by Sebastian Misiewicz (Bitbucket: SMisiewicz, GitHub: sebastian-misiewicz).

---

Hi

I have a RPG program which sends data to 'stdout' via QtmhWrStout API (which is a part of QZHBCGI). So the data flow is

Web browser <-> HTTP Server <-> QZHBCGI ( my RPG program)

my intention is to get rid of HTTP server and use Node.js instead.

Web browser <-> Node.js <-> ???????? ( my RPG program)

I have no idea how to interact with stdout/stdin from Node.js
I've tried iSh with system command, iPgm with 'regular call'.
No matter how I call my program each time I got QPRINT with content.
I've tried also to add QIBM_USE_DESCRIPTOR_STDIO=Y to Qp2RunPase - no effect.

Any idea how to interact with ILE 'stdout'/'stdin'
On the other hand QtmhPutEnv (part of QZHBCGI) works ok.
Regards
Sebastian

How to deal with DBCS chars at IBMi db2

XMLService how to parse DBCS (double-byte character set) chars?? Can u give us hint?

Below is our xml example:

<?xml version="1.0"?><xmlservice><myscript><sql><prepare stmt="main" error="fast"><![CDATA[SELECT TSKID, TSKNM, ENHTXT FROM DBNAMW.TK WHERE TSKNM = ?]]></prepare><execute stmt="main" error="fast"><parm><![CDATA[DEMO1]]></parm></execute><fetch stmt="main" block="all" desc="on" error="fast"></fetch><describe stmt="main"></describe><free></free></sql></myscript></xmlservice>

Thanks