Issue with array of DS

[kadler]

Issue with array of DS

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

No problem. Again, I recommend engaging Club Seiden. A place where cool IBM i php experts live. Including other php folk been down road with RPG misinformation ... whoops says your RPG guy (after 2 days struggling).

[kadler]

Original comment by Guido Sangiovanni (Bitbucket: gsangiov, GitHub: sangio90).

---

Thank you for everything.
I never thought the real issue was in the XMLSERVICE but this looked like a good place to find help, and I was right.
Now everything works as expected, so summing up my main issues were 2:

1. Wrong DS definition (I'm still angry for wasting like 2 days trusting my colleague's DS definition)
2. plugSize

You can close this issue ;-)

Best wishes,
Young grasshopper.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

array="on" means nothing to xmlservice ... just php toolkit

Ok, we finally have language you are coding (php). You only need to pick a larger stored procedure call (plugSize below).

#!php

try { $ToolkitServiceObj = ToolkitService::getInstance($db, $user, $pass, $extension); }
catch (Exception $e) { echo  $e->getMessage(), "\n"; exit(); }
$options = array('stateless'=>true,'plugSize'=>'10M');
$ToolkitServiceObj->setToolkitServiceParams($options);

There are multiple forums to help you with php toolkit. I recommend Club Seiden

I am going to be politically incorrect here (and spooky), but only to help your wandering thinking. XMLSERVICE is never at fault, NEVER, always your php script. This is very helpful advice to avoid wandering off road looking for xmlservice problem (rare to never exists). This is not brag, simply fact that XMLSERVICE has served millions of ILE call hamburgers. You should never consider your XML hambuger to be anything but another hamburger, nothing uniquer, nothing new, not an xmlservice issue. To wit, you should never end up here in XMLSERVICE for PHP problems. In fact, only the great man Alan Seiden himself should ever bring a problem to XMLSERVICE. I meet with Alan's team once a week. Aka, you, young grasshopper, crawled into xmlservice sewer drain and met xmlservice author clown named 'It' ... boo ... Happy Halloween.

[kadler]

Original comment by Guido Sangiovanni (Bitbucket: gsangiov, GitHub: sangio90).

---

Hi,
array="on" is produced by XMLWrapper class provided by Zend's PHP implementation.

What I'm trying to read is an array of max 999 elements, each of them must be a DS composed by 179 fields, making it 3698(bytes?).

Everything works fine now, I'm just facing a size issue since it always reads 999 big DS, is 3.694.302 bytes of data too much for a single XMLSERVICE request?

Below the snippet of PHP preparing parameters, 999 is "dimension", anything below 53 works, when I exceed that value it crashes, producing an error regarding teraspace and memory.

#!PHP

$params[] = $toolkitServiceObj->AddDataStruct($callParams, 'INPUTDS');
$params[] = $toolkitServiceObj->AddDataStruct($outputParams, 'OUTPUTDS', 999, '', true);

Sadly I'm a PHP developer and don't know much about DS and RPG.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

I have no idea what means array="on", not an xmlservice code artefact (completely crazy).

Appears you are wondering completely wrong, and, getting worse each day. This time showing a nested array (dim="3698") of arrays (dim="999") of 'DS' structures (of something we can not see). Your previous data for DSOUTPUT is 'truncated', but indicates only a single dim(999) array of a 'bigger' record that you say is 3698 bytes long ( last 30 parms of OUTPUTDS ). You really need to post entire DSOUTPUT and DSINPUT structure/layout in this issue for me to help you any further ( last 30 parms of OUTPUTDS + whatever actual DSINPUT structure really looks like). Aka, you are lost at sea Dude!

You must match your XML 'ds' structure to all 32 elements in the array. We are using the additional fields JUST, GET, LUCKY, 30MORE in this example to try to put you back on the right path (your actual names are 30 more of something we can not see in this issue).

#!bash

REQID 13A         
FIELDTWO 15P3     
JUST 12A          
GET 12P2 
LUCKY 32A
30MORE 10i0
: correct match, only one dim(999) of a 'bigger' record (30 more fields)
<ds dim='999'>
 <data var='REQID' type='13A'/>
 <data var='FIELDTWO' type='15p3'/> 
 <data var='JUST' type='12A'/>
 <data var='GET' type='12p2'/>
 <data var='LUCKY' type='32A'/>
 <data var='30MORE' type='10i0'/>
</ds>

[kadler]

Original comment by Guido Sangiovanni (Bitbucket: gsangiov, GitHub: sangio90).

---

Ok, I managed to solve part of this issue.
My RPG colleague forgot to sent me the definition of the last 30 parms of OUTPUTDS...

Is it possible to receive 999 DS which are long 3698 or is it too big?

I'm using

#!XML

<ds var='OUTPUTDS' dim="3698" array="on">
                <ds dim="999" array="on">
                </ds>
</ds>

It seems to be using too much memory, am I right?

BTW Really thanks a lot for your great stuff and for your help!

[kadler]

Original comment by Guido Sangiovanni (Bitbucket: gsangiov, GitHub: sangio90).

---

So, my output DS is composed by 140+ fields (I double checked them on my XML and they are properly defined).
The total lenght of a single DS should be 3434 and I can have max 999 DS.

What I don't understand is how should my XML be. I've seen in many examples online a tag wrapping another tag.
Something like

#!XML

<ds>
    <ds dim='999' //sometimes I read array='on' don't know if it's a standard>
        <data ...>
    </ds>
</ds>

Should I instead use the single DS size somewhere?
Thanks

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

I removed some of the fields since they're almost one like another with no exceptions.

This presents a error in logic. You cannot 'remove' fields' out of a 'DS' (DSINPUT and/or DSOUTPUT). That is, 'more' fields in any given structure, then XML must account for all fields. Technically, we need every field described in XML to calculate actual size of the structure. Aka, dim(999) of a 'DS' of absolute size elements, no field(s) missing, no field(s) added, exact match XML to 'DS'. Basically, there is no such thing as a 'truncated' DS structure of dim(999) size, every field in a 'DS' must be described in XML for it to work.

BTW -- This may explain why dim(1) will work for you. That is, by pure luck you may be able to pull off first two fields of first element in a 'DS', and all the remaining simply a big hole of zeros (as long as you do not touch the zeros in the RPG program).

#!bash
  DSOUPUT (ACTUAL)    DSOUTPUT (YOUR BAD XML - removed some fields)
dim(1) -- actual
1 REQID 13A      <--- 1 REQID 13A      <== dim(1) works fine
1 FIELDTWO 15P3  <--- 1 FIELDTWO 15P3  <== dim(1) works fine
1 JUST 12A       <--- 2 REQID 13A      <== dim(2) almost, but wrong (junk)
1 GET 12P2       <--- 2 FIELDTWO 15P3  <== dim(2) decimal error (boom)
1 LUCKY 32A      <---(n)<== each dim(n) actual data more incorrect (boom, boom, boom)
dim(2) -- actual
2 REQID 13A         
2 FIELDTWO 15P3     
2 JUST 12A          
2 GET 12P2 
2 LUCKY 32A
: correct (match)     : incorrect (mismatch 'true' data)
<ds dim='999'>        <ds dim='999'>
 <data type='13A'/>     <data type='13A'/>
 <data type='15p3'/>    <data type='15p3'/>    
 <data type='12A'/>   </ds>    
 <data type='12p2'/>
 <data type='32A'/>
</ds>

Note: 
Left -> Actual is the real data (add fields 'JUST' 'GET' 'LUCKY' ... just example)
Right -> bad XML not describing ALL elements (missing 'JUST', 'GET', 'LUCKY' fields)

[kadler]

Original comment by Guido Sangiovanni (Bitbucket: gsangiov, GitHub: sangio90).

---

I'm sorry, DSOUTPUT is ike:

#!RPG

PHYSICAL FILE........... DSOUTPUT
LIBRARY................. MY_LIB
KEY    FIELD   LENGTH   DEC  TYPE  FROM   TO      TEXT....................
          REQID            13                   A         1       13         FIELD DESC1
          FIELDTWO         15        3          P         8       15         FIELD DESC2

DSINPUT is like:

#!RPG

PHYSICAL FILE  . . . . .: DSINPUT 
LIBRARY  . . . . . . . .: MY_LIB
KEY    FIELD   LENGTH   DEC  TYPE  FROM   TO      TEXT....................
             ONE           3            0        P         1       2             DESC1
             TWO          13                     A         3      15             DESC2

I removed some of the fields since they're almost one like another with no exceptions.

Thanks

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Structure DSOUTPUT is not in cut/paste above. All you have is declaration of dim(999) of 'something' (something we can not see).

BTW -- we also can not see DSINPUT.

[kadler]

Original comment by Guido Sangiovanni (Bitbucket: gsangiov, GitHub: sangio90).

---

Hi Tony,
Thanks for your reply.
I'm sure as well that my problem lies in the RPG program (I'm no RPG developer and I must rely on others dev stuff).

Here's the source code:

#!RPG

** SOURCE 1

DOCCURX         DS                  OCCURS(999) LIKEDS(DSOUTPUT)
 *                                                                
DI                S              5U 0                             
 /COPY *LIBL/QCPYSRC,PGM009R                                     
DCallPGM009R      PI                                              
D entryPar                            LIKEDS(DSINPUT) CONST      
D backArray                           LIKEDS(DSOUTPUT) DIM(999)   
 *------------------------------------------------                
 /free                                                            
     *INRT = *ON  ;                                               
    CLEAR  DSOUTPUT   ;                                           
    FOR  I=1  TO  %ELEM(backArray) ;                              
           backArray(i) = DSOUTPUT   ;                            
    ENDFOR  ;                                          
  /end-free                                             
 C                   CALL      'PGM001R'               
 C                   PARM      entryPar      DSINPUT   
 C                   PARM                    OCCURX   
  /free                                                 
     FOR  I=1  TO  %ELEM(OCCURX)  ;                   
            %OCCUR(OCCURX)  = I   ;                   
            IF  OCCURX.REQID = *BLANK  ;	          
                LEAVE ;                       
            ENDIF  ;                          
            backArray(i) = OCCURX ;         
     ENDFOR  ;                                
  /end-free   

** SOURCE 2 (/COPY *LIBL/QCPYSRC, PGM009R

D DSINPUT       E DS                  EXTNAME(INPUTDS) INZ        
D DSOUTPUT      E DS                  EXTNAME(OUTPUTDS) INZ        
DsendDS           DS                  LIKEDS(DSINPUT) INZ         
DgetDS            DS                  LIKEDS(DSOUTPUT) DIM(999) INZ
                                                                   
DCallPGM009R      PR                  EXTPGM('PGM009R')           
D  entry                              LIKEDS(DSINPUT) CONST       
D  return                             LIKEDS(DSOUTPUT)  DIM(999)   

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

So, quickly, thousands of applications running nested array DS structures (php toolkit, python tooliit, node toolkit, etc.).You are not doing anything new. Action to be taken, something is going wrong in your RPG program, so you need to debug your RPG program (not xmlservice).

Only action i can take this end is request could post your RPG parameters and structures, then perhaps i can see what you are doing wrong.