Use `make` to build/install

[ibm]

Use `make` to build/install

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

done

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Ok, I added a generic Makefile that works with either GNU gmake or PASE make (/QOpenSys/usr/bin/make).

#!shell

/QOpenSys/usr/bin/make xmlservice
/QOpenSys/usr/bin/make yips
...

gmake xmlservice
gmake yips
...

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Aaron, can you look at issue in litmis/system400?

Before i do anything with 'Makefile' xmlservice, i want your opinion on open issue for system400 + utilities (crtrpgmod, etc). I want to write little scripts once and re-use for any purpose (chroot or not).

Also, maybe watch issues in litmis/system400 (talking to myself is not healthy).

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

The more we can automate the faster IBM i shops will be able to deliver solutions and the less an individual will need to be an expert to pursue these things.

MMM ... political campaign "GNU tools in every IBM i pot". Ok, perhaps, RPM build, install, and IBM i repo host will make that a lot easier. However, bending GNU tools to serve ILE lib/mbr(.pgm, .srvpgm, .cmd, etc.), is 'new art'. Very well! I shall paint with my Makefile/utility brush. Perhaps next GNU Makefile Picasso, Monet or simply a kid with crayons. Well, I think it's pretty.

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I began to wonder about 'who is target for RPG Makefile example'?

The more we can automate the faster IBM i shops will be able to deliver solutions and the less an individual will need to be an expert to pursue these things. The dependencies add up and they start choking progress, even for experts that know how to solve them.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

You seem to have a general narrative of README.md 'taming GNU tools' for beginners. That is, 'info' from perzl, 'gmake' from yips, 'system400' from yips, so on. In fact, you suggested afore mentioned install.sh gather up all 'tools'.

I began to wonder about 'who is target for RPG Makefile example'?

1. Beginners -- So, my observation, folks that need ton of README help are better served with CL programs (CRTXML, etc.).

2. Daredevils -- Maybe help toe dipping GNU folks, aka, gmake hybrid ILE environment (kinda linux-like).

3. Experts -- These folks do not really need much help. At best, demonstrating a very easy way to add little utilities to Makefile with out all the hoopla of over engineering the planet (aka, bob is good, bob is complex, bob fails in chroot).

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I used info as 'talk' utility user about compile environment (absence of a 'configure" command). Did you want me to remove 'info' from Make file??

Your call, but I'd leave it for now to save you time. If somebody gets a build fail because info is missing then they can log an issue. I can add an RFE for info to be part of the future package manager repo.

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I do not understand. I intend to post system400 to YIPS as a 'binary' download? Yes?

I misunderstood your previous comment. I thought you were looking for a public chroot environment to test the build process. Disregard my comment. [Aaron goes to drink more coffee]

Do you want to create a project in litmis/system400 (public source)?

Done. I've added you as admin: https://bitbucket.org/litmis/system400/overview

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

info

I used info as 'talk' utility user about compile environment (absence of a 'configure" command). Did you want me to remove 'info' from Makefile??

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

system400

Do you want to create a project in litmis/system400 (public source)?

Feel free to use the free/public spaces.litmis.com.

I do not understand. I intend to post system400 to YIPS as a 'binary' download? Yes?

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

but ... i do not have a public system400 (chroot friendly system)

Feel free to use the free/public spaces.litmis.com. This is what Jesse and I have been using for Python, Node.js and PASE labs.

On my machine the info command comes from perzl (I had not yet used this command so I went looking). Just wanted to note that so we can include info install instructions as part of the README.md.

% which info | xargs ls -al
lrwxrwxrwx    1 aaron    0                54 Dec 17 2015  /usr/bin/info -> ../../opt/freeware/bin/info

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

So, basic idea ... pretty simple .. but ... i do not have a public system400 (chroot friendly system)

Makefile

#!shell

### ./make_libdb400.sh
### > export INICHROOT=/QOpenSys/db2sock
### > export INILIB=XMLSERVICE
### > export INICONF=plugconf1
### > gmake -f Makefile

### env vars
ifndef INICHROOT
INICHROOT   = NULL
endif
$(info chroot INICHROOT=$(INICHROOT))

ifndef INILIB
INILIB      = XMLSERVICE
endif
$(info ILE library INILIB=$(INILIB))

ifndef INICONF
INICONF     = plugconf1
endif
$(info XMLSERVICE config INICONF=$(INICONF))

### ILE compile scripts
CCRPG       = ./maketools/CRTRPGMOD
CCPGM       = ./maketools/CRTPGM

### *PGM - $(INILIB)/xmlmain.pgm
### xmlmain.rpgle xmlthis.rpgle xmlthat.rpgle ...
XMLMAINPGM  = xmlmain.pgm
XMLMAINOBJS = xmlmain.mod

### tells make all things to do (order)
all: $(XMLMAINPGM)

### CRTRPGMOD
.SUFFIXES: .mod .rpgle
.rpgle.mod:
	$(CCRPG) --root $(INICHROOT) --lib $(INILIB) --rpgle $<

### -- CRTPGM
$(XMLMAINPGM): $(XMLMAINOBJS)
	$(CCPGM) --pgm $(XMLMAINPGM ) --mods $(XMLMAINOBJS) --lib $(INILIB) (... so on ...)

### -- housekeeping(optional)
clean:
	rm -f $(XMLMAINPGM)
	rm -f *.mod

CRTRPGMOD

#!shell

#!/bin/sh
# Arguments
# -r|--root  - chroot location (/QOpenSys/db2sock)
# -l|--lib   - ILE Library output (XMLSERVICE)
# -r|--rpgle - ILE RPG program (xmlmain.rpgle)
IFSROOT=NULL
IFSPATH=$(pwd)
ILELIB=NULL
ILESRC=NULL
ILENAM=NULL
ILEMOD=NULL
while [[ $# -gt 1 ]]
do
 key="$1"
 case $key in
 -r|--root)
 IFSROOT="$2"
 shift # past argument
 ;;
 -l|--lib)
 ILELIB="$2"
 shift # past argument
 ;;
 -r|--rpgle)
 ILESRC="$2"
 shift # past argument
 ILENAM=$(echo "$ILESRC" | cut -d'.' -f1 | cut -d'/' -f2)
 ILEMOD="$ILENAM.mod"
 ;;
 *)
 # unknown option
 ;;
 esac
 shift # past argument or value
done

# which system command?
CHROOT=
SYSCMD=
case $IFSROOT in
 NULL)
 CHROOT=
 SYSCMD="system"
 ;;
 *)
 CHROOT="$IFSROOT"
 SYSCMD="system400"
 ;;
esac

# process command
touch "./$ILEMOD"
arg="CRTRPGMOD MODULE($ILELIB/$ILENAM) SRCSTMF('$CHROOT$IFSPATH/$ILESRC') DBGVIEW(*SOURCE) OUTPUT(*PRINT) REPLACE(*YES) TGTCCSID(37)"
echo "$SYSCMD '$arg'"
RES=$($SYSCMD "$arg")
OK=$(echo "$RES" | grep -c "00 highest severity")
if (($OK==0))
then
  echo "===================================="
  echo "*** ERROR $ILELIB/$ILENAM"
  echo "===================================="
  echo "$RES"
  BAD=$(echo "$RES" | grep "*RNF")
  echo "$BAD"
  exit -1
else
  echo "==> $ILELIB/$ILENAM -- 00 highest severity"
fi

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

BTW (off topic) -- litmis/db2sock -- i will take your suggestion to build multiple 'objects'. DB2 driver base (libdb400.a), Toolkit (toolkit400.so), json parser (json400.so), etc., all different dirs in project. Also, any given add-on thing400.so can be replaced, aka, don't like default json400.so parser, put your own in from some Open Source site (any which way open). I hope to update the project soon. If you want more discussion (after you see update), please switch to db2sock issues for this non-xmlservice topic.

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Which did you want???

I vote #2; walk before we run. Learn the ins and outs of how it works with various RPG/CL/DDS/MSGF/etc scenarios.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

autoreconf isn't necessary given RPG only runs on a single OS and chip architecture.

Correct, However, tool does a lot more, aka, essentially does pick up laundry, cook dinner, mow yard, and chat with dear dad. All with 'billions' of years of human experience generated into the configure script.

So, to be clear, i must ask.

1. Do we want a custom configure script that generates Makefile (library target, etc.). Start the ball rolling on template for creating configure files 'by hand', later some enterprising young man (cough), could build a tool to generate the configure.

2. Only need custom Makefile, wherein people learn the make language (env vars parms, etc.).

Which did you want???

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

The big question ... What do you want to do? That is, you want to set an example, is a custom configure script 'the right example'??

A custom Makefile is a good 'walk before we run' strategy. After using the custom Makefile approach for awhile we can determine if the other tooling helps. Besides, I am guessing a lot of the stuff in autoreconf isn't necessary given RPG only runs on a single OS and chip architecture.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Preamble (in case you do not know):

So i took a quick look at end-2-end building for configure, make, make install. The most important 'technology' is the idea of 'hands off' building Makefile from a few 'hint/config' files (Makefile.am, configure.ac).

1. Tool autoreconf --install does all the work to create configure, Makefile, install.sh, etc. (thousands of lines).

2. distribution is usually a tar file make distcheck and out pops your source name-1.0.tar.gz.

3. download name-1.0.tar.gz and user normal compile using configure, make, make install.

The bad news ... 'hands off' approach using these linux tools will not work on IBM i ILE (especially ILE RPG). This whole process is completely c programming. Specifically, 'billions' of years of expert opinion on how to compile stuff in unix without problems. Fantastic, but completely useless for compile ILE RPG. Ahh ... also useless for ILE c (not to be a heart breaker).

The next best thing ... custom Makefile.

1. As stated before, directly manipulating a Makefile to use small utilities like CRTCMOD, CRTRPGMOD, CRTPGM, etc., is very easy. Essentially, 'the human' is the configure step.

2. Alternatively, it would be possible to build a custom configure command (built by human named Tony Ranger Carins), that would give the appearance of a full Linux configure, make, make install. To be clear, make, make install would be the real deal (processed by gmake), only configure script would have to be 'hand made'.

The big question ... What do you want to do? That is, you want to set an example, is a custom configure script 'the right example'??

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I don't think i want to sign up for that. Perhaps make process could simply warn about missing tools?

I think that will be "good enough", yes.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

The install.sh script would obtain anything and everything necessary to build xmlservice-rpg, including downloading/installing gmake or db2util if it didn't yet exist. It could also include chroot creation so as to not worry ...

Mmm ...

... pick up the laundry?

... cook dinner?

... mow yard?

... call dad for a nice visit?

I don't think i want to sign up for that. Perhaps make process could simply warn about missing tools?

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

I do not understand what install.sh is suppose to do. Can you elaborate?

The install.sh script would obtain anything and everything necessary to build xmlservice-rpg, including downloading/installing gmake or db2util if it didn't yet exist. It could also include chroot creation so as to not worry the person downloading as to whether they would "hose PASE" (had that happen to a customer recently, first time for me :-).

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

"common approach" Linux takes to building, and if that means m4, then we should do that.

Spoken true Linux Borg (forgive again). Understood.

Further, all of these utilities should be bootstrapable via DB2's HTTP_GETCLOB functions so it is dirt simple to obtain for anybody on a supported IBM i release.

I understand SYSTOOLS.HTTPGETCLOB.

I do not understand what install.sh is suppose to do. Can you elaborate?

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Do you need MyProject.m4 Linux compliance for your opinion of 'model citizen' (*)?

Good question. My usage of the make approach is based on manually creating, not automatically generating, a Makefile. Said another way, I've only been a consumer of configure/m4 and not an author.

I believe RPG in the IFS is our new reality, because Git. Using make, and all its tooling, becomes a natural/sensical next step and introduces a chance to teach IBM i folk. I think we should go the route of the "common approach" Linux takes to building, and if that means m4, then we should do that.

Further, all of these utilities should be bootstrapable via DB2's HTTP_GETCLOB functions so it is dirt simple to obtain for anybody on a supported IBM i release. For example...

$ db2 "SELECT SYSTOOLS.HTTPGETCLOB('https://bitbucket.org/inext/xmlservice-rpg/src/master/install.sh','') FROM SYSIBM.SYSDUMMY1" | sh

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

The BIG question ... do we really want a RPG project like xmlservice to require bunch of PASE Open Source tools to build (m4, gmake, etc.)? I think you already answered yes. Mmm ... emphasised by 'model citizen'.

Here's one that was recently announced. I haven't looked into it yet. https://github.com/s4isystems/Bob

Good: Bob is cool. Lot's of work in thing. Demo hooked up with RDI look 'slick'.

Bad: PASE 'system' based ... chroot 'unfriendly'.

Also, what is system400? Not sure I've heard of that yet.

Simpler db2util that builds inside db2sock, so db2sock can 'one flow' compile everything and ILE parts in a chroot.

... model citizen of this (In My Opinion) preferred approach ... Further, I believe whatever we come up with should be chroot friendly (db2util or system400).

Well, 'model citizen' probably exceeds my grasp. However, if you look at litmis/db2sock ILE-CGI Makefile, you will see i am a fan of minimalistic Makefiles, wherein Bob seems over engineered to me (no offence).

Here are some of my key 'wants' as seen in litmis/db2sock ILE-CGI Makefile ...

CRTthing - Most key concept is Makefile called tiny 'wrapper' utilities CRTCMOD, CRTRPGMOD, CRTPGM, etc. These utilities are easily be made chroot friendly 'redirect' work system<>db2util (system400). Also adding utilities is a snap.

Makefile object.mod -- Second concept in my Makefile is 'object.mod list' proxy in the IFS to allow for 'make only changed things'.

#!shell
### ILE
### CCRPG       = ./CRTRPGMOD
CCC         = ./CRTCMOD
CCPGM       = ./CRTPGM

### *PGM CGI - $(INILIB)/DB2JSON
CGI400PGM  = db2json.pgm
CGI400MOD  = $(INILIB)/db2json
CGI400OBJS = db2json.mod another.mod another1.mod another2.mod
CGI400MODS = $(INILIB)/db2json

Makefile ... model citizen?

Linux traditional configure, make, make install, where configure actually creates Makefile. Most typically something like 'MyProject.m4' is used to generate platform appropriate Makefile. I do not have anything like this in my litmis/db2sock ILE-CGI project (have a custom make_libdb400.sh).

Do you need MyProject.m4 Linux compliance for your opinion of 'model citizen' (*)?

(*) Forgive me -- Linux compliance Star Trek Borg "Resistance is futile. You will be assimilated into configure, make, make install".

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

Do we want to invest in 'make' technology?

I believe xmlservice-rpg should invest in make approach. This approach (building QSYS.LIB from the IFS) is starting to gain momentum (publicly and privately) and xmlservice-rpg has the opportunity to be a model citizen of this (In My Opinion) preferred approach.

Here's one that was recently announced. I haven't looked into it yet. https://github.com/s4isystems/Bob

Further, I believe whatever we come up with should be chroot friendly. I am wondering if we could do that by simply adding a shim of sorts when in chroot; specifically, a shim that redirects system to db2util.

Also, what is system400? Not sure I've heard of that yet.

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

So, that said, back to topic. What do you want to do here in xmlservice? Do we want to invest in 'make' technology?

For readers ... put a clearer definition on 'make' technology.

Make favours CL program create and install disappear in favour of 'make' Open Source tools to compile RPG in XMLSERVICE (probably gmake). Although, I suppose it could be argued that both CL and 'make' could exist in the project concurrently.

Secondary issue is 'make' from chroot, where access to system '/QSYS.LIB' is restricted (can't see errors in program compile). This involves a work around utility like db2util or system400, which, use database stored procedure technology to proxy QSQSRVR as escape for chroot restrictions (DB2 QSQ jobs are started outside the choort).

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Will this repo (xmlservice-rpg) eventually be sunset?

No. The repo will probably exist indefinitely. This project is stable (many toolkits). There are no outstanding issues (beyond 'go faster'). So, this technology works as designed.

I am wondering if db2sock should remain only an async DB2 PASE driver and then other (rpm) packages would/could be added to wrapper db2sock for addtional capability (fast-cgi, WebSocket, etc).

Clearly, my thinking/hope is newer technology in db2sock (incomplete), will offer a better/faster platform toolkits. Hopefully we simply "choose-by-switch" to run either xmlservice or new db2sock technologies under an existing toolkit. Anyway, If you wish to discuss packaging of litmis/db2sock please shift discussion to to that project issues.

[kadler]

Original comment by Aaron Bartell (Bitbucket: aaronbartell, GitHub: aaronbartell).

---

i am thinking xmlservice functions eventually move to litmis/db2sock with built-in toolkit functions (faster).

Will this repo (xmlservice-rpg) eventually be sunset? I don't want to read into the above comment and need more understanding as to what it actually means.

Thinking out loud, and thinking about the long term... We can now openly discuss that IBM is shipping a package manager (Jesse announced at COMMON) sometime in the future. This means we can put together (rpm) packages with dependencies as standard practice in the IBM i community (vs. PTF or *SAVF download). I am wondering if db2sock should remain only an async DB2 PASE driver and then other (rpm) packages would/could be added to wrapper db2sock for addtional capability (fast-cgi, WebSocket, etc).

Thoughts?

[kadler]

Original comment by Tony Cairns (Bitbucket: rangercairns, GitHub: rangercairns).

---

Are there plans to have this be built from the IFS using something like make?

Possible. We do 'gmake' for ILE parts in litmis/db2sock. ILE c, but nothing different about RPG. However, we need a 'chroot friendly' system utility runner.

1. litmis/db2sock system400 'chroot friendly' IBM i command processor.

2. litmis/db2util also 'chroot friendly' IBM i command processor.

Future.

i am thinking xmlservice functions eventually move to litmis/db2sock with built-in toolkit functions (faster).

What do you think?