ibm/zowe-cli-cics-deploy-plugin

Provides the cics-deploy plug-in for Zowe CLI https://github.com/zowe/zowe-cli to deploy Node.js and other applications from a workstation to IBM CICS Transaction Server within a CICS bundle. Documentation is available at https://ibm.github.io/zowe-cli-cics-deploy-plugin/

zowe-cli-cics-deploy-plugin

This project provides a plug-in for Zowe CLI to deploy applications developed on a workstation to IBM® CICS® Transaction Server for z/OS® (CICS). It aims to provide an experience similar to deploying to a cloud platform when deploying to CICS. It will also provide low-level commands for performing individual steps of the deployment process that could be used as part of a CI/CD pipeline.

Installing

Install the plug-in by following the steps in installing.

Documentation

You can find information and tutorials on using this plug-in in our documentation.

Contributing

Contributions are welcome - see the contribution guidelines. If you have a question or encounter a problem, please search the issues before raising a New issue.

zowe-cli-cics-deploy-plugin's Issues

cics-deploy command with missing parameters shows profile error instead of help

zowe zos-files list data-set shows extensive help about all the missing and optional parameters:

zowe zos-files list data-set

Syntax Error:
Missing Positional Argument: dataSetName
Argument Description: The name or pattern of the data set that you want to list

Syntax Error:
Missing Required Option:
--host (-H)

Option Description:
The z/OSMF server host name.

Syntax Error:
Missing Required Option:
--user (-u)

Option Description:
Mainframe (z/OSMF) user name, which can be the same as your TSO login.

Syntax Error:
Missing Required Option:
--password (--pass,--pw)

Option Description:
Mainframe (z/OSMF) password, which can be the same as your TSO password.

Example:

 - Show the data set "ibmuser.asm":

      $ bright zos-files list data-set "ibmuser.asm"

Use "bright zos-files list data-set --help" to view command description, usage, and options.

It would be useful if zowe cics-deploy deploy bundle showed similar help, rather than an error for a missing profile:

zowe cics-deploy deploy bundle
Command Preparation Failed:
No default profile set for type "zosmf".```

Windows - can not use cics-deploy bundle due to invalid bundleDir

If I run without escaped directorys on windows - the home directory get U: prefix.

$ zowe cics-deploy deploy bundle --name LOOPER --bundledir /u/atkinc/bundles/Looper_1.0.0/ --cics-deploy-profile allregs --zosmf-p zosmf2c
16:17:51.479313 : DFHDPLOY CICS TS APPLICATION DEPLOYMENT 2019/04/08 4:17pm
16:17:51.480097 : RELEASE: HCI7300. SERVICE LEVEL: HCI7300.
16:17:51.481305 : *
16:17:51.481312 : SET CICSPLEX(CAPLEX);
16:17:53.539701 : DFHRL2119I Connection to CMAS(CACMASA ) successful.
16:17:55.578787 : DFHRL2013I Connection to CICSPLEX(CAPLEX) successful.
16:17:55.593846 :
16:17:55.593856 : *
16:17:55.593862 : DEPLOY BUNDLE(LOOPER)
16:17:55.593866 : BUNDLEDIR(U:/atkinc/bundles/Looper_1.0.0/)
16:17:55.593871 : SCOPE(ALLREGIN)
16:17:55.593875 : STATE(ENABLED)
16:17:55.611459 : DFHRL2020E Unable to read contents of file U:/atkinc/bundles/Looper_1.0.0/META-INF/cics.xml. File could not be found.
16:17:55.611506 : DFHRL2055I Errors have occurred, processing terminated.
16:17:55.613770 : DFHRL2014I Disconnecting from CICSPLEX(CAPLEX).

If I escape the directories - I get invalid bundleDir

$ zowe cics-deploy deploy bundle --name LOOPER --bundledir //u//atkinc//bundles//Looper_1.0.0// --cics-deploy-profile allregs --zosmf-p zosmf2c
16:20:55.227152 : DFHDPLOY CICS TS APPLICATION DEPLOYMENT 2019/04/08 4:20pm
16:20:55.227838 : RELEASE: HCI7300. SERVICE LEVEL: HCI7300.
16:20:55.229048 : *
16:20:55.229055 : SET CICSPLEX(CAPLEX);
16:20:57.290464 : DFHRL2119I Connection to CMAS(CACMASA ) successful.
16:20:59.339345 : DFHRL2013I Connection to CICSPLEX(CAPLEX) successful.
16:20:59.355429 :
16:20:59.355463 : *
16:20:59.355504 : DEPLOY BUNDLE(LOOPER)
16:20:59.355544 : BUNDLEDIR(//u//atkinc//bundles//Looper_1.0.0//)
16:20:59.355582 : SCOPE(ALLREGIN)
16:20:59.355622 : STATE(ENABLED)
16:21:05.514483 : DFHRL2049I Creating BUNDLE definition.
16:21:05.516579 : DFHRL2301E BUNDLE(LOOPER) cannot be deployed as the following attributes contain invalid characters: BUNDLEDIR .
16:21:05.517883 : DFHRL2055I Errors have occurred, processing terminated.
16:21:05.521436 : DFHRL2014I Disconnecting from CICSPLEX(CAPLEX).

Unexpected Command Error:
Please review the message and stack below.
Contact the creator of handler:
"C:\Users\CHRISTOPHERAtkinson\Documents\ZoweStuff\ZOWEDEV\CICS\cics-deploy\zowe-cli-cics-deploy-plugin\lib\cli\deploy\bundle/DeployBundle.handler"
Message:
A failure occurred during CICS bundle deployment.
Reason = DFHDPLOY stopped processing due to an error.
Stack:
Error: A failure occurred during CICS bundle deployment.
Reason = DFHDPLOY stopped processing due to an error.
at DeployBundleHandler. (C:\Users\CHRISTOPHERAtkinson\Documents\ZoweStuff\ZOWEDEV\CICS\cics-deploy\zowe-cli-cics-deploy-plugin\lib\cli\shared\BundleParent.handler.js:70:23)
at Generator.throw ()
at rejected (C:\Users\CHRISTOPHERAtkinson\Documents\ZoweStuff\ZOWEDEV\CICS\cics-deploy\zowe-cli-cics-deploy-plugin\lib\cli\shared\BundleParent.handler.js:15:65)
at process._tickCallback (internal/process/next_tick.js:68:7)

generate bundle fails using node 10.15.3

zowe cics-deploy generate bundle --port 27501 --overwrite
NODEJSAPP "cics-nodejs-invoke" defined for startscript "server.js"
Command Error:
A failure occurred during CICS bundle generation.
 Reason = An error occurred attempting to write nodejsapp file '/Users/cockerma/git/cics-nodejs-invoke/projects/cics-nodejs-invoke/nodejsapps/cics-nodejs-invoke.nodejsapp': The module '/Users/cockerma/git/zowe-cli-cics-deploy-plugin/node_modules/node-expat/build/Release/node_expat.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 67. This version of Node.js requires
NODE_MODULE_VERSION 64. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).

node --version
v10.15.3
npm --version
6.9.0

I have tried rebuilding the plugin with the following, but with the same results:

cd ~/git/zowe-cli-cics-deploy-plugin
npm install
npm run build
zowe plugins install .
zowe plugins list

Installed plugins: 

 -- pluginName: zowe-cli-cics-deploy-plugin 
 -- package: zowe-cli-cics-deploy-plugin 
 -- version: 0.5.0 
 -- registry: https://registry.npmjs.org/

zowe plugins validate

_____ Validation results for plugin 'zowe-cli-cics-deploy-plugin' _____
This plugin was successfully validated. Enjoy the plugin.

I got a similar error when doing a merge

zowe cics-deploy generate bundle --port 27501 --merge --overwrite
Command Error:
A failure occurred during CICS bundle generation.
 Reason = Parsing error occurred reading a CICS manifest file: The module '/Users/cockerma/git/zowe-cli-cics-deploy-plugin/node_modules/node-expat/build/Release/node_expat.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 67. This version of Node.js requires
NODE_MODULE_VERSION 64. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).

The user should be supplied with all logs to enable debugging.

Currently the DEPLOY step returns only the DFHDPLY output.

However as this may well be run in automation - and there are many possible causes of error.

It would be nice to have.

The bunlde error logs, MSGUSR from CICS region etc.

--version parameter doesn't work

If I try to use the --version parameter to customise the bundle version, I get the following error:

Syntax Error:
You specified the following unknown values: "1.8.5".

 Could not interpret them as a group, command name, or positional option.

Use "bright cics-deploy generate bundle 1.8.5 --help" to view command description, usage, and options

I think this is because the top-level zowe command has a --version already. I think we'll need to rename our parameter.

Npm output not streamed back if --verbose is specified.

It is expected that npm output should be streamed back if verbose is specified - it is not.

Eg

Undeployed existing bundle from CICS
Removing the contents of the remote bundle directory
Uploading the bundle to the remote bundle directory
Running npm install for the remote bundle
Deploying the bundle to CICS
███_______| 33% O | Waiting for JOB51833 to enter OUTPUT

Can we skip one of the commands?

I don't see how or why you would.

But it's a scenarion in the test plan for PUSH.

How should generate bundle behave with existing bundle metadata?

generate bundle currently tries to do some clever merging if you run it in a directory with an existing bundle.
I think the main usecase for this command is generating a new bundle. You might run it in a directory with an existing bundle by accident and get strange results.
So my feeling is that the default behaviour should be to fail if there's existing bundle metadata, and provide an option to add a new bundlepart.

--bundleid overrides the NODEJSAPP name from package.json

If I have an app with a package.json in it and run the following:

zowe cics-deploy generate bundle --bundleId myNodeBundle

The generated bundle uses myNodeBundle for both the name of the NODEJSAPP and the bundle ID. I was expecting the name from package.json to be used for the NODEJSAPP, and just the ID to be overridden.

PUSH does not work in GIT BASH on Windows due to the directory issue.

This error on the --targetDir is the problem

$ zowe cics-deploy push bundle --name CICSJS03 --targetdir /u/atkinc/pushtest --zosmf-profile zosmf2c --cics-deploy-profile deploy --ssh-profile ssh2Ckey --ow
Command Error:
A failure occurred during CICS bundle pushing.
Reason = A problem occurred attempting to create directory 'U:/atkinc/pushtest/CICSJSON_1.0.0'. Problem is: z/OSMF REST API Error:
Rest API failure with HTTP(S) status 500
category: 8
rc: -1
reason: 93651005
message: mkdir() error

Invalid bundle version is silently ignored

If I specify an invalid version using --bundleversion, it's silently ignored and 1.0.0 is used instead. I think it should give an error message and fail to generate the bundle, otherwise the user might get a nasty surprise!

zowe cics-deploy generate bundle --bundleversion foo.bar.baz
Anonymous CICS Bundle generated
[mattwil@oc7236824711 scratch]$ cat META-INF/cics.xml 
<manifest xmlns="http://www.ibm.com/xmlns/prod/cics/bundle" bundleVersion="1" bundleRelease="0" bundleMajorVer="1" bundleMinorVer="0" bundleMicroVer="0"></manifest>

Return code could reflect which step failed.

Whilst PUSH reports on the stage eg npm fails etc - should it issue an error code for automation to use ?

--verbose option results in repeated messages to the console

Using the --verbose option on zowe cics-deploy push bundle --name INVOKE --targetdir /u/cicprov/mnt/CICPY000/bundles --overwrite --verbose results in the same messages repeated many times to the console. I copied the output into log.txt that is 544KB. I would expect CI-CD pipelines to commonly use the --verbose option to capture a full logs.

The log shows a second problem. I deliberately remove write permission from the bundle directory:

$ ls -l /u/cicprov/mnt/CICPY000/bundles/
/u/cicprov/mnt/CICPY000/bundles/:
total 16
dr-xr-xr-x   6 COCKERM  TSOUSER     8192 Apr 18 15:58 cics-nodejs-invoke_1.0.0

and the command zowe cics-deploy push bundle --name INVOKE --targetdir /u/cicprov/mnt/CICPY000/bundles --overwrite --verbose duly shows an error when it attempts to remove the files within the bundle, but instead of failing, it continues :

Removing the contents of the remote bundle directory


$ cd /u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0 && rm -r *
rm: FSUM9196 cannot remove directory "node_modules": EDC5111I Permission denied.
rm: FSUM9196 cannot remove directory "nodejsapps": EDC5111I Permission denied.
rm: FSUM9195 cannot unlink entry "package.json": EDC5111I Permission denied.
rm: FSUM9195 cannot unlink entry "package-lock.json": EDC5111I Permission denied.
rm: FSUM9196 cannot remove directory "public": EDC5111I Permission denied.
rm: FSUM9195 cannot unlink entry "server.js": EDC5111I Permission denied.
rm: FSUM9196 cannot remove directory "META-INF": EDC5111I Permission denied.
$ 
Uploading the bundle to the remote bundle directory

Deploy with long bundledir doesn't work

zowe cics-deploy deploy bundle --name LONGDIR --bundledir /u/wilson/anIncrediblyLongBundleDirectoryThatWillForceContinutationInDFHDPLOYJCL
Command Error:
A failure occurred during CICS bundle deployment.
 Reason = Failure occurred submitting DFHDPLOY JCL: 'z/OSMF REST API Error:
Rest API failure with HTTP(S) status 400
message:  Job submission error. Record length 98 too long for JCL submission, maxlen=80
category: 6
reason:   9
rc:       4

'. Most recent status update: 'Submitting DFHDPLOY JCL'.

Deploying to multiple regions - and one region fails to enable. Is this message good enough?

I am deploying to a scope with more than one region. The second region fails to enable to the nodejsapp ( due to the port being in use on the first ). The PUSH command reports this from the DFHDPLY output. You have to delve deeper to see the reason. Should we return anymore output to the user than what it currently does.

Eg to find the reason you need to look in STDERR.

The error is shown in the node STDERR

events.js:160
throw er; // Unhandled 'error' event
^

Error: listen EADDRINUSE :::30701

With MSGUSR giving.

DFHAP1900 04/23/2019 10:58:33 CALMAS3 NONE CICSUSER CONL SET BUNDLE(CICSMULT) ENABLESTATUS(ENABLED) RESP(NORMAL) RESP2(0).
DFHSJ1304 E 04/23/2019 10:58:35 CALMAS3 CICSUSER NODEJSAPP CICSAPP1 has ended abnormally with return code 1.

Zowe push shows this error from DFHDPLY

C:\Users\CHRISTOPHERAtkinson\Documents\ZoweStuff\ZOWEDEV\PUSH1\tests\CICSMULT>zowe cics-deploy push bundle --name CICSMULT --targetdir /u/atkinc/pushtest --zosmf-profile zosmf2c --cics-deploy-profile allregs --ssh-profile ssh2Ckey --ow
WARNING: No .zosAttributes file found in the bundle directory, default values will be applied.
WARNING: No .zosAttributes file found in the bundle directory, default values will be applied.
WARNING: No .zosAttributes file found in the bundle directory, default values will be applied.
WARNING: No .zosAttributes file found in the bundle directory, default values will be applied.
10:57:59.057965 : DFHDPLOY CICS TS APPLICATION DEPLOYMENT 2019/04/23 10:57am
10:57:59.058603 : RELEASE: HCI7300. SERVICE LEVEL: HCI7300.
10:57:59.059913 : *
10:57:59.059920 : SET CICSPLEX(CAPLEX);
10:58:01.078042 : DFHRL2119I Connection to CMAS(CACMASA ) successful.
10:58:03.086085 : DFHRL2013I Connection to CICSPLEX(CAPLEX) successful.
10:58:03.101003 :
10:58:03.101013 : *
10:58:03.101019 : DEPLOY BUNDLE(CICSMULT)
10:58:03.101023 : BUNDLEDIR(/u/atkinc/pushtest/CICSMULT_1.1.1)
10:58:03.101027 : SCOPE(ALLREGIN)
10:58:03.101031 : STATE(ENABLED)
10:58:09.149777 : DFHRL2049I Creating BUNDLE definition.
10:58:09.156515 : DFHRL2024I BUNDLE definition(CICSMULT) VERSION(1) successfully created.
10:58:10.160109 : DFHRL2052I Installing BUNDLE definition.
10:58:10.160445 : DFHRL2131I Waiting for BUNDLE(CICSMULT) to be installed in active regions in SCOPE(ALLREGIN).
10:58:29.277383 : DFHRL2130I BUNDLE(CICSMULT) installed in 2 of 2 regions in SCOPE(ALLREGIN).
10:58:29.277432 : DFHRL2054I Setting BUNDLE state to ENABLED.
10:58:40.346790 : DFHRL2067W BUNDLE(CICSMULT) did not reach state ENABLED in all systems in SCOPE(ALLREGIN).
10:58:40.351913 : DFHRL2129I BUNDLE(CICSMULT) state is INSTALLED on 2 CICS regions in SCOPE(ALLREGIN).
10:58:40.351954 : DFHRL2129I BUNDLE(CICSMULT) state is DISABLED on 2 CICS regions in SCOPE(ALLREGIN).
10:58:40.351961 :
10:58:40.351964 : Status of BUNDLE(CICSMULT) in SCOPE(ALLREGIN):
10:58:40.351968 : CICS STATUS AVAILABILITY BUNDLE
10:58:40.351971 : CALMAS1 DISABLED NONE CICSMULT
10:58:40.351975 : CALMAS3 DISABLED NONE CICSMULT
10:58:40.353341 :
10:58:40.353346 : All Bundle part records for BUNDLE(CICSMULT):
10:58:40.353350 : CICS BUNDLE STATUS AVAILABILITY BUNDLEPART
10:58:40.354049 : CALMAS1 CICSMULT ENABLED NONE CICSAPP1 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.354849 : CALMAS3 CICSMULT ENABLED NONE CICSAPP2 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.355767 : CALMAS1 CICSMULT ENABLED NONE CICSAPP3 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.356510 : CALMAS3 CICSMULT ENABLED NONE CICSAPP4 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.357295 : CALMAS3 CICSMULT DISABLED NONE CICSAPP1 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.358242 : CALMAS1 CICSMULT DISABLED NONE CICSAPP2 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.358972 : CALMAS3 CICSMULT DISABLED NONE CICSAPP3 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.359758 : CALMAS1 CICSMULT DISABLED NONE CICSAPP4 http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP
10:58:40.359763 :
10:58:40.359795 : DFHRL2055I Errors have occurred, processing terminated.
10:58:40.367040 : DFHRL2014I Disconnecting from CICSPLEX(CAPLEX).

Command Error:
A failure occurred during CICS bundle pushing.
Reason = DFHDPLOY stopped processing for JOBID JOB63569 due to an error.

Do we need async access to DFHDPLOY messages?

The DEPLOY/UNDEPLOY actions can take a while to run, DEPLOY regularly takes about 30 seconds for me. Watching a static screen while waiting for it to complete isn't great. I've implemented a progress bar, and maybe that's sufficient, but should we be looking for something better? We could (attempt) to poll the JES output and display in an approximation of real-time.

Is it worth the effort to do that? It wont make things run any faster, but might be less tedious.

Parameter names should use hyphens to separate words

From looking at the guidelines CommandFormatStandards under heading General Naming Conventions suggest hyphenation to separate words in options. Examples can be seen in the CLI Reference Document eg. --region-name.

The cics-deploy plugin by comparison does not use hyphens for most parameter names eg. --csdgroup and --targetstate except for parameters that already exist elsewhere such as profile and global parameters eg. --cics-deploy-profile.

As Zowe CLI users will typically use a mixture of cics-deploy and other commands, conforming to the Zowe standards will help usability.

NLS Enable messages

Should we consider allocating DFHRLxxxx message numbers for each message issued? Something from a range adjacent to that of DFHDPLOY; it'll allow us to provide a more detailed user-response for each such message. We'd document them in the CICS KC, but they would at least by Googleable.

As part of that work we should extract the text of the messages into a format that can be translated.

Npm should not run when the push command is used on a bundle that does not contain a NODEJAPP bundlepart

Npm should not run when the push command is used on a bundle that does not contain a NODEJAPP bundlepart. But it does.

Eg
$ cd /u/atkinc/pushtest/NoNode_1.0.3 && rm -r *
$
Uploading the bundle to the remote bundle directory

Running npm install for the remote bundle

<$PATH:/usr/lpp/IBM/cnj/IBM/node-latest-os390-s390x/bin" && npm install
[email protected] /u/atkinc/pushtest/NoNode_1.0.3
+-- [email protected]
| +-- [email protected]
| +-- [email protected]
| +-- [email protected]

Example bundle attached.

NoNodeAPP.zip

Error message from deploy bundle with an overlength jobcard is not nice

zowe cics-deploy deploy bundle --name FREYJA --bundledir /u/wilson/freyjademo --jobcard "//MWDEPLOY JOB a very long jobcard which is an exceeding bad thing to do because it will break"
Command Error:
A failure occurred during CICS bundle deployment.
 Reason = Failure occurred submitting DFHDPLOY JCL: 'z/OSMF REST API Error:
Rest API failure with HTTP(S) status 400
message:  Job submission error. Record length 94 too long for JCL submission, maxlen=80
category: 6
reason:   9
rc:       4

Suggest it would be worth checking the length of the jobcard parameter ourselves

Error scenarios should set return code and output to stderr

In error scenarios we're currently setting rc=0. For scripting of the command it would be useful to tell if the generate has failed, so that the script can skip any further actions and report errors.
For example, if I don't have write permission to the current directory so the bundle metadata can't be created I'd like to see rc > 0.

Also it would be useful if error messages could be sent to stderr. This allows scripts to extract error messages from informational messages.

Build should generate a .md file that summarise the help for all commands

It would be useful for the build to create a .md file that includes the same source as the command line help for all of the cics-deploy commands, with an TOC/index at the start and HTML anchors into each command and each option. We could then link into this file from the wiki and other locations as the definitive source of parameter help, and it can be found and indexed by search engines. This would also allow the wiki to focus on use cases, rather than duplicating reference information that could become stale.

I have prototyped what the file could look like in the wiki Commands.md

This could use a similar approach to that used by the Zowe CLI that generate Zowe CLI Help.

Should deploy bundle perform an undeploy beforehand? Do we need an --overwrite option?

The current behaviour of zowe cics-deploy deploy bundle is that it tells DFHDPLOY to do an UNDEPLOY BUNDLE followed by a DEPLOY BUNDLE.

As this is one of our 'atomic commands', I'm wondering if it would be better to to just do the DEPLOY. It's then up to the user to script an undeploy bundle followed by a deploy bundle if they want to do that.

A use-case where you might not want to UNDEPLOY first is a first-time deployment you're not sure if you've got a unique BUNDLE resource name. You're expecting the DEPLOY to just work, but if you picked a name that was already used you will inadvertently undeploy someone else's application.

An alternative would be to provide a command line argument to perform the UNDEPLOY first (maybe --force?)

I'd expect our cics-deploy push command to combine the undeploy and the deploy.

Push should create directory specified by --targetdir option

At the moment the z/OS directory specified by --targetdir option is required to exist and the bundle directory is created within it using mkdir. If targetdir directory not exist you get the error:

Command Error:
A failure occurred during CICS bundle pushing.
 Reason = A problem occurred attempting to create directory '/u/cicprov/mnt/CICPY000/bundles/cics-nodejs-invoke_1.0.0'. Problem is: z/OSMF REST API Error:
Rest API failure with HTTP(S) status 500
category: 8
rc:       -1
reason:   93651005
message:  mkdir() error

It would be useful if any necessary parent directories were automatically created, for example using the mkdir -p option when creating the bundle directory. This enhancement would be useful when deploying a bundle to the file system created by z/OS Provisioning Toolkit. For example:

zowe cics-deploy push bundle --name INVOKE --targetdir /u/cicprov/mnt/CICPY000/bundles --overwrite

would work even though /bundles does not exist.

Generate of bundle ignores a profile in my directory.

When I generate a bundle - it creates a sub directory with the nodejsapp bundle part, and a profile ( sample)

I have a profile already setup in my directory - why can't I use it ?!

The directory ( top level after running the generate bundle)

How does the user know where stdout/stderr are?

Suppose a Node.js Bundle is deployed successfully to CICS without error, that's awesome! Now what? As the user, how do I know where my stderr/stdout output gets written by CICS? Or suppose something goes wrong, how do I find the error messages issued to stderr, SYSPRINT, MSGUSR, JESMSGLG?

Maybe we need an extra activity that can be scripted after the DEPLOY (or perhaps as part of DEPLOY) that uses CMCI to query aspects of (one of) the region(s) that the Bundle was deployed to. It'd be good to provide the user with a zowe command they could issue to view the JESMSGLG, etc. (perhaps starting at a particular timestamp within the logs), and to view the output files on zFS.

If we require CMCI in order to do this then that will be a 4th profile to configure, the first three being zosMF, SSH and cics-deploy.

Generated .profile has %INCLUDE path that needs updating

The generated .profile includes:

%INCLUDE=&USSCONFIG;/nodejsapps/general.profile

that should be

%INCLUDE=&USSCONFIG;/nodejsprofiles/general.profile

to match the naming for existing JVM profiles directory (albeit all lower case) <cics_install>/JVMProfiles.

Update dependencies to @zowe

A recent commit on Zowe CLI changed the prefix for both core and imperative to @zowe instead of @brightside.
We'll need to update all our dependencies and imports to reflect this.

Incidentally, I think this was the root of the issues we had with importing Session/AbstractSession.

deploy bundle does not support multiline jobcards

I think it should be possible to have a multi-line jobcard by simply beginning a subsequent line with // and continuing in column 7 (see https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.ieab600/iea3b6_Examples_of_JOB_statements.htm)
However this doesn't seem to work. Looks like we'll probably need to split the jobcard to allow multiple lines:

zowe cics-deploy deploy bundle --name FREYJA --bundledir /u/wilson/freyjademo --jobcard "//MWDEPLOY JOB MSGCLASS=A \n//          TIME=NOLIMIT,MSGLEVEL=(1,1),NOTIFY=&SYSUID,CLASS=Y"
Command Error:
A failure occurred during CICS bundle deployment.
 Reason = Failure occurred submitting DFHDPLOY JCL: 'z/OSMF REST API Error:
Rest API failure with HTTP(S) status 400
message:  Job submission error. Record length 90 too long for JCL submission, maxlen=80
category: 6
reason:   9
rc:       4

'. Most recent status update: 'Submitting DFHDPLOY JCL'.

Allow DFHDPLOY to manage the bundle definition

We currently require either --resgroup or --csdgroup to be specified. However, the DFHDPLOY doc states that if you don't specify either, it will create the definition in BAS and discard it afterwards. I think we should support this approach, it simplifies things for Freyja.

Online help shows "bright" instead of "zowe"

zowe cics-deploy undeploy bundle --help and others displays "bright" or "$ bright" instead of "zowe" command in the usage and examples sections, eg:

USAGE
 -----

   bright cics-deploy undeploy bundle [options]

and

EXAMPLES
 --------

   - Undeploy a CICS bundle using the default cics-deploy
   profile:

      $ bright cics-deploy undeploy bundle --name EXAMPLE

Retrieve output from DFHDPLOY progressively

The progress bar on deploy and undeploy provides at least some reassurance that something is happening, but it would be nice if we could retrieve the output from DFHDPLOY as it happens. This would give the user some handy feedback on what exactly is going on.

bundle generate should generate .zosattributes file

it would be helpful cics-deploy generate bundle generated a tempalte .zosattributes file with a basic set of filetypes, including things like .profile.

Inaccurate comment in generated .zosattributes

The zowe cics-deploy generate command creates the following .zosattributes file. The comment # Upload images in binary is inaccurate because .ttf, .woff, .eot, and .zip are not image files. Suggest changing to # Upload following files in binary.

As this file type will be unfamiliar, can we add a comment at the start about the purpose & format.

# Don't upload node_modules
node_modules -
# Don't upload things that start with dots
.* -

# Upload images in binary
*.jpg binary binary
*.png binary binary
*.gif binary binary
*.zip binary binary
*.eot binary binary
*.svg binary binary
*.ttf binary binary
*.woff binary binary

# Convert CICS Node.js profiles to EBCDIC
*.profile ISO8859-1 IBM-1047

push names bundle-dir with the BUNDLE name rather than bundle ID

If I do this:
zowe cics-deploy push bundle --targetdir /u/wilson/zowe-tests --name INVOKAPP

my bundle is uploaded to /u/wilson/zowe-tests/INVOKAPP i.e. the directory is named after the CICS BUNDLE resource rather than the (potentially more descriptive) bundle ID.

I suggest we use bundle ID instead, perhaps appending the version to be consistent with CICS Explorer.

Unhelpful error message if bundle is uploaded without .zosattributes ( or ones that would be invalid)

11:26:52.448922 : DFHRL2131I Waiting for BUNDLE(CICSPH02) to be installed in active regions in SCOPE(CALMAS1).
11:26:54.461144 : DFHRL2300E BUNDLE(CICSPH02) cannot be deployed. The reason for the failure could not be determined.
11:26:54.467623 : DFHRL2055I Errors have occurred, processing terminated.
11:26:54.472497 : DFHRL2014I Disconnecting from CICSPLEX(CAPLEX).

Command Error:
A failure occurred during CICS bundle pushing.
Reason = DFHDPLOY stopped processing due to an error.

Actual cause of this error - ( I just copied on from explorer and it did not have a .zosattribute files with it)

DFHRL0107 I 04/17/2019 11:26:52 CALMAS1 CICSUSER The CICS resource lifecycle manager has started to create the BUNDLE resource
CICSPH02.
DFHRL0112 E 04/17/2019 11:26:52 CALMAS1 COIE The encoding of the manifest /u/atkinc/pushtest/CICSPH02/META-INF/cics.xml in the root
directory of the bundle CICSPH02 is not valid.
DFHRL0110 E 04/17/2019 11:26:52 CALMAS1 COIE The CICS resource lifecycle manager has failed to create the BUNDLE resource CICSPH02.

BundleMocked.test is too tightly bound to implementation

See #51 (comment)

The timer on PUSH goes up and down. It does not increment nicely

The timer on the PUSH command seems related to the step it is on.

So you see the time go to 34% 70% 18% etc.

It should step nicely upwards.

Mangled NODEJSAPP name is not used for path in manifest

If I specify a NODEJSAPP name on the command line which needs to mangled, the mangled name is ued for .nodesjapp file, but not for the path attribute in cics.xml. Therefore the bundle would not work in CICS.

zowe cics-deploy generate bundle --nodejsapp foo$€@ --startscript foo.js
NODEJSAPP "foo$€@" defined for startscript "foo.js"
Anonymous CICS Bundle generated
[mattwil@oc7236824711 scratch]$ cat META-INF/cics.xml 
<manifest xmlns="http://www.ibm.com/xmlns/prod/cics/bundle" bundleVersion="1" bundleRelease="0"><define name="fooXXX" type="http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP" path="nodejsapps/foo$€@.nodejsapp"></define></manifest>
[mattwil@oc7236824711 scratch]$ ls nodejsapps
fooXXX.nodejsapp  fooXXX.profile

Should --port be renamed for Generate Bundle?

We have a CMCI port, a zosMF port, maybe an SSH port... perhaps these things need a little disambiguation when they're all used together? We could rename --port to --nodeport perhaps?

Create cics-deploy-profile is missing examples in the help

It would be nice to add examples in the output from
zowe profiles create cics-deploy-profile --help

Generated files do not have a newline at the end

None of the generated files (cics.xml, profile or NODEJSAPP) have a newline at the end. It's a minor niggle, but when viewing the files in a terminal (which seems like a reasonably likely thing to do to see what the tool has generated) it screws up the terminal output a bit.

 cat META-INF/cics.xml 
<manifest xmlns="http://www.ibm.com/xmlns/prod/cics/bundle" bundleVersion="1" bundleRelease="0"><define name="foo" type="http://www.ibm.com/xmlns/prod/cics/bundle/NODEJSAPP" path="nodejsapps/foo.nodejsapp"></define></manifest>[mattwil@oc7236824711 scratch]$ 
```

If CICS does not have permission to the bundle directory DFHDPLOY gives unknown error

Waiting for JOB69328 to enter OUTPUT (Processing DFHDPLOY DEPLOY action)

11:46:00.651039 : DFHDPLOY CICS TS APPLICATION DEPLOYMENT 2019/04/24 11:46am
11:46:00.651788 : RELEASE: HCI7300. SERVICE LEVEL: HCI7300.
11:46:00.653012 : *
11:46:00.653019 : SET CICSPLEX(CAPLEX);
11:46:02.670416 : DFHRL2119I Connection to CMAS(CACMASA ) successful.
11:46:04.683749 : DFHRL2013I Connection to CICSPLEX(CAPLEX) successful.
11:46:04.700320 :
11:46:04.700339 : *
11:46:04.700346 : DEPLOY BUNDLE(CICSJS02)
11:46:04.700351 : BUNDLEDIR(/u/atkinc2/CICSJSON_1.0.0)
11:46:04.700355 : SCOPE(CALMAS1)
11:46:04.700359 : STATE(ENABLED)
11:46:04.700363 : CSDGROUP(PUSHTEST);
11:46:04.706854 : DFHRL2132I Analyzing CICS regions and CSD attributes.
11:46:12.771460 : DFHRL2051I Creating BUNDLE definition on the CSD in system(CALMAS1 ).
11:46:12.891344 : DFHRL2049I Creating BUNDLE definition.
This is the error that DFHDPLY gives - should the error (MSGUSR) be sent to the user ?

11:46:12.897641 : DFHRL2024I BUNDLE definition(CICSJS02) VERSION(3) successfully created.
11:46:13.901118 : DFHRL2052I Installing BUNDLE definition.
11:46:13.901520 : DFHRL2131I Waiting for BUNDLE(CICSJS02) to be installed in active regions in SCOPE(CALMAS1).
11:46:15.916293 : DFHRL2300E BUNDLE(CICSJS02) cannot be deployed. The reason for the failure could not be determined.
11:46:15.922147 : DFHRL2055I Errors have occurred, processing terminated.
11:46:15.927066 : DFHRL2014I Disconnecting from CICSPLEX(CAPLEX).

The actual error is found in MSGUSR

DFHRL0106 E 04/24/2019 11:46:13 CALMAS1 COIE The CICS resource lifecycle manager failed to create the BUNDLE resource CICSJS02
because CICS is not authorized to read the manifest /u/atkinc2/CICSJSON_1.0.0/META-INF/cics.xml in the root directory of
the bundle.
DFHRL0110 E 04/24/2019 11:46:13 CALMAS1 COIE The CICS resource lifecycle manager has failed to create the BUNDLE resource CICSJS02.

Is DEPLOY too verbose in golden path scenarios? Or too techie?

DFHDPLOY will generate quite a lot of text that might not mean very much to our Freyja persona, should we hide some of it by default? It's probably needed if something goes wrong, but if everything works we could suppress much of the output.

On a related note, are the messages and other externals too CICS specific? Would it be better for our user not to even know about DFHDPLOY, perhaps we should seek generic terms in which to describe what's happening?

Error messages could be improved for problems parsing package.json

I have a few scenarios involving missing/empty/bad package.json that could do with clearer messages:

• When there's no main or scripts/start property in package.json I get this rather misleading message:

A failure occurred during CICS Bundle generation.
    Reason = BundlePart "cics-nodejs-invoke" references a file that does not exist: "/home/mattwil/git/zowe-cics-bundle-plugin-public/__tests__/__results__/data/afae1a9c-09f3-4c56-aaab-8969d5a5f633_generate_bundle_command/apps/minimal-package-json/undefined"

• When package.json is empty I get this:

A failure occurred during CICS Bundle generation.
    Reason = Unexpected end of JSON input

and when it's malformed I get:

A failure occurred during CICS Bundle generation.
    Reason = Unexpected token : in JSON at position 9

I think it would helpful in both cases to say that the problem occurred when parsing package.json.

No NODEJSAPP generated without package.json

I have an app with no package.json, and I want to specify the relevant parameters and get my bundle generated including the NODEJSAPP and profile. I'm specifying --startscript, --bundleid and --nodjesapp, but I don't see a NODEJSAPP or profile generated.

We should allow the user to specify the LPAR DFHDPLOY runs on

It would be good if we could allow a SYSPARM to allow DFHDPLY to run on a different LPAR than the zOS/MF is running on.

What should go in the generated Node.js profile?

The generate bundle command currently generates a Node.js profile with an include statement. This means it won't work by itself, someone needs to create that include file or edit the generated profile. The name of the include file we generate contains the name of the NODEJSAPP, so you'd need to create a unique file for each app following this scheme.

Other options include:

• Generate a profile with no includes. This might work out of the box if the defaults are suitable. This could be the template profile we ship with CICS including all the comments.
• Generate a profile with a 'static' include. This patterns suggests that you put everything system-related like WORK_DIR and NODE_HOME in a single include file that could be included by all apps. You'd still need to set PORT in the app profile.

Here's the complete profile we're currently generating:

# This is a profile for a CICS NODEJSAPP resource

# Import the provisioned configuration file for this NODEJSAPP,
# this file must be created before this Bundle can be installed in CICS
INCLUDE=&USSCONFIG;/nodejsapps/cics-nodejs-invoke.included.profile

# Set the PORT envionment variable, application code should reference this
# value in preference to a hard-coded port number, the value references an
# environment variable that will be configured within the provisioned configuration file.
PORT=&HTTP_PORT;

Should PUSH detect if npm_modules directory is removed from .zosattributes ?

If you edit the .zosatttribute file to allow npm_modules directory to be uploaded the npm_modules are uploaded.

The PUSH then does a npm install of the .package.json file.

If the modules are all the same they are just reinstalled. So wastes time.

Maybe a warning should be given, as perhaps there maybe a use case for the user uploading something that is not in the package.json ?

DFHRL2069W from DFHDPLOY on deploy bundle. Should we support both ENABLED/AVAILABLE as target states?

When I perform a deploy bundle command, I get the following warning from DFHDPLOY:

DFHRL2069W BUNDLE(NODETEST) cannot be set to AVAILABLE as it contains no entry points.

I think we should set the target state to ENABLED rather than AVAILABLE to avoid this.

The deploy still succeeds, it's just that the warning might be confusing to the user.