intersystems/ipm

InterSystems ObjectScript Package Manager

ObjectScript Package Manager Client - ZPM

Helps to install ObjectScript classes and routines, globals, Embedded Python modules, CSP and Frontend packages, and any files into InterSystems IRIS published on the official ZPM Registry or private ZPM registry of your own.

Documentation

• The official documenation in the wiki
• Articles on the InterSystems Developer Community
• Videos on YouTube

Installing ObjectScript Package Manager Client:

0. Use one-liner in terminal call or programmatically:

s r=##class(%Net.HttpRequest).%New(),r.Server="pm.community.intersystems.com",r.SSLConfiguration="ISC.FeatureTracker.SSL.Config" d r.Get("/packages/zpm/latest/installer"),$system.OBJ.LoadStream(r.HttpResponse.Data,"c")

OR:

1. Download the latest version of zpm from the registry
2. Import the zpm.xml into IRIS and compile via any desired way (Management Portal, Studio or Terminal)

After that you can use PackageManager to install modules from community repository in any namespace.

3. Check if you call a zpm in command line and get the following: USER>zpm

zpm: USER>

How to Install a ZPM Module:

1. Call this command to open zpm shell:

USER> zpm

2. See the list of available modules:

zpm: USER>repo -list-modules -n registry

deepseebuttons 0.1.7 dsw 2.1.35 holefoods 0.1.0 isc-dev 1.2.0 mdx2json 2.2.0 objectscript 1.0.0 pivotsubscriptions 0.0.3 restforms 1.6.1 thirdpartychartportlets 0.0.1 webterminal 4.8.3 zpm 0.0.7

3. You can load any module that resides in any of the defined repos into IRIS. E.g. here is the way to install webterminal:

zpm: USER> install webterminal

To uninstall a module:

USER> zpm

zpm: USER> uninstall webterminal

How to submit modules

This is described in the following set of articles

The simplest and template repository can be found here. Here is the alternative supported folder structure.

Support and Collaboration

ObjectScript Package Manager is a community supported project and thus open to collaboration via Pull Requests. Issues and feature requests are very welcome

ipm's Issues

Use IAM to manage calls to /registry/package

We need to filter calls for an update to only from openexchange.intersystems.com

Can't install on Iris for Health

Hi,

I try to install zpm on iris for health with those commands :

• cd /tmp

• wget -q https://pm.community.intersystems.com/packages/zpm/latest/installer -O zpm.xml

• iris session irishealth

• USER>Do $system.OBJ.Load("/tmp/zpm.xml","ck")

Error :

Compiling routine %ZPM.PackageManager.Developer.ResourceReferenceAttributes.1
ERROR #6281: XMLTYPE of class %ZPM.PackageManager.Developer.Processor.CSPApplication must be able to differentiate child classes of %Installer.Activity.
ERROR #5090: An error has occurred while creating projection %ZPM.PackageManager.Developer.Processor.CSPApplication:XMLEnabled.
Detected 1 errors during compilation in 1.812s.
ERROR #6281: XMLTYPE of class %ZPM.PackageManager.Developer.Processor.CSPApplication must be able to differentiate child classes of %Installer.Activity.
ERROR #5090: An error has occurred while creating projection %ZPM.PackageManager.Developer.Processor.CSPApplication:XMLEnabled.
Detected 1 errors during load.
2019-09-30 11:35:43 0 %ZPM.Installer: ERROR #6281: XMLTYPE of class %ZPM.PackageManager.Developer.Processor.CSPApplication must be able to differentiate child classes of %Installer.Activity.
ERROR #5090: An error has occurred while creating projection %ZPM.PackageManager.Developer.Processor.CSPApplication:XMLEnabled.
2019-09-30 11:35:43 0 %ZPM.Installer: Installation failed at 2019-09-30 11:35:43
2019-09-30 11:35:43 0 %Installer: Elapsed time 4.49325s
Time spent 4.519202
Load finished successfully.

Tested on :

• IRIS for UNIX (Ubuntu Server LTS for x86-64 Containers) 2019.2.0S (Build 260U) Thu Jun 13 2019 01:59:29 EDT

• IRIS for UNIX (Apple Mac OS X for x86-64) 2019.1.0S (Build 111U) Wed Jan 16 2019 00:31:57 EST

Make the option to setup REST apps without folders setup

Sometimes we need only the name and dispatch class to setup the REST app.
But now ZPM needs PATH and Directory, and controls, if it exists in source, even it's not needed for the REST App.
example https://github.com/intersystems-community/RESTForms2/blob/master/module.xml
Let's simplify REST apps setup to give the option not use PATH and DIRECTORY params.

Investigate leverage of github packages with zpm

Fix: <DIRECTORY> in ResolveChildren for long paths on Windows

#19 moved to use StudioOpenDialog to resolve children of a .PKG resource (to include .inc/.mac as well); the filter on this should be more careful to avoid CSP files, where a long filename (e.g., node_modules) can cause errors and fail installation.

@pravinb1 observed this trying to install mdx2json on an internal application and will take care of the PR.

verify lifecycle phase failing

zpm: USER>objectscript-math verify -only                                                                                             
[objectscript-math]     Verify START
[objectscript-math]     Verify FAILURE - ERROR #19: the file was already mounted
ERROR #19: the file was already mounted

Needs investigation.

Issue with publishing declarative-objectscript package via OEX

We received the following error while publishing your package declarative-objectscript:
ERROR #6901: XSLT XML Transformer Error: unable to determine file base pathname in at line 0 offset 0
discussion

Add actions to the module-action: download module archive tar to specific directory

For example, to transfer a module from one register to another

Make a convenient call for available modules to install

E.g.
list -remote
to display what's available on the current registry

Introduce an option to generate module.xml for the package

something like:

zpm load /path
zpm generate module.xml

automatically: go through all the /src and form Objectscript resources
automatically: go through tests and form tests
interactively: ask for web apps and load the info and form CSPAPPLICATION sections

UPDATE!
Ask for the name (and dependencies?)
Scan the NAMESPACE and generate
Resources for all classes(not mapped) and routines,
Globals (no mapped),
and CSP Apps.

Setup a droplet for stage server

Digital ocean?
Once we have an IP we setup the domain name

Add an option to transfer parameters to modules installation

For example the name of the REST application, or parameters for post-install methods

#21 breaks Directory attribute for class resource processor

For example:

    <Resource Name="AppS.Exception.CLS">
	  <Attribute Name="Directory">cls</Attribute>
    </Resource>
    <Resource Name="AppS.Exception.PKG">
    </Resource>

The package manager attempts to load AppS.Exception from the module root rather than /cls (which used to be the default).

ZPM Client doesn't work with alternative registry

Error install: blocksexplorer, isc-apptools-admin

%SYS>zpm "list"
zpm 0.1.8

zpm: USER>install blocksexplorer

[blocksexplorer] Reload START
[blocksexplorer] Reload SUCCESS
[blocksexplorer] Module object refreshed.
[blocksexplorer] Validate START
[blocksexplorer] Validate SUCCESS
[blocksexplorer] Compile START
[blocksexplorer] Compile FAILURE - ОШИБКА #5559: Определение класса Blocks.BlocksMap имеет ошибки, поэтому не может быть скомпилировано

ОШИБКА #5030: Возникла ошибка при компиляции класса Blocks.BlocksMap
ОШИБКА #5559: Определение класса Blocks.BlocksMap имеет ошибки, поэтому не может быть скомпилировано
ОШИБКА #5030: Возникла ошибка при компиляции класса Blocks.BlocksMap
ОШИБКА #5559: Определение класса Blocks.BlocksMap имеет ошибки, поэтому не может быть скомпилировано
ОШИБКА #5030: Возникла ошибка при компиляции класса Blocks.BlocksMap
zpm: USER>install webterminal

Error install to windows $zv=IRIS for Windows (x86-64) 2019.1.1 (Build 609U) Thu Sep 12 2019 22:26:55 EDT

zpm: %SYS>list
zpm 0.1.7

REST2>zpm "install contest-rest-template"

[swagger-ui] Reload START
[swagger-ui] Reload SUCCESS
[swagger-ui] Module object refreshed.
[swagger-ui] Validate START
[swagger-ui] Validate SUCCESS
[swagger-ui] Compile START
[swagger-ui] Compile SUCCESS
[swagger-ui] Activate START
[swagger-ui] Configure START
[swagger-ui] Configure FAILURE - ОШИБКА #875: Основная роль : не существует.
[swagger-ui] Activate FAILURE - ОШИБКА #875: Основная роль : не существует.
ОШИБКА #875: Основная роль : не существует.
ОШИБКА #875: Основная роль : не существует.
ОШИБКА #875: Основная роль : не существует.
REST2>

Add actions to the module-action: description get a description of the module taken from the repository

uninstall allows you to remove dependent class

I suggest showing an alert if there are modules that depend on deleted module

Add the support for method calling

Introduce the support for tag in module.xml which accepts the call for class method with parameters
Similar as it introduced to %Installer.cls

We need it e.g. to call methods which setup initial data

Add actions to the module-action: transfer module from one register to another

Let dependencies omit version requirement

now you need to introduce a version of the dependent module.
let's omit that to look for any latest module

Add support ${parameter} too

Now {$parameter} is supported in ZPM
Introduce ${parameter} syntax support too cause it's the same as in %Installer

ZPM's %Installer manifest does not fail in case of errors

When some errors happen during install of ZPM through %Installer manifest, it ignores any errors.
Excerpt of the log with this issue

2020-04-10T10:22:54.2764064Z [zpm]	Reload START
2020-04-10T10:22:54.2764330Z [zpm]	Reload SUCCESS
2020-04-10T10:22:54.2764452Z [zpm]	Module object refreshed.
2020-04-10T10:22:54.2786471Z [zpm]	Validate START
2020-04-10T10:22:54.2790253Z [zpm]	Validate FAILURE - ERROR #5001: Error validating resource '/tests/unit_tests/'
2020-04-10T10:22:54.2792318Z   > ERROR #7200: Datatype value 'test' failed IsValidDT validation
2020-04-10T10:22:54.2793291Z   > ERROR #5802: Datatype validation failed on property '%ZPM.PackageManager.Developer.Processor.UnitTest:Phase', with value equal to "test"
2020-04-10T10:22:54.2794062Z ERROR #5001: Error validating resource '/tests/unit_tests/'
2020-04-10T10:22:54.2794897Z   > ERROR #7200: Datatype value 'test' failed IsValidDT validation
2020-04-10T10:22:54.2799321Z   > ERROR #5802: Datatype validation failed on property '%ZPM.PackageManager.Developer.Processor.UnitTest:Phase', with value equal to "test"
2020-04-10T10:22:54.2800131Z ERROR #5001: Error validating resource '/tests/unit_tests/'
2020-04-10T10:22:54.2800814Z   > ERROR #7200: Datatype value 'test' failed IsValidDT validation
2020-04-10T10:22:54.2807554Z     > ERROR #5802: Datatype validation failed on property '%ZPM.PackageManager.Developer.Processor.UnitTest:Phase', with value equal to "test"
2020-04-10T10:22:55.5316330Z [zpm]	Reload START
2020-04-10T10:22:55.5317712Z [zpm]	Reload SUCCESS
2020-04-10T10:22:55.5318477Z [zpm]	Module object refreshed.
2020-04-10T10:22:55.5354532Z [zpm]	Validate START
2020-04-10T10:22:55.5356236Z [zpm]	Validate FAILURE - ERROR #5001: Error validating resource '/tests/unit_tests/'
2020-04-10T10:22:55.5358319Z   > ERROR #7200: Datatype value 'test' failed IsValidDT validation
2020-04-10T10:22:55.5359358Z   > ERROR #5802: Datatype validation failed on property '%ZPM.PackageManager.Developer.Processor.UnitTest:Phase', with value equal to "test"
2020-04-10T10:22:55.5359956Z ERROR #5001: Error validating resource '/tests/unit_tests/'
2020-04-10T10:22:55.5360486Z   > ERROR #7200: Datatype value 'test' failed IsValidDT validation
2020-04-10T10:22:55.5361182Z     > ERROR #5802: Datatype validation failed on property '%ZPM.PackageManager.Developer.Processor.UnitTest:Phase', with value equal to "test"2020-04-10 10:22:55 0 %ZPM.Installer: Installation succeeded at 2020-04-10 10:22:55
2020-04-10T10:22:55.5361760Z 2020-04-10 10:22:55 0 %Installer: Elapsed time 7.633499s
2020-04-10T10:22:55.5390993Z 
2020-04-10T10:22:55.5395206Z %SYS>
2020-04-10T10:22:55.5395621Z 
2020-04-10T10:22:55.5395916Z %SYS>
2020-04-10T10:22:55.5396302Z 
2020-04-10T10:22:55.5396610Z %SYS>
2020-04-10T10:22:55.5444012Z 
2020-04-10T10:22:55.5444433Z %SYS>
2020-04-10T10:22:55.5500844Z 
2020-04-10T10:22:55.5501132Z %SYS>
2020-04-10T10:23:05.4127825Z Removing intermediate container cc08c8149fcf
2020-04-10T10:23:05.4128986Z  ---> 529fbe0deb68
2020-04-10T10:23:05.4196053Z Successfully built 529fbe0deb68

It shows that build was successful, but error ERROR #5802: Datatype validation failed on property '%ZPM.PackageManager.Developer.Processor.UnitTest:Phase', with value equal to "test" happened there

Modify ZPM Analytics

Event Category - module name
Event action - version
Event Label - IRIS Product, Platform, Version

Introduce External CLI for ZPM client

Introduce a stage brunch and CI for testing and deploying of ZPM client

globals are not published with package

E.g. this package:
https://github.com/isc-zpm/Samples-Aviation

Introduce automatic submits for approved packages in open-exchange

If the package has module.xml and the version is approved on Open Exchange let the package be automatically submitted to ZPM repo server
Expose the API to let this calls and accept the repo as a parameter

add command testpackage

The command will do the following:

it will publish the package on a remote test registry (the official one) and will return the result.
it will start an IRIS instance with ZPM and install the package from test registry and will return the result (success or error log)

UPDATE!!!
Make the option to run integrity tests (deployment tests) for the package automatically:
create registry, publish package, create client IRIIS with zpm, deploy package, run tests, return result.
In ONE command.

if the module has the heavy globals it doesn't publish the package remotely

E.g. this repo not publishes itself.
It silently reports on success but doesn't work in fact.

Introduce usage tracking

Track installations and send it to the registry if possible
The tracking feature is enabled by default but could be turned off

Introduce logging for csp application setup

If we have CSP and REST applications settings in Module it's not visible now e.g. with

install package -v

Create official test server for zpm registry

It should be rebuilt every night

Add synonyms to Path and Directory parameters of CSPApplication

Path means directory in repo where are csp files are situated
Directory means -where to deploy csp files on a client machine
Add synonym SourcePath to Path
and synonym DeployPath to Directory

Add zpm extension features for OS-level scripting and error reporting

https://github.com/timleavitt/ObjectScript-Math/blob/master/.travis.yml is a motivating use case:

echo "set good = ##class(%ZPM.PackageManager).Shell(\"load /source $flags\") write ! if good<1 { do \$System.Process.Terminate(\$job,1) } halt" > build
docker exec --interactive $instance iris session $instance -B < build

For ease of OS-level scripting, it would be nice if this could be reduced to:

echo "zpm \"load /source $flags\":1:1" > build
docker exec --interactive $instance iris session $instance -B < build

Where the second positional parameter to ZPM indicates that the process should terminate with an OS-level error upon an error, and the third positional parameter to ZPM indicates that the process should halt after success.

specify minimum IRIS version

Add the parameter to module.xml to specify the minimum IRIS version for this module

add the option to build package from /package/subpackage/class.cls path

Today, if you add the source for module.xml resources to the folder X ZPM, expects resources to be placed in:
X/cls/Package/class.cls
X/int/Package/routine.int

Let's introduce the option to form a package if resources are in:
X//Package/class.cls
X/Package/routine.int
X/Package/routine.mac
X/Package/macro.inc

Introduce IRIS BI or DataStudio Analytics for ZPM

Datastudio is done

Restore verify lificycle phase for zpm itself

command zpm verify should successfully test ZPM itself with integration tests.

zpm verify should be also included in CI

NAMEADD error during module installation

zpm: IRISAPP>install iris-history-monitor

zWrite+13^%Stream.FileBinary.1

Globals are expected in /gbl but not in /src/gbl

Why is that?
If globals couldn't be part of the source - e.g. like configuration data?

Also, zpm doesn't support dot-syntax for mapping global resource to folders - so all the globals are expected to be in /gbl

Add wildcards support for INC, MAC, INT and GBL

Now it is the matter of individual resources

Use IAM in deploy, for production

https://docs.intersystems.com/irislatest/csp/docbook/apimgr/index.html

Get info about github repo package uses

It would be nice if there was a command or if there was information available somewhere that produced a link to the github repo that a specific project was using. This would make it easier to contribute/report bugs encountered

Installing modules fails on IRIS Windows

I'm getting an error if I try installing a module on IRIS for Windows:

zpm: APPSIGHT>install dsw

ERROR #5001: Error creating directory :

I traced this down to a line in ##class(%ZPM.PackageManager.Developer.File).RemoveDirectoryTree() which calls ##class(%File).NormalizeDirectory(##class(%File).TempFilename()). This returns an empty string on IRIS because of a change in how the TempFilename method works.

ZPM installs with error on 2019.1

With 0.1.6 version import it shows:

ERROR #5002: ObjectScript error: zGetInstallerProperties+11^%ZPM.PackageManager.Developer.Utils.1 *%GetData,%SQL.StatementResult

ERROR #5574: Error reported while running generator for parameter '%ZPM.PackageManager.Developer.Processor.CSPApplication:ATTRIBUTES'
> ERROR #5030: An error occurred while compiling class '%ZPM.PackageManager.Developer.Processor.CSPApplication'
Detected 1 errors during load.
2020-02-13 13:54:06 0 %ZPM.Installer: ERROR #5002: ObjectScript error: zGetInstallerProperties+11^%ZPM.PackageManager.Developer.Utils.1 *%GetData,%SQL.StatementResult
ERROR #5574: Error reported while running generator for parameter '%ZPM.PackageManager.Developer.Processor.CSPApplication:ATTRIBUTES'
> ERROR #5030: An error occurred while compiling class '%ZPM.PackageManager.Developer.Processor.CSPApplication'

Add documentation for ZPM

Command reference
Use cases
Add registry documentation

cannot make ZPM package from Python Gateway

Add CI for ZPM client release

If dev version is ready send it to stage brunch where auto testing procedure will take place

test zpm-client with standalone registry (docker?) for all the typical functionality:
search
install module

Fix the ZPM Client Download

Now you see the XML in a browser which is NOT obvious that PRODUCT should be 'saved as'
Convert to Downloadable file. Please.