Giter Site home page Giter Site logo

qlik-oss / corectl Goto Github PK

View Code? Open in Web Editor NEW
29.0 11.0 11.0 10.91 MB

corectl is a command line tool to perform reloads, fetch metadata and evaluate expressions in Qlik Core apps.

License: Other

Shell 0.97% Go 98.49% Dockerfile 0.10% Makefile 0.44%
blackduck dockerbump

corectl's Introduction

CircleCI Go Report Card Latest Version

corectl

Corectl has been archived in favour of the new tool Qlik-CLI!

Qlik-CLI contains all functionality of corectl while also adding a plentora of new features related to the Qlik SaaS platform.

Old releases of corectl will be kept, but no further development will be done in this tool.


Corectl is a command line tool to perform reloads, fetch metadata and evaluate expressions in Qlik Core apps.


Install

macOS

If you want to install corectl using brew you have to first tap our homebrew repo with the following:

brew tap qlik-oss/taps

after which you can install corectl by running:

brew install qlik-corectl

Note that the binary is still installed as corectl.

If you prefer, you can install using curl instead. Simply run:

 curl --silent --location "https://github.com/qlik-oss/corectl/releases/latest/download/corectl-Darwin-x86_64.tar.gz" | tar xz -C /tmp && mv /tmp/corectl /usr/local/bin/corectl

Linux

If you want to install corectl using snap you can run:

sudo snap install qlik-corectl

This installs qlik-corectl with the alias corectl, so you can still use it as normal (without the qlik- prefix).

If you prefer, you can install using curl instead. Simply run:

 curl --silent --location "https://github.com/qlik-oss/corectl/releases/latest/download/corectl-Linux-x86_64.tar.gz" | tar xz -C /tmp && mv /tmp/corectl /usr/local/bin/corectl

Windows

Using the git-bash shell you can install corectl with curl by running:

curl --silent --location "https://github.com/qlik-oss/corectl/releases/latest/download/corectl-windows-x86_64.zip" > corectl.zip && unzip ./corectl.zip -d "$HOME/bin/" && rm ./corectl.zip

You can also download the binary manually from releases.

Examples

This sections describes some commands and configuration that can be used with the corectl tool.

To simplify usage of corectl, basic configurations such as: engine connection details, app and objects, can be described in a configuration file. We have added an example configuration file to this repo here.

corectl will automatically check for a corectl.yml | corectl.yaml file in your current directory, removing the need to pass the config file using flags for each command.

Example configuration:

engine: localhost:9076 # URL and port to running Qlik Associative Engine instance
app: corectl-example.qvf # App name that the tool should open a session against.
script: ./script.qvs # Path to a script that should be set in the app
connections: # Connections that should be created in the app
  testdata: # Name of the connection
    connectionstring: /data # Connectionstring (qConnectionString) of the connection. For a folder connector this is an absolute or relative path inside of the engine docker container.
    type: folder # Type of connection
objects:
  - ./object-*.json # Path to objects that should be created from a json file. Accepts wildcards.

For more information regarding which additional options that are configurable are further described here.

Also check out the blog post about utilizing corectl and catwalk to build your data model here.

Usage

Usage documentation can be found here.

bash & zsh

corectl provides auto completion of commands and flags for bash and zsh. To load completion in your shell add the following to your ~/.bashrc or ~/.zshrc file depending on shell.

if [ $(which corectl) ]; then
  . <(corectl completion <shell>)
fi

(Substitute <shell> with bash or zsh.) Auto completion requires bash-completion to be installed.

If you want add an alias for corectl, you can add the following snippet into your rc file aswell

alias <myalias>=corectl
complete -o default -F __start_corectl <myalias>

where <myalias> should be substituted for whatever you wish to call corectl.

PowerShell

corectl has some basic auto completion for PowerShell. It can be used by doing the following.

First generate the PowerShell script by running:

corectl completion ps > <file-path.ps1>

followed by adding the following to your PowerShell profile

. <file-path.ps1>

Development

Prerequisite

  • golang >= 1.13

Build

Fast and easy - corectl will be built into the $GOPATH/bin and executable directly from bash using corectl

make install

If you want to keep the previous installed version you can use make build and get the binary to the current working directory

make build

You can also build/install using go as previously:

go build
go install

but this will not include additional build information into corectl (version, branch and commit).

Test

The unit tests are run with the go test command:

go test ./...

The integration tests depend on external components. Before they can run, you must accept the Qlik Core EULA by setting the ACCEPT_EULA environment variable, you start the services by using the docker-compose.yml file. The tests are run with the test script:

ACCEPT_EULA=<yes/no> docker-compose up -d
go test corectl_integration_test.go

The tests are by default trying to connect to the engines running on localhost as defined in the docker-compose.yml file. By setting a series of command line parameters you can run the engines on different hosts and ports:

go test corectl_integration_test.go --engineStd HOST:PORT --engineJwt HOST:PORT --engineAbac HOST:PORT --engineBadLicenseServer HOST:PORT

If the reference output files need to be updated, run the test with --update flag.

go test corectl_integration_test.go --update

The full test-suite can also easily be run with make.

ACCEPT_EULA=<yes/no> make test

For more granular control however the separate commands, mentioned above, should still be used.

Release

You create a release by pushing a commit and a tag with semantic versioning. CircleCi will then run a release build that uses goreleaser to release corectl with the version set as the git tag.

To create a release for e.g. version v1.0.0:

RELEASE_VERSION=v1.0.0 ./release.sh

The script will update the API specification with the new version, create a commit with a tag and push to origin.

Documentation

The usage documentation is generated using cobra/doc. To regenerate the documentation and the API specification run:

make docs

Contributing

We welcome and encourage contributions! Please read Open Source at Qlik R&D for more info on how to get involved.

corectl's People

Contributors

alexanderpalsson avatar axelssonhakan avatar carlioth avatar fredrikfolkesson avatar gabbaxx avatar glooms avatar muiatqlik avatar qlikossbuild avatar renovate[bot] avatar sublibra avatar wennmo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

corectl's Issues

Formatting issue in meta command

Note the ---------- Tables ---------- output:

[...]
Expression1   445/1984    3.5k    $numeric, $integer         445/1984                                 1(58), 0(49), 2(45), 4(34), 5(31)
Expression2   10/1984     80      $numeric, $integer         10/1984                                  0(844), 1(571), 2(229), 3(155), 4(86)
Expression3   1951/1984   15.2k   $numeric                   1951/1984                                0.20980(3), 0.21729(2), 0.54495(2)

Total RAM                 35.1k                              12.2k          508          46---------- Tables ----------
Name           Row count   RAM     Fields
Characters     26          46      Num, Alpha
ASCII          191         508     Num, AsciiAlpha, AsciiNum
Transactions   1984        12.2k   Num, TransLineID, TransID, Dim1, Dim2, Dim3, Expression1, Expression2, Expression3

Total RAM                  35.1k

---------- Associations ----------
Field(s)   Linked tables
Num        Characters <--> ASCII <--> Transactions

Consider coloring output

Today we're using regular terminal colors ("black/white"), we should consider using colors to help the user get a quick overview without having to look closely at all the information outputted.

Change the update command to support separate update tasks

Refactor the update command to support separate tasks:

update - Update script, connection, generic objects if defined in the config or in parameters
update script - only update the script
update objects - only update the generic objects
update connection - only update connections

Add example configuration in repo

Create a standard project with boiler plate code.

  • example csv code
  • example app
  • example connector
  • connection to engine with correct flags

eval fails without a preceeding = sign

Fails:

corectl eval "rand()" -a "Executive Dashboard.qvf" -v
No config file
---------- Connecting to app ----------
Engine: ws://localhost:9076/app/engineData/ttl/30
SessionId: Q29yZWN0bC1ha2wtU0VTVEgtQUtMLU1BQy5sb2NhbC1FeGVjdXRpdmUgRGFzaGJvYXJkLnF2Zg==
App: Executive Dashboard.qvf(reconnected)
Meta: http://localhost:9076/v1/apps/Executive+Dashboard.qvf/data/metadata
Failed to evaluate expression with error code: 7000

Works:

corectl eval "=rand()" -a "Executive Dashboard.qvf" -v
No config file
---------- Connecting to app ----------
Engine: ws://localhost:9076/app/engineData/ttl/30
SessionId: Q29yZWN0bC1ha2wtU0VTVEgtQUtMLU1BQy5sb2NhbC1FeGVjdXRpdmUgRGFzaGJvYXJkLnF2Zg==
App:  Executive Dashboard.qvf(opened)
Meta: http://localhost:9076/v1/apps/Executive+Dashboard.qvf/data/metadata
=rand()
0.3970550140366

Passing a connection via parameters

I would like to be able to create a basic connection via the command line parameter instead of pointing to a file having that information.
Today:
corectl -a app.qvf reload --script ./myscript.qvs --connections ./connectionsyml
Change example:
corectl -a app.qvf reload --script ./myscript.qvs --connect myconnection:/data

Perhaps a key value passed or similar

Add .rc file for most common parameters

Add possibility to set common parameters such as (--no-save) in a configuration file in the users home directory

DoD

  • Decide upon supported parameters
  • Decide file location, name, design
  • Document and create example config

No error message when engine has expired

Using corectl against an expired engine throws no valuable error message.

E.g. trying to build an application returns app not found. I would expect it to tell me that the engine has expired.

Related to #90

Apps command doesn't work in 0.3

Res:

Error: unknown command "apps" for "qli"
Run 'qli --help' for usage.
unknown command "apps" for "qli"

Also doesn't show up in help

Reload without specified app reloads something (default app?)

If the reload command is run without any app defined (in any connection) the tool still gives reload success:

Reload finished successfully
Saving...Done

Doing the same with meta results in:
Error when fetching system table: websocket: close 1006 (abnormal closure): unexpected EOF

{"timestamp":"2018-09-28T08:23:04.749Z","log_type":"Session","logseverity":"INFO","thread_id":4085245696,"user_id":"Personal\Me ","tenant_id":"","version":"12.225.50600.0409.10","app_id":"/home/nobody/Qlik/Sense/Apps/unnamed-app.qvf","title":"","doc_modifi ed":"20180928T082107.000Z","exit_reason":"Session expired","session_start":"20180928T082225.000Z","session_duration":0.000451,"c pu_spent":0.001265,"bytes_received":288,"bytes_sent":638,"calls":3,"selections":0,"secure_protocol":"Off"}

Catch no license message on ws

When engine is configuered to run with a license service and the license service isn't working / no licenses left, engine will return on the websocket a message explaining this.
Catch this error and display in the console. Today we will get:
websocket: close 1006 (abnormal closure): unexpected EOF

Refactor the integration test

Currently, the integration tests are not atomic.

One test could test the possibility to create an app and the next could test that the created app has some objects inside it.

We should look into refactoring the tests so they could be run independently of each other.

Ability to adjust output verbosity

I would like to be able to set the verbosity of the output when reloading. Today when I'm reloading a smaller app from api insights, my command window will drown with more than 5000 lines.

Error in help output

  • In main help output: (corectl -h) qli mentioned instead of corectl

  • In wrong command
    Error: unknown command "***" for "qli"
    Run 'qli --help' for usage.

  • No info about how to list apps

Dependency to local running engine

At startup there is a dependency of having an engine running on 127.0.0.1:9076

Handle this in a better way that just displaying:
Could not connect to engine:localhost dial tcp 127.0.0.1:9076: connect: connection refused

Add support for generic objects

Add support for listing, getting object content and creating generic objects through the tool.

Sample design

Listing objects

corectl objects
output
ID Type AF0D barchart 4AD4 listbox

Getting object content

corectl object <id>
output
{ <properties of object> }

Creating and updating objects

By setting the --objects flag objects in the app can be updated

--objects
The --objects flag is used to specify a list of objects to be set in the app. It can either be used as a parameter to the reload command, or the special update-objects command that does just that.
corectl --objects=/path/to/genericobject1.json:/path/to/genericobject2.json reload
corectl --objects=/path/to/genericobject1.json:/path/to/genericobject2.json update-objects

Setup publish new release flow

Analyse different ways of creating a new release on github and decide on one for this project. Document how we do releases.

Add data paging in field view

Add possibility to navigate in the field output.

E.g.

  • scrolling
  • page up / page down
  • end / home
  • search

Investigate what libraries exists and what they support.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.