Giter Site home page Giter Site logo

the-doctor's Introduction

The Doctor | Cloud Elements Asset Management Tool


Version Downloads

Overview

The Doctor is a CLI tool to allow easy control management with Cloud Elements. This provides the ability to move between environments and to backup files locally or to GitHub.

Installation

$ npm install -g ce-util

PROTIP: node version must be >= v6.3.0

Setup

$ doctor init

The init function will create a hidden directory .doctor in your home directory. This folder will have a config.json that will store all your accounts.

An account consists of a name, orgSecret, userSecret and baseUrl. Make sure the baseUrl is formatted like: https://staging.cloud-elements.com, https://console.cloud-elements.com, or https://console.cloud-elements.co.uk

Usage

The doctor can download/upload assents from your Cloud Elements accounts to a directory or to a file.

GitHub

For use with GitHub, it is recommended to use the --directory option. This will export assets to an intuitive directory structure, and extract any JavaScript to its own file.

Versioning

Versioning in the doctor can be used to manage VDRs or Formulas in github using the -v flag. It allows deploying an updated versioned object to an environment without affecting the current deployed version. For upload the version is applied by appending "_{version}" to the name of the object being uploaded. For downloads the version will split on a "_" delimiter. Example download result "myFormulaName_v2" saves as "myFormulaName". Versioning currently only supports downloading of single objects using the -n

Continuous Deployment

For use with continuous deployment or just routine backups its recommended to just use the --file option. This will save space complexity as an entire CE environment can be represented as one file.

Commands

download    save objects locally
upload      imports objects into your account
delete      delete objects from your account
graph       graph a formula flowchart
accounts    manage accounts to use
init        initialize account
help [cmd]  display help for [cmd]

Objects

all
elements
formulas
vdrs
formulasInstances (delete only)
instances (delete only)

Options

-d, --directory for downloading/uploading to/from and directory
-f, --file for downloading/uplaoding to/from and file
-n, --name for downloading/uploading a specific entity or multiple comma-separated entities to/from and directory/file
-v, --version for downloading/uploading versioned objects

Examples

# list all accounts you have available to interact with
$ doctor accounts list

# removes an account from the list of accounts by the given account nickname or -n
$ doctor accounts remove -n devStaging

# add an account in one line
$ doctor accounts add -n devStaging -u <user secret> -o <organization secret> -b https://staging.cloud-elements.com

# import vdrs from a specified file path to an account (via nickname from your account list) Note: you can replace vdrs with formulas, elements, or all)
$ doctor upload vdrs staging -f ~/Desktop/vdrs-staging.json

# export vdrs from a specified account (note the account should be from your accounts list and you just need to denote the account name) to the given file path. Again: you can replace vdrs with formulas, elements, or all)
$ doctor download vdrs staging -f ~/Desktop/vdrs-staging.json

# export an vdr from the list of specified account vdrs to the given folder path. For elements, you need to pass element key using -n. Again: you can replace vdrs with formulas, elements
$ doctor download vdrs staging -n myVdr -D ~/Desktop/vdrs/

# export vdrs (comma-separated) from the list of specified account vdrs to the given folder path. For elements, you need to pass element key using -n. Again: you can replace vdrs with formulas, elements
$ doctor download vdrs staging -n vdr1,vdr2,vdr3 -D ~/Desktop/vdrs/

# export vdrs from specified account vdrs to the given folder path. (Note: use -u or --useNew boolean flag to download vdrs in new folder structure for more readability)
$ doctor download vdrs staging -D ~/Desktop/vdrs/ --useNew

# import an vdr from the list of specified account vdrs. For elements, you need to pass element key using -n. Again: you can replace vdrs with formulas, elements
$ doctor upload vdrs staging -n myVdr -D ~/Desktop/vdrs/

# import vdrs (comma-separated) from the list of specified account vdrs. For elements, you need to pass element key using -n. Again: you can replace vdrs with formulas, elements
$ doctor upload vdrs staging -n vdr1,vdr2,vdr3 -D ~/Desktop/vdrs/

# doctor delete has similar functionality but will not allow you to delete all. Please see doctor delete --help for more
$ doctor delete formulas accountName 

# export formulas separating out JS into files for easier version control
$ doctor download formulas staging -d ~/Desktop/formulas

# import or export specific entities by their name using -n, --name (You can pass list of comma-separated names)
$ doctor upload formulas staging -d ~/formulas -n specific\ formula\ name

# export a versioned object (example formula named myFormula_2)
$ doctor download formulas staging -f ~/formula.json -n myFormula -v 2

# import a versioned object (example formula named myFormula, version 3)
$ doctor upload formulas staging -f ~/formula.json -v 3

# generate flow chart for the specified formula
$ doctor graph formulas -f ~/Desktop/doctor/formula.json

Limitations

The doctor can not export instance or account level entities i.e. instances, instance or account level object definitions/transformations, accounts, users and formula instances.

Versioning is not supported for elements

Setup Local Development Environment

$ npm uninstall ce-util -g 
$ git clone https://github.com/CloudElementsOpenLabs/the-doctor.git
$ cd the-doctor
$ npm install -g
$ npm link 

the-doctor's People

Contributors

mohdrafiuddin avatar jonnathancharpentiergl avatar mukundbayyaram avatar jayme-t avatar rrothenb avatar pradheep-arul avatar mmairs9 avatar mjhirsch avatar kellytgold avatar g33g avatar harikrishnav avatar nick-bair avatar rahul5595 avatar saisravan84 avatar venkat202 avatar

Stargazers

Christian Nicholls avatar Juraj Veverka avatar Chandu Manda avatar Sravan Ananthula avatar Enrico Simonetti avatar Afrah avatar Steven Petsinger avatar Jacob Jackson avatar Luke Vance avatar Vijay Rajasekaran avatar

Watchers

Robbie Burda avatar James Cloos avatar Ramana Lashkar avatar  avatar Matt Trice avatar Greg Lindahl avatar Ross Caubarreaux avatar Brian Rothhaar avatar Vijay Rajasekaran avatar Ben Pepper avatar Raphael Barbo avatar  avatar  avatar Ricky Amparo avatar Raj avatar

the-doctor's Issues

API is not supported for v2 VDRs

I getting below error while downloading the VDRs:

Failed to complete operation: 400 - {"requestId":"5eb4f575e4b0bc475fc02b34","message":"This API is not supported for v2 VDRs"}

Please look into the issue.

Windows BAT file

I tried putting multiple "doctor" commands in a bat file. It will run the first one successfully, but then the bat file exits without an error or reason why it is not running the rest of the commands.

doctor download formulas staging -n "Formula name 1" -f formula1.json
doctor download formulas staging -n "Formula name 2" -f formula2.json
doctor download formulas staging -n "Formula name 3" -f formula3.json

Support `doctor upload` for subformulas

doctor does not seem to support upload of formulas that call subformulas. It would be great to extend the tool to support this, so we can fully leverage the power of subformulas.

Issue - Doctor install / uninstall, Add accounts etc

$$:CE_Staging rajan$ sudo npm install -g
npm ERR! code ENOLOCAL
npm ERR! Could not install from "" as it does not contain a package.json file.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/<<>>/.npm/_logs/2020-09-22T00_39_31_008Z-debug.log
$$ npm uninstall ce-util -g
npm WARN deprecated [email protected]: this package has been deprecated
npm WARN checkPermissions Missing write access to /usr/local/lib/node_modules/ce-util
npm WARN checkPermissions Missing write access to /usr/local/lib/node_modules
npm ERR! code ENOENT
npm ERR! syscall access
npm ERR! path /usr/local/lib/node_modules/ce-util
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, access '/usr/local/lib/node_modules/ce-util'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/<<>>/.npm/_logs/2020-09-22T00_39_51_740Z-debug.log

doctor init on Windows as Service Account

Issue

Executing doctor commands under the context of a Windows built-in service account results in exceptions. I'm trying ideally to upload via an on-prem Azure DevOps agent via the doctor cli.

OS: Microsoft Windows Server 2019 Datacenter

Error Message:

PS C:\windows\ServiceProfiles\NetworkService> doctor init
? Nickname of account: staging
? User secret: <user_secret>
? Organization secret: <org_secret>
? Base url (format like https://staging.cloud-elements.com): https://staging.cloud-elements.com
node:fs:585
  handleErrorFromBinding(ctx);
  ^

Error: ENOENT: no such file or directory, open 'undefined\.doctor\config.json'
    at Object.openSync (node:fs:585:3)
    at Object.readFileSync (node:fs:453:35)
    at C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\ce-util\src\util\readFile.js:6:65
    at new Promise (<anonymous>)
    at module.exports (C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\ce-util\src\util\readFile.js:6:10)
    at module.exports (C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\ce-util\src\core\addAccount.js:13:28)
    at addAccount (C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\ce-util\src\cli\doctor-init.js:34:3)
    at saveCreds (C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\ce-util\src\cli\doctor-init.js:52:5)
    at C:\Windows\ServiceProfiles\NetworkService\AppData\Roaming\npm\node_modules\ce-util\src\cli\doctor-init.js:62:44
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  errno: -4058,
  syscall: 'open',
  code: 'ENOENT',
  path: 'undefined\\.doctor\\config.json'
}

Node.js v17.6.0

I was expecting the .doctor folder and config json file to be created in the home directory for the service account: C:\Windows\ServiceProfiles\NetworkService\.doctor

In reality the tool throws an exception which indicates that it cannot seem to find the $HOME account for the service account: no such file or directory, open 'undefined\.doctor\config.json'

This is likely to do with the use of Windows specific environment variables in doctor-init.js. Could probably be replaced with os.homedir() for a true cross-platform solution.

Solution

Replace instances of:

const homeDir = (process.platform === 'win32') ? process.env.HOMEPATH : process.env.HOME

with

const homeDir = require('os').homedir()

in:

/src/cli/doctor-init.js
/src/core/addAccount.js
/src/core/listAccounts.js
/src/core/removeAccount.js
/src/core/loadAccount.js

Tested on Windows Server 2019 Datacentre, Windows Server 2016, Ubuntu 20.04, Ubuntu 18.04, CentOS 7.

Add default account behavior

Just a suggestion, but passing the account nickname to all commands seems unintuitive and a bit cumbersome, I would like to suggest adding a default account behavior as well as an optional flag for using non-default accounts. i.e.

$ doctor upload formulas --account myAccountName -d ~/formulas

HOME missing in Windows environment

This tool doesn't work in windows because you reference HOME in the init script.

  const folderPath = `${process.env.HOME}/.doctor`
  const filePath = `${process.env.HOME}/.doctor/config.json`;

If you use HOMEPATH if HOME is undefined it will work.

Print a message/throw an error when the "name" flag does not return any artifacts

Currently when using the --name flag to specify a formula or VDR artifact by name, if nothing is found with that name in the specified CE account, an empty file or director will still be created.

Preferred functionality would be user feedback to allow the user to correct the name used in order to accurately retrieve the asset they were looking for.

i.e.

$ doctor download formulas staging -d ./myFormula -n badAtSepllinng
>> error: no formula with name "badAtSeplling" found in account "staging"

$ doctor download formulas staging -d ./myFormula -n iCanDoSpelling
>> formula "iCanDoSpelling" successfully downloaded in directory ./myFormula


add default alias for "doctor"

As we have more external people using this tool, it seems like we should have a more formal usage term for it. I propose that the npm package name is also used as the command with the optional alias as doctor.

so the following two commands would be equivalent:

$ doctor --version

$ ceutil --version

Alias for import and export

import and export require a frame of reference for example: am I importing from staging to my desktop or am I exporting from staging to my desktop
We should creates alias for these that are upload and download because those are more standard terminology and I know that I am downloading from staging to my computer and uploading from my computer to prod or whatever the case may be

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.