lightblue-platform/lightblue-metadata-manager

A command line tool to manage Lightblue metadata.

lightblue-metadata-manager's Issues

Log metadata changes

Logging metadata changes will help us manage metadata changes across environments without relying on version control. Alternatively, this could be done in Lightblue (might be a better option).

Push metadata update events to metadataChangesLog entity. An event should include date of the change, environment (hostname extracted from metadata url?), previous metadata and new metadata. Once lightblue-platform/lightblue-core#460 is implemented, we can capture who made the change.

Running integration tests requires LB_CLIENT_DEV environment variable set

It looks like the test is supposed to used embedded lightblue on localhost, but I think it is actually using whatever lightblue the user has configured in LB_CLIENT_DEV configuration.

I started working on a PR to inject a FileSystem to the cli so you could perform this test against an in memory filesystem like jimfs (https://github.com/google/jimfs)

Promotion functionality

Make the tool understand the order of environments: dev->qa->stage->prod. This will allow a single command promotion to next environment.

lbmd prmote --env stage -e user -v newest

will promote newest user version from stage to prod. Before any changes land, it should print a diff and ask: "Are you sure you want to push those changes in prod? [Y/n]".

Show diffs in unified format

https://www.gnu.org/software/diffutils/manual/html_node/Unified-Format.html

Output to stdout instead of file (or allow via option)

Outputting to stdout is more flexible and more standard I think. Easier to work with other tools (like json formatters) and stay agnostic of your current directory.

An option makes sense, though I would default to stdout personally.

Push only entityInfo or only schema

# Push both entityInfo and schema
lbmd push --env dev -e user
# Push only schema (new feature)
lbmd push-schema --env dev -e user
# Push only entityInfo (new feature)
lbmd push-entityInfo --env dev -e user

Push in bulk

Right now you can push entity one at a time:

lbmd push --env dev -e user

Push in bulk (new feature):

lbmd push --env dev -e "/^user.*/"

will push or locally downloaded entities matching the pattern.

Diff does not sort json arrays

Resulting in lots of false positives. Let's sort them to make the diff more readable.

Verify promotion

After promotion, could use diff to check that what was promoted actually landed. This is a manually available step right now, but promotion should fail automatically if diff shows something wasn't promoted.

Diff remote metadata

Currently, the diff functionality allows comparing local metadata against remote metadata. This RFE is to be able to compare metadata in different Lightblue environments.

lbmd diff --env stage -e user -v newest --env2 prod -v2 newest

If -v2 is not given, assume the same as -v.

Updating versions (including references)

When you bump entity version, it's easy to miss references in other entities. The tool could do that for you.

Example:

# download all entities
lbmd pull --env dev -e "/.*/"
# set user entity version to 0.0.9 and update all references to user in all downloaded entities
lbmd set-version -e user -v 0.0.9 --includeReferences 

Add reasonable defaults

For pull, -e should default to $all.
Version -v should always default to newest.

Integration tests

If this tool is to be used for actual releases, it needs some tests. Add lightblue-client-integration-test to the project and create some integration tests, preferably at the MetadataManagerApp level (including argument parsing).

Does push work with regex?

$ lbmd push --env dev2 -e '/redacted(redacted|redacted|redacted|redacted)/' --entityInfoOnly
Exception in thread "main" java.io.FileNotFoundException: /redacted(redacted|redacted|redacted|redacted)/.json (No such file or directory)

Add disable metadata version functionality

But first: lightblue-platform/lightblue-client#333

Changesets

Describe metadata changes for entire release. Apply multi-entity patch with a single command.