javanile/yamlinc

Compose multiple YAML files into one with $include tag. Split Swagger/OpenAPI into multiple YAML files.

⭐ Give me a star, please! ⭐

---

Yamlinc

Create a composed YAML file using $include tag.

Simple usage

STEP #1 - Install global yamlinc command-line utility

$ npm install -g yamlinc

STEP #2 - Create "my_swagger_doc.yml" and split it into multiple file

## file: my_swagger_doc.yml
version: '2.0'
$include: ./tags.yml
$include: ./paths.yml

## file: tags.yml
tags:
- FirstTag
- SecondTag

## file: paths.yml
paths:
  /api/me:
    get: ...      
  /api/you:
    post: ...
  $include: others-paths.yml

## file: others-paths.yml
/api/other/one:
  get: ...      
/api/other/two:
  post: ...

STEP #3 - Simply compile the entry point 'my_swagger_doc.yml'

$ yamlinc my_swagger_doc.yml

STEP #4 - Get your compiled file 'my_swagger_doc.inc.yml'

NOTICE: Yamlinc appends '*.inc.yml' extension to compiled file.

Development watcher

During development you need constantily updated compiled file by watching changes of dependencies

$ yamlinc --watch spectacle -d my_swagger_doc.yml

This example generates documentation with spectacle

Feed your .inc.yml file

If your application needs a compiled file as parameter you can simply compound and feed

$ yamlinc --exec docker-compose -f docker-compose.yml

Redirect output to another command

If your application needs send output to another command or chaining using pipe follow this example

$ yamlinc --output - input.yml | nc seashells.io 1337

Parse files to find syntax errors

If your application needs stop after a syntax error or missing file inclusion use strict mode

$ yamlinc --strict settings.yml

Use an external schema

If you have your own or a third party schema you can pass it to yamlinc like this

$ yamlinc --schema ../node_modules/cloudformation-schema-js-yaml

Create your scenario

If you have custom scenario with YAML file please place issues on the following page

https://github.com/javanile/yamlinc/issues/new

yamlinc's Issues

It would be good to support .yaml file extension

This is a nice little tool, but I initially had trouble with it because all of my YAML files use .yaml instead of .yml as the file extension. When I tried to use yamlinc with one of those files, all I got was

Arguments error >> missing file name.

After trying a few other things, I eventually guessed that yamlinc might only recognize .yml file extensions, which turned out to be correct.

If you get a chance, please modify yamlinc to recognize .yaml file extensions as valid.

Can I use this tool to merge two openapi specs?

Hi!
Let's say that two people develop two independent API documents (each has standard openapi schema: openapi, info, paths). Does this tool merge these two files into a single openapi spec?

Reverse Merge

Hi,
when i'm doing the merge of one yml fill to another one with another properties the yml file will always import in the end.
ex:
customer.yml
type: customer
name: John
age: 30

users.yml
$include: ./customer-yml
api:
server_url: xpto.api.com

the output of users.inc.yml will be:

api:
server_url: xpto.api.com
type: customer
name: John
age: 30

so is inverse to the place were i put the include.

How can i include for the order that i want?

Syntax for Pulling in Subcomponents

Currently, $include pulls in entire files. Many YAML tools support the JSON Reference syntax for pulling in pieces of other files as part of the URI fragment. This would extend the syntax to be: ${file}[#${reference}] where reference is a JSON reference.

As an example, if my file a.yml looks like:

foo:
  potato: 6
bar:
  taco: 10

And my referring file looks like:

blub:
  $include: './a.yml#/bar'

The output would look like:

blub:
  taco: 10

yamlinc changes example dates when merging

Here's a piece of a request definition in one of my source files:

      - name: date
        description: The date
        in: path
        required: true
        schema:
          type: string
          format: date
        example: 2019-12-25

And here's the same section after merging with yamlinc:

        - name: date
          description: The date
          in: path
          required: true
          schema:
            type: string
            format: date
          example: '2019-12-25T00:00:00.000Z'

Note how the example date in the last line has been changed from just the date to a full date and time format, which is not what I want in this case. Is there a way to prevent this from happening?

Post body parameters partially not visible

I use spectacle as the docs recommend but when I use inside of my paths an inline block to define (for example the POST) parameters, spectacle shows only the description inside the Request Body section without the post body parameters and parameter definition.

 post:
    summary: '[POST] Create project'
    description: Creates a new project.
    parameters:
      - in: body
        name: name
        description: The project name.
        type: string
    tags:
      - Projects
    responses:
        ....

But when I use refs, everything is working.

 post:
    summary: '[POST] Create project'
    description: Creates a new project.
    parameters:
      - $ref: '#/parameters/projectNameBodyParam'
    tags:
      - Projects
    responses:
        ....

I'm also using swagger-ui-express and everything is working fine here. Maybe it's a problem at spectacle, but maybe its a bug and you can help me.

yamlinc depend on corrupts npm libs 'colors'

Run yamlinc with mess output after using npm install -g yamlinc to install globally

An error is furnished when !Ref tag from SAM is encountered and the file compliation is terminated.

Analize : template.yaml

Problem : Error on file 'template.yaml' unknown tag !<!Ref> at line 25, column 46:
... nment: !Equals [!Ref Environment, dev]
^
Compile : template.inc.yaml

comments are stripped from output file

It would be much nicer if the comments in the included files were also able to appear in the output file.

Our output cloudformation file is both functional and documentation. But now people have to go dig up the sub-files in order to read the comments.

Export as JSON

Add command-line argument to export as JSON instead YAML

es.

yamlinc --json main.yml

Support options for filename out and STDOUT

Currently the yamlinc generates a <file>.inc.yml file in the current directory.
Greater flexibility is needed where the output file can be specified.

For example

yamlinc -o some/path/to/MyType.yml src/types/MyType.yml

or
Writing to STDOUT
yamlinc -o - src/types/MyType.yml

Some includes fail

It doesn't seem to be directly related with yamlinc, but with a dependency of it:
https://www.npmjs.com/package/cuid
As I'm writing this, "Published 8 hours ago".

I use Yamlinc as a dependency for my swagger definitions and some of the includes are not working since today. In every deploy I reinstall the dependencies, so it all points to cuid package working differently than yesterday.

Inline code parser

New feature to develop is Inline code parser.
This enables the possibility to parse YAML string from command-line and use $include tag to compound multiple files without the main file like this

$ yamlinc --inline '{ $import: a.yml, $import: b.yml }'

other examples

$ yamlinc --inline '{ $import: basefile.yml, version: '0.0.1-new' }'

$ yamlinc --inline 'sample:\
  node: "test"\
  $include: rest.yml'

Commented out $include should still work

In some yml usages (Swagger/OpenAPI), it is very strict on what tags you can use. My goal is to have my base file still be a valid Swagger while decorating endpoints with external files. If instead of $include it was #include or even #$include then swagger would see it as a comment while the tool would correctly import what it needs.

IS it possible to make the$include string be customizable?

Support path glob

Hey there,

would be nice if the $include could handle a glob value.
Maybe like .gitignore, where you can set something like:

**/*.yaml
**/foo.yaml

Which would checkout the following example paths:

src/foo/bar/baz.yaml
src/foo/bar.yaml
src/foo.yaml

rename output file.

hi!

1. I would like to be able to set my own name for the final Yaml file. comand yamlinc --output - input.yml didn't work, it also continues to take the name from the source file and add "inc"
   2 ) and also wanted to set my address for the final Yaml file

Include JSON file

A new feature is "Include JSON file"
add the possibility to use $include with JSON files

Question about watching

Hi,

I'm wondering how the watch works.

I'm trying :
yamlinc --watch doc/src/index.yaml

But I get a spawn error:

{PROJECT_PATH}/node_modules/yamlinc/src/helpers.js:69
        wrapper.stdout.on('data', (data) => {
                       ^

TypeError: Cannot read property 'on' of undefined
    at Object.spawn ({PROJECT_PATH}/node_modules/yamlinc/src/helpers.js:69:24)
    at Object.spawnLoop ({PROJECT_PATH}/node_modules/yamlinc/src/yamlinc.js:335:17)
    at Timeout.setTimeout [as _onTimeout] ({PROJECT_PATH}/node_modules/yamlinc/src/yamlinc.js:304:18)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)

Any idea how I can fix that ?

Problems with Cloudformation templates

I'd like to use this in a Cloudformation template. But receive validation errors such as
Problem : Error on file '../template.yaml' unknown tag !<!Ref> at line 150, column 31:
Name: !Ref Name

[BUG] Yamlinc must fail when $include path was't found

Version: v0.0.65
Steps to reproduce:

1. Create file main.yml

some_key: "some_value"
some_data:
    $include: included_file_that_does_not_exists.yml

2. Run yamlinc main.yml && cat main.inc.yml
   Output:

## --------------------
## DON'T EDIT THIS FILE
## --------------------
## Engine: [email protected]
## Source: main.yml

some_key: some_value
some_data: {}

Expected result:
Yamlinc must exit with status 1 and print error message: "Included file included_file_that_does_not_exists.yml was't found"

Program can not find file symlinks

If the target file of an $include is a symbolic link, the program fails to find the file.

Steps

Environment

I made a clean environment of Ubuntu 18.04 with Docker for minimal testing (feel free to jump to "Recreation" if this isn't applicable).

docker --rm -it ubuntu:18.04
apt-get update && apt-get install npm
npm install -g [email protected]
mkdir testbed && cd testbed

Recreation

echo '$include: "./link.yaml"' > root.yaml
echo 'tacos: {}' > target.yaml
ln -s target.yaml link.yaml
yamlinc --strict root.yaml

Expected Behavior

I would expect the output to look something like this:

## --------------------
## warning stuff here...
tacos: {}

Actual Behavior

The program exits with code 1 and gives the following message:

   Analize : root.yaml
 > Problem : file not found './link.yaml' on 'root.yaml' at line 1.

Problem

The issue appears to be in helpers.js:91 with the usage of fs.lstatSync(file).isFile(), specifically, I do not think that lstat should be used. I do not think that a tool like yamlinc means to look at the link stats, but cares about the target file. I believe fs.statSync(file).isFile() is correct here.

excessive quoting

input yaml:

  OrdersCancel:
    Type: AWS::Serverless::Function

output yaml:

OrdersCancel:
  Type: 'AWS::Serverless::Function'

Notice the spurious single quotes added in the above code block
Because of this I get the following error message when trying to use the cloudformation yaml:
Error: Template does not have any APIs connected to Lambda functions

Additionally:
input yaml:

Globals:
  Api:
    # Allows an application running locally on port 8080 to call this API
    Cors:
      AllowMethods: "'*'"
      AllowHeaders: "'*'"
      AllowOrigin: "'*'"

output yaml:

Globals:
  Api:
    Cors:
      AllowMethods: '''*'''
      AllowHeaders: '''*'''
      AllowOrigin: '''*'''

Why are my double quotes being randomly turned into single quotes?

yamlinc fails in unrecognized tag

Analize : template-api.yaml

Problem : Error on file 'template-api.yaml' unknown tag !<!Ref> at line 32, column 27:
- !Ref Environment
^
Compile : template-masterdata-api.inc.yaml

This is aws sam logic and it shouldn't fail.