mermade/openapi-filter

Filter internal paths, operations, parameters, schemas etc from OpenAPI/Swagger/AsyncAPI definitions

openapi-filter's Issues

feat: allow filtering individual fields in response

Feature request to allow filtering individual fields in a response. The use case for this would be cases where the API either secretly or temporarily returns a specific field, which you need to specify in the spec due to API generation but doesn't want to document on the public docs.

I could possibly try make a PR, but wanted to post it as Issue to hear thoughts first.

execution failes from 1.4.0

After update yargs from 10.x to 14.x calling openapi-filter --tags x-public x-hidden -- infile.yaml fails with

Usage: openapi-filter [options] {infile} [{outfile}]

Options:
---snip---

Not enough non-option arguments: got 0, need at least 1

How to extract out all the associated definitions for a particular API?

I want to extract out a single API from source.yaml into target.yaml, but along with its models.

API use fails as of 1.2.2

I was working on upgrading to 2.0.0 from 1.2.1 and I've traced a problem to the upgrade from 1.2.1 to 1.2.2.

With 1.2.1 I can do this:

const apiFilter = require('openapi-filter');

async function filterInternalRoutes(docPromise, filterTags) {
  const openApiDocument = await Promise.resolve(docPromise);
  return apiFilter.filter(openApiDocument, {
    tags: filterTags || ['x-internal', 'x-alpha'],
  });
}

When I do that in 1.2.2, I get this error:

Usage: openapi-filter [options] {infile} [{outfile}]

Options:
  -h, --help     Show help                                             [boolean]
  --version      Show version number                                   [boolean]
  -i, --inverse  output filtered elements only                         [boolean]
  -t, --tags     tags to filter by             [array] [default: ["x-internal"]]

Not enough non-option arguments: got 0, need at least 1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.

Did the way openapi-filter API is invoked change in 1.2.2? Same error with 2.0.0.

ReferenceError: Excessive alias count indicates a resource exhaustion attack

When YAML API specifications that contain many aliases openapi-filter may fail. This is caused by a more strict maxAliasCount in YAML

https://github.com/eemeli/yaml/blob/440aadcd703c76f8920f75bde9d0ea522eb3a08a/src/index.js#L14

Since openapi-filter is a command line tool I think it's safe to turn off this check by setting maxAliasCount=-1 in yaml parse options. I will submit a PR for this later today.

Error: options.overrides is not iterable

When running without any overrides argument, the following error is emitted:

/path/to/openapi-filter/index.js:19
        for (let override of options.overrides) {
                                     ^

TypeError: options.overrides is not iterable
    at /path/to/openapi-filter/index.js:19:38
    at recurse (/path/to/openapi-filter/node_modules/reftools/lib/recurse.js:37:9)
    at Object.filter (/path/to/openapi-filter/index.js:18:5)
    at Object.<anonymous> (/path/to/openapi-filter/openapi-filter.js:44:25)
    at Module._compile (internal/modules/cjs/loader.js:654:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
    at Module.load (internal/modules/cjs/loader.js:566:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:506:12)
    at Function.Module._load (internal/modules/cjs/loader.js:498:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:695:10)

Output when using the inverse option should be valid

I am using the --inverse option to output both external and internal API documentation. When I output the external documentation everything works as expected, however when I run openapi-filter with the --inverse option the output is not a validOpenAPI object. Because of this I cannot run it through a documentation generator to output human-readable documentation.

Per #1 (comment) this is a known issue. I believe there is an easy fix for the 90% use case: the openapi and info fields are required in the OpenAPI object, so these fields should always be included in the output of openapi-filter.

x-internal does not filter the referenced properties from object

User.yaml

type: object
properties:
  name: 
    type: string
  userSecret:
    x-internal: true
    $ref: 'UserSecretInfo.yaml'

The user secret is not removed from target yaml after applying the filter

PR Idea - pass options as file

Right now the options have to be passed as arguments to the CLI.

This works fine for when you have a limited set of options to set, but if you want to filter out a large number of methods/operationIds, it would make more sense to provide a "options" file as a parameter.

something like --optionsFile ./config/openapi-filter-options.json

Which would results in

openapi-filter source.yaml target.yaml --optionsFile ./config/openapi-filter-options.json

Do you have a suggestion how the openapi-filter-options.json should be structured?

Question - Filter by path & methods

I was wondering if it would be possible to filter out specific "path" methods using the tags options.

Example

paths:
  /pet:
    post:
      tags:
        - pet
      summary: Add a new pet to the store
      description: ""
      operationId: addPet
      responses:
        "405":
          description: Invalid input
      security:
        - petstore_auth:
            - write:pets
            - read:pets
      requestBody:
        $ref: "#/components/requestBodies/Pet"
    put:
      tags:
        - pet
      summary: Update an existing pet
      description: ""
      operationId: updatePet
      responses:
        "400":
          description: Invalid ID supplied
        "404":
          description: Pet not found
        "405":
          description: Validation exception
      security:
        - petstore_auth:
            - write:pets
            - read:pets
      requestBody:
        $ref: "#/components/requestBodies/Pet"

I want to filter out the /pet but on the the PUT.

The only option that I have right now, is to add a specific flag (like x-visibility) to the put method .
Or is there an option to specific target a path & method?

if there is not, I'll try to create a PR to support this filtering option.

Inverse and valid options cause nested definitions to be removed

Hi guys,

what I was trying to achieve with openapi-filter was to limit OpenAPI spec to resources flagged with x-internal, still keeping the whole spec valid. I used following command:
npx openapi-filter --inverse --valid --flags x-internal
Unfortunately I found that as long as it works for simple examples it does not support nested definitions/components.
On the screenshot below you can see one of your test samples adjusted to my scenario:

When you run above command on that sample, definitions b1, b2 and c1 won't appear in the output.

It will be really interesting if you can provide a reverse switch as well. Only to keep the elements with x-internal: true.

It will be really interesting if you can provide a reverse switch as well. Only to keep the elements with x-internal: true. Useful for a quick check around what's getting filtered out.

3.2.1 update broke running through npx

Since yesterday cant run the script neither in CI pipelines, nor on a local machine;

Need to install the following packages:
  [email protected]
Ok to proceed? (y) y
node:internal/modules/cjs/loader:557
      throw e;
      ^

Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './types' is not defined by "exports" in /__REDACTED__/.npm/_npx/b725999c9659cba2/node_modules/yaml/package.json
    at new NodeError (node:internal/errors:399:5)
    at exportsNotFound (node:internal/modules/esm/resolve:266:10)
    at packageExportsResolve (node:internal/modules/esm/resolve:602:9)
    at resolveExports (node:internal/modules/cjs/loader:551:36)
    at Module._findPath (node:internal/modules/cjs/loader:620:31)
    at Module._resolveFilename (node:internal/modules/cjs/loader:1039:27)
    at Module._load (node:internal/modules/cjs/loader:898:27)
    at Module.require (node:internal/modules/cjs/loader:1120:19)
    at require (node:internal/modules/helpers:112:18)
    at Object.<anonymous> (/__REDACTED__/.npm/_npx/b725999c9659cba2/node_modules/openapi-filter/openapi-filter.js:7:22) {
  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
}

OpenAPI v3.1 support

I'm guessing this package probably supports OpenAPI v3.1 ok as the changes being made are not too valid to the overall way this works, but I wanted to double check.

I want to update openapi.tools if so.

Filter regular tags in OpenAPI 3.0 yaml?

Hi there !
Thanks for this great little tool.
I am trying to filter tags like Public in this example:

/add/pet:
    post:
      summary: Add
      description: Add a pet
      requestBody:
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/AddPet"
        required: true
      responses:
        "200":
          description: "OK"
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/AddPetResponse"
        "400":
          description: "Error"
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: "#/components/schemas/BadRequestResponse"
      tags:
        - Public

Unfortunately, when I am only getting {} when executing :
openapi-filter --inverse --tags Public -- openapi.yaml

Could you help me please?

Can't run module globally

Installing with npm install -g openapi-filter or running with npx openapi-filter results in command not found: openapi-filter error.

`--` option to separate tags from positional args no longer works

Example:

$ ./node_modules/openapi-filter/openapi-filter.js --valid --tags x-private -- dist/Internal.yaml dist/Public.yaml
openapi-filter.js <infile> [outfile]

Positionals:
  infile   the input file
  outfile  the output file

Options:
  --info           include complete info object with --valid           [boolean]
  --inverse, -i    output filtered elements only                       [boolean]
  --servers        include complete servers object with --valid        [boolean]
  --tags, -t       tags to filter by           [array] [default: ["x-internal"]]
  --overrides, -o  tags to override fields                 [array] [default: []]
  --valid          try to ensure inverse output is valid               [boolean]
  --strip, -s      strip the tags from the finished product            [boolean]
  --lineWidth, -l  max line width of yaml output          [number] [default: -1]
  --maxAliasCount  maximum YAML aliases allowed          [number] [default: 100]
  --help           Show help                                           [boolean]
  --version        Show version number                                 [boolean]

Not enough non-option arguments: got 0, need at least 1

Work around exists:

Move --valid to after the --tags arguements.

./node_modules/openapi-filter/openapi-filter.js --tags x-private --valid dist/Internal.yaml dist/Public.yaml

--strip option not recognized

I tried using the --strip option to remove tags after filtering, but that just caused openapi-filter to show the usage information, and to say that strip is an unknown argument. Same with -s.

Enhancement: Instead of hardcoding the name of vendor extension 'x-internal' will it be possible to optionally accept the list of extensions to filter out as user input?

Default can remain 'x-internal' but also allow users to specify a list of custom vendor extensions (such ['x-private', 'x-hidden', 'x-do-not-show'] etc.) on command line, based on which openapi-filter can filter out the elements.

Inverse filtered documents do not include main level security definitions

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch [email protected] for the project I'm working on.

Inverse filtered paths with following flags produce an non valid api document that does not include the securitySchemes under to components when there are active security definitions under the paths->path->method structure

-i --valid --strip --servers

Here is the diff that solved my problem:

diff --git a/node_modules/openapi-filter/index.js b/node_modules/openapi-filter/index.js
index dd1f690..d342663 100644
--- a/node_modules/openapi-filter/index.js
+++ b/node_modules/openapi-filter/index.js
@@ -139,10 +139,40 @@ function filter(obj,options) {
         if (options.servers && !filtered.servers && Array.isArray(src.servers)) {
             filtered.servers = src.servers;
         }
+        const activeSecuritySchemes= 
+        (src.paths?Object.keys(src.paths):[])
+            .flatMap(pathUrl =>src.paths[pathUrl])
+            .flatMap(pathElement => Object.keys(pathElement)
+            .flatMap(method =>pathElement[method]))
+            .filter(path => Object.keys(path).filter(value => options.flags.includes(value)))
+            .flatMap(path =>{
+                if (!filtered.security && Array.isArray(path.security)) {
+                    return path.security.flatMap(securityItem => Object.keys(securityItem));
+                }else{
+                    return [];
+                }
+            })
+            .filter(filterUnique)
+
+        // OAS2
+        if (src.securityDefinitions) {
+            filtered.securityDefinitions = Object.fromEntries(Object.entries(src.securityDefinitions).filter(([key]) => activeSecuritySchemes.includes(key)));
+        }
+        // OAS3
+        if (src.components && src.components.securitySchemes) {
+            if(!filtered.components){
+                filtered.components ={};
+            }
+            filtered.components.securitySchemes = Object.fromEntries(Object.entries(src.components.securitySchemes).filter(([key]) => activeSecuritySchemes.includes(key)));
+        }
     }
     return (options.inverse ? filtered : src);
 }
 
+function filterUnique(value, index, array) {
+    return array.indexOf(value) === index;
+  }
+
 module.exports = {
     filter : filter
 };

This issue body was partially generated by patch-package.

Documentation: Provide command-line example for usage of tags

Due to the way yargs handles array parameters (yargs/yargs#162), the command-line below fails:

node openapi-filter --tags x-internal x-hidden swagger.yaml filtered.yaml

But this works:
node openapi-filter --tags x-internal x-hidden -- swagger.yaml filtered.yaml

Might want to update the readme with an example, so that other users don't trip over this.

`tags` array and inverse mode; multiple elements filtered

This may be a regression from #26.
We use a top-level tags with multiple filtered elements.
Reproduce:

input

openapi: 3.0.0
info:
  title: API
  version: 1.0.0
tags:
  - name: Tag 1
  - "x-public": true
    name: Tag 2
  - "x-public": true
    name: Tag 3
  - "x-public": true
    name: Tag 4
paths: {}

output

tags:
  - x-public: true
    name: Tag 4

But we expected:

tags:
  - x-public: true
    name: Tag 2
  - x-public: true
    name: Tag 3
  - x-public: true
    name: Tag 4

Thanks in advance.

Single/Double quotes removed from example

I'm trying to use an example for a property, whose type is date-time:
e.g.: example: '20:30'

But it's removing the single/double quotes in the output. The same happens with date format.

input:

...
components:
  schemas:
    Order:
      type: object
      properties:
        id:
          type: integer
          format: int64
        shipDateTime:
          type: string
          format: date-time
          example: '20:30'
        shipDate:
          type: string
          format: date
          example: '2020-05-05'
...

output:

...
components:
  schemas:
    Order:
      type: object
      properties:
        id:
          type: integer
          format: int64
        shipDateTime:
          type: string
          format: date-time
          example: 20:30
        shipDate:
          type: string
          format: date
          example: 2020-05-05
...

expected:

...
components:
  schemas:
    Order:
      type: object
      properties:
        id:
          type: integer
          format: int64
        shipDateTime:
          type: string
          format: date-time
          example: '20:30'
        shipDate:
          type: string
          format: date
          example: '2020-05-05'
...

Thanks in advance.

Filter ignores some $refs

Hello!
I wanted to use the openapi-filter to remove some internal headers from our public openapi specifications.
Unfortunately, it doesn't seem to fully work.

An example:

openapi: 3.0.2
info:
  title: Swagger Petstore - OpenAPI 3.0
  description: omit
  termsOfService: 'http://swagger.io/terms/'
  contact:
    email: [email protected]
  license:
    name: Apache 2.0
    url: 'http://www.apache.org/licenses/LICENSE-2.0.html'
  version: 1.0.5
servers:
  - url: /api/v3
paths:
  /pet:
    post:
      tags:
        - pet
      summary: Add a new pet to the store
      description: Add a new pet to the store
      operationId: addPet
      parameters:
        - $ref: '#/components/parameters/InternalHeader'
        - $ref: '#/components/parameters/OtherInternalHeader'
        - in: header
          name: X-ThirdInternalHeader
          schema:
            type: string
          required: true
          x-internal: true
      requestBody:
        description: Create a new pet in the store
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Pet'
        required: true
      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Pet'
        '405':
          description: Invalid input

components:
  parameters:
    InternalHeader:
      in: header
      name: X-InternalHeader
      schema:
        type: string
      required: true
      x-internal: true
    OtherInternalHeader:
      in: header
      name: X-OtherInternalHeader
      schema:
        type: string
      required: true
      x-internal: true
  schemas:
    Pet:
      required:
        - name
        - photoUrls
      type: object
      properties:
        id:
          type: integer
          format: int64
          example: 10
      xml:
        name: pet

You can see there are three headers (InternalHeader, OtherInternalHeader & ThirdInternalHeader).
After running the openapi-filter the result looks like this:

... omitted because irrelevant
paths:
  /pet:
    post:
      tags:
        - pet
      summary: Add a new pet to the store
      description: Add a new pet to the store
      operationId: addPet
      parameters:
        - $ref: "#/components/parameters/OtherInternalHeader"

...
components:
  parameters: {}
... omitted because irrelevant

In the end all the parameter definitions are removed but the reference to ONE of the two headers is still there.

CVE-2020-7608

There is a vulnerability CVE-2020-7608 on yargs-parser which is a dependency of yargs. Is there a chance the yargs version can be bumped up to pick up the fix?

Inverse filtering of tags not working correctly

first of all thanks for the great tool.

When using --inverse on a file containing tags the results are invalid.

Example 1:

openapi: 3.0.0
tags:
- name: pet
  x-internal: true
  description: Everything about your Pets
- name: store
  description: Access to Petstore orders
- name: user
  description: Operations about user
  x-internal: true

Result of openapi-filter -i

tags:
  - name: pet
    x-internal: true
    description: Everything about your Pets
  - null
  - name: user
    description: Operations about user
    x-internal: true

Expected:

tags:
  - name: pet
    x-internal: true
    description: Everything about your Pets
  - name: user
    description: Operations about user
    x-internal: true

Example 2:

openapi: 3.0.0
tags:
- name: pet
  description: Everything about your Pets
- name: store
  x-internal: true
  description: Access to Petstore orders
- name: user
  description: Operations about user
  x-internal: true

Result of openapi-filter -i

tags:
  "1":
    name: store
    x-internal: true
    description: Access to Petstore orders
  "2":
    name: user
    description: Operations about user
    x-internal: true

Remove a specific parameter and examples

Hi, I have a lot of examples and x-examples in my file that I would I to remove. Also, there is a parameter user-id that needs to be removed. Please provide a CLI command example for the same.

Can openapi-filter be used to remove specifications by tag?

(not really and issue but a question)

Trying to consume (import into Azure API Management) a subset of the GitHub API. Because this specification is huge, it cannot be imported. I only need those operations related to the 'issues' tags. Can this tool be used to process the specification file and produce a smaller specification with only operations and schemas related to this tag? (see - name: issues below).

Complete specification: https://github.com/github/rest-api-description/blob/main/descriptions/ghes-3.8/ghes-3.8.json

openapi: 3.0.3
info:
  version: 1.1.4
  title: GitHub v3 REST API
  description: GitHub's v3 REST API.
  license:
    name: MIT
    url: https://spdx.org/licenses/MIT
  termsOfService: https://docs.github.com/articles/github-terms-of-service
  contact:
    name: Support
    url: https://support.github.com/contact?tags=dotcom-rest-api
  x-github-plan: ghes
  x-github-release: 3.9
tags:
  - name: actions
    description: Endpoints to manage GitHub Actions using the REST API.
  - name: activity
    description: Activity APIs provide access to notifications, subscriptions, and timelines.
  - name: apps
    description: Information for integrations and installations.
  - name: billing
    description: Monitor charges and usage from Actions and Packages.
  - name: checks
    description: Rich interactions with checks run by your integrations.
  - name: code-scanning
    description: Retrieve code scanning alerts from a repository.
  - name: codes-of-conduct
    description: Insight into codes of conduct for your communities.
  - name: emojis
    description: List emojis available to use on GitHub.
  - name: dependabot
    description: Endpoints to manage Dependabot.
  - name: dependency-graph
    description: Endpoints to access Dependency Graph features.
  - name: gists
    description: View, modify your gists.
  - name: git
    description: Raw Git functionality.
  - name: gitignore
    description: View gitignore templates
  - name: issues
    description: Interact with GitHub Issues.
  - name: licenses
    description: View various OSS licenses.
  - name: markdown
    description: Render GitHub flavored markdown
  - name: merge-queue
    description: Interact with GitHub Merge Queues.
  - name: meta
    description: Endpoints that give information about the API.
  - name: migrations
    description: Move projects to or from GitHub.
  - name: oidc
    description: Endpoints to manage GitHub OIDC configuration using the REST API.
  - name: orgs
    description: Interact with GitHub Orgs.
  - name: packages
    description: Manage packages for authenticated users and organizations.
  - name: projects
    description: Interact with GitHub Projects.
  - name: pulls
    description: Interact with GitHub Pull Requests.
  - name: rate-limit
    description: Check your current rate limit status
  - name: reactions
    description: Interact with reactions to various GitHub entities.
  - name: repos
    description: Interact with GitHub Repos.
  - name: search
    description: Look for stuff on GitHub.
  - name: secret-scanning
    description: Retrieve secret scanning alerts from a repository.
  - name: teams
    description: Interact with GitHub Teams.
  - name: users
    description: Interact with and view information about users and also current user.
  - name: scim
    description: Provisioning of GitHub organization membership for SCIM-enabled providers.
  - name: enterprise-admin
    description: Enterprise Administration
  - name: oauth-authorizations
    description: OAuth Authorizations API

With `--inverse`, The openapi `tags` array is converted to an object if first element doesn't contain the tag(s) being filtered on

When using --inverse in combination with an tags array at the top level of the spec, the structure of the resulting tags property's value gets converted incorrectly to an object instead of an array if the first element in the input array does not have the property being filtered on ("x-public" in the following examples).

Minimal repro spec:

{
  "tags": [
    {
      "name": "Tag 1"
    },
    {
      "x-public": true,
      "name": "Tag 2"
    }
  ]
}

Running openapi-filter --inverse --tags "x-public" -- repro.json on this results in:

{
  "tags": {
    "1": {
      "x-public": true,
      "name": "Tag 2"
    }
  }
}

If the "x-public" property is moved to the first item in the tags array, the resulting output is correct, though:

Input spec:

{
  "tags": [
    {
      "x-public": true,
      "name": "Tag 1"
    },
    {
      "name": "Tag 2"
    }
  ]
}

Command: openapi-filter --inverse --tags "x-public" -- repro.json
Output:

{
  "tags": [
    {
      "x-public": true,
      "name": "Tag 1"
    }
  ]
}

Accidental 3.0 release?

Hey there,
Dependabot opened a 3.0.0 update for me this morning, was this an accidental publish? I don't see a tag or changelog as has been what is historically done in this project.

--valid not keeping entire info section

This nifty tool could save me some headaches, so I really appreciate your effort in making it. I need to produce three or four flavors of an API spec, so it's easier for me to tag operations with what flavor they should be in rather than what flavors they should be filtered out of. Consequently, I'm trying to use the -inverse option. I quickly discovered that I also need to use the --valid option, which put the openapi and info sections at the bottom of the output file.

I have a suggestion and bug report:

Suggestion: I know that putting those sections at the bottom of the file results in a valid OpenAPI file, but people are used to seeing them at the beginning of the file. Any chance you can put them at the top of the output for people who open the file in a text editor?

Bug: The info section in my source file contains title, version, and description, but the output from openapi-filter only contains title and version. The description should also be included in the output.

Also, I just noticed that the version value is being truncated if it ends in .0. In my source file it's 3.0, and in the output of openapi-filter it's just 3.