watermarkchurch/contentful-schema-diff

Command-line tool which generates Contentful migration files from the diff between two spaces or environments

contentful-schema-diff's Issues

CI-friendly migration

I was wondering what's the best way to set up something like

https://www.contentful.com/developers/docs/tutorials/general/continuous-integration-with-circleci/

Approach they use there is a singleton entry on Contentful which stores the current schema migration version. Then when you push code to an environment, it would know which migrations it would need to run.

This repo could potentially add dev-friendly commands like:

npm run migrate
npm run migrate:down
etc
https://edgeguides.rubyonrails.org/active_record_migrations.html#running-migrations <-- Rails had a really nice setup

How do you approach CI? What's your migration process like?

I feel like if I get this missing piece, I'll have an amazing setup between Contentful and your tool.

Doesn't handle default value

  {
+    defaultValue: {
+      en-US: true
+    }
-    required: false
+    required: true
   }

Vulnerability of contentful-management dependency

The project I'm working on currently has High vulnerability: "Server-Side Request Forgery" because of axios version 0.19.2 used in contentful-management 5.0.0-beta2.

contentful-management dependency has update to 7.3.1. and that version uses axios 0.21.1 which is not vulnerable.

Migration of ID change is incorrect - `id()` is not a function

When changing a field's ID, the output migration file uses id as the function to call, when I believe it's newId

Produced:

  myField.editField("headerContent").id("headerContentFoo");

should be

  myField.editField("headerContent").newId("headerContentFoo");

field add / remove improperly generate update sometimes

I removed a field and added a field, that just so happened to look like a modification to the diff algorithm. See generated migration below:

/*
   {
-    id: "tag"
+    id: "internalTitle"
-    name: "Tag"
+    name: "Internal Title"
+    required: true
   }
   ...
 ]
 */

sectionBlockText.editField('internalTitle')
  .id('internalTitle')
  .name('Internal Title')
  .required(true)

Package causes ts linter to go crazy

So i am experiencing this really odd behaviour with this package. First of all, it seems to work as expected BUT, having this package in my repository makes the typescript linter, after running it, fail almost every file and react component of the project and complaining about stuff that is not even there, like the usage of var in a file where only const is used e. g.
this only happens when this module is installed. when i remove it from my package.json, delete node_modules and make fresh install everything runs normal again and the linter does not find any issues.

"tslint": "~5.20.1",
"tslint-config-prettier": "~1.18.0",
"typescript": "~3.7.3",

Can't find module

Steps to reproduce:

1. Be sure you are running node v14.15.4
2. install contentful-schema-diff
3. run: $ contentful-schema-diff --from --to --token

Expected: migration scripts are created

Actual:
_http_agent.js:444
options = req[kRequestOptions];
^

TypeError: Cannot read property 'Symbol(requestOptions)' of undefined
at Agent.removeSocket (_http_agent.js:444:20)
at TLSSocket.onClose (_http_agent.js:371:11)
at TLSSocket.emit (events.js:327:22)
at net.js:673:12
at TCP.done (_tls_wrap.js:563:7)

Changes to content type name and/or description are not picked up

Updates to the name and/or description of a content type are not picked up by Contentful Schema Diff.

This screenshot shows the difference between two exports from two environments where only the name and description was updated:

But upon running the script, no diff to this content type is generated.

Bug: Content type argument doesn't work

When including --content-type MYCONTENTTYPE in the command, the generated file still includes every file type from the export.

Example:
contentful-schema-diff --from MY-EXPORT-1.json --to MY-EXPORT-2.json -c image --one-file --out diff-output/

I expect that the above command would generate a diff file that only contains changes to image content type, but instead, it includes changes from all content types in the export files.

Test fails: modify › writes created fields

When I run the tests I get the following error:

npm run test

> [email protected] test /Users/mlindsey/workspace/contentful-schema-diff
> tsc --project tsconfig.test.json && nyc ava 'src/**/*.test.js'


  ✔ diff › isDiff handles simple diff
  ✔ diff › isDiff returns false for empty diff
  ✔ diff › isDiff returns false for non-diff
  ✔ diff › isDiff handles complex diff
  ✔ diff › isDiffItem returns false for empty array
  ✔ diff › isDiffItem returns false for array with wrong length
  ✔ diff › isDiffItem returns false for array with wrong key
  ✔ diff › isDiffItem handles corner case
  ✔ diff › isDiffObj returns false for empty obj
  ✔ diff › isDiffObj returns false for non-obj
  ✔ diff › isDiffObj returns true for simple diff
  ✔ diff › isDiffObj returns true for diff with sub-diffs
  ✔ diff › isDiffObj returns true for complex diff
  ✔ create › dumps content type
  ✔ create › dumps simple fields
  ✔ editor_interface › writes nothing if no diff
  ✔ editor_interface › explicitly writes default on initial
  ✔ editor_interface › does not reopen content type if variable already was written
  ✔ editor_interface › writes changes for diffs
  ✔ editor_interface › opens content type for edit if varname not set in context
  ✔ editor_interface › writes help text if present
  ✔ editor_interface › writes change if settings changed
  ✔ modify › writes content type without def (176ms)
  ✔ modify › sets varname on context (157ms)
  ✔ modify › dumps diff as comment (153ms)
  ✖ modify › writes created fields 
  ✔ modify › moves newly created fields (144ms)
  ✔ modify › writes deleted fields (130ms)
  ✔ modify › writes moved fields (125ms)
  ✔ modify › writes change to top-level field details (123ms)
  ✔ modify › writes change to items
  ✔ modify › writes change to moved field
  ✔ modify › properly moves field when new field inserted above
  ✔ modify › also picks up field changes when new field inserted above
  ✔ runners › async_writer › rejects the promise if write sends an error
  ✔ runners › async_writer › waits for the drain event if draining
  ✔ runners › async_writer › writes chunks to the stream
  ✔ runners › async_writer › recursively writes the chunk after drain event
  ✔ runners › async_writer › catches errors after a drain event
  ✔ runners › file_per_content_type › writes timestamped file for each content type (1.1s)
  ✔ runners › write_single_file › writes a chunk to the specified file (1.4s)

  1 test failed

  modify › writes created fields

  /Users/mlindsey/workspace/contentful-schema-diff/src/modify.test.js:369

   368:                     name: 'Meta-Description',            
   369:                     validations: [{ size: { max: 160 } }]
   370:                 }));                                     

  Value must match expression:

  `␊
    var menu = migration.editContentType('menu')␊
    ␊
    /*␊
   [␊
     {␊
  -    type: "Symbol"␊
  +    type: "Text"␊
     }␊
  +  {␊
  +    id: "movedField"␊
  +    name: "Moved Field"␊
  +    type: "Number"␊
  +    validations: [␊
  +      {␊
  +        in: [␊
  +          0␊
  +          1␊
  +          2␊
  +        ]␊
  +      }␊
  +    ]␊
  +  }␊
  +  {␊
  +    id: "newField"␊
  +    name: "New Field"␊
  +    type: "Symbol"␊
  +    required: true␊
  +    validations: [␊
  +    ]␊
  +  }␊
     {␊
       validations: [␊
         {␊
  -        message: "The Top Button must be a..."␊
  +        message: "A new message"␊
         }␊
       ]␊
     }␊
  -  {␊
  -    id: "movedField"␊
  -    name: "Moved Field"␊
  -    type: "Number"␊
  -    required: true␊
  -    validations: [␊
  -    ]␊
  -  }␊
     {␊
  +    disabled: true␊
       items: {␊
         validations: [␊
  -        {␊
  -          range: {␊
  -            min: 1␊
  -            max: 4␊
  -          }␊
  -        }␊
  +        {␊
  +          range: {␊
  +            min: 1␊
  +            max: 5␊
  +          }␊
  +        }␊
           ...␊
         ]␊
       }␊
     }␊
  -  {␊
  -    id: "sideMenu"␊
  -    name: "Side Menu"␊
  -    type: "Link"␊
  -    validations: [␊
  -      {␊
  -        linkContentType: [␊
  -          "menu"␊
  -        ]␊
  -        message: "The Side Menu must be a Menu"␊
  -      }␊
  -    ]␊
  -    linkType: "Entry"␊
  -  }␊
   ]␊
   */␊
    ␊
      menu.editField('name')␊
          .type('Text')␊
  ␊
      menu.editField('topButton')␊
          .validations([ { linkContentType:␊
       [ 'menuButton' ],␊
      message:␊
       'A new message' } ])␊
  ␊
      menu.editField('items')␊
          .disabled(true)␊
          .items({ type:␊
     'Link',␊
    validations:␊
     [ { range:␊
          { min:␊
             1,␊
            max:␊
             5 } },␊
       { linkContentType:␊
          [ 'menu',␊
            'menuButton' ],␊
         message:␊
          'The items must be either buttons or drop-down menus' } ],␊
    linkType:␊
     'Entry' })␊
  ␊
      menu.createField('newField', { name:␊
     'New Field',␊
    type:␊
     'Symbol',␊
    localized:␊
     false,␊
    required:␊
     true,␊
    validations:␊
     [],␊
    disabled:␊
     false,␊
    omitted:␊
     false })␊
    ␊
      menu.moveField('newField')␊
          .afterField('movedField')␊
      menu.moveField('movedField')␊
          .afterField('name')␊
      menu.editField('movedField')␊
          .required(false)␊
          .validations([ { in:␊
       [ 0,␊
         1,␊
         2 ] } ])␊
  ␊
      menu.deleteField('sideMenu')␊
    `

  Regular expression:

  /menu.createField\('newField', { name: 'New Field',/

  src/modify.test.js:369:13
  step (src/modify.test.ts:101:11)
  Object.next (src/modify.test.ts:23:11)
  fulfilled (src/modify.test.js:7:24)



  `--fail-fast` is on. At least 4 tests were skipped.

---------------------------|----------|----------|----------|----------|-------------------|
File                       |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
---------------------------|----------|----------|----------|----------|-------------------|
All files                  |    48.65 |    29.54 |    55.75 |    55.36 |                   |
 src                       |    39.78 |    27.68 |    44.94 |    46.63 |                   |
  create.ts                |      100 |      100 |      100 |      100 |                   |
  delete.js                |        0 |      100 |        0 |        0 |             2,4,6 |
  diff.ts                  |    95.24 |    95.45 |      100 |    95.24 |                66 |
  editor_interface.ts      |    96.43 |    91.67 |      100 |     96.3 |                20 |
  index.js                 |        0 |        0 |        0 |        0 |... 43,44,45,54,56 |
  main.js                  |        0 |        0 |        0 |        0 |... 1,93,94,95,100 |
  model.js                 |        0 |      100 |      100 |        0 |                 2 |
  modify.ts                |     89.8 |    87.23 |      100 |    89.36 |... 9,85,86,88,175 |
  source.js                |        0 |        0 |        0 |        0 |... 10,111,121,122 |
  test-support.js          |        0 |        0 |        0 |        0 |... 6,7,9,62,64,75 |
  utils.ts                 |    76.09 |       50 |    70.59 |    76.92 |... 52,53,54,56,72 |
 src/runners               |    93.48 |       80 |    95.83 |    93.02 |                   |
  async_writer.ts          |    94.74 |      100 |     87.5 |    94.74 |                35 |
  file_per_content_type.ts |    97.44 |      100 |      100 |    97.22 |                55 |
  index.js                 |        0 |      100 |      100 |        0 |                 2 |
  write_single_file.ts     |    90.91 |       50 |      100 |       90 |          19,20,44 |
---------------------------|----------|----------|----------|----------|-------------------|

Default value for widgetNameSpace for changeFieldControl

I was wondering if a default value for widgetNameSpace can be applied instead of setting it to undefined?

I'm not sure on how this is done on contentful, when no custom field controls are applied to any fields - as far as I know it should be using the default value which is builtIn, however that is not being set.
(checked the JSON preview in contentful - we have no fields with any custom field controls nor does it specify which field controls are being in use)

Sometimes when the migration scripts are generated, they have the value of "built in" and sometimes "undefined" - hence why I suggested to place in the default value in case it does come to be undefined.

Currently the issue is when trying to transpile the typescript to javascript - it errors on changeFieldControl when trying to pass undefined to a param that expects either 'builtin' | 'extension'.

script could not be parsed

Hello,
Thanks for creating this script!
I'm trying to call runMigration() on the generated ts file from the script, and it says "The 20200125001610_generated_diff_event.ts script could not be parsed, as it seems to contain syntax errors."
I also tried via contentful space migration, and I'm getting the same error with more detail:

import Migration, { MigrationFunction } from "contentful-migration";  
^^^^^^  
SyntaxError: Cannot use import statement outside a module  

I tried creating a similar tsconfig.json as in the project, and running ts-node --project tsconfig.json migrateEvents.ts, but it still gives the same error.

Changing identity title is not registered

Setting "This field represents the Entry title" under "Field options" is not reflected in the migration when no other changes are made to the field/content type.

🐛 When a content type is removed no migration file created

When generating migration files, a removed content type is not accounted for. I'm guessing this is because it's not in the destination's set of types enumerated looking for property changes?

Syntax error

After generating migration scripts to execute via CLI via the contentful space migration tool, I get a lot of errors by the generated typescript files - however when I convert the typescript to javascript it seems to work. The console also doesn't actually list out what the actual errors are either.

Any reason why or has anyone else experienced this issue?

Here's the output in the console.

Error: The /Usersbob.dillan/Documents/workspaces/community-hub/contentful-migration/scripts/20200225142341_generated_diff_case_studies.ts script could not be parsed, as it seems to contain synta… │ │ errors. │ │ │ │ at run (/Users/bob.dillan/.nvm/versions/node/v10.15.3/lib/node_modules/contentful-cli/node_modules/contentful-migration/built/bin/cli.js:55:19) │ │ at migration (/Users/bob.dillan/.nvm/versions/node/v10.15.3/lib/node_modules/contentful-cli/lib/cmds/space_cmds/migration.js:94:10) │ │ at Object.argv [as handler] (/Users/bob.dillan/.nvm/versions/node/v10.15.3/lib/node_modules/contentful-cli/lib/utils/async.js:7:10) │ │ at innerArgv.then.argv (/Users/bob.dillan/.nvm/versions/node/v10.15.3/lib/node_modules/contentful-cli/node_modules/yargs/lib/command.js:241:49)