duo-labs/markdown-to-confluence

Syncs Markdown files to Confluence

markdown-to-confluence's Issues

Error when running in a git repo

When running the script in a local git repo with all MD files stored in docs/I get the error
raise InvalidGitRepositoryError(epath)
To solve this, I added search_parent_directories=True to the Repo object,
so that line 247 in markdown-to-confluence.py is now:
repo = git.Repo(args.git, search_parent_directories=True)
This seems to avoid the error, but now the script tries this:
Attempting to deploy docs/__pycache__/convert.cpython-39.pyc
which results in
Skipping docs/__pycache__/convert.cpython-39.pyc since it's not a supported format.
Do you have any hint what do do?

Disable sections

Need to use this, but have no need in Authors or ToC. Is there an option to disable those sections? I could not find that, maybe add it as a feature?

Getting the error with modules

Please refer to the screen for python in virtual environment. I have tried with both 3.8 and 3.9

Getting error with ModuleNotFoundError, even though it is installed.

Mistune v2 breaks convert.py

MR #21 broke convert.py. This is because mistune in versions 2.0.0+ no longer has a Renderer object, which this code tries to use.
To fix, we need to either:

• Rewrite convert.py to use up-to-date objects, or
• Downgrade mistune back to 0.8.4

Dependencies conflict

When trying to use the Docker image, the requirements fail to install because of a dependency issue between the versions of urllib3 and requests:

[5/6] RUN pip install --no-cache-dir -r requirements.txt:
#9 1.265 Collecting certifi==2019.9.11
#9 1.384 Downloading certifi-2019.9.11-py2.py3-none-any.whl (154 kB)
#9 1.427 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 154.4/154.4 kB 3.7 MB/s eta 0:00:00
#9 1.458 Collecting chardet==3.0.4
#9 1.484 Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
#9 1.491 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.4/133.4 kB 23.6 MB/s eta 0:00:00
#9 1.537 Collecting gitdb2==2.0.6
#9 1.565 Downloading gitdb2-2.0.6-py2.py3-none-any.whl (63 kB)
#9 1.567 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.0/64.0 kB 225.9 MB/s eta 0:00:00
#9 1.603 Collecting GitPython==3.0.4
#9 1.638 Downloading GitPython-3.0.4-py3-none-any.whl (454 kB)
#9 1.666 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 454.6/454.6 kB 18.7 MB/s eta 0:00:00
#9 1.692 Collecting idna==2.8
#9 1.712 Downloading idna-2.8-py2.py3-none-any.whl (58 kB)
#9 1.713 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.6/58.6 kB 338.6 MB/s eta 0:00:00
#9 1.739 Collecting mistune==0.8.4
#9 1.762 Downloading mistune-0.8.4-py2.py3-none-any.whl (16 kB)
#9 1.805 Collecting PyYAML==5.4
#9 1.830 Downloading PyYAML-5.4.tar.gz (174 kB)
#9 1.836 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 174.8/174.8 kB 40.4 MB/s eta 0:00:00
#9 1.918 Installing build dependencies: started
#9 3.600 Installing build dependencies: finished with status 'done'
#9 3.601 Getting requirements to build wheel: started
#9 4.209 Getting requirements to build wheel: finished with status 'done'
#9 4.212 Preparing metadata (pyproject.toml): started
#9 4.351 Preparing metadata (pyproject.toml): finished with status 'done'
#9 4.390 Collecting requests==2.22.0
#9 4.409 Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB)
#9 4.411 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.0/58.0 kB 247.4 MB/s eta 0:00:00
#9 4.434 Collecting smmap2==2.0.5
#9 4.454 Downloading smmap2-2.0.5-py2.py3-none-any.whl (24 kB)
#9 4.504 Collecting textwrap3==0.9.2
#9 4.528 Downloading textwrap3-0.9.2-py2.py3-none-any.whl (12 kB)
#9 4.576 Collecting urllib3==1.26.5
#9 4.596 Downloading urllib3-1.26.5-py2.py3-none-any.whl (138 kB)
#9 4.602 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 138.1/138.1 kB 26.8 MB/s eta 0:00:00
#9 4.636 Collecting yapf==0.28.0
#9 4.657 Downloading yapf-0.28.0-py2.py3-none-any.whl (180 kB)
#9 4.663 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 180.4/180.4 kB 36.0 MB/s eta 0:00:00
#9 4.683 INFO: pip is looking at multiple versions of pyyaml to determine which version is compatible with other requirements. This could take a while.
#9 4.683 INFO: pip is looking at multiple versions of mistune to determine which version is compatible with other requirements. This could take a while.
#9 4.683 INFO: pip is looking at multiple versions of idna to determine which version is compatible with other requirements. This could take a while.
#9 4.683 INFO: pip is looking at multiple versions of gitpython to determine which version is compatible with other requirements. This could take a while.
#9 4.683 INFO: pip is looking at multiple versions of to determine which version is compatible with other requirements. This could take a while.
#9 4.684 INFO: pip is looking at multiple versions of gitdb2 to determine which version is compatible with other requirements. This could take a while.
#9 4.684 INFO: pip is looking at multiple versions of chardet to determine which version is compatible with other requirements. This could take a while.
#9 4.684 INFO: pip is looking at multiple versions of certifi to determine which version is compatible with other requirements. This could take a while.
#9 4.684 ERROR: Cannot install -r requirements.txt (line 8) and urllib3==1.26.5 because these package versions have conflicting dependencies.
#9 4.684
#9 4.684 The conflict is caused by:
#9 4.684 The user requested urllib3==1.26.5
#9 4.684 requests 2.22.0 depends on urllib3!=1.25.0, !=1.25.1, <1.26 and >=1.21.1
#9 4.684
#9 4.684 To fix this you could try to:
#9 4.684 1. loosen the range of package versions you've specified
#9 4.684 2. remove package versions to allow pip attempt to solve the dependency conflict
#9 4.684
#9 4.685 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
#9 4.781
#9 4.781 [notice] A new release of pip available: 22.2.1 -> 22.2.2
#9 4.781 [notice] To update, run: pip install --upgrade pip

Deploy a package on Pypi

It would be more simple to use this library if it was deployed on the official pypi repository. This would allow me to simply call pip install markdown-to-confluence in a CI/CD pipeline instead of having to clone the project and build it there.

Occasional 409s?

Most of the time this tool works great. Seemingly randomly I sometimes run into a failure where I get a "409 Conflict" response code.

This seems relevant: https://community.developer.atlassian.com/t/replacing-a-page-with-copy-single-page-api-fails-with-optimisticlockexception/37622/3

Maybe some retry logic would help?

b'{"statusCode":409,"data":{"authorized":false,"valid":true,"errors":[],"successful":false},"message":"com.atlassian.confluence.api.service.exceptions.ConflictException: javax.persistence.OptimisticLockException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1"}'
b'{"statusCode":409,"data":{"authorized":false,"valid":true,"errors":[],"successful":false},"message":"com.atlassian.confluence.api.service.exceptions.ConflictException: javax.persistence.OptimisticLockException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1"}'

--dry-run fails

When I run with --dry-run the _create and _update methods fail

markdown-to-confluence/confluence.py

Line 107 in 8ce8d5d

return {}

returns {}

The following expects:

markdown-to-confluence/confluence.py

Line 323 in 8ce8d5d

page_id = response['id']

markdown-to-confluence/confluence.py

Line 324 in 8ce8d5d

page_url = urljoin(self.api_url, response['_links']['webui'])

and

markdown-to-confluence/confluence.py

Line 401 in 8ce8d5d

page_url = urljoin(self.api_url, response['_links']['webui'])

Expect the id and _links key to exist