njvack / markdown-to-json Goto Github PK
View Code? Open in Web Editor NEWA tool to turn Markdown into a nested JSON structure.
License: MIT License
A tool to turn Markdown into a nested JSON structure.
License: MIT License
#stuff
##Test
a
b
breaks with
File "/usr/bin/md_to_json", line 11, in <module>
load_entry_point('markdown-to-json==1.0.0', 'console_scripts', 'md_to_json')()
File "/usr/lib/python3.8/site-packages/markdown_to_json/scripts/md_to_json.py", line 86, in main
return jsonify_markdown(
File "/usr/lib/python3.8/site-packages/markdown_to_json/scripts/md_to_json.py", line 68, in jsonify_markdown
rendered = renderer.stringify_dict(nested)
File "/usr/lib/python3.8/site-packages/markdown_to_json/markdown_to_json.py", line 114, in stringify_dict
for k, v in d.items()
AttributeError: 'list' object has no attribute 'items'
expectations
{"stuff": "Test":["a","b"]}
or
{"stuff": "Test":["a\nb"]}
or
{"stuff": "Test":["a \nb "]}
if whitespace is preserved
Hello,
thank you for this great library :).
I'm wondering if you would like to handle comments in your tool. I've the issue that I'm getting the following markdown file:
# latest
<!-- comment -->
* some feature
* bugfix: various bugfixes
# 1.0.0.0
* initiale Version
Currently I'm getting the following JSON from your tool:
{
"latest": "<!-- comment -->\n\n['some feature', 'bugfix: various bugfixes']"
"1.0.0.0": [
"initiale Version"
]
}
I would expect this:
{
"latest": [
"some feature",
"bugfix: various bugfixes"
],
"1.0.0.0": [
"initiale Version"
]
}
Consider this markdown:
## GUID
2db62bb2-8ac0-4137-b26f-78a12bff449d
## Title
Some title
## Summary
A Summary
## Priority
- Must
## Detailed Description
- Lorem Ipsum Blabla
- Lorem Ipsum Blabla
## Reference Requirements
> na
## Categories
- Cat6
The command md_to_json Test.md
yields
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\Scripts\md_to_json-script.py", line 11, in <module>
load_entry_point('markdown-to-json==1.0.0', 'console_scripts', 'md_to_json')()
File "C:\ProgramData\Anaconda3\lib\site-packages\markdown_to_json-1.0.0-py3.6.egg\markdown_to_json\scripts\md_to_json.py", line 94, in main
File "C:\ProgramData\Anaconda3\lib\site-packages\markdown_to_json-1.0.0-py3.6.egg\markdown_to_json\scripts\md_to_json.py", line 72, in jsonify_markdown
File "C:\ProgramData\Anaconda3\lib\site-packages\markdown_to_json-1.0.0-py3.6.egg\markdown_to_json\markdown_to_json.py", line 59, in nest
File "C:\ProgramData\Anaconda3\lib\site-packages\markdown_to_json-1.0.0-py3.6.egg\markdown_to_json\markdown_to_json.py", line 66, in _dictify_blocks
File "C:\ProgramData\Anaconda3\lib\site-packages\markdown_to_json-1.0.0-py3.6.egg\markdown_to_json\markdown_to_json.py", line 80, in _ensure_list_singleton
markdown_to_json.markdown_to_json.ContentError: Error at line 14: Can't mix lists and other content
I accidentally wrote a python-to-markdown serializer/deserializer markpickle before realizing that your code does the deserializing part much the same way or better & my ATX-headers-as-dict code now follows your pattern.
I sort of would like to fix the 10 issues and 3 pull requests so I can validate that my serializer/deserializer can handle the same cases that markdown-to-json can handle & make that part of the build process.
I could
Having done this a few time before...
I asked another dude if they needed help & they gave me github only rights & disappeared for a few years. Not useful! I'd have to still publish to pypi under a different name.
I asked yet another dude if they needed help & they gave me just pypi collaboration rights- good enough, I forked & could publish, but can't do the new 'trusted publisher thing'
Another dude just didn't reply for years, so I forked and published under a similar name.
Anyhow, before I take that last option, I wanted to get your feedback.
Can't open gg.md for reading
when running on Windows. 😢
Hi @njvack ,
I have tried your package for converting rasa bot framework .md file to json convertion
.md file is as follows
## happy path
* greet
- utter_greet
* request_restaurant
- restaurant_form
- form{"name": "restaurant_form"}
- form{"name": null}
- utter_slots_values
* thankyou
- utter_noworries
## unhappy path
* greet
- utter_greet
* request_restaurant
- restaurant_form
- form{"name": "restaurant_form"}
* chitchat
- utter_chitchat
- restaurant_form
- form{"name": null}
- utter_slots_values
* thankyou
- utter_noworries
but while converting json i am facing following
markdown_to_json.markdown_to_json.ContentError: Error at line 3: Can't mix lists and other content
Any solutions ?
I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/md_to_json", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/scripts/md_to_json.py", line 89, in main
indent)
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/scripts/md_to_json.py", line 68, in jsonify_markdown
rendered = renderer.stringify_dict(nested)
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 114, in stringify_dict
for k, v in d.items()
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 120, in _valuify
return self.stringify_dict(cm_vals)
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 114, in stringify_dict
for k, v in d.items()
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 120, in _valuify
return self.stringify_dict(cm_vals)
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 114, in stringify_dict
for k, v in d.items()
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 120, in _valuify
return self.stringify_dict(cm_vals)
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 114, in stringify_dict
for k, v in d.items()
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 133, in _render_block
return method(block)
File "/usr/local/lib/python2.7/site-packages/markdown_to_json/markdown_to_json.py", line 137, in _render_generic_block
return "\n".join(block.strings)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 83: ordinal not in range(128)
When passing in this string:
#### When someone unexpectedly hands over to you in the middle of a presentation and you’ve never even seen the slides before
If I change the ’
to '
, the error goes away and it works fine. Similarly, Chinese characters or anything non-ASCII also breaks md-to-json.
Is this compatible with the Pandoc JSON format?
I have an md with a table and tried using this tool to convert it. But seems this only supports basic simple mds. I get the following error,
"markdown_to_json.markdown_to_json.ContentError: Error at line 47: Can't mix lists and other content"
Do you have any plans on improvements?
Thanks for this great job!
I tried the cases below, and it missed several important descriptions.
text = """# REPO
intro
## Description
This is an example file
### Application
application
## Authors
* Nate Vack
* Vendor Packages
* docopt
* CommonMark-py
## Versions
### Version 1
Here's something about Version 1; I said "Hooray!"
### Version 2
Here's something about Version 2"""
import markdown_to_json
markdown_to_json.dictify(text)
Here is the output:
OrderedDict([('REPO',
OrderedDict([('Description',
OrderedDict([('Application', 'application')])),
('Authors',
['Nate Vack',
'Vendor Packages',
['docopt', 'CommonMark-py']]),
('Versions',
OrderedDict([('Version 1',
'Here\'s something about Version 1; I said "Hooray!"'),
('Version 2',
"Here's something about Version 2")]))]))])
the tokens intro
and application
are not parsed.
Thanks for your help if you could give some advice!
What are you using a markdown parser?
It seems to be strict markdown without github or gitlab flavor support, table are no supported:
$ md_to_json index.md
Traceback (most recent call last):
File "/home/noraj/.local/share/virtualenvs/table-iZ6bjDlQ/bin/md_to_json", line 10, in <module>
sys.exit(main())
File "/home/noraj/.local/share/virtualenvs/table-iZ6bjDlQ/lib/python3.7/site-packages/markdown_to_json/scripts/md_to_json.py", line 89, in main
indent)
File "/home/noraj/.local/share/virtualenvs/table-iZ6bjDlQ/lib/python3.7/site-packages/markdown_to_json/scripts/md_to_json.py", line 68, in jsonify_markdown
rendered = renderer.stringify_dict(nested)
File "/home/noraj/.local/share/virtualenvs/table-iZ6bjDlQ/lib/python3.7/site-packages/markdown_to_json/markdown_to_json.py", line 114, in stringify_dict
for k, v in d.items()
AttributeError: 'list' object has no attribute 'items'
My source is something like that:
Contest | Date | Location | Links
--- | --- | --- | ---
SigSeg V1 | Sat, 01 Dec. 2018 Sun, 02 Dec. 2018 | Paris, IdF, France | [website][sigsegv1]
Hack.lu CTF 2018 | Tue, 16 Oct. 2018 Thu, 18 Oct. 2018 | On-line | [CTFTime][ctftimeHacklu2018] - [website][hacklu]
picoCTF 2018 | Fri, 28 Sep. 2018 Fri, 12 Oct. 2018 | On-line | [CTFTime][ctftimePicoCTF2018] - [website][picoCTF]
[sigsegv1]:https://sigsegv1.rtfm.re/
[ctftimeHacklu2018]:https://ctftime.org/event/699
[hacklu]:https://arcade.fluxfingers.net/
[ctftimePicoCTF2018]:https://ctftime.org/event/681
[picoCTF]:https://picoctf.com/
For example
cat test.md
# issues
## 紧急bug
md_to_json test.md
{
"issues": {
"\u7d27\u6025bug": ""
}
}
hope it should be:
{
"issues": {
"紧急bug": ""
}
}
I'm getting an error when i run the tool on some (admittedly large) markdown, so i thought i'd take a look at the repo.
very kindly you've defined a devcontainer, but it's creation is failing for me...
It looks like pipx is (somehow) being considered missing from the underlying image
<snip of lots of stuff>
=> [dev_containers_target_stage 1/8] RUN mkdir -p /tmp/dev-container-fea 1.2s
=> [dev_containers_feature_content_normalize 1/2] COPY --from=dev_contai 1.2s
=> [dev_containers_feature_content_normalize 2/2] RUN chmod -R 0755 /tmp 0.3s
=> [dev_containers_target_stage 2/8] COPY --from=dev_containers_feature_ 0.0s
=> [dev_containers_target_stage 3/8] RUN echo "_CONTAINER_USER_HOME=$( ( 0.4s
[2023-10-26T03:24:50.880Z] => ERROR [dev_containers_target_stage 4/8] RUN --mount=type=bind,from=d 12.3s
------
> [dev_containers_target_stage 4/8] RUN --mount=type=bind,from=dev_containers_feature_content_source,source=black_0,target=/tmp/build-features-src/black_0 cp -ar /tmp/build-features-src/black_0 /tmp/dev-container-features && chmod -R 0755 /tmp/dev-container-features/black_0 && cd /tmp/dev-container-features/black_0 && chmod +x ./devcontainer-features-install.sh && ./devcontainer-features-install.sh && rm -rf /tmp/dev-container-features/black_0:
0.424 ===========================================================================
0.424 Feature : Black (via pipx)
[2023-10-26T03:24:50.881Z]
0.424 Description : Black is an uncompromising Python code formatter.
0.424 Id : ghcr.io/devcontainers-contrib/features/black
0.424 Version : 2.0.17
0.424 Documentation : http://github.com/devcontainers-contrib/features/tree/main/src/black
0.424 Options :
0.424 VERSION="latest"
0.424 ===========================================================================
3.148 nanolayer
11.74 cd /tmp/tmpach6m_fq && chmod +x -R . && _REMOTE_USER="vscode" _REMOTE_USER_HOME="/home/vscode" PACKAGE="black" VERSION="latest" INJECTIONS="" INCLUDEDEPS="false" INTERPRETER="" NANOLAYER_VERBOSE="" NANOLAYER_FORCE_CLI_INSTALLATION="" NANOLAYER_PROPAGATE_CLI_LOCATION="1" NANOLAYER_CLI_LOCATION="/tmp/nanolayer-MS9NFAbxeD/nanolayer" bash -i +H ./install.sh
11.74
11.74 [notice] A new release of pip is available: 23.0.1 -> 23.3.1
11.74 [notice] To update, run: pip install --upgrade pip
11.86 Traceback (most recent call last):
11.86 File "/root/.local/bin/pipx", line 5, in <module>
11.86 from pipx.main import cli
11.86 ModuleNotFoundError: No module named 'pipx'
11.90 Traceback (most recent call last):
11.90 File "/root/.local/bin/pipx", line 5, in <module>
11.90 from pipx.main import cli
11.90 ModuleNotFoundError: No module named 'pipx'
11.95 Traceback (most recent call last):
11.95 File "<string>", line 1, in <module>
11.95 File "nanolayer.__main__", line 50, in main
11.95 File "typer.main", line 328, in __call__
11.95 File "typer.main", line 311, in __call__
11.95 File "click.core", line 1157, in __call__
11.95 File "typer.core", line 778, in main
11.95 File "typer.core", line 216, in _main
11.95 File "click.core", line 1688, in invoke
11.95 File "click.core", line 1688, in invoke
11.95 File "click.core", line 1434, in invoke
11.95 File "click.core", line 783, in invoke
11.95 File "typer.main", line 683, in wrapper
11.95 File "nanolayer.cli.install", line 65, in install_devcontainer_feature
11.95 File "nanolayer.installers.devcontainer_feature.oci_feature_installer", line 131, in install
11.95 File "nanolayer.utils.invoker", line 59, in invoke
11.95 nanolayer.utils.invoker.Invoker.InvokerException: The command 'cd /tmp/tmpach6m_fq && chmod +x -R . && _REMOTE_USER="vscode" _REMOTE_USER_HOME="/home/vscode" PACKAGE="black" VERSION="latest" INJECTIONS="" INCLUDEDEPS="false" INTERPRETER="" NANOLAYER_VERBOSE="" NANOLAYER_FORCE_CLI_INSTALLATION="" NANOLAYER_PROPAGATE_CLI_LOCATION="1" NANOLAYER_CLI_LOCATION="/tmp/nanolayer-MS9NFAbxeD/nanolayer" bash -i +H ./install.sh' failed. error: Return Code: 1. see logs for details.
12.31 ERROR: Feature "Black (via pipx)" (ghcr.io/devcontainers-contrib/features/black) failed to install! Look at the documentation at http://github.com/devcontainers-contrib/features/tree/main/src/black for help troubleshooting this error.
------
[2023-10-26T03:24:50.881Z] Dockerfile-with-features:30
--------------------
29 |
30 | >>> RUN --mount=type=bind,from=dev_containers_feature_content_source,source=black_0,target=/tmp/build-features-src/black_0 \
31 | >>> cp -ar /tmp/build-features-src/black_0 /tmp/dev-container-features \
32 | >>> && chmod -R 0755 /tmp/dev-container-features/black_0 \
33 | >>> && cd /tmp/dev-container-features/black_0 \
34 | >>> && chmod +x ./devcontainer-features-install.sh \
35 | >>> && ./devcontainer-features-install.sh \
36 | >>> && rm -rf /tmp/dev-container-features/black_0
37 |
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c cp -ar /tmp/build-features-src/black_0 /tmp/dev-container-features && chmod -R 0755 /tmp/dev-container-features/black_0 && cd /tmp/dev-container-features/black_0 && chmod +x ./devcontainer-features-install.sh && ./devcontainer-features-install.sh && rm -rf /tmp/dev-container-features/black_0" did not complete successfully: exit code: 1
View build details: �]8;;docker-desktop://dashboard/build/default/default/w3kxnf2msyaha4jntwrrp3z59�\docker-desktop://dashboard/build/default/default/w3kxnf2msyaha4jntwrrp3z59�]8;;�\
[2023-10-26T03:24:50.889Z] Stop (150291 ms): Run: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /tmp/devcontainercli-badger/container-features/0.51.3-1698290531190/Dockerfile-with-features -t vsc-markdown-to-json-f8ad31d84594ef91756778208ae96bf09cbbb588331486895ec5534ee2ffea76 --target dev_containers_target_stage --build-context dev_containers_feature_content_source=/tmp/devcontainercli-badger/container-features/0.51.3-1698290531190 --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label --build-arg _DEV_CONTAINERS_IMAGE_USER=vscode --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp /home/badger/git/markdown-to-json
[2023-10-26T03:24:50.894Z] Error: Command failed: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /tmp/devcontainercli-badger/container-features/0.51.3-1698290531190/Dockerfile-with-features -t vsc-markdown-to-json-f8ad31d84594ef91756778208ae96bf09cbbb588331486895ec5534ee2ffea76 --target dev_containers_target_stage --build-context dev_containers_feature_content_source=/tmp/devcontainercli-badger/container-features/0.51.3-1698290531190 --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label --build-arg _DEV_CONTAINERS_IMAGE_USER=vscode --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp /home/badger/git/markdown-to-json
[2023-10-26T03:24:50.894Z] at BAA (/home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js:463:1860)
[2023-10-26T03:24:50.895Z] at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[2023-10-26T03:24:50.895Z] at async Ww (/home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js:462:1691)
[2023-10-26T03:24:50.895Z] at async SK (/home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js:462:610)
[2023-10-26T03:24:50.895Z] at async SAA (/home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js:479:3660)
[2023-10-26T03:24:50.896Z] at async GC (/home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js:479:4775)
[2023-10-26T03:24:50.896Z] at async ZeA (/home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js:611:12251)
[2023-10-26T03:24:50.896Z] at async VeA (/home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js:611:11992)
[2023-10-26T03:24:50.906Z] Stop (161777 ms): Run: /usr/share/code/code --ms-enable-electron-run-as-node /home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/badger/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-4b84e523-30af-4dfe-9c50-0eb79f04b2341698290527924 --workspace-folder /home/badger/git/markdown-to-json --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/badger/git/markdown-to-json --id-label devcontainer.config_file=/home/badger/git/markdown-to-json/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/badger/git/markdown-to-json/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2023-10-26T03:24:50.907Z] Exit code 1
[2023-10-26T03:24:50.910Z] Command failed: /usr/share/code/code --ms-enable-electron-run-as-node /home/badger/.vscode/extensions/ms-vscode-remote.remote-containers-0.315.1/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/badger/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-4b84e523-30af-4dfe-9c50-0eb79f04b2341698290527924 --workspace-folder /home/badger/git/markdown-to-json --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/badger/git/markdown-to-json --id-label devcontainer.config_file=/home/badger/git/markdown-to-json/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/badger/git/markdown-to-json/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2023-10-26T03:24:50.910Z] Exit code 1
[2023-10-26T03:25:29.317Z] Error opening dev container configurations: CodeExpectedError: ENOPRO: No file system provider found for resource 'vscode-remote://dev-container%2B7b22686f737450617468223a222f686f6d652f6261646765722f6769742f6d61726b646f776e2d746f2d6a736f6e222c226c6f63616c446f636b6572223a66616c73652c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f686f6d652f6261646765722f6769742f6d61726b646f776e2d746f2d6a736f6e2f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f686f6d652f6261646765722f6769742f6d61726b646f776e2d746f2d6a736f6e2f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f686f6d652f6261646765722f6769742f6d61726b646f776e2d746f2d6a736f6e2f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d/workspaces/markdown-to-json'
at r.j (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:645:11397)
at async r.exists (vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:645:13603)
at async Promise.all (index 0)
at async vscode-file://vscode-app/usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1861:2796
any thoughts?
markdown-to-json does not work with python 3.9. I applied a fix to CommonMark.py but are not allowed to push it
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.