markusstraub/mdsplit

mdsplit is a python command line tool to split markdown files into chapters at a given heading level

mdsplit's Issues

Assume utf-8 as default encoding

Hi,

first of all thanks for this really useful script! Unfortunately I got a encoding errors when I ran it for the first time as Python 3.8 under windows assumes the cp1252 encoding on my computer.

  File "C:\dev\tmsspec-doc-py\mdsplit.py", line 341, in <module>
    main()
  File "C:\dev\tmsspec-doc-py\mdsplit.py", line 333, in main
    splitter.process()
  File "C:\dev\tmsspec-doc-py\mdsplit.py", line 147, in process
    self.process_file(self.in_path, self.out_path)
  File "C:\dev\tmsspec-doc-py\mdsplit.py", line 166, in process_file
    self.process_stream(stream, in_file_path.name, out_path)
  File "C:\dev\tmsspec-doc-py\mdsplit.py", line 88, in process_stream
    file.write(line)
  File "C:\Program Files\Python38\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\ue60f' in position 4: character maps to <undefined>`

Wouldn't make it sense if the script assumed utf-8 encoding by default - maybe overrideable via CLI parameter? Had to monkey-patch the file.open() calls in your script with encoding="utf-8" to make it work.

Thanks,
Franz

Truncate file path length

If one heading is very long, the software will fail due to OSError in linux.

Traceback (most recent call last):
File "mdsplit", line 8, in
sys.exit(main())
^^^^^^
File "mdsplit.py", line 342, in main
splitter.process()
File "mdsplit.py", line 148, in process
self.process_file(self.in_path, self.out_path)
File "mdsplit.py", line 167, in process_file
self.process_stream(stream, in_file_path.name, out_path)
File "mdsplit.py", line 75, in process_stream
if not chapter_path.exists():
^^^^^^^^^^^^^^^^^^^^^
File "pathlib.py", line 1235, in exists
self.stat()
File "pathlib.py", line 1013, in stat
return os.stat(self, follow_symlinks=follow_symlinks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 36] File name too long:

More than 6 levels to split into

Is there some particular limitation to the number of accepted levels to descent into? I have a file with 8 levels, and I would really like to split it for my Obsidian vault. Thanks.

Correct internal cross-references when splitting the markdown file

It would be great if mdsplit could also fix markdown cross-references like [text](#target) (where target is a link to an internal heading) if link and target are no longer in the same file.

Same goes for relative image links in the input markdown that are no longer valid in the split output markdown.

Gitlab Wiki Usage

• File names should not contain spaces. This is fixed in 81d5925.
• TODO: For the Wiki a link syntax [link](<basename.md>) does not get the proper MIME type assigned by the server. .md needs to be removed. Together with above change such links are served correctly: [link](basename)

Assignees: @prismv
Labels:
Milestone:

Turn mdsplit into a Python library

The CLI interface of mdsplit is really neat, but for some use cases I'd really need a python library version of it:

• rewrite relative image links in the split out markdown files so that they work in the new directory structure
• clean up markdown text (background: I am working with many markdown files that have been converted from .docx via pandoc)
• decide whether a split markdown file should be written out or merged with the next document (think of a h1-level headline immediately followed by a h2-level headline, currently the h1 headline would be put into a document of its own)

Thus the library version needs to represent the split markdown documents as objects and offer methods to access their contents as well as some merge logic.