zk graph --format json is not handling note titles with inner double quotes

[tjex]

zk graph --format json is not handling note titles with inner double quotes

[tjex]

@mcDevnagh @julio-lopez @jurica

Would it be a case of manually sanitizing the note titles pre-json output? e.g, in the internal/core/note_format.go file?

I had a look at the go json.Marshal documentation.

It states something about a syntax like below being possible?

	Filename     string                 `json:"filename,string"`

Which should format it appropriately for wrapping in double quotes, but it didn't work on first try.

[mcDevnagh]

We shouldn't have to sanitize.
encoding/json does the sanitation for us.
For example,

package main

import "encoding/json"

type Test struct {
	Title string
}

func main() {
	json, err := json.Marshal(Test{
		Title: `"`,
	})

	if err != nil {
		panic(err)
	}
	print(string(json))
}

outputs

{"Title":"\""}

Is json.Marshal outputting everything in zk graph --json? This would be an upstream issue, but I'd find that very surprising. More likely we're calling json.Marshal for each note, and ignoring errors (or simply logging them). If that is the case, the best place to start fixing this would be to look at that error.

[tjex]

Ok great, thanks for this. As a heads up, I won't be able to look at this (or anything else) until March 17. I'll be on holiday 🏝️

[tjex]

note: narrowed this down a little bit and it actually looks like the culprit is osEnv() call in newNoteFormatter.

When it's getting called in graph.go, line 70, something about the double quoted title is breaking it. This is why it spits out the entire shell env variables to stdout. The double quotes in the file name is not the culprit as I'm using ids as file names.

Changing the file title from: what does "this" mean? to what does 'this' mean? executes without error.

[tjex]

It's actually due to the quotes not being escaped in link_format.go.

These functions are used to render the links within documents as well. So escaping the double quotes here will render links within notes with the backslashes: [[a linked \"note\"]]. This of course breaks the links.

[mcDevnagh]

Does that mean that the true culprit is links to titles with quotes and not titles with quotes themselves?

[tjex]

In other words, quotes are being escaped properly everywhere in json output except the string for the filename's own link.

If I hardcode the problem field output to foo, the call to zk graph --format json returns json without error.

{
  "filename": "another \"test\" note.md",
  "filenameStem": "another \"test\" note",
  "path": "another \"test\" note.md",
  "absPath": "/Users/tjex/.local/src/zk-org/workbench/test-vault/another \"test\" note.md",
  "title": "another \"test\" note",
  "link": "foo",
  "lead": "",
  "body": "",
  "snippets": [],
  "rawContent": "# another \"test\" note\n\n\n",
  "wordCount": 4,
  "tags": [],
  "metadata": {},
  "created": "2024-04-01T03:11:03.540446352Z",
  "modified": "2024-04-01T03:11:05.217630323Z",
  "checksum": "a121f8d06da1bff78c5e5eb4816d0417ab2ec22317bad43baedb51a20c60df68"
}

Otherwise it would in effect return this invalid json:

{
  "filename": "another \"test\" note.md",
  "filenameStem": "another \"test\" note",
  "path": "another \"test\" note.md",
  "absPath": "/Users/tjex/.local/src/zk-org/workbench/test-vault/another \"test\" note.md",
  "title": "another \"test\" note",
+  "link": "[[another "test" note]]", <- the quotes around test are not escaped
  "lead": "",
  "body": "",
  "snippets": [],
  "rawContent": "# another \"test\" note\n\n\n",
  "wordCount": 4,
  "tags": [],
  "metadata": {},
  "created": "2024-04-01T03:11:03.540446352Z",
  "modified": "2024-04-01T03:11:05.217630323Z",
  "checksum": "a121f8d06da1bff78c5e5eb4816d0417ab2ec22317bad43baedb51a20c60df68"
}

With the initial idea for the fix in #400 , the output looks like this:

{
  "filename": "another \"test\" note.md",
  "filenameStem": "another \"test\" note",
  "path": "another \"test\" note.md",
  "absPath": "/Users/tjex/.local/src/zk-org/workbench/test-vault/another \"test\" note.md",
  "title": "another \"test\" note",
  "link": "[[another \"test\" note]]",
  "lead": "",
  "body": "",
  "snippets": [],
  "rawContent": "# another \"test\" note\n\n\n",
  "wordCount": 4,
  "tags": [],
  "metadata": {},
  "created": "2024-04-01T03:11:03.540446352Z",
  "modified": "2024-04-01T03:11:05.217630323Z",
  "checksum": "a121f8d06da1bff78c5e5eb4816d0417ab2ec22317bad43baedb51a20c60df68"
}

Links are still rendered correctly within documents:

# note title
[another "test" note](another%20%22test%22%20note)
[[another "test" note]]