masukomi/private_comments

a tool for managing private comments on, but not in, your files

Note: The lack of activity here doesn't mean this repo is abandoned. It's just very stable code that doesn't need poking. I rely on it daily. ;)

Table of Contents

• Special Design Features
• Installation
• MacOS via Homebrew
• Other
• Usage
• Editor Plugins
• Specifying a Port
• Specifying a Directory
• Additional Information
• Creating An Editor Plugin
• Example Client
• Building From Source
• Troubleshooting
• Pre-Commits Warning
• Contributing

Private comments allows you to leave “private comments” on specific lines of a codebase that are not stored in the codebase.

Imagine being dropped into a new codebase and having the freedom to leave whatever todo items and breadcrumbs you want without cluttering the codebase. Imagine working on a client’s codebase and not having to worry about what you say, or who sees it.

Imagine leaving yourself a todo comment on a line, and having it disappear when you commit a change to that line. No more obsolete comments!

Editor Plugins speak to a tiny Private Comments REST server running in your computer. The server uses a little under 7MB of RAM and essentially no CPU. The animated GIF on this page is taking up more resources.

For information about the APIs and creating a plugin for you favorite editor please read "Creating An Editor Plugin" below.

Note: Private Comments currently assumes you're using git for your version control system. It's possible to use with editor plogins that know about other version control systems, but instructions are currently not provided for writing those.

Special Design Features

• Comments are tied to a specific commit. Each comment is associated with the commit that produced the line you're commenting on. When that line is no longer in the file, the comment will disappear. However, if you roll back your git repository to a prior commit, it will reappear so you can see any comments that were relevant at the time.
• Comments created with Private Comments are stored in a sharable git repository. A group of freelancers working on a client codebase can leave inline comments for each other as they learn the codebase without altering the source code itself.
• Project names and file names are never stored. This way you can back up your private comments repo to a third party server without violating your NDA. If the third party server is compromised the hackers won't be able to derive anything from a leaked filename or project name. If you're not encrypting your comments then you just need to be careful to not include any secrets in your comments.
• Comments can be encrypted. Encryption is optionally handled on the client side. The Private Comments server only cares that you're passing it a string. It doesn't care what that string is. Encryption / Decryption is fairly easy to add to any private comments browser plugin.

Installation

MacOS via Homebrew

Execute the following lines in your terminal.

brew tap masukomi/homebrew-apps
brew install private_comments

Other

For other operating systems you'll need to build it from source (see below).

Usage

Private comments is a simple web server that runs locally and provides an API for editor plugins to interact with.

I'd recommend you launch it by running private_comments && disown This will boot the server in the background and allow you to close the window and not worry about it. If you need to stop the server you can run pkill private_comments

Once you've got the server running, access the endpoints as described in the API docs or use an editor plugin or the pc command line client.

If you've forgotten if the server is running or not you can check by running pgrep private_comments That will return the id of the process if it's running or nothing if it isn't.

Editor Plugins

There is a Vim Plugin which leverages the pc client instead of talking directly to the APIs. This is the easiest way to bootstrap a plugin for your favorite editor.

The Emacs plugin does not use the pc client and speaks directly to the server. This is the preferred way of doing things.

See below for more details on creating a plugin for your favorite editor.

If you don't use Vim or Emacs you can still view private comments with the pc client but the experience is sub-optimal. The pc client is really designed for testing and bootstrapping if making http calls from your favorite editor is difficult.

Once the private_comments server is running locally you can follow the instructions for the pc command line client (see below), or better yet, a plugin for your favorite editor.

Specifying a Port

By default private comments runs on port 5749. To change this, set the PRIVATE_COMMENTS_PORT environment variable to the desired port.

For example: In bash you could change it to 3111 by executing it like this.

PRIVATE_COMMENTS_PORT=3111 private_comments

Alternately you could define it in your ~/.bashrc with this line

export PRIVATE_COMMENTS_PORT=3111

Specifying a Directory

By default private_comments will store its data in ~/.config/private_comments

To specify another directory just set the PRIVATE_COMMENTS_DIR environment variable to a valid path.

Additional Information

• Backing Up Your Private Comments
• Competitors

Creating An Editor Plugin

There's not a lot to creating an editor plugin, and the Example Client (see below) can be used to rapidly bootstrap the process. Under the covers it implements all the steps in the diagram below. The Vim plugin is an example of quickly bootstrapping a plugin. The Emacs plugin speaks directly to the local server, which makes it dramatically easier to inline the comments.

Please contact me if you'd like help creating a plugin for your favorite editor.

This diagram provides a high level overview of how data flows through the system. If you choose to interact with the API directly (recommended) you can find all the details in the API docs.

I recommend using the Example Client (pc) as a reference to confirm that your plugin is doing the right thing, and that the data you generate is readable by another client. The source to pc can be a reasonable blueprint for how to go about building a client.

Example Client

As a tool for testing the server, and bootstrapping plugin development Private Comments comes with an example command line client called pc

Following the usage guide below you can record, retrieve, and delete comments. Note that Private Comments can only record comments about lines that have been committed in git. So, if it's a new line, or you've changed the line, you'll need to commit it before commenting. Please reference the diagram above for how this should work.

Note: pc is not intended to be the tool you use to record comments. It's a testing/proof-of-concept client implementation. With that said, it certainly can be done and the Vim plugin does exactly that.

❯ pc --help
Private Comments client v1.3
Usage: pc -f file-path [-fcdlsp] [option values]
     -f --file=<path>                   Relative path from root of git repo. Ex. "src/pc.scm"
     -c --comment=<Comment>             A comment to be stored
     -d --delete                        Will delete comment at specified location (line & file)
     -l --line=<Line Number>            The line number of the comment to be stored
     -p --port=<Server Port>            Private Comments Server Port [default: 5749]
     -s --server=<Server URL>           Private Comments Server Url [default: http://0.0.0.0]
     -x --debug                         display comments with debugging output
     -h --help                          Display this text

Report bugs at https://github.com/masukomi/private_comments/issues

To retrieve comments for a given file you'd say something like this:

pc -f my/awesome/ruby/file.rb

Assuming the Private Comments server was running it'd return a human readable list of comments preceeded by their line number. For example:

12: Ask Mary about this.
22: see docs for how this call works
106: OMGWTFBBQ!!!

If there aren't any comments it'll just return nothing.

To record a comment on a particular line of code you could say:

pc -f my/awesome/ruby/file.rb -l 44 -c "TODO refactor me" 

NOTE: the current implementation only supports commenting on lines of code that have been previously committed to git.

It will exit with an error code if you're commenting on an uncommitted line.

You could use this as a quick starting point for an editor plugin, or tweak the code to make it generate different output. In the long run using the API is going to be a better solution.

Building From Source

Once you've cloned this repo you'll need to install Chicken Scheme and then run the install_chicken_eggs.sh script in the src directory.

If you're hacking on it I recommend running it with csi

$ csi private_comments.scm

When your ready to do a final compile just use the build.sh script a private_comments and pc executable.

The tests are written in Bash and use the bash_unit framework. Once you've installed it, added it to your PATH, and compiled your changes with build.sh you can run the tests like this:

# run from within the src directory

$ bash_unit tests/test_server
$ bash_unit tests/test_client

If it's running private_comments will be shut down, and a new instance will be run. The new instance will store its test data separately so you don't have to worry about messing up, or loosing, any existing comments you may have created with private comments.

Contributing

Private Comments is written in Chicken Scheme. Pull requests with new features, or improved code, are happily welcomed. Fork the repo. Make it better. Submit your changes. Note that all changes must be backwards compatible. We can't break existing plugins. If you're interested in a making a change that would break backwards compatibility please open a ticket to discuss it first. We'll create a v2 version of the API if it's worth it.

If you're new to Scheme, or Chicken Scheme, don't worry. Just do your best and submit what you come up with. It's all good. I'm not an expert Schemer either.

Please add a test to src/tests/test_server (for server changes) or src/tests/test_client (for client changes) to confirm your changes are good, and that existing features haven't been broken.

private_comments's Issues

Keeping comments in another branch instead of an API server

Wouldn't it make sense for devs to simply have another branch on the repo they are commenting on, to add the private comments? They don't need to push that branch if they want to keep it secret. I.e why have a web server?

private_comments needs to support DELETE rest action

we should be able to delete a comment by sending a DELETE request to /v1/comments endpoint.

params would be essentially the same as creating a comment only without the comment key. For example:

{
  "project_name_hash": "7135459ae30c0d5180b623986c420bf20856461cb6b9b860986a22c7654ed755",
  "file_path_hash": "faf3e6fc36b8524dad3aa4a317e734623f0dfcf6934a659015827406ebfb0c87",
  "treeish": "777a58e942164e94964999bcc7cd20bbcda28fe55ac38be28e530f58ddad5ad8",
  "line_number": 4
}

If there is currently not comment on that treeish + line_number tuple it should just return success because the user's intent of there being no comment on that line in that version has been achieved (through no action).

pc client has no option for deleting existing comment

This is blocked by Issue #3

pc client should have an command line option to delete an existing comment on a given line.

proposed interface

# short form
$ pc -f path/to/file.scm -l 26 -d
# long form
$ pc --file=path/to/file.scm --line=26 --delete

both are instructing the pc client to delete an existing comment on line 26

if there is no comment on line 26 of the current version of the specified file then the code should follow the "success" path because the user's intent of having no comments on line 26 has been achieved (by doing nothing).

errors arising from the private_comments server and / or communicating with it should have output akin to any other command that encounters this issue.

successful deletion should result in a message like "Comments deleted from line of "

Comments can persist after line deleted

Given a file with multiple lines like the following

blame commit	line number	code
a12345	1	#!/usr/bin/env bash
a12345	2	function foo() {
a12345	3	echo "bar"
a12345	4	}
a12345	5
a12345	6	echo "done"

if we leave a comment on line two, it is stored with the treeish a12345 and line number 2.

if we then delete lines 2-5, commit, and ask the client to display comments it will request comments for treeish a12345 on this file, and will be passed one for line 2.

Line two will now be the blank line above echo "done". It's a valid line, but it's not at all what we had left a comment on.

As such i think an additional piece of information needs to be recorded for each comment:

The sha256 hash of the line itself. We should probably generate the hash after stripping leading and trailing whitespace.

In that scenario, if there were still lines in the file that were last touch in the same commit as the line the comment was left on originally, then the comment will be returned, and while it's treeish won't match, the hash of the lines content would.

The plugin would then be making the following check on each returned comment.

Does the specified line number have the same hash of its contents? If so, show it. If not, don't.

Consequences:

If the line is from the same commit as before it is still NOT guaranteed to have the same hash (of the line's contents). The example above demonstrates that. It would have the same commit hash, but a different content hash.

If it's from a different commit it may still have the same hash if there were changes to the leading / trailing white-space. In which case, it's close enough and should still be displayed.

If the hash of the line's content is different then we can infer that it's from a different commit and thus we have 2 reasons this comment shouldn't be displayed.

Breakdown

The file may have been edited since the comment was left.

• the line commented on may still exist, but may have a different line number.
• the line commented on may have been edited and have a new treeish
  • IF the changes were only to trailing / leading whitespace we should still display it
• there may be 2 comments for the same line number &
  • one may have a matching treeish & line hash
  • neither has a matching treeish or line hash
  • neither has a matching treeish but 1 has a matching line hash
  • neither has a matching treeish and BOTH have a matching line hash

Because of the last case the server will now be storing a new entry in the json with saved_at + <epoch time of save in seconds>. Much easier than asking git when it was saved because responses don't involve querying git so much as just returning a json file found on disk.

Task

• add support for line_hash parameter
• guarantee said parameter is always returned
  • existing users will have old comments that won't have this. In those cases it should exist in the response, but with a null value.

New Tests

I think we need a test that actually sets up a repo adds then and edits a file.

• new repo
• add and commit test file above
• comment on line 2
• delete lines 2-5
• confirm that comment isn't displayed on the new line 2 (formerly line 6)
• comment on the new line 2
• make a whitespace change to line 2 (leading or trailing)
• confirm that the comment is still displayed on line 2
  • explanation:
    • when the comment on the new line 2 was left, line two came from the initial commit and thus had its treeish.
    • After editing it has a new commit, but the 1st line still has the treeish from the inital commit
    • because line 1 is still from the same commit that the last comment was left against the comment will be returned
    • line two comes from a new treeish (which has no comments) BUT has a line_hash and line_number that matches the comment that was returned.

Related Notes

Unfortunately this will necessitate updating the plugins. On the upside it won't be a breaking change.

The server should already be storing additional fields it's not prepared for, so an old server would work with a new plugin.

The question is should a new server accept input from an old plugin. I think, that since we control the plugins, and the server isn't auto-updating, we should make the new server require the additional new hash. Maybe we should have the plugins start sending in an API version number too.

What happens if a different user connects to my running private comments server?

Is such a scenario possible?

Also release `private_comments` as a 64-bit linux binary

Would be useful to avoid having to build from source for 64-bit machines.

Thanks!

better collision handling

In a shared environment it's possible to have 2 people attempt to leave a comment on the same line of the same file.

To my knowledge there are no teams using this so fixing this is going to be low priority. Holler if you're using / want to use PC in a team environment.

Problem 1

There's a potential race condition here where person A leaves a comment, but person B has not yet received it, or has simply opened the file before the comment was added. There are a variety of reasons this could happen depending on how data is being shared.

In this scenario the PC server's repo is updated, but person B's plugin hasn't registered the change by the time they decide to leave a comment.

B leaves a comment, submits it to PC.

At this point i think PC will simply clobber person A's comment. I don't think there will be a conflict because it will probably just be considered an edit.

Problem 2

B makes their change, but then A's earlier commit is downloaded to B's system. We then have a merge conflict.

Solutions

I think the solution to Problem 2 is to have PC recognize that there's a merge conflict and take both changes. HOWEVER these will be conflicting JSON objects so data processing has to happen to make sure it remains valid JSON and that only the actual comment portion has both authors stuff in it.

I think the solution to Problem 1 is to check who the author is on the file we're about to replace ( "edit" ) and see if it's the same as the person who just submitted a comment. If they're the same just take the new one (as per usual). If they're different, merge the data, provide attribution.

In both cases the full comment should be returned to the submitter.

Side Effects

After submitting a comment the full comment must be returned to the submitter. The plugin should then show the contents of what came back, not what was submitted, because there may now be additional information. In practice they'll probably want to show what was submitted, until the response comes back and then replace it, although usually it'll be the same data.

add ability to generate a file with the comments inlined.

I'm thinking something like

pc -f /path/to/file.rb -g

g for "generate"

this would build a version of the file with the comments inserted inline.

open questions:

• should they be commented using the single line comment format of the language of the given file?
  • if so, what about files with unknown language extensions?
• should they be before the line they are commenting on or after? I'm leaning towards before.

[high] sometimes comment files end up staged but not committed

i tried deleting a comment and got this error. The file was created with v1.2.4

error: the following file has changes staged in the index:
    19e5df/c950fe18f67af13fb7ef0dc55a7d0427dd00dfbf9bb6d764a4a7d55f37f6f715-542.json
(use --cached to keep the file, or -f to force removal)

investigation reveals that the problem is that git config --get init.templatedir is defined and i have a pre-commit hook in there that checks for "bad" words. In this case i left a comment with "TODO" which is a bad word which prevents a commit from happening.

Any pre-commit hook is potentially problematic because it could prevent the commit from succeeding and private comments doesn't currently check the exit status and error appropriately.

500 error should be JSON or HTML based on requested mime type

Right now the 500 error is always HTML which is ... kinda 🤦🏻‍♀️ considering most all of the calls are expecting JSON.

• Convert it to JSON when that's the requested mime type
• standardize & document the JSON returned by errors so that plugins can leverage the error messages.

test git author recording

in the test_everything file:

• test the server records the git commit with the user specified in the request
• test that when no user is specified in the request an unknown user is specified instead of the default user on the system.

Note: I don't think either of these are actually implemented in private_comments.scm please create a new ticket for whatever needs to be done.

pc & private_comments will allow you to comment on uncommitted lines

I'm not entirely sure how this is happening BUT the resulting treeish-dir is 00000000

For example

~/.config/private_comments/<project_hash>/00000000/<file_hash>-<line_no>.json

fixes:

1. pc should be examining the git blame (or doing so better) and confirming that there's a legit treeish for the current line
2. private_comments should a) do the same and b) reject 00000000 as a treeish (in all cases).

I'm calling this a bug despite it being somewhat useful, because when you DO commit the line the comment will disappear from view because the line will have a different treeish.

The build scripts should not require `sudo` to install

Following build instructions from the README.md, trying to build private_comments for 64-bit linux, I noticed that the build scripts want to install into paths that are typically not writable for a regular user.

It would be nice for that requirement to be removed, it does not seem necessary.

I could sudo run the install scripts, but that seems like an overreach.

improper handling of deletion of staged files [edge case]

I've encountered a problem where a file has been staged, but not committed. This is a significant edge case that happened as a result of pre-commit hooks that got embedded into the private comments repo as a result of a templatedir line in the global git config. When the new project's pc repo was created everything in that templatedir got added to the .git dir of the new comments repo. The pre-commit failed for reasons that aren't relevant here. I think very few people are even aware of the templatedir directive, never-mind actually use it. And even if it is used, it would also need to contain pre-commit hooks that fail on one of our json files.

If you've got PC server running you'll see something like the following when you try and delete the staged comment. I did C-c C-d in emacs.

Error

error: the following file has changes staged in the index:
    93569b7b/04c515dfbde50ac08e5d6906e1ba50a315d70b53322f98c572fcec3ff2989461-30.json
(use --cached to keep the file, or -f to force removal)

Proposed Fix

always rm with -f because loosing that data is precisely what the user is asking for.

Given the embedded treeish in the file name this isn't a situation where we're deleting an old comment. We're deleting the current comment on the current line shown so there'd never be the possibility of seeing it if we took the alternate approach of committing the staged file and then git rming it.

guarantee there are no pre-commit hooks

issue #14 came into existence because of failing pre-commit hooks that were created via a templatedir git config directive.

As a general statement there should never be pre-commit hooks running in the private_comments repos, but templatedir usage is valid, and we shouldn't force users to not use it.

Instead, when a new project repo is created, we should check if there is an executable .git/pre-commit file if so we should remove its executable bit.

Additionally there should be a PRIVATE_COMMENT_ALLOW_PRE_COMMIT environment variable that is checked. If it's set to true this behavior should be skipped.

Testing

1. define a templatedir that contains an executable at hooks/pre-commit & confirm that initializing a new repo on your computer adds that file at .git/hooks/pre-commit and that it's executable (ls -lash .git/hooks/pre-commit)
2. leave a comment on a repo you haven't commented on before.
3. check if the new repo has the expected copy of that pre-commit file at .git/hooks/pre-commit. If not, ... something's wonky that probably isn't anything to do with PC
4. confirm that the file in the new PC repo is not executable, but is readable and writable. chmod 660 is what should have been run on it.
5. repeat steps 2 & 3 but this time set the PRIVATE_COMMENTS_ALLOW_PRE_COMMIT environment variable to true before starting the private_comments server and confirm that the pre-commit remains executable.

remove project name and hash from display output

I forgot that the Vim plugin bootstraps itself with the pc client. I don't think this breaks anything but it's definitely an inferior experience.

Instead, create a new command line option. I'm thinking -x. That will provide the project name, project hash, and the metadata for each comment.

That can then be the reference tool for debugging new plugins.

enable logging of requests

allow the server to be launched in a logging mode.

in this mode it would log the JSON for all requests to a log file, along with a timestamp. I'm sure there's some already existing standard for storing server request logs in JSON format. If so, we should just follow that.

I'm imagining options to enable this like

private_comments --log-to foo/bar/baz.log

the first would log to the default location. the 2nd would log to wherever you told it to.

For bonus points we could add a new request like status that'd toggle logging.

Ability to list all comments, even if place where they were made is gone

Love your app, but inability to check comments I wrote before without playing with revisions is very inconvenient for me. Is it possible to add something like list of the all comments for file/project?
I use Private Comments with Emacs, if it matters

status page should provide app info & be verified by client

This is a 2 parter:

1. the private_comments server's status page (/status) should have additional fields which indicate the name and version of the app.
2. the pc client should verify that the JSON coming back is actually from a private comments and that it's from a version that the client is compatible with.

Add a Plugin for VSCode

Following the documentation add a plugin (in a separate repo) for VS Code that speaks to this server ( Simple JSON API calls), allows users to display, edit, and delete comments provided by the server.

If you're comfortable with Lisp the emacs plugin may be a useful guide, but it'll probably be more straightforward to just read the docs and implement it without looking at other codebases.

NOTE: this would be created in your own separate repo. not in this one. ;)

pc should have option to ignore errors from files not known to git

Sometimes you don't care that the file isn't known to git. This is frequently the case with a plugin that may not be checking.

We should add a new option that indicates that pc should not return an exit status indicating failure if the only problem is that the file isn't known to git.

Instead it should return the same thing as a request for a file known to git with no comments in it.

maybe -q && --quiet ? I'm not sure if "quiet" is a good thing because its a word whose meaning might expand over time. -g & --gitless ? dunno, that seems worse. I'm open to suggestions.