sarcasm/run-clang-format

A wrapper script around clang-format, suitable for linting multiple files and to use for continuous integration

run-clang-format.py

Lint files and directories with clang-format

Introduction

A wrapper script around clang-format, suitable for linting multiple files and to use for continuous integration.

This is an alternative API for the clang-format command line. It runs over multiple files and directories in parallel. A diff output is produced and a sensible exit code is returned.

How to use?

Copy run-clang-format.py in your project, then run it recursively on directories, or specific files:

./run-clang-format.py -r src include foo.cpp

It's possible to exclude paths from the recursive search:

./run-clang-format.py -r \
    --exclude src/third_party \
    --exclude '*_test.cpp' \
    src include foo.cpp

These exclude rules can be put in a .clang-format-ignore file, which also supports comments.

An example configuration is available in this repo:

$ cat .clang-format-ignore
# ignore third_party code from clang-format checks
src/third_party/*

Continuous integration

Check .travis.yml.

For an example of failure in logs, click the badge (build is broken on purpose):

FAQ

Can I check only changed files?

No, and this is what this repository was initially about. However, once working around a few shortcommings of git clang-format, I opted to try an alternative strategy which expects the whole project to be correctly formatted.

It would make sense to support this feature as well, so that the coding style does not need to be enforced but merely suggested.

run-clang-format's Issues

Support cmake-format

It maybe not directly related, but usually people check native code with clang-format (this script) and cmake files with cmake-format. The tools have relatively similar interface and both of them don't support scanning folders :) While this scripts solves is for clang-format, it would be nice to have the similar wrapper on cmake-format.

Possible options:

• Completely different script/repo
• Integrate the possibility to run both tools from this script

I think latter will be useful for a lot of people, so there won't be the need to different wrappers and there can be one to rule them all.

Failure when run with python 3.5

Hello, I would love to use this tools as it does exactly what I want. But when running it on my CI builds I'm getting the following error:

run-clang-format.py: error: OpenEast/include/Platform/Win32/Win32_Registry.hpp: TypeError: init() got an unexpected keyword argument 'encoding'
Traceback (most recent call last):
File "/home/travis/build/AMS21/OpenEast/deps/run-clang-format/run-clang-format.py", line 118, in run_clang_format_diff_wrapper
ret = run_clang_format_diff(args, file)
File "/home/travis/build/AMS21/OpenEast/deps/run-clang-format/run-clang-format.py", line 163, in run_clang_format_diff
**encoding_py3)
TypeError: init() got an unexpected keyword argument 'encoding'

Full log

Not sure what to make of this and if it even has something todo with my Win32_Registry.hpp file. Could also be possible that I miss configured something.
Install script
Run script

Hopefully you'll be able to help me.
Kind regard André

Create a pypi package?

Hi!
Thanks a lot for writing this script. Would you mind creating a Pypi package for it? Since many people are already using conan, it would make installing this script easier as well.

Please let me know if you want me to create a PR.

Seems like "clang-format --dry-run -Werror foo.cpp" basically already does exactly this?

Seems like "clang-format --dry-run -Werror foo.cpp" basically already does exactly this?

Run clang-format in parallel:

$ find . -iname "*.cpp" -or -iname "*.h" | xargs -P0 -I{} clang-format --dry-run -Werror {}

If any file is not formatted:

$ echo $?
123

Ambiguous error output when clang-format executable doesn't exist

If you run the application with the optional --clang-format-executable and the executable doesn't exist you get the error message run-clang-format.py: error: [Errno 2] No such file or directory. While this is technically true, it does not truly represent what is wrong with the application.

If the application is run on a directory of files a message will appear for every file:

$ ls ./src
foo.cpp  foo2.cpp
$ python run-clang-format.py ./src -r --clang-format-executable clang-format-wrong
run-clang-format.py: error: [Errno 2] No such file or directory
run-clang-format.py: error: [Errno 2] No such file or directory

This could lead people to believe that there's something wrong with the file structure or source-file-finding function of the application instead of a missing package, or executable. This is especially annoying to debug when running on CI with limited resources.

Checking that the executable is present (with a --version call, or something) and having a relevant error message for this specific error would fix the issue.

---

Thank you for the excellent application. I'm surprised that this isn't done by the LLVM team, given how ubiquitous the use case is.

Error when using style argument in the form {key: value}

Because on how extend works on list in python, we can't use style argument like this:
"{BasedOnStyle: Microsoft,UseTab: Always,ColumnLimit: 180,Language: Cpp}"

A simple fix would be to change line 140 from:

    if args.style:
        invocation.extend(['--style', args.style])

to

    if args.style:
        invocation.extend(['--style=' + args.style])

Thanks

Doesn't work on Win32 platforms

I'm excited about this as a potential feature, but it doesn't work on my Windows desktop. The use of signal.signal(signal.SIGPIPE, signal.SIG_DFL) crashes Python when I try to run. I was able to get past this by commenting out that line. However, the readlines method appears to produce a list of lines terminated by \n (this is somewhat confusing to me, since the file seems to have \r\n terminators) while the output of the clang-format process on windows produces output with \r\n. The python difflib then marks everything as changed because of the line terminators.

It would be preferable if the python code ensured that the two sets of lines had consistent line terminators, regardless of the output of readlines or clang-format

Create package for pypi

This is an awesome tool and it'd be nice if this was a pip-installable from https://pypi.org/.

How to modify the .clang-format file to use Google Code Style

Modify the configuration file：

BasedOnStyle: Google

How to make the modified configuration file take effect

Formatting only diffed files

I saw in the readme that a goal was to be able to format only changed files. Here is a snippet :

import os
import subprocess

cmd = ['git', 'diff-index', '--name-only', 'HEAD']
output = subprocess.Popen(cmd, stdout=subprocess.PIPE).communicate()[0]
files = iter(output.splitlines())

You can then loop through this output to get each file.

It may not be the most elegant way but it does work. You can also check if there are changed files with "git diff-index --quiet HEAD"

I can supply a patch if this seems ok. : )

running script fails with "UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 111: ordinal not in range(128)"

Only python2.7, introduced by commit 01038d3

Work-around is to use python3

OS:
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

Python:
python --version
Python 2.7.12

git clone --depth 1 https://github.com/Sarcasm/run-clang-format.git run_clang_format
Cloning into 'run_clang_format'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 10 (delta 0), reused 6 (delta 0), pack-reused 0
Unpacking objects: 100% (10/10), done.
Checking connectivity... done.
+ ./run_clang_format/run-clang-format.py -r <redacted>
Traceback (most recent call last):
  File "./run_clang_format/run-clang-format.py", line 253, in <module>
    sys.exit(main())
  File "./run_clang_format/run-clang-format.py", line 230, in main
    outs, errs = it.next()
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 668, in next
    raise value
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 111: ordinal not in range(128)
```

Add in-place reformat

Clang-format allows to do in-place reformatting with -i option. Would be useful to have the same in the script as well.

Example - using in git pre-commit hook to auto-format all the files. Probably it should still exit with non-zero exit code if at least one file was reformatted though.

UnicodeDecodeError on files with unicode strings

Running the script on this source file will error:

int main()
{
	auto t = u8"遊";
	return 0;
}

Error:

run-clang-format.py: error: test.cpp: UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 32: character maps to <undefined>
Traceback (most recent call last):
  File "X:\run-clang-format.py", line 93, in run_clang_format_diff_wrapper
    ret = run_clang_format_diff(args, file)
  File "X:\run-clang-format.py", line 126, in run_clang_format_diff
    outs = list(proc_stdout.readlines())
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 32: character maps to <undefined>

Running clang-format on the file works fine.

Adding run-clang-format to PyPI

First of all, thanks for creating this great wrapper around clang-format, it makes running it locally and on CI much more easy and useful!

Now that clang-format is packaged in PyPI (https://pypi.org/project/clang-format/) would you consider adding this package to PyPI and configuring clang-format as a dependency? That way users could just pip install run-clang-format (or even better using pipx we could pipx install run-clang-format!), and be able to use this tool very easily.

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 102: invalid continuation byte

For some files script crashes with UTF-8 reading error.

For me it happened at least for files:

• https://github.com/wxWidgets/wxWidgets/blob/master/src/stc/scintilla/lexers/LexMatlab.cxx
• https://github.com/wxWidgets/wxWidgets/blob/master/src/stc/scintilla/lexers/LexCSS.cxx

Seems like the issue is in unicode symbols (like Vrána). Please handle them properly

Environment:
Docker python:3.7 image
Python 3.7

Getting error message when fetching clang-format version

When running the script I get the following error message:
run-clang-format.py: error: Command '[u'clang-format', '--version']' returned non-zero exit status 1`

I'm running python 2.7.5 and LLVM 3.4.2

When removing the return statement in line 309. The script will (of cause) continue and succeed.

Do not run diff when using -q

See #22. This issue can be seen when using -q as well. However it doesn't sound right, because without the need to show the diff there should be no need to open the files from Python side at all - just run clang-format and check it's exitcode probably.

Support for .clang-format-ignore

I stumbled upon 2 projects using a .clang-format-ignore file.
I'm not sure if there are many tools supporting that, but it could be appropriate for run-clang-format.

Mozilla / gecko-dev:

Pass the file to hg diff --exclude, or parses it manually when using git.

• example: https://github.com/mozilla/gecko-dev/blob/15a188107012c425e674aff3a4fbee79fd1bf5f5/.clang-format-ignore
• code: https://github.com/mozilla/gecko-dev/blob/2eae45b21475d741879a2a20bb6aff72742356a8/python/mozbuild/mozbuild/code-analysis/mach_commands.py#L1731-L1762

Facebook / yoga

Used by ???

• example: https://github.com/facebook/yoga/blob/cc27d851101b962518fac1dfbe0fc82dd893866c/.clang-format-ignore

Increase contrast on the last line of the screenshot in the README.md

I do like the screenshot given in https://github.com/Sarcasm/run-clang-format/blob/master/screenshot.png, but there's at least one instance of a user who didn't see the last line because it's dark grey on black background. Increasing the contrast (i.e. use lighter grey) a little would help making the message fully self-explanatory.

argument to point to .clang-format-ignore file

When using the script in a build server environment, it is a limitation that I need to place a .clang-format-ignore in the same directory as the python script.

I would be happy if I could get an input argument for the script that points to the .clang-format-ignore file. This way it would be possible to keep the script in a docker image and point to a .clang-format-ignore file in the version controlled repository.

Add summary info

I think it would be useful to have some summary of the execution in addition to exitcode, like

XXX files checked
Y of them are not formatted correctly / Everything is OK

With maybe even list of files having issues.

The use case is when I integrate the script i.e. in git pre-commit hook and I don't want to see the full diff, but I'd like to have some useful human-readable output, so developers know where are the issues

Is there any idea for maintaining a offical docker image

there is three docker image wrapper now.
https://hub.docker.com/search?q=clang-format-checker&type=image