source-foundry/fdiff

An OpenType table diff tool for fonts. Based on the fontTools TTX format.

An OpenType table diff tool for fonts

About

fdiff is a Python command line comparison tool for assessment of granular differences in the OpenType table data between font files. The tool provides cross-platform support for local and remote font diffs on macOS, Windows, and GNU/Linux systems with a Python v3.7+ interpreter.

Looking for a high-level overview of OpenType table differences rather than low-level changes? Check out Just van Rossum's fbdiff tool.

What it does

• Takes two font file path arguments (or URL for remote fonts) for the font comparison
• Dumps OpenType table data in the fontTools library TTX format (XML)
• Compares the OpenType table data across the two files using the unified diff format with 3 lines of surrounding context

Optional Features

• Filter OpenType tables with the --include or --exclude options
• Modify the number of context lines displayed in the diff with the -l or --lines option
• Display the first n lines of the diff output with the --head option
• Display the last n lines of the diff output with the --tail option
• Execute the diff with an external diff tool using the --external option

Run fdiff --help to view all available options.

Contents

• Installation
• Usage
• Issues
• Contributing
• Acknowledgments
• License

Installation

fdiff requires a Python 3.7+ interpreter.

Installation in a Python3 virtual environment is recommended.

Use any of the following installation approaches:

pip install from PyPI

$ pip3 install fdiff

pip install from source

$ git clone https://github.com/source-foundry/fdiff.git
$ cd fdiff
$ pip3 install -r requirements.txt .

Developer install from source

The following approach installs the project and associated optional developer dependencies, so that source changes are available without the need for re-installation.

$ git clone https://github.com/source-foundry/fdiff.git
$ cd fdiff
$ pip3 install --ignore-installed -r requirements.txt -e ".[dev]"

Usage

Local font files

$ fdiff [OPTIONS] [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

Remote font files

fdiff supports GET requests for publicly accessible remote font files. Replace the file path arguments with URL:

$ fdiff [OPTIONS] [PRE-FONT FILE URL] [POST-FONT FILE URL]

fdiff works with any combination of local and remote font files. For example, to compare a local post font file with a remote pre font file to assess local changes against a font file that was previously pushed to a remote, use the following syntax:

$ fdiff [OPTIONS] [PRE-FONT FILE URL] [POST-FONT FILE FILE PATH]

⭐ Tip: Remote git repository hosting services (like Github) support access to files on different git branches by URL. Use these repository branch URL to compare fonts across git branches in your repository.

Options

Filter OpenType tables

To include only specified tables in your diff, use the --include option with a comma-separated list of table names:

$ fdiff --include head,post [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

To exclude specified tables in your diff, use the --exclude option with a comma-separated list of table names:

$ fdiff --exclude glyf,OS/2 [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

Do not include spaces between the comma-separated table name values!

Change number of context lines

To change the number of lines of context above/below lines that have differences, use the -l or --lines option with an integer value for the desired number of lines. The following command reduces the contextual information to a single line above and below lines with differences:

$ fdiff -l 1 [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

Display the first n lines of output

Use the --head option followed by an integer for the number of lines at the beginning of the output. For example, the following command displays the first 20 lines of the diff:

$ fdiff --head 20 [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

Display the last n lines of output

Use the --tail option followed by an integer for the number of lines at the end of the output. For example, the following command displays the last 20 lines of the diff:

$ fdiff --tail 20 [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

Use an external diff tool

Please Note: This feature has not been tested across all supported platforms. Please report any issues that you come across on the project issue tracker.

By default, fdiff performs diffs with Python source. If you run into performance issues with this approach, you can use compiled diff executables that are available on your platform. fdiff will dump the ttx files and run the command that you provide on the command line passing the pre and post font ttx dump file paths as the first and second positional arguments to your command.

For example, you may run the diff -u command on GNU/Linux or macOS like this:

$ fdiff --external="diff -u" [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

fdiff supports built-in color formatting and OpenType table filtering when used with external diff tools. The context line, head, and tail options are not supported with the use of external diff tools.

Disable color diffs

ANSI escape code colored diffs are performed by default in terminal environments.

To view a diff without ANSI escape codes in your terminal, include the --nocolor option in your command:

$ fdiff --nocolor [PRE-FONT FILE PATH] [POST-FONT FILE PATH]

Other Options

Use fdiff -h to view all available options.

Issues

Please report issues on the project issue tracker.

Contributing

Contributions are warmly welcomed. A development dependency environment can be installed in editable mode with the developer installation documentation above.

Please use the standard Github pull request approach to propose source changes.

Source file linting

Python source files are linted with flake8. See the Makefile test-lint target for details.

Testing

The project runs continuous integration testing on the GitHub Actions service with the pytest toolchain. Test modules are located in the tests directory of the repository.

Local testing by Python interpreter version can be performed with the following command executed from the root of the repository:

$ tox -e [PYTHON INTERPRETER VERSION]

Please see the tox documentation for additional details.

Test coverage

Unit test coverage is executed with the coverage tool. See the Makefile test-coverage target for details.

Acknowledgments

fdiff is built with the fantastic fontTools free software library and performs text diffs of binary font files using dumps of the TTX OpenType table data serialization format as defined in the fontTools library.

License

Copyright 2019 Source Foundry Authors and Contributors

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

fdiff's Issues

CI: Disable duplicate run on Push and PR

Annoyingly, both Travis and Appveyor must be configured to not run CI twice if you push to a PR branch on the main repo.

travis.yml and appveyor.yml:

# The following prevents Travis from running CI on pull requests that come from a
# branch in the same repository. Without this, it will run the same CI for the
# pull request branch _and_ the pull request itself, which makes no sense.
branches:
  only:
    - master
    # We want to build wip/* branches since these are not usually used for PRs
    - /^wip\/.*$/
    # We want to build version tags as well.
    - /^v\d+\.\d+.*$/

Add support for definition of OpenType tables to exclude in the diff

Add support for diffs that exclude user-specified OT tables in the font files. These tables will be defined using a command line argument.

This can be implemented throughfontTools.ttLib.ttFont.saveXML argument skipTables=[table defs]:

https://github.com/fonttools/fonttools/blob/cc096ccef080f9786878ac1898f18e0e7bfdffb8/Lib/fontTools/ttLib/ttFont.py#L217-L234

The 'skipTables' argument may be a list of tables
to skip, but only when the 'tables' argument is false.

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Add remote font file diff support

Support automated pulls of one (to compare with a local build) or two (for remote build vs. remote build comparisons) remote font files for TTX diffs

Default to external diff?

fdiff is great!

Internal python diff is way too slow on large strings. Maybe default to external unix diff command if available? That one is instantanous.

Add head support

Add support for filtering of diff output by first n lines of text

Consider ANSI escape code colored diff output as the default

ANSI color output is optional and requires a -c or --colorflag. It might be helpful to turn this on by default and detect when output is piped to another process (in which case the ANSI color codes should not be added to the strings).

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Add "quick diff" support with hash digest comparisons

Add support for cross git branch diffs

It would be helpful to be able to diff font files with the same file path locally across git branches rather than relying on remote pushes and comparisons with remote hosting service URL.

Minor: "# -*- coding: utf-8 -*-" not needed anymore for Python 3

Running pyupgrade --py36-plus will delete it.

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Add support for definition of OpenType tables to include in the diff

Add support for diffs that only include user-specified OT tables in the font files. These will be defined using a command line argument.

This can be implemented throughfontTools.ttLib.ttFont.saveXML argument tables=[table defs]:

https://github.com/fonttools/fonttools/blob/cc096ccef080f9786878ac1898f18e0e7bfdffb8/Lib/fontTools/ttLib/ttFont.py#L217-L234

The 'tables' argument must either be false (dump all tables) or a
list of tables to dump.

Add tail support

Add support for filtering of diff output by last n lines of text.

Diff containing "differences" for identical lines

I'm finding an issue where fdiff reports lines as being different, even though they aren't. In the fragment below, only the EndSize lines are actually different. I've included a "manual" diff fragment as well.

The full diff of this GPOS table is pretty large in both cases, but the fdiff one is a lot larger than the manual diff:

• fdiff: 84459 lines
• diff -u: 25151 lines

$ fdiff font_A.ttf font_B.ttf --include=GPOS:

--- font_A.ttf	2020-04-30T07:14:55.087104+02:00
+++ font_B.ttf	2020-05-03T22:51:25.100047+02:00
@@ -285,57 +285,57 @@
               <Value1 XAdvance="-72">
                 <XAdvDevice>
                   <StartSize value="8"/>
-                  <EndSize value="654"/>
-                  <DeltaFormat value="32768"/>
-                </XAdvDevice>
-              </Value1>
-            </PairValueRecord>
-            <PairValueRecord index="1">
-              <SecondGlyph value="g"/>
-              <Value1 XAdvance="-51">
-                <XAdvDevice>
-                  <StartSize value="8"/>
-                  <EndSize value="749"/>
-                  <DeltaFormat value="32768"/>
-                </XAdvDevice>
-              </Value1>
[ many lines snipped ]
+                  <EndSize value="656"/>
+                  <DeltaFormat value="32768"/>
+                </XAdvDevice>
+              </Value1>
+            </PairValueRecord>
+            <PairValueRecord index="1">
+              <SecondGlyph value="g"/>
+              <Value1 XAdvance="-51">
+                <XAdvDevice>
+                  <StartSize value="8"/>
+                  <EndSize value="751"/>
+                  <DeltaFormat value="32768"/>
+                </XAdvDevice>
+              </Value1>
[ many lines snipped ]

$ diff -u gpos_old.ttx gpos_new.ttx:

--- gpos_old.ttx	2020-05-04 06:39:29.000000000 +0200
+++ gpos_new.ttx	2020-05-04 06:39:45.000000000 +0200
@@ -285,7 +285,7 @@
               <Value1 XAdvance="-72">
                 <XAdvDevice>
                   <StartSize value="8"/>
-                  <EndSize value="654"/>
+                  <EndSize value="656"/>
                   <DeltaFormat value="32768"/>
                 </XAdvDevice>
               </Value1>
@@ -295,7 +295,7 @@
               <Value1 XAdvance="-51">
                 <XAdvDevice>
                   <StartSize value="8"/>
-                  <EndSize value="749"/>
+                  <EndSize value="751"/>
                   <DeltaFormat value="32768"/>
                 </XAdvDevice>
               </Value1>

I can't publicly share this font, though :(

Add summary view (?)

I just tried fdiff for the first time. Super easy to install and use! Really nice work.

I'm not sure whether this makes sense for you to add, but I thought I'd share something that I wish I could do right now: starting at a high level to understand where diffs might be in two fonts, and then "zooming in" to inspect those more closely. Basically, I wish fdiff have more of a summary view of some kind.

My use case is this: I changed the axis label in Recursive from "Proportion" to "Monospace," but tried to change nothing else. However, the one with the same axis called "Monospace" is about 1% bigger in filesize. This isn't very significant, but I'd still like to understand why the filesize changed. I guess that somehow, the label adjustment reoriented my GSUB features in such a way that affected the fvar deltas? However, there are too many changed lines for the fdiff view for me to make sense of.

"Before" font with PROP axis

"After" font with MONO axis

Here's a very rough, inaccurate sketch of what I'm imagining this summary view might be:

▶️ fdiff --summary font-betas/recursive-prop_xprn_weight_slnt_ital--2019_09_26.ttf font-betas/rec-mono_xprn_wght_slnt_ital--2019_09_26.ttf

BEFORE: recursive-prop_xprn_weight_slnt_ital--2019_09_26.ttf (595 KB)

AFTER:  rec-mono_xprn_wght_slnt_ital--2019_09_26.ttf (598 KB, +1.005%)

Diffs in tables:

  GSUB: -189 lines
        +213 lines

  gvar: -348 lines
        +549 lines

  name: -3 lines
        +3 lines

Basically, I think I just need enough information for me to make sense of what tables have changed, so that I can then use the --include option to look at one table at a time. Honestly, I'm not quite sure that this would get me all the way to understanding, or whether there needs to be some other view, like a "medium-level" view, or something I haven't yet imagined.

Of course, it's also possible that I'm trying to use this tool for something outside its intent and that you shouldn't try to satisfy this use case. Just thought I'd report my experience, in case it's helpful.

Add static type checks

Add type annotations to source + mypy based static type checks to CI testing

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Support external diff tools in place of the CPython difflib approach

The Python diff approach will be slow for large files. It would be helpful to be able to use diff -u and similar executables for the diff execution.