syrusakbary/fastdiff

A fast native implementation of diff algorithm with a pure Python fallback

fastdiff

fastdiff is a re-implementation of difflib in pure Rust compiled to WebAssembly to speedup different language integrations:

• Python: via Wasmer, falling back to difflib compare.

Install

Python

To install fastdiff in Python, you just need to do:

pip install fastdiff

And then, use it in Python like this:

from fastdiff import compare

str1 = 'hello\nwasm\n'
str2 = 'hello\npython\n'

print(compare(str1, str2))

Benchmarks

Python

When comparing strings with 300 lines, the WebAssembly based approach is about 100 times faster than the pure Python approach.

---------------------------------------------------------------------------------------------- benchmark: 3 tests ---------------------------------------------------------------------------------------------
Name (time in ms)                        Min                   Max                  Mean             StdDev                Median                IQR            Outliers      OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_content_native        47.0735 (1.0)         50.7082 (1.0)         47.4401 (1.0)       0.7944 (1.0)         47.1680 (1.0)       0.2133 (1.0)           2;2  21.0792 (1.0)          21           1
test_benchmark_compile_native     1,187.2395 (25.22)    1,305.0581 (25.74)    1,222.2197 (25.76)    48.3778 (60.90)    1,198.9380 (25.42)    51.4950 (241.44)        1;0   0.8182 (0.04)          5           1
test_benchmark_content_base       5,480.3568 (116.42)   5,623.0508 (110.89)   5,516.5008 (116.28)   60.7623 (76.49)    5,487.1852 (116.33)   57.0271 (267.38)        1;0   0.1813 (0.01)          5           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Building

For building the WebAssembly file, you need Rust and the wasm32 target.

# Install Rustup
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Add wasm32 target
rustup target add wasm32-unknown-unknown

And then, run:

make build

Testing

For testing in Python you can do:

make test_python

fastdiff's Issues

Differences / comparison between `testfixtures`?

testfixtures seems to have a wider scope and more flags/options

• https://github.com/simplistix/testfixtures
• https://testfixtures.readthedocs.io/en/latest/comparing.html#mapping-comparison-objects

Perhaps either for inspiration, collaboration, etc.

New Python release

Would it be possible to have a new patch release of the Python version which includes #6?

Install fails on Alpine linux with Python version 3.5 or higher

When trying to install fastdiff as a dependency for snapshottest, I get the following error:

Collecting wasmer; python_version >= "3.5" and platform_machine == "x86_64" and sys_platform == "linux" (from fastdiff>=0.1.4<1->snapshottest~=0.5.0->-r requirements.txt (line 19))
  Could not find a version that satisfies the requirement wasmer; python_version >= "3.5" and platform_machine == "x86_64" and sys_platform == "linux" (from fastdiff>=0.1.4<1->snapshottest~=0.5.0->-r requirements.txt (line 19)) (from versions: )
No matching distribution found for wasmer; python_version >= "3.5" and platform_machine == "x86_64" and sys_platform == "linux" (from fastdiff>=0.1.4<1->snapshottest~=0.5.0->-r requirements.txt (line 19))

For Alpine linux, it appears it can't compile wasmer (see syrusakbary/snapshottest#88) and it's causing an issue. I am also able to install snapshottest when using the Alpine image with Python 3.4.

test_fastdiff.test_native without wasmer fails on 0.2.0

Over on this build of the conda-forge feedstock for fastdiff, I've come upon:

============================= test session starts ==============================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.8.0, pluggy-0.12.0
benchmark: 3.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=5.00us max_time=1.00s calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: $SRC_DIR, inifile:
plugins: benchmark-3.0.0, cov-2.7.1
collected 6 items

tests/test_fastdiff.py ..F...                                            [100%]

=================================== FAILURES ===================================
_________________________________ test_native __________________________________

    def test_native():
        compared = compare_native("one\ntwo\nthree", "one\ntwo\nfour")
>       assert compared == ['  one', '  two', '- three', '+ four']
E       AssertionError: assert [] == ['  one', '  two', '- three', '+ four']
E         Right contains more items, first extra item: '  one'
E         Use -v to get the full diff

tests/test_fastdiff.py:31: AssertionError

I haven't had a chance to check it out locally, but will get to it when i have a chance... will at least turn the verbosity up...

`pytest-runner` is deprecated

https://github.com/pytest-dev/pytest-runner/#deprecation-notice

This cause snapshottest install failure on my machine without proxy because dependencies in setup_requires not using configured pip index mirror.