iscc/iscc-core

ISCC - Codec & Algorithms

iscc-core's Issues

Add chain code for polygon registry

ISCC-ID should depend on wallet-id of declaring party

The wallet-id of the declaring party should be mixed into the ISCC-ID (in a reversible manner) to support pre-assigments without race conditions.

Add tests for code in documentation strings (doctests)

Collapse whitespace in Text-Code generation algorithm

Add statistics to documentation site.

Improve clarity of toplevel api

update architecture illustrations

Support upcoming Semantic-Code in gen_iscc_code

Integration of new work from ISCC Schema v0.3.0

ISCC metadata should be a superset of standard NFT metadata

Add deterministic random code generator

Add sha256 support to Instance-Code via SubType

Add ISCC metadata signing spec/algo

publish source wheels

Pydantic import error

It seems that the package is incompatible with recent changes in Pydantic. I just installed iscc-core into a fresh Python 3.10 venv and then tried to import it. This was the result:

Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import iscc_core
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/xxx/proj/iscc/venv/lib/python3.10/site-packages/iscc_core/__init__.py", line 2, in <module>
    from iscc_core.options import core_opts, conformant_options
  File "/home/xxx/proj/iscc/venv/lib/python3.10/site-packages/iscc_core/options.py", line 15, in <module>
    from pydantic import BaseSettings, Field
  File "/home/xxx/proj/iscc/venv/lib/python3.10/site-packages/pydantic/__init__.py", line 363, in __getattr__
    return _getattr_migration(attr_name)
  File "/home/xxx/proj/iscc/venv/lib/python3.10/site-packages/pydantic/_migration.py", line 296, in wrapper
    raise PydanticImportError(
pydantic.errors.PydanticImportError: `BaseSettings` has been moved to the `pydantic-settings` package. See https://docs.pydantic.dev/2.5/migration/#basesettings-has-moved-to-pydantic-settings for more details.

For further information visit https://errors.pydantic.dev/2.5/u/import-error

There is currently no version restriction defined for the Pydantic dependency:

iscc-core/pyproject.toml

Line 50 in fb114bb

pydantic = {extras = ["dotenv"], version = "*"}

The packages that were automatically installed by pip are:

Package           Version
----------------- -------
annotated-types   0.6.0
base58            2.1.1
bitarray          2.8.5
blake3            0.3.4
data-url          1.0.7
iscc-core         1.0.5
jcs               0.2.1
loguru            0.7.2
more-itertools    10.1.0
pip               22.0.2
pybase64          1.3.1
pydantic          2.5.2
pydantic_core     2.14.5
setuptools        59.6.0
typing_extensions 4.9.0
uvarint           1.2.0
xxhash            3.4.1

The solution for this and similar issues is to specify more specific version requirements for dependencies.

Prevent non-deterministic JSON serialization

Additional dependencies required for Windows: Windows build failure (Microsoft Visual C++ 14.0 or greater is required)

I'll have to work on this my end, although I'm not able to be admin of this particular machine. Logging for anyone who may also happen upon this issue:

• pip 22.1.2
• Python 3.9.6

Running: python -m pip install iscc-core

Collecting iscc-core
  Using cached iscc-core-0.2.8.tar.gz (56 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting data-url<2.0,>=1.0
  Using cached data_url-1.0.3-py3-none-any.whl
Collecting bitarray-hardbyte
  Using cached bitarray_hardbyte-2.3.8-cp39-cp39-win_amd64.whl (103 kB)
Collecting blake3
  Using cached blake3-0.3.1-cp39-none-win_amd64.whl (193 kB)
Collecting jcs<0.3.0,>=0.2.0
  Using cached jcs-0.2.1-py3-none-any.whl (7.6 kB)
Collecting pybase64
  Using cached pybase64-1.2.2-cp39-cp39-win_amd64.whl (38 kB)
Collecting pydantic[dotenv]
  Using cached pydantic-1.9.1-cp39-cp39-win_amd64.whl (2.0 MB)
Collecting more-itertools
  Using cached more_itertools-8.13.0-py3-none-any.whl (51 kB)
Collecting xxhash
  Using cached xxhash-3.0.0-cp39-cp39-win_amd64.whl (29 kB)
Collecting uvarint<2.0,>=1.2
  Using cached uvarint-1.2.0-py3-none-any.whl (4.4 kB)
Collecting loguru<0.7,>=0.6
  Using cached loguru-0.6.0-py3-none-any.whl (58 kB)
Collecting bases<0.3,>=0.2
  Using cached bases-0.2.1-py3-none-any.whl (35 kB)
Collecting base58<3.0,>=2.1
  Using cached base58-2.1.1-py3-none-any.whl (5.6 kB)
Collecting typing-validation
  Using cached typing_validation-0.0.1.post7-py3-none-any.whl (9.5 kB)
Requirement already satisfied: typing-extensions in c:\users\spencer\apps\python\lib\site-packages (from bases<0.3,>=0.2->iscc-core) (4.1.1)
Requirement already satisfied: colorama>=0.3.4 in c:\users\spencer\apps\python\lib\site-packages (from loguru<0.7,>=0.6->iscc-core) (0.4.4)
Collecting win32-setctime>=1.0.0
  Using cached win32_setctime-1.1.0-py3-none-any.whl (3.6 kB)
Collecting python-dotenv>=0.10.4
  Using cached python_dotenv-0.20.0-py3-none-any.whl (17 kB)
Building wheels for collected packages: iscc-core
  Building wheel for iscc-core (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for iscc-core (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [75 lines of output]
      A setup.py file already exists. Using it.
      C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\iscc_core\cdc.pxd
        tree = Parsing.p_module(s, pxd, full_module_name)
      C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\iscc_core\minhash.pxd
        tree = Parsing.p_module(s, pxd, full_module_name)
      C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\iscc_core\simhash.pxd
        tree = Parsing.p_module(s, pxd, full_module_name)
      Compiling iscc_core/cdc.py because it changed.
      Compiling iscc_core/minhash.py because it changed.
      Compiling iscc_core/simhash.py because it changed.
      [1/3] Cythonizing iscc_core/cdc.py
      [2/3] Cythonizing iscc_core/minhash.py
      [3/3] Cythonizing iscc_core/simhash.py
      ************************************************************
      Succeeded to compile Cython/C accelerator modules :)       *
      ************************************************************
      running build
      running build_py
      creating C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build
      creating C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39
      creating C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\cdc.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\check.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\codec.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_content_audio.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_content_image.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_content_mixed.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_content_text.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_content_video.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_data.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_flake.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_instance.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\code_meta.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\conformance.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\constants.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\dct.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\iscc_code.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\iscc_id.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\minhash.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\models.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\options.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\simhash.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\utils.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\wtahash.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\__init__.py -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\cdc.c -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\cdc.pxd -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\data.json -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\minhash.c -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\minhash.pxd -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\simhash.c -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      copying iscc_core\simhash.pxd -> C:\Users\Spencer\AppData\Local\Temp\pip-install-g8z84mcf\iscc-core_94f9d89354664a52b747d9b61b700b0a\build\lib.win-amd64-cpython-39\iscc_core
      running build_ext
      building 'iscc_core.cdc' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      Traceback (most recent call last):
        File "C:\Users\Spencer\Apps\python\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
          main()
        File "C:\Users\Spencer\Apps\python\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "C:\Users\Spencer\Apps\python\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\poetry\core\masonry\api.py", line 68, in build_wheel
          return unicode(WheelBuilder.make_in(poetry, Path(wheel_directory)))
        File "C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 78, in make_in
          wb.build()
        File "C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 110, in build
          self._build(zip_file)
        File "C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 162, in _build
          self._run_build_command(setup)
        File "C:\Users\Spencer\AppData\Local\Temp\pip-build-env-wrk8753n\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 190, in _run_build_command
          subprocess.check_call(
        File "C:\Users\Spencer\Apps\python\lib\subprocess.py", line 373, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['C:/Users/Spencer/Apps/python/python.exe', 'C:\\Users\\Spencer\\AppData\\Local\\Temp\\pip-install-g8z84mcf\\iscc-core_94f9d89354664a52b747d9b61b700b0a\\setup.py', 'build', '-b', 'C:\\Users\\Spencer\\AppData\\Local\\Temp\\pip-install-g8z84mcf\\iscc-core_94f9d89354664a52b747d9b61b700b0a\\build']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for iscc-core
Failed to build iscc-core
ERROR: Could not build wheels for iscc-core, which is required to install pyproject.toml-based projects

Implementation of canonical ISCC metadata IPFS-CID hashing

Support for ISCC-CODE aggregation

A way to aggregate multiple ISCC-CODEs into a single ISCC-CODE.

Use Case:

• create a canonical ISCC-CODE for a collection of multiple files
• create a canonical ISCC-CODE for directory of files including subdirectories
• create a canonical ISCC-CODE for container formats that embed other files

see also: https://github.com/andhus/dirhash

Support JSON-LD triplet binding

Add and document timestamp for conformance test data.

Add decomposing similarity/distance function

Implement `explain` function

Add new algorithm for ISCC-SUM code type with 256-bit

ISCC metadata should support canonical serialization

Add examples to docstrings

Unified metadata schema

Combine return types of gen_* code generator functions into a single unified type.

Rename Meta-Code input `properties` to `metadata`.

Add URN representation

Clarify license

Finalize Meta-Code algorithm

Remove Schema related code (now in iscc-schema)

Update documentation and architecture illustration with new ISCC-UNIT

Digital asset description metadata should support markdown

Use rfc2397 - Data URL to support embedding of arbitrary metadata

ISCC metadata should have a json-ld context

Add ISCC string validation

Move Code class to separate module

Change meta-, data-, and top-hashes to multihashes

Create diagram to visualize Meta-Code

Accept chain id int/str/enum for gen_iscc_id_v0

Move documentation to core.iscc.codes

Complete documentation

• Review existing documentation
• Add glossary of terms

Optimize soft_hash_audio for codes shorter than 256-bit

ISCC metadata canonical hashing with IPFS and NFT tokon-id support

Define a canonical hashing scheme for ISCC metadata such that the resulting hash is a IPFS CIDv1 that can be used as token-id in ERC721 or ERC-1155 contracts. See also: https://forum.openzeppelin.com/t/how-to-erc-1155-id-substitution-for-token-uri/3312/14

Release v0.2.0 package on pypi