fast translation error

[opencybersecurityalliance]

fast translation error

[subbyte]

Python 3.9.6 on macOS with pip list results:

Package                          Version
-------------------------------- -----------
adal                             1.2.7
antlr4-python3-runtime           4.8
anyio                            3.6.2
anytree                          2.8.0
appdirs                          1.4.4
appnope                          0.1.3
argon2-cffi                      21.3.0
argon2-cffi-bindings             21.2.0
asttokens                        2.2.1
attrs                            22.2.0
Babel                            2.12.1
backcall                         0.2.0
beautifulsoup4                   4.12.2
bleach                           6.0.0
boto3                            1.26.84
botocore                         1.29.111
cattrs                           22.2.0
certifi                          2022.12.7
cffi                             1.15.1
charset-normalizer               3.1.0
click                            8.1.3
colorama                         0.4.6
colorlog                         6.7.0
comm                             0.1.3
cpe                              1.2.1
cryptography                     39.0.2
debugpy                          1.6.7
decorator                        5.1.1
deepmerge                        1.1.0
defusedxml                       0.7.1
docker                           6.0.1
exceptiongroup                   1.1.1
executing                        1.2.0
fastjsonschema                   2.16.3
firepit                          2.3.15
Flask                            2.2.3
flatten-json                     0.1.13
idna                             3.4
ijson                            3.2.0.post0
importlib-metadata               6.3.0
ipykernel                        6.22.0
ipython                          8.12.0
ipython-genutils                 0.2.0
itsdangerous                     2.1.2
jedi                             0.18.2
Jinja2                           3.1.2
jmespath                         1.0.1
json-fix                         0.5.1
json5                            0.9.11
jsonmerge                        1.9.0
jsonschema                       4.17.3
jupyter_client                   8.1.0
jupyter_core                     5.3.0
jupyter-server                   1.23.6
jupyterlab                       3.5.3
jupyterlab-pygments              0.2.2
jupyterlab_server                2.15.2
kestrel-jupyter                  1.0.7
kestrel-lang                     1.5.12
lark                             1.1.5
lark-parser                      0.12.0
lxml                             4.9.2
MarkupSafe                       2.1.2
matplotlib-inline                0.1.6
mistune                          2.0.5
nbclassic                        0.5.5
nbclient                         0.7.3
nbconvert                        7.3.1
nbformat                         5.8.0
nest-asyncio                     1.5.6
notebook                         6.5.4
notebook_shim                    0.2.2
numpy                            1.24.2
packaging                        23.1
pandas                           2.0.0
pandocfilters                    1.5.0
parso                            0.8.3
pexpect                          4.8.0
pickleshare                      0.7.5
pip                              23.0.1
platformdirs                     3.2.0
prometheus-client                0.16.0
prompt-toolkit                   3.0.38
psutil                           5.9.4
ptyprocess                       0.7.0
pure-eval                        0.2.2
pyarrow                          11.0.0
pycparser                        2.21
Pygments                         2.15.0
PyJWT                            2.6.0
pyOpenSSL                        23.0.0
pyrsistent                       0.19.3
python-dateutil                  2.8.2
pytz                             2023.3
PyYAML                           6.0
pyzmq                            25.0.2
requests                         2.28.2
requests-cache                   1.0.1
requests-toolbelt                0.10.1
s3transfer                       0.6.0
Send2Trash                       1.8.0
setuptools                       67.6.1
simplejson                       3.19.1
six                              1.16.0
sniffio                          1.3.0
soupsieve                        2.4
stack-data                       0.6.2
stix-shifter                     4.6.2
stix-shifter-modules-elastic-ecs 4.6.2
stix-shifter-utils               4.6.2
stix2-matcher                    3.0.0
stix2-patterns                   1.3.2
stix2-validator                  3.0.2
tabulate                         0.9.0
terminado                        0.17.1
tinycss2                         1.2.1
tomli                            2.0.1
tornado                          6.2
traitlets                        5.9.0
typeguard                        3.0.2
typer                            0.7.0
typing_extensions                4.5.0
tzdata                           2023.3
ujson                            5.7.0
url-normalize                    1.4.3
urllib3                          1.26.15
wcwidth                          0.2.6
webencodings                     0.5.1
websocket-client                 1.5.1
Werkzeug                         2.2.3
wheel                            0.40.0
xmltodict                        0.13.0
zipp                             3.15.0

[pcoccoli]

I think the problem involves stix-shifter mappings that map multiple native result fields to the same STIX property; I will need to handle that in the ingest() function. Maybe combining the duplicate columns into 1 (taking the first non-null value).
I noticed the problem doesn't happen with stix-shifter 4.6.1 but it does with 4.6.2. I can't explain why though; the mappings in question don't seem to have changed between those versions.

[pcoccoli]

Fixed by 464f7d0