Build fails

[robguinness]

Build fails

[GMishx]

Please check with #46

[robguinness]

Hi @GMishx ,

Thanks. I checked out your branch, and when I run python setup.py build I get a different error:

running build_deps
Traceback (most recent call last):
  File "atarashi/build_deps.py", line 30, in <module>
    from atarashi.libs.ngram import createNgrams
  File "/home/rob/projects/forks/atarashi/atarashi/../atarashi/libs/ngram.py", line 27, in <module>
    from tqdm import tqdm
ModuleNotFoundError: No module named 'tqdm'
Traceback (most recent call last):
  File "setup.py", line 189, in <module>
    setup(**metadata)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 137, in run
    self.run_command('build_deps')
  File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 126, in run
    ], check = True)
  File "/usr/lib/python3.7/subprocess.py", line 468, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/rob/projects/forks/atarashi/.venv/bin/python', 'atarashi/build_deps.py', '-t', '4']' returned non-zero exit status 1.

However, if I install the requirements first (pip install -r requirements.txt), then it builds successfully.

[GMishx]

Before building Atarashi, you need to install the dependencies. You can install either through requirements.txt or by running sudo python3 setup.py install_deps

I forgot to add it to the README

[mcjaeger]

I think it has been added to the readme @robguinness if you would like to help us, you could consider confirming it is working and we could merge the PR.

[robguinness]

There is a mention of pip install -r requirements.txt, but only under the heading "Installing just dependencies". In any case, I can confirm that @GMishx's PR solves the initial problem related to this issue, so I think it can be merged and this issue closed. I could open a separate PR about the documentation issue, if you'd like.

[robguinness]

Sorry, I'm actually getting another issue now when trying to build:

Successfully installed numpy-1.15.4 pandas-0.23.4 python-dateutil-2.7.5 pytz-2018.9 setuptools-40.6.3 six-1.12.0 tqdm-4.29.0
Obtaining code_comment from git+https://github.com/amanjain97/code_comment#egg=code_comment
  Updating ./.venv/src/code-comment clone
Installing collected packages: code-comment
  Found existing installation: code-comment 0.1.0
    Not uninstalling code-comment at /home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages, outside environment /home/rob/projects/forks/atarashi/.venv
    Can't uninstall 'code-comment'. No files were found to uninstall.
  Running setup.py develop for code-comment
    Complete output from command /home/rob/projects/forks/atarashi/.venv/bin/python -c "import setuptools, tokenize;__file__='/home/rob/projects/forks/atarashi/.venv/src/code-comment/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps --user --prefix=:
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help
    
    error: option --user not recognized
    
    ----------------------------------------
  Can't roll back code-comment; was not uninstalled
Command "/home/rob/projects/forks/atarashi/.venv/bin/python -c "import setuptools, tokenize;__file__='/home/rob/projects/forks/atarashi/.venv/src/code-comment/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps --user --prefix=" failed with error code 1 in /home/rob/projects/forks/atarashi/.venv/src/code-comment/
Traceback (most recent call last):
  File "setup.py", line 187, in <module>
    setup(**metadata)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/setuptools/__init__.py", line 143, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.7/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 134, in run
    self.run_command('install_deps')
  File "/usr/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "setup.py", line 92, in run
    subprocess.run(install_options + ['-e', package], check = True)
  File "/usr/lib/python3.7/subprocess.py", line 468, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/rob/projects/forks/atarashi/.venv/bin/python', '-m', 'pip', 'install', '--upgrade', '--ignore-installed', '--user', '-e', 'git+https://github.com/amanjain97/code_comment#egg=code_comment']' returned non-zero exit status 1.

It seems this is a similar issue as above, except now it is related to building the dependency code_comment. Any ideas on how to resolve that?

[GMishx]

I have opened another PR #48. It implements PEP 518 to install dependencies. Can you please test it @robguinness? Hope this will work for you.

[robguinness]

Great! I will test this tomorrow, as it's getting late here.

[robguinness]

The installation goes through with no errors, but when I try to run the program, e.g.:

$ atarashi -a DLD data/file.c

I get a FileNotFoundError error. Here is the traceback:

Traceback (most recent call last):
  File "/home/rob/projects/forks/atarashi/.venv/bin/atarashi", line 11, in <module>
    load_entry_point('atarashi==0.0.9', 'console_scripts', 'atarashi')()
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/atarashi/atarashii.py", line 122, in main
    result = atarashii_runner(inputFile, processedLicense, agent_name, similarity, ngram_json, verbose)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/atarashi/atarashii.py", line 59, in atarashii_runner
    scanner = DameruLevenDist(processedLicense)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/atarashi/agents/atarashiAgent.py", line 36, in __init__
    self.licenseList = LicenseLoader.fetch_licenses(licenseList)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/atarashi/license/licenseLoader.py", line 37, in fetch_licenses
    licenseDataFrame = pd.read_csv(licenseList)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 678, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 440, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 787, in __init__
    self._make_engine(self.engine)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 1014, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/pandas/io/parsers.py", line 1708, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 384, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 695, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: File b'/home/rob/projects/forks/atarashi/.venv/lib/python3.7/site-packages/licenses/processedLicenses.csv' does not exist

So it seems that somewhere the path to the license files is configured incorrectly, since these files are not copied to site-packages.

[GMishx]

😨 I anticipated this but it did not occurred with me.

Anyway, let me check.

[GMishx]

Hey, have you tried the new commit I pushed to #48? It worked for me in a virtual environment and as a standard install as well.

[robguinness]

I get a new error:

running install_data
creating /data
error: could not create '/data': Permission denied

[GMishx]

Thank you for your patience @robguinness !
I have changed the location of non-python files and it should be working now. Can you please check once again?

[robguinness]

Hi! Sorry for the delays. I got pulled in other directions. I'm trying to get back to this work this week, and I will update you when I've tried it. Thanks for YOUR patience! ;-)