Check tests work on master, and assess PR: 68

[agstephens]

Check tests work on master, and assess PR: 68

[alaniwi]

What I tried:

on an Ubuntu box:

Testing ESGF/master

• install as follows:
  • create and activate a Python 3 virtual env, clone the repo, then in master do pip install --ugrade pip, pip install -r requirements.txt, pip install .
• test using: cd tests and py.test
• many tests fail, for example:

============================================= FAILURES =============================================
_________________________________ TestConnection.test_blank_query __________________________________
Traceback (most recent call last):
  File "/path/to/esgf-pyclient/tests/test_connection.py", line 24, in test_blank_query
    json = conn.send_search({})
  File "/tmp/venv/lib/python3.8/site-packages/pyesgf/search/connection.py", line 156, in send_search
    self.open()
  File "/tmp/venv/lib/python3.8/site-packages/pyesgf/search/connection.py", line 98, in open
    self._passed_session, requests_cache.core.CachedSession)):
AttributeError: module 'requests_cache' has no attribute 'core'
_____________________________ TestConnection.test_connection_instance ______________________________
Traceback (most recent call last):
  File "/path/to/esgf-pyclient/tests/test_connection.py", line 80, in test_connection_instance
    session = requests_cache.core.CachedSession(self.cache,
AttributeError: module 'requests_cache' has no attribute 'core'

(a path munged above for privacy)

Overall: 35 failed, 13 passed, 12 skipped, 1 warning in 1.06s

Testing the larsbuntemeyer/requests_cache branch

• Then in the clone, add the remote https://github.com/larsbuntemeyer/esgf-pyclient . Check out the requests_cache from this remote and rerun the installation and the tests

• This time: most pass: 2 failed, 46 passed, 12 skipped, 1 warning in 336.30s (0:05:36)

Remaining failures are in: TestConnection.test_get_shard_list and TestResults.test_batch_size_has_no_impact_on_results. (There was also a failure in TestResults.test_shards_constrain4 due to timeout from esgf-node.llnl.gov, which was not repeatable when rerun.)

Both of these remaining failures were included in the 35 tests that were failing with module 'requests_cache' has no attribute 'core' in master (i.e. tests which were not being run properly in master).

Of these:

• The failure in TestConnection.test_get_shard_list is an artefact of a previous wrong change - in this commit, line 46 of tests/test_connection.py (change is not valid because replica shards at CEDA live on esgf-index2, hence only one shard on esgf-index1 and the assertion fails). We can separately revert this line. Failing at:

  File "/path/to/esgf-pyclient/tests/test_connection.py", line 46, in test_get_shard_list
    assert len(shards['esgf-index1.ceda.ac.uk']) > 1
AssertionError: assert 1 > 1

• The failure in test_batch_size_has_no_impact_on_results may be a genuine failure that we still need to fix. At least it is being tested properly now. Fails at:

Traceback (most recent call last):
  File "/path/to/esgf-pyclient/tests/test_results.py", line 256, in test_batch_size_has_no_impact_on_results
    assert len(ids_batch_size_50) == len(ids_batch_size_100)
AssertionError: assert 50 == 100

Conclusion

Recommend merge (and also revert Carsten's change to line 46 of tests/test_connection.py)

[agstephens]

Leaving open until tests have been addressed, tagged: @alaniwi

[valeriupredoi]

the install and test procedure is now changed to the modern conda+mamba env creation, pip install and pytest:

• mamba env create -n esgf-pyclient -f environment.yml
• pip install -e .[develop]
• pytest

There are number of tests that fail from test_results.py module, and I have #98 open for that and a number of transitory fails due to ESGF servers' 503's that I opened #99 for, all other tests pass, and this can now be closed 👍