Comments (3)
Hi @mechaman !
Could you try running the command make start_services
on your host? Some of the tests need Elasticsearch to be launched. Maybe this should be clearer in the docs.
from cosr-back.
Hey @sylvinus .
Thank you for replying :) So I did as you said and end up with only 1 error and 1 skipped.
tests/sparktests/test_plugin_linkgraph.py::test_spark_link_graph_parquet[0] FAILED
tests/sparktests/test_sources.py::test_spark_multiple_sources[0] PASSED
tests/testdata/html_newspaper_testcases/test_urls.txt SKIPPED
Perhaps you should also add make start_services
in the 3. Optional: Run the tests section before running make docker_test_coverage
. As for the test that has an error and the one that is skipped, any idea on how I could fix it?
Also, I seem to be having an issue when I run make import_local_data
. I have had to pip install
many modules and have come to a halt due to a particular one :
ImportError: No module named pyfaup.faup
Would you happen to know how to solve this issue as well?
Thanks for your patience.
- J
from cosr-back.
Great! Sorry for the lag, I'll try to be faster to reply now.
The pyfaup
error is a dependency we just added in the Dockerfile. If you do make docker_build
it will rebuild the container. It is also the reason test_spark_link_graph_parquet
is failing. I will think about options to warn users that their container is not up to date with the current code to avoid similar confusion in the future.
Making the services optional for the tests and adding a message is indeed a very good idea, thanks!
test_urls.txt
is not an actual test, we should actually exclude the whole testdata/ directory in test collection.
Thanks for your feedbacks, it really helps!
from cosr-back.
Related Issues (20)
- Structure of ES clusters HOT 9
- Questions on deployment HOT 1
- Spark-submit uses only 1 core. HOT 4
- Speed up Travis builds HOT 1
- Improve host-level PageRanks HOT 1
- Investigate MyHTML parser
- Add GDELT document source
- Add a Github document source HOT 2
- Add a Reddit data source
- Add Stack Overflow document source HOT 1
- Add new Malware/Phishing Blacklists HOT 1
- Advertising Lists HOT 2
- Add Makefile commands to save/load elasticsearch snapshots
- PageRank & other jobs: check if output directory already exists
- Integrate the new Common Crawl News dataset HOT 1
- Add docker-compose for the local tests
- Deduplicate URLs in backlinks plugin results
- Investigate annotating all the code with mypy
- Getting error running 'make virtualenv' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cosr-back.