Comments (8)
It's probably because of your python version
from scrapydweb.
i second this. a fresh install like described in the docs is not working.
this is the error log after
pip install scrapydweb
scrapydweb
`(.venv) scrapy@:~/scrapydweb$ scrapydweb
[2022-12-21 23:31:53,636] INFO in apscheduler.scheduler: Scheduler started
[2022-12-21 23:31:53,649] INFO in scrapydweb.run: ScrapydWeb version: 1.4.0
[2022-12-21 23:31:53,649] INFO in scrapydweb.run: Use 'scrapydweb -h' to get help
[2022-12-21 23:31:53,649] INFO in scrapydweb.run: Main pid: 4523
[2022-12-21 23:31:53,649] DEBUG in scrapydweb.run: Loading default settings from /home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/scrapydweb/default_settings.py
Traceback (most recent call last):
File "/home/scrapy/scrapydweb/.venv/bin/scrapydweb", line 8, in
sys.exit(main())
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/scrapydweb/run.py", line 32, in main
app = create_app()
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/scrapydweb/init.py", line 81, in create_app
handle_db(app)
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/scrapydweb/init.py", line 123, in handle_db
db.create_all()
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/flask_sqlalchemy/extension.py", line 868, in create_all
self._call_for_binds(bind_key, "create_all")
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/flask_sqlalchemy/extension.py", line 839, in _call_for_binds
engine = self.engines[key]
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/flask_sqlalchemy/extension.py", line 628, in engines
app = current_app._get_current_object() # type: ignore[attr-defined]
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/werkzeug/local.py", line 513, in _get_current_object
raise RuntimeError(unbound_message) from None
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
the current application. To solve this, set up an application context
with app.app_context(). See the documentation for more information.
2022-12-21 23:31:53,668 INFO sqlalchemy.engine.Engine BEGIN (implicit)
[2022-12-21 23:31:53,668] INFO in sqlalchemy.engine.Engine: BEGIN (implicit)
2022-12-21 23:31:53,671 INFO sqlalchemy.engine.Engine SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?
[2022-12-21 23:31:53,671] INFO in sqlalchemy.engine.Engine: SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?
2022-12-21 23:31:53,672 INFO sqlalchemy.engine.Engine [generated in 0.00070s] ('1.4.0', 1, 0)
[2022-12-21 23:31:53,672] INFO in sqlalchemy.engine.Engine: [generated in 0.00070s] ('1.4.0', 1, 0)
2022-12-21 23:31:53,673 INFO sqlalchemy.engine.Engine ROLLBACK
[2022-12-21 23:31:53,673] INFO in sqlalchemy.engine.Engine: ROLLBACK
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
self.dialect.do_execute(
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: no such table: metadata
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/scrapydweb/common.py", line 85, in handle_metadata
metadata = Metadata.query.filter_by(version=version).first()
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2824, in first
return self.limit(1)._iter().first()
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 2916, in _iter
result = self.session.execute(
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1714, in execute
result = conn._execute_20(statement, params or {}, execution_options)
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
return connection._execute_clauseelement(
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
ret = self._execute_context(
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
self.handle_dbapi_exception(
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2124, in handle_dbapi_exception
util.raise(
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise
raise exception
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
self.dialect.do_execute(
File "/home/scrapy/scrapydweb/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: metadata
[SQL: SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?]
[parameters: ('1.4.0', 1, 0)]
(Background on this error at: https://sqlalche.me/e/14/e3q8)`
from scrapydweb.
Which python version are you using?
Check this:
#205
from scrapydweb.
Thanks eracle!
For anyone interested the solution is here: #205 (comment)
from scrapydweb.
Please have a try for the latest scrapydweb v1.4.1, which supports Python 3.6 to 3.9
from scrapydweb.
Fixed in scrapydweb v1.4.1
from scrapydweb.
在 scrapydweb v1.4.1 中修复
并没有,还是会报这个错:
[root@localhost scrapydweb]# scrapydweb
[2023-09-13 17:51:32,029] INFO in apscheduler.scheduler: Scheduler started
[2023-09-13 17:51:32,039] INFO in scrapydweb.run: ScrapydWeb version: 1.4.1
[2023-09-13 17:51:32,039] INFO in scrapydweb.run: Use 'scrapydweb -h' to get help
[2023-09-13 17:51:32,039] INFO in scrapydweb.run: Main pid: 445615
[2023-09-13 17:51:32,039] DEBUG in scrapydweb.run: Loading default settings from /usr/local/lib/python3.9/site-packages/scrapydweb-1.4.1-py3.9.egg/scrapydweb/default_settings.py
Traceback (most recent call last):
File "/usr/local/bin/scrapydweb", line 33, in
sys.exit(load_entry_point('scrapydweb==1.4.1', 'console_scripts', 'scrapydweb')())
File "/usr/local/lib/python3.9/site-packages/scrapydweb-1.4.1-py3.9.egg/scrapydweb/run.py", line 32, in main
app = create_app()
File "/usr/local/lib/python3.9/site-packages/scrapydweb-1.4.1-py3.9.egg/scrapydweb/init.py", line 81, in create_app
handle_db(app)
File "/usr/local/lib/python3.9/site-packages/scrapydweb-1.4.1-py3.9.egg/scrapydweb/init.py", line 123, in handle_db
db.create_all()
File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/extension.py", line 900, in create_all
self._call_for_binds(bind_key, "create_all")
File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/extension.py", line 871, in _call_for_binds
engine = self.engines[key]
File "/usr/local/lib/python3.9/site-packages/flask_sqlalchemy/extension.py", line 687, in engines
app = current_app._get_current_object() # type: ignore[attr-defined]
File "/usr/local/lib/python3.9/site-packages/werkzeug/local.py", line 508, in _get_current_object
raise RuntimeError(unbound_message) from None
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
the current application. To solve this, set up an application context
with app.app_context(). See the documentation for more information.
2023-09-13 17:51:32,057 INFO sqlalchemy.engine.Engine BEGIN (implicit)
[2023-09-13 17:51:32,057] INFO in sqlalchemy.engine.Engine: BEGIN (implicit)
2023-09-13 17:51:32,060 INFO sqlalchemy.engine.Engine SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?
[2023-09-13 17:51:32,060] INFO in sqlalchemy.engine.Engine: SELECT metadata.id AS metadata_id, metadata.version AS metadata_version, metadata.last_check_update_timestamp AS metadata_last_check_update_timestamp, metadata.main_pid AS metadata_main_pid, metadata.logparser_pid AS metadata_logparser_pid, metadata.poll_pid AS metadata_poll_pid, metadata.pageview AS metadata_pageview, metadata.url_scrapydweb AS metadata_url_scrapydweb, metadata.url_jobs AS metadata_url_jobs, metadata.url_schedule_task AS metadata_url_schedule_task, metadata.url_delete_task_result AS metadata_url_delete_task_result, metadata.username AS metadata_username, metadata.password AS metadata_password, metadata.scheduler_state AS metadata_scheduler_state, metadata.jobs_per_page AS metadata_jobs_per_page, metadata.tasks_per_page AS metadata_tasks_per_page, metadata.jobs_style AS metadata_jobs_style
FROM metadata
WHERE metadata.version = ?
LIMIT ? OFFSET ?
2023-09-13 17:51:32,060 INFO sqlalchemy.engine.Engine [generated in 0.00030s] ('1.4.1', 1, 0)
[2023-09-13 17:51:32,060] INFO in sqlalchemy.engine.Engine: [generated in 0.00030s] ('1.4.1', 1, 0)
2023-09-13 17:51:32,061 INFO sqlalchemy.engine.Engine ROLLBACK
[2023-09-13 17:51:32,061] INFO in sqlalchemy.engine.Engine: ROLLBACK
from scrapydweb.
Can you show me the result of “pip list”, or create a new virtual env and try again?
from scrapydweb.
Related Issues (20)
- Not able to see stats section of the job HOT 1
- scrapydweb failed to run on python 3.8 HOT 5
- 启动报错:sqlite3.OperationalError: no such table: metadata HOT 13
- Is it possible to run multiple spider at the same time in a tmux machine with scrapydweb automatically
- items Oops! Something went wrong. HOT 1
- APScheduler 3.10 causing 500 errors HOT 2
- How to Change Timezone of scrapydweb? HOT 3
- Which scrapyd image you use? HOT 2
- Clean install on clean Ubuntu VM. Whatever I do it is not working. HOT 2
- Docker compose scrapdweb with scrapyd the log url use docker name
- Processes dont stop after finishing HOT 1
- v1.4.1 submit cron job can't run HOT 1
- ('Connection aborted.', timeout('timed out',))
- ERROR: Package 'scrapydweb' requires a different Python: HOT 4
- Error while installing scrapydweb HOT 2
- spiders are closed but showing as running/warning in the tasks page
- web界面可以使用中文吗? HOT 1
- DATABASE_URL配置连接域名:端口的mysql失败 HOT 2
- Jobs are killed without a clear reason HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scrapydweb.