Passing arguments to scraper

[my8100]

Passing arguments to scraper

[echoocking]

schedule.json is correct. __init__ func needs change like this could solve my problem.

 def __init__(self, mgo_db=None, mgo_table=None, *args, **kwargs):
        mgo_conn = get_mgo_client()
        self.mgo_table = mgo_conn[mgo_db][mgo_table]
        super(MtopSpider, self).__init__()

init take any spider arguments and copy them to spider as attrbutes

thank you very much 👍 hhhhhhh~~

[my8100]

It should work if you add '-d symbol=APPL' in the additional text box!

[ohenrik]

Thank you, this worked 🎉

[echoocking]

i want pass params to spider , so i use init func to get -a params like this

    def __init__(self, mgo_db, mgo_table):
        mgo_conn = get_mgo_client()
        self.mgo_table = mgo_conn[mgo_db][mgo_table]
        super(MtopSpider, self).__init__()

and add
'-d mgo_db=mtop
-d mgo_table=taolijin'
in the additional text box

but i got

2019-03-20 21:33:09 [twisted] CRITICAL: 
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/scrapyd-server-b9lo-2yl/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/root/.local/share/virtualenvs/scrapyd-server-b9lo-2yl/lib/python3.7/site-packages/scrapy/crawler.py", line 79, in crawl
    self.spider = self._create_spider(*args, **kwargs)
  File "/root/.local/share/virtualenvs/scrapyd-server-b9lo-2yl/lib/python3.7/site-packages/scrapy/crawler.py", line 102, in _create_spider
    return self.spidercls.from_crawler(self, *args, **kwargs)
  File "/root/.local/share/virtualenvs/scrapyd-server-b9lo-2yl/lib/python3.7/site-packages/scrapy_redis/spiders.py", line 153, in from_crawler
    obj = super(RedisSpider, self).from_crawler(crawler, *args, **kwargs)
  File "/root/.local/share/virtualenvs/scrapyd-server-b9lo-2yl/lib/python3.7/site-packages/scrapy/spiders/__init__.py", line 51, in from_crawler
    spider = cls(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument '_version'

what shoud i do for this? thanks a lot :)

[my8100]

Check out https://doc.scrapy.org/en/latest/topics/spiders.html#spider-arguments