just keep on fighting
crawl vr content
usage: command line example: crawl movie -o movie_result -t csv --logfile=log "movie" is the spider member varibale name's value. you can choose the spider to crawl by spider's name, like "movie", "android_app" and so on.
item export: change FIELDS_TO_EXPORT in setting.py if you want to export specified fields or remember change it if you choose different spider
code description: we have base class VrItem and ItemSpider because the app and movie html have some same fields and same processes so we abstract VrItem and ItemSpider for clean code looking
sql_scrip: when 'load data infile' is used, remember transfer windows format to unix format