[ ] uri_spider_and_gather_metadata: Id: 1333c4e5-744c-4470-b0b9-ad7df4352d4b
[ ] uri_spider_and_gather_metadata: Entity: Uri#http://www.iastate.edu
[ ] uri_spider_and_gather_metadata: Options: [{"name"=>"threads", "value"=>"5"}, {"name"=>"depth_limit", "value"=>"3"}, {"name"=>"obey_robots", "value"=>"false"}, {"name"=>"create_urls", "value"=>"false"}, {"name"=>"show_source_uri", "value"=>"true"}, {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}]
[ ] uri_spider_and_gather_metadata: Calling Setup
[ ] uri_spider_and_gather_metadata: Got user options list: [{"name"=>"threads", "value"=>"5"}, {"name"=>"depth_limit", "value"=>"3"}, {"name"=>"obey_robots", "value"=>"false"}, {"name"=>"create_urls", "value"=>"false"}, {"name"=>"show_source_uri", "value"=>"true"}, {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}]
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"threads", "value"=>"5"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Verifying this user option: threads
[ ] uri_spider_and_gather_metadata: Regex should match an integer
[ ] uri_spider_and_gather_metadata: Converting threads to an integer
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"threads", "value"=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"depth_limit", "value"=>"3"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Verifying this user option: depth_limit
[ ] uri_spider_and_gather_metadata: Regex should match an integer
[ ] uri_spider_and_gather_metadata: Converting depth_limit to an integer
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"depth_limit", "value"=>3}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"obey_robots", "value"=>"false"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Verifying this user option: obey_robots
[ ] uri_spider_and_gather_metadata: Regex should match a boolean
[ ] uri_spider_and_gather_metadata: Converting obey_robots to a bool
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"obey_robots", "value"=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"create_urls", "value"=>"false"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Verifying this user option: create_urls
[ ] uri_spider_and_gather_metadata: Regex should match a boolean
[ ] uri_spider_and_gather_metadata: Converting create_urls to a bool
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"create_urls", "value"=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"show_source_uri", "value"=>"true"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Verifying this user option: show_source_uri
[ ] uri_spider_and_gather_metadata: Regex should match a boolean
[ ] uri_spider_and_gather_metadata: Converting show_source_uri to a bool
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"show_source_uri", "value"=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Verifying this user option: user_agent
[ ] uri_spider_and_gather_metadata: Regex should match an alpha-numeric string
[ ] uri_spider_and_gather_metadata: No need to convert user_agent to a string
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Task configured with the following options: [{"threads"=>5}, {"depth_limit"=>3}, {"obey_robots"=>false}, {"create_urls"=>false}, {"show_source_uri"=>true}, {"user_agent"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}]
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent