intrigueio / intrigue-core Goto Github PK
View Code? Open in Web Editor NEWDiscover Your Attack Surface!
Home Page: https://core.intrigue.io
License: Other
Discover Your Attack Surface!
Home Page: https://core.intrigue.io
License: Other
Probably needs an update to the latest Pipl Api
48:37 task-worker.1 | 2016-05-01T15:48:37.662Z 47284 TID-oxtuqze8g Intrigue::SearchPiplTask JID-609091bc20847785a7ad6065 INFO: fail: 1.86 sec
08:48:37 task-worker.1 | 2016-05-01T15:48:37.662Z 47284 TID-oxtuqze8g WARN: {"class"=>"Intrigue::SearchPiplTask", "args"=>[2472, []], "retry"=>true, "queue"=>"task", "backtrace"=>true, "jid"=>"609091bc20847785a7ad6065", "created_at"=>1462117715.5040362, "enqueued_at"=>1462117715.5041, "error_message"=>"403 FORBIDDEN", "error_class"=>"OpenURI::HTTPError", "failed_at"=>1462117717.457272, "retry_count"=>0, "error_backtrace"=>["/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:359:in `open_http'", "/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'", "/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'", "/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:210:in `catch'", "/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'", "/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'", "/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:717:in `open'", "/Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:35:in `open'", "/Users/jcran/work/intrigue/projects/intrigue-core/lib/client/search/pipl.rb:20:in `search'", "/Users/jcran/work/intrigue/projects/intrigue-core/lib/tasks/search_pipl.rb:40:in `run'", "/Users/jcran/work/intrigue/projects/intrigue-core/lib/tasks/base.rb:67:in `perform'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:151:in `execute_job'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:133:in `block (2 levels) in process'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/logging.rb:31:in `with_context'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/server/logging.rb:7:in `call'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:132:in `invoke'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:128:in `block in process'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:167:in `stats'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:127:in `process'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:79:in `process_one'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:67:in `run'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:16:in `watchdog'", "/Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:24:in `block in safe_thread'"]}
08:48:37 task-worker.1 | 2016-05-01T15:48:37.662Z 47284 TID-oxtuqze8g WARN: OpenURI::HTTPError: 403 FORBIDDEN
08:48:37 task-worker.1 | 2016-05-01T15:48:37.662Z 47284 TID-oxtuqze8g WARN: /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:359:in `open_http'
08:48:37 task-worker.1 | /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'
08:48:37 task-worker.1 | /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'
08:48:37 task-worker.1 | /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:210:in `catch'
08:48:37 task-worker.1 | /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'
08:48:37 task-worker.1 | /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'
08:48:37 task-worker.1 | /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:717:in `open'
08:48:37 task-worker.1 | /Users/jcran/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:35:in `open'
08:48:37 task-worker.1 | /Users/jcran/work/intrigue/projects/intrigue-core/lib/client/search/pipl.rb:20:in `search'
08:48:37 task-worker.1 | /Users/jcran/work/intrigue/projects/intrigue-core/lib/tasks/search_pipl.rb:40:in `run'
08:48:37 task-worker.1 | /Users/jcran/work/intrigue/projects/intrigue-core/lib/tasks/base.rb:67:in `perform'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:151:in `execute_job'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:133:in `block (2 levels) in process'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/logging.rb:31:in `with_context'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/server/logging.rb:7:in `call'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/middleware/chain.rb:132:in `invoke'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:128:in `block in process'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:167:in `stats'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:127:in `process'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:79:in `process_one'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/processor.rb:67:in `run'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:16:in `watchdog'
08:48:37 task-worker.1 | /Users/jcran/.rvm/gems/ruby-2.3.0@core/gems/sidekiq-4.1.1/lib/sidekiq/util.rb:24:in `block in safe_thread'
08:48:38 task-worker.1 | [ ] : Port: 22
Hi,
I'm trying to install the project on Kubuntu 15.4
Ruby version is ruby 2.1.5p273 (2014-11-13) [x86_64-linux-gnu]
All required gems are installed using bundle.
Once I run the application, I get the following error
sudo foreman start
00:35:26 web.1 | started with pid 28570
00:35:26 worker.1 | started with pid 28571
00:35:26 web.1 | /usr/lib/ruby/vendor_ruby/bundler/source/git.rb:191:in `rescue in load_spec_files': https://github.com/intrigueio/arachnid.git (at master) is not yet checked out. Run `bundle install` first. (Bundler::GitError)
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/source/git.rb:188:in `load_spec_files'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/source/path.rb:92:in `local_specs'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/source/git.rb:159:in `specs'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:211:in `block (2 levels) in index'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:209:in `each'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:209:in `block in index'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/index.rb:9:in `build'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:206:in `index'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:200:in `resolve'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:140:in `specs'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:185:in `specs_for'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:174:in `requested_specs'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/environment.rb:18:in `requested_specs'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/runtime.rb:13:in `setup'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler.rb:127:in `setup'
00:35:26 web.1 | from /usr/lib/ruby/vendor_ruby/bundler/setup.rb:18:in `<top (required)>'
00:35:26 worker.1 | /usr/lib/ruby/vendor_ruby/bundler/source/git.rb:191:in `rescue in load_spec_files': https://github.com/intrigueio/arachnid.git (at master) is not yet checked out. Run `bundle install` first. (Bundler::GitError)
00:35:26 web.1 | from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/source/git.rb:188:in `load_spec_files'
00:35:26 web.1 | from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/source/path.rb:92:in `local_specs'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/source/git.rb:159:in `specs'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:211:in `block (2 levels) in index'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:209:in `each'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:209:in `block in index'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/index.rb:9:in `build'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:206:in `index'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:200:in `resolve'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:140:in `specs'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:185:in `specs_for'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/definition.rb:174:in `requested_specs'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/environment.rb:18:in `requested_specs'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/runtime.rb:13:in `setup'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler.rb:127:in `setup'
00:35:26 worker.1 | from /usr/lib/ruby/vendor_ruby/bundler/setup.rb:18:in `<top (required)>'
00:35:26 worker.1 | from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
00:35:26 worker.1 | from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
00:35:26 web.1 | exited with code 1
00:35:26 system | sending SIGTERM to all processes
00:35:26 worker.1 | exited with code 1
Should print the actual task name:
20:45:50 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/share/current-minimum-wage-isn-t-enough-workers-get-out-poverty
from https://github.com/intrigueio/intrigue-core/wiki/Setting-up-a-Test-Environment-on-Ubuntu-Linux
sudo apt-get -y install redis-sever E: Unable to locate package redis-sever
sudo apt-get -y install redis-server Much better :)
20:39:23 web.1 | 2015-08-24 20:39:23 - NoMethodError - undefined method `form' for false:FalseClass:
20:39:23 web.1 | /Users/jcran/work/intrigue/projects/intrigue-core/app/views/index.erb:45:in `block in singleton class'
20:39:23 web.1 | /Users/jcran/work/intrigue/projects/intrigue-core/app/views/index.erb:-6:in `instance_eval'
20:39:23 web.1 | /Users/jcran/work/intrigue/projects/intrigue-core/app/views/index.erb:-6:in `singleton class'
20:39:23 web.1 | /Users/jcran/work/intrigue/projects/intrigue-core/app/views/index.erb:-8:in `__tilt_70289781187520'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/tilt-2.0.1/lib/tilt/template.rb:155:in `call'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/tilt-2.0.1/lib/tilt/template.rb:155:in `evaluate'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/tilt-2.0.1/lib/tilt/template.rb:96:in `render'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:822:in `render'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:666:in `erb'
20:39:23 web.1 | /Users/jcran/work/intrigue/projects/intrigue-core/core.rb:106:in `block (2 levels) in <class:IntrigueApp>'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in `call'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in `block in compile!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:974:in `[]'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:974:in `block (3 levels) in route!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:993:in `route_eval'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:974:in `block (2 levels) in route!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1014:in `block in process_route'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in `catch'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in `process_route'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:972:in `block in route!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:971:in `each'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:971:in `route!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1084:in `block in dispatch!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `block in invoke'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `catch'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `invoke'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:1081:in `dispatch!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6/lib/sinatra/base.rb:906:in `block in call!'
20:39:23 web.1 | /Users/jcran/.rvm/gems/ruby-2.1.5@core/gems/sinatra-1.4.6
As a user, i'd like to be able to download and use an up-to-date docker container.
It'd be super handy if we looked up providers for IPAddresses as part of the enrichment process. This would allow us to cluster based on this information and better understand what lives on which provider.
As a user, it'd be handy if you could paginate results in the UI for me so i could more easily load pages.
To repro:
NoMethodError at /v1/
undefined method `form' for false:FalseClass
file: index.erb location: block in singleton class line: 45
It looks like we need to accelerate plans to database-back entities vs passing them around as form parameters.
Too noisy:
0:47:06 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/immigration/earned-citizenship
20:47:07 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/immigration/streamlining-immigration
20:47:07 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/immigration/economy
20:47:07 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/inmigracion
20:47:07 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/healthreform/healthcare-overview
20:47:07 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/healthreform/relief-for-americans-and-businesses
20:47:08 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/healthreform/map
20:47:08 worker.1 | [+] task_name: Creating entity: PhoneNumber, {"name"=>"800-318-2596"}
20:47:08 worker.1 | [ ] task_name: Adding entity #<Intrigue::Entity::PhoneNumber:0x007ff979672f30 @id="1fd3420e-fef3-4314-b25f-d85dd649445f", @type="PhoneNumber", @attributes={"name"=>"800-318-2596"}>
20:47:08 worker.1 | [ ] task_name: 301 Redirect on https://www.whitehouse.gov/healthreform/myths-and-facts
20:47:08 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/healthreform/myths-and-facts
20:47:08 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/healthreform
20:47:08 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/family
20:47:09 worker.1 | [+] task_name: Creating entity: Info, {"name"=>"Metadata for https://www.whitehouse.gov/sites/default/files/imagecache/gallery_img_full/image/image_file/1_p080615ps-0425.jpg", "metadata"=>{"Comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "Component 1"=>"Y component: Quantization table 0, Sampling factors 2 horiz/2 vert", "Component 2"=>"Cb component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Component 3"=>"Cr component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Compression Type"=>"Baseline", "Content-Type"=>"image/jpeg", "Data Precision"=>"8 bits", "Image Height"=>"436 pixels", "Image Width"=>"654 pixels", "Jpeg Comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100", "Number of Components"=>"3", "Resolution Units"=>"none", "X Resolution"=>"1 dot", "X-Parsed-By"=>["org.apache.tika.parser.DefaultParser", "org.apache.tika.parser.jpeg.JpegParser"], "Y Resolution"=>"1 dot", "comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "tiff:BitsPerSample"=>"8", "tiff:ImageLength"=>"436", "tiff:ImageWidth"=>"654", "w:comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n"}}
20:47:09 worker.1 | [ ] task_name: Adding entity #<Intrigue::Entity::Info:0x007ff979f525d8 @id="d4a79f0d-423a-4a30-b63a-546d95a11f10", @type="Info", @attributes={"name"=>"Metadata for https://www.whitehouse.gov/sites/default/files/imagecache/gallery_img_full/image/image_file/1_p080615ps-0425.jpg", "metadata"=>{"Comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "Component 1"=>"Y component: Quantization table 0, Sampling factors 2 horiz/2 vert", "Component 2"=>"Cb component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Component 3"=>"Cr component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Compression Type"=>"Baseline", "Content-Type"=>"image/jpeg", "Data Precision"=>"8 bits", "Image Height"=>"436 pixels", "Image Width"=>"654 pixels", "Jpeg Comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100", "Number of Components"=>"3", "Resolution Units"=>"none", "X Resolution"=>"1 dot", "X-Parsed-By"=>["org.apache.tika.parser.DefaultParser", "org.apache.tika.parser.jpeg.JpegParser"], "Y Resolution"=>"1 dot", "comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "tiff:BitsPerSample"=>"8", "tiff:ImageLength"=>"436", "tiff:ImageWidth"=>"654", "w:comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n"}}>
20:47:10 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/sites/default/files/imagecache/gallery_img_full/image/image_file/1_p080615ps-0425.jpg
20:47:12 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/poverty
20:47:12 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/economy/reform
20:47:13 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/share/behind-scenes-watch-president-obama-prepare-take-action-immigration
20:47:13 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/foreign-policy/cross-cutting-issues
20:47:14 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/ebola-response
20:47:14 worker.1 | [ ] task_name: Parsing text from http://www.whitehouse.gov/the-press-office/2014/09/24/remarks-president-obama-address-united-nations-general-assembly
20:47:14 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2014/09/24/remarks-president-obama-address-united-nations-general-assembly
20:47:15 worker.1 | [ ] task_name: Parsing text from http://www.whitehouse.gov/ebola-response
20:47:15 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/ebola-response
20:47:15 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/blog/2014/09/10/president-obama-we-will-degrade-and-ultimately-destroy-isil
20:47:15 worker.1 | [+] task_name: Creating entity: Info, {"name"=>"Metadata for https://www.whitehouse.gov/sites/default/files/imagecache/gallery_img_full/image/image_file/p080415ps-0181.jpg", "metadata"=>{"Comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "Component 1"=>"Y component: Quantization table 0, Sampling factors 2 horiz/2 vert", "Component 2"=>"Cb component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Component 3"=>"Cr component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Compression Type"=>"Baseline", "Content-Type"=>"image/jpeg", "Data Precision"=>"8 bits", "Image Height"=>"436 pixels", "Image Width"=>"654 pixels", "Jpeg Comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100", "Number of Components"=>"3", "Resolution Units"=>"none", "X Resolution"=>"1 dot", "X-Parsed-By"=>["org.apache.tika.parser.DefaultParser", "org.apache.tika.parser.jpeg.JpegParser"], "Y Resolution"=>"1 dot", "comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "tiff:BitsPerSample"=>"8", "tiff:ImageLength"=>"436", "tiff:ImageWidth"=>"654", "w:comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n"}}
20:47:15 worker.1 | [ ] task_name: Adding entity #<Intrigue::Entity::Info:0x007ff974e72cf8 @id="6bc960b8-9c03-496f-9850-ac3a661aa9fc", @type="Info", @attributes={"name"=>"Metadata for https://www.whitehouse.gov/sites/default/files/imagecache/gallery_img_full/image/image_file/p080415ps-0181.jpg", "metadata"=>{"Comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "Component 1"=>"Y component: Quantization table 0, Sampling factors 2 horiz/2 vert", "Component 2"=>"Cb component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Component 3"=>"Cr component: Quantization table 1, Sampling factors 1 horiz/1 vert", "Compression Type"=>"Baseline", "Content-Type"=>"image/jpeg", "Data Precision"=>"8 bits", "Image Height"=>"436 pixels", "Image Width"=>"654 pixels", "Jpeg Comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100", "Number of Components"=>"3", "Resolution Units"=>"none", "X Resolution"=>"1 dot", "X-Parsed-By"=>["org.apache.tika.parser.DefaultParser", "org.apache.tika.parser.jpeg.JpegParser"], "Y Resolution"=>"1 dot", "comment"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n", "tiff:BitsPerSample"=>"8", "tiff:ImageLength"=>"436", "tiff:ImageWidth"=>"654", "w:comments"=>"CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 100\n"}}>
20:47:16 worker.1 | [ ] task_name: Parsing text from http://www.whitehouse.gov/blog/2014/09/10/president-obama-we-will-degrade-and-ultimately-destroy-isil
20:47:16 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/blog/2014/09/10/president-obama-we-will-degrade-and-ultimately-destroy-isil
20:47:16 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/foreign-policy/cuba
20:47:16 worker.1 | [ ] task_name: Parsing text from http://www.whitehouse.gov/the-press-office/2011/03/21/remarks-president-obama-latin-america-santiago-chile
20:47:16 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2011/03/21/remarks-president-obama-latin-america-santiago-chile
20:47:16 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/mexico-costa-rica-trip-2013
20:47:16 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/sites/default/files/imagecache/gallery_img_full/image/image_file/p080415ps-0181.jpg
20:47:16 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/prime-minister-shinzo-abe-japan-state-visit
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/june-2015-photo-day[ ] task_name: Parsing text from https://www.whitehouse.gov/blog/2013/05/31/vice-president-biden-discusses-start-new-era-us-brazilian-relations
20:47:17 worker.1 |
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/may-2015-photo-day
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/march-2015-photo-day
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/first-lady-michelle-obama-travels-asia
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=1
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/april-2015-photo-day
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=3
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=2
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=4
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=7
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=5
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=6
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/photogallery/august-2015-photo-day?page=8
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/live/press-briefing-press-secretary-josh-earnest-138
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/file/31
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/issues/foreign-policy/iran-deal
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/live/president-obama-speaks-national-clean-energy-summit
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/live/president-obama-honor-wnba-champion-phoenix-mercury
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/feed/media/video-audio
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/24/president-speaks-national-clean-energy-summit
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/22/it-s-time-congress-pass-responsible-budget
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/blog/2015/07/01/president-obama-announces-us-will-reopen-our-embassy-cuba
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/24/press-briefing
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/21/west-wing-week-082115-or-look-ahead
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/us-re-opens-our-embassy-havana-cuba
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/21/president-obama-previews-his-upcoming-trip-alaska
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=1
20:47:17 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=2
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=4
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=3
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/continuing-work-improve-community-policing
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=5
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=6
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=7
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2015/08/14/west-wing-week-81415-or-thats-why-we-do-what-we-do?page=8
20:47:18 worker.1 | [ ] task_name: Parsing text from http://www.whitehouse.gov/interactive-budget
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/interactive-budget
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/blog/issues/Western-Hemisphere-
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/22/weekly-address-it%E2%80%99s-time-congress-pass-responsible-budget
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/15/weekly-address-continuing-work-improve-community-policing
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/08/weekly-address-reaffirming-our-commitment-protecting-right-vote
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/01/weekly-address-celebrating-fifty-years-medicare-and-medicaid
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/07/25/weekly-address-wall-street-reform-working
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/07/18/weekly-address-comprehensive-long-term-deal-iran
20:47:18 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/07/11/weekly-address-making-our-communities-stronger-through-fair-housing
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/07/04/weekly-address-have-safe-and-happy-fourth-july
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/06/27/weekly-address-affordable-care-act-here-stay
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/06/20/weekly-address-creating-new-pathways-opportunity-americans-you
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/06/13/weekly-address-stand-american-workers-and-pass-taa
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/briefing-room/weekly-address?page=1
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/07/remarks-president-signing-sawtooth-national-recreation-area-and-jerry
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/06/remarks-president-voting-rights-act
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/04/remarks-president-white-house-demo-day
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/05/remarks-president-iran-nuclear-deal
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/04/remarks-president-and-un-secretary-general-ban-ki-moon-after-bilateral
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/03/remarks-president-announcing-clean-power-plan
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/briefing-room/speeches-and-remarks?page=1
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/17/executive-order-presidential-innovation-fellows-program
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/03/remarks-president-young-african-leaders-initiative-presidential-summit
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/14/presidential-proclamation-national-employer-support-guard-and-reserve
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/07/presidential-proclamation-national-health-center-week-2015
20:47:19 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/the-press-office/2015/08/05/presidential-memorandum-continuation-us-drug-interdiction-assistance
20:47:22 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/blog/2015/06/17/promoting-security-and-transparency-central-america-high-level-security-dialogue-nor
20:47:22 worker.1 | [ ] task_name: Parsing text from https://www.whitehouse.gov/photos-and-video/video/2014/11/14/west-wing-week-111414-or-future-we-see
Currently, gives:
RuntimeError at /v1/task_results/TODO
Unable to find task_result:TODO
file: task_result.rb location: find line: 39
[ ] : Id: 2490
[ ] : Entity: IpAddress#50.31.225.93
[ ] : Calling setup()
[ ] : Options: []
[ ] : Calling run()
[ ] : Got result: {"ip"=>"x.x.x.x", "protocols"=>["80/http", "22/ssh"]}
[+] : Creating entity: IpAddress, {"name"=>"x.x.x.x", "additional"=>{"ip"=>"50.31.225.93", "protocols"=>["80/http", "22/ssh"]}}
[+] : Run complete. Ship it!
[ ] : Calling cleanup!
spidr (https://github.com/postmodern/spidr) is actively maintained, and appears to follow more link types than arachnid (https://github.com/dchuk/Arachnid). It may also help with memory usage. Give it a test run.
Handle this error and see if you can parse out the nameservers. May affect the 'whois' module too.
[ ] : Id: e724de46-9524-4613-847a-477bbc9e302c
[ ] : Entity: {"type"=>"DnsRecord", "attributes"=>{"name"=>"amazon.in"}}
[ ] : Calling Setup
[ ] : Task configured with the following options: []
[ ] : Calling Run
[ ] : Error querying whois: Unexpected token: Access to .IN WHOIS information is provided to assist persons in determining the contents of a domain name registration record in the .IN registry database. The data in this record is provided by .IN Registry for informational purposes only, and .IN does not guarantee its accuracy. This service is intended only for query-based access. You agree that you will use this data only for lawful purposes and that, under no circumstances will you use this data to(a) allow, enable, or otherwise support the transmission by e-mail, telephone, or facsimile of mass unsolicited, commercial advertising or solicitations to entities other than the data recipient's own existing customers; or (b) enable high volume, automated, electronic processes that send queries or data to the systems of Registry Operator, a Registrar, or Afilias except as reasonably necessary to register domain names or modify existing registrations. All rights reserved. .IN reserves the right to modify these terms at any time. By submitting this query, you agree to abide by this policy.
Domain ID:D15860-AFIN
Domain Name:AMAZON.IN
Created On:11-Feb-2005 11:14:14 UTC
Last Updated On:01-May-2014 21:37:11 UTC
Expiration Date:11-Feb-2017 11:14:14 UTC
Trademark Name:amazon
Trademark Date:1999-09-15
Trademark Country:india
Trademark Number:277580
Sponsoring Registrar:Mark Monitor (R84-AFIN)
Status:CLIENT DELETE PROHIBITED
Status:CLIENT TRANSFER PROHIBITED
Status:CLIENT UPDATE PROHIBITED
Registrant ID:mmr-28993
Registrant Name:Hostmaster, Amazon Legal Dept.
Registrant Organization:Amazon Technologies, Inc.
Registrant Street1:P.O. Box 8102
Registrant Street2:
Registrant Street3:
Registrant City:Reno
Registrant State/Province:NV
Registrant Postal Code:89507
Registrant Country:US
Registrant Phone:+1.2062664064
Registrant Phone Ext.:
Registrant FAX:+1.2062667010
Registrant FAX Ext.:
Registrant Email:[email protected]
Admin ID:mmr-28993
Admin Name:Hostmaster, Amazon Legal Dept.
Admin Organization:Amazon Technologies, Inc.
Admin Street1:P.O. Box 8102
Admin Street2:
Admin Street3:
Admin City:Reno
Admin State/Province:NV
Admin Postal Code:89507
Admin Country:US
Admin Phone:+1.2062664064
Admin Phone Ext.:
Admin FAX:+1.2062667010
Admin FAX Ext.:
Admin Email:[email protected]
Tech ID:mmr-28993
Tech Name:Hostmaster, Amazon Legal Dept.
Tech Organization:Amazon Technologies, Inc.
Tech Street1:P.O. Box 8102
Tech Street2:
Tech Street3:
Tech City:Reno
Tech State/Province:NV
Tech Postal Code:89507
Tech Country:US
Tech Phone:+1.2062664064
Tech Phone Ext.:
Tech FAX:+1.2062667010
Tech FAX Ext.:
Tech Email:[email protected]
Name Server:PDNS1.ULTRADNS.NET
Name Server:PDNS2.ULTRADNS.NET
Name Server:PDNS3.ULTRADNS.ORG
Name Server:PDNS4.ULTRADNS.ORG
Name Server:PDNS5.ULTRADNS.INFO
Name Server:PDNS6.ULTRADNS.CO.UK
Name Server:NS1.P31.DYNECT.NET
Name Server:NS2.P31.DYNECT.NET
Name Server:
Name Server:
Name Server:
Name Server:
Name Server:
DNSSEC:Unsigned
[ ] : Attempting Zone Transfer on amazon.in against nameserver 8.8.8.8
[-] : Task Execution Timed out
[+] : Ship it!
[ ] : Writing to file: dns_transfer_zone-amazon.in
Now using rbenv for all development environments, docs need to be updated.
/32 means something very different in IPv6. Make sure you don't try to scan these with masscan during a SimpleScan / InternalScan.
As reported in the Gitter chan:
fyoorer @aditya_gujar_twitter 02:05
just set up intrigue docker in OSX , but not able to login using the generated passwd
Jonathan Cran @jcran 02:06
hmm, services may not have restarted with the new config
one sec
try... AUTOGENERATED
all caps
fyoorer @aditya_gujar_twitter 02:20
i just did docker build . and then docker run
fyoorer @aditya_gujar_twitter 02:21
here you go:
Starting PostgreSQL 9.6 database server G G[ OK ]
Starting redis-server: redis-server.
Starting intrigue-core processes
[+] Setup initiated!
[+] Generating system password: cnznqhgpoqlidrts
[+] Copying puma config....
[ ] File already exists, skipping: /core/config/puma.rb
[+] Copying system config....
[ ] File already exists, skipping: /core/config/config.json
[+] Copying database config....
[+] Creating.... /core/config/database.yml
[+] Setting up task worker config....
[+] Copying: /core/config/sidekiq-task-interactive.yml.default
[+] Copying: /core/config/sidekiq-task-autoscheduled.yml.default
[+] Copying: /core/config/sidekiq-app.yml.default
[+] Obtaining latest data...
Getting data files (will fail if we don't have internet)
Getting latest MaxMind GeoLiteCity database
Getting latest web_accounts_list.json
[+] Copying control script...
[ ] File already exists, skipping: /core/util/control.sh
[+] Schema Version: 5
Starting interactive task processing...
Starting autoscheduled task processing...
Starting background worker processing...
Starting puma...
Setting puma environment to: development
Puma starting in single mode...
Version 3.6.2 (ruby 2.3.1-p112), codename: Sleepy Sunday Serenity
Min threads: 0, max threads: 16
Environment: development
Listening on tcp://0.0.0.0:7777
Use Ctrl-C to stop
Jonathan Cran @jcran 02:22
perfect
for some reason it thought your /core/config/config.json already existed
so it didn't overwrite with the new pass
fyoorer @aditya_gujar_twitter 02:25
i am checking in docker shell, core/config/puma.rb is not there
Jonathan Cran @jcran 02:25
can you ls your /core/config?
fyoorer @aditya_gujar_twitter 02:27
sorry. i was checking somewhere else
config.json database.yml puma.rb sidekiq-app.yml sidekiq-task-autoscheduled.yml sidekiq-task-interactive.yml config.json.default database.yml.default puma.rb.default sidekiq-app.yml.default sidekiq-task-autoscheduled.yml.default sidekiq-task-interactive.yml.default
Jonathan Cran @jcran 02:28
kk, and can you do the same in the repo (on the docker host, not inside the container)
ls /path/to/intrigue-core/config
fyoorer @aditya_gujar_twitter 02:29
config.json database.yml.default sidekiq-app.yml.default sidekiq-task-interactive.yml.default config.json.default puma.rb.default sidekiq-task-autoscheduled.yml.default
Potential use case - when spidering a site, youtube videos are often linked. By knowing the video ID, we should be able to collect all video metadata in JSON from a uri like http://gdata.youtube.com/feeds/api/videos/9-nXT8lSnPQ?alt=json
More detail: http://blog.sedica.es/?p=8
Sequel (http://sequel.jeremyevans.net/) looks very promising as an improvement to the existing homegrown ORM functionality. A couple reasons to look at Sequel:
This would mean Redis would go away as the primary data store, which is a good thing.
Also - datamapper was my first bet here, but there seems to be a general consensus that Sequel is more full-featured, and just as easy to use.
The app is starting to take an MVC shape, implement an ORM for simple task_result and scan_result management.
TransUnion's TLOxp bills themselves as ... the most powerful technology in the market for locating and researching connections between individuals, businesses and assets.You can access TLOxp’s massive data repository through the intuitive, online interface and secure API connections. Comprised of public and proprietary records.
Investigate what a module for TLOxp would look like.
Today, entities are not checked for uniqueness within the context of a running task before being created. This is not ideal, as you end up with tons of duplicate entities being created.
Potential solution: hash entity type + name + attributes and store in an array for checking while the task is running.
Solution:
This probably just needs some good ole fashion TLC notes and all but I took fresh kali on AMD64 docker instance and ran through installation of redis etc etc on a docker instance. A note to add to the docs would be to run a gem update first or install the non kali packed ruby version. There also seems to be a bug in that puma is looking for ./config/puma.rb but only ./config/puma.rb.default exists no big deal just needs to be moved from puma.rb.default to puma.rb or add it to the readme...
The problem:
I ran into a few bundle errors. zlib and a bunch of other stuff were throwing errors..I don't think some of the libs were installed by default on kali so it required libgcrypt11-dev and zlib1g-dev along with a host of other libraries due to ruby gem libs being required and ended up getting bundle errors due to missing libraries etc etc, it basically ended up requiring a great deal of libs like the following....
sudo apt-get install build-essential libreadline-dev libssl-dev libpq5 libpq-dev libreadline5 libsqlite3-dev libpcap-dev openjdk-7-jre git-core autoconf postgresql pgadmin3 curl zlib1g-dev libxml2-dev libxslt1-dev vncviewer libyaml-dev curl zlib1g-dev
I didn't really want to do that because I wanted a default Kali installation for the docker instance to not mess with the libraries but I finally just decided to do a full gem update....
Then started formman start and obtained a puma error
root@8696ab6be916:/intrigue-core# foreman start
06:35:17 web.1 | started with pid 20596
06:35:17 worker.1 | started with pid 20597
06:35:19 web.1 | /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/dsl.rb:20:in read': No such file or directory @ rb_sysopen - ./config/puma.rb (Errno::ENOENT) 06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/dsl.rb:20:in
_load_from'
06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/dsl.rb:9:in block in load' 06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/dsl.rb:8:in
tap'
06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/dsl.rb:8:in load' 06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/configuration.rb:56:in
load'
06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/cli.rb:531:in parse_options' 06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/lib/puma/cli.rb:189:in
run'
06:35:19 web.1 | from /var/lib/gems/2.1.0/gems/puma-2.13.4/bin/puma:10:in <top (required)>' 06:35:19 web.1 | from /usr/local/bin/puma:23:in
load'
06:35:19 web.1 | from /usr/local/bin/puma:23:in `
Original Installation errors from bundle errors were a list of things like the following...
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
/usr/bin/ruby2.1 extconf.rb
checking if the C compiler accepts ... yes
Building nokogiri using packaged libraries.
checking for gzdopen() in -lz... no
zlib is missing; necessary for building libxml2
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/ruby2.1
--help
--clean
--use-system-libraries
--enable-static
--disable-static
--with-zlib-dir
--without-zlib-dir
--with-zlib-include
--without-zlib-include=${zlib-dir}/include
--with-zlib-lib
--without-zlib-lib=${zlib-dir}/lib
--enable-cross-build
--disable-cross-build
extconf failed, exit code 1
Gem files will remain installed in /var/lib/gems/2.1.0/gems/nokogiri-1.6.6.2 for inspection.
Results logged to /var/lib/gems/2.1.0/extensions/x86_64-linux/2.1.0/nokogiri-1.6.6.2/gem_make.out
An error occurred while installing nokogiri (1.6.6.2), and Bundler cannot continue.
Make sure that gem install nokogiri -v '1.6.6.2'
succeeds before bundling.
root@8696ab6be916:/intrigue-core# ruby -v/intrigue-core# cat /var/lib/gems/2.1.0/extensions/x86_64-linux/2.1.0/nokogiri-1.6.6.2/mkmf.log
ruby 2.1.5p273 (2014-11-13) [x86_64-linux-gnu]
root@8696ab6be916:
"gcc -o conftest -I/usr/include/x86_64-linux-gnu/ruby-2.1.0 -I/usr/include/ruby-2.1.0/ruby/backward -I/usr/include/ruby-2.1.0 -I. -D_FORTIFY_SOURCE=2 -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fPIC conftest.c -L. -L/usr/lib/x86_64-linux-gnu -L. -Wl,-z,relro -L/build/ruby2.1-S44MX0/ruby2.1-2.1.5/debian/lib -fstack-protector -rdynamic -Wl,-export-dynamic -lruby-2.1 -lpthread -lgmp -ldl -lcrypt -lm -lc "
20:41:28 worker.1 | 2015-08-25T03:41:28.533Z 49412 TID-ounue4bxg WARN: {"class"=>"Intrigue::UriSpiderAndGatherMetadataTask", "args"=>["05199dab-4109-40f5-bf1d-1aa7cc17941e", "b6518062-70c8-4de4-895c-440f6e7c4d7d", [], []], "retry"=>true, "queue"=>"default", "jid"=>"33e0624f0e306f404138056d", "created_at"=>1440473773.8000069, "enqueued_at"=>1440474068.777426, "error_message"=>"undefined method `to_string' for \"uri\":String", "error_class"=>"NoMethodError", "failed_at"=>1440473904.579612, "retry_count"=>2, "retried_at"=>1440474088.532373}
20:41:28 worker.1 | 2015-08-25T03:41:28.534Z 49412 TID-ounue4bxg WARN: NoMethodError: undefined method `to_string' for "uri":String
20:41:28 worker.1 | 2015-08-25T03:41:28.534Z 49412 TID-ounue4bxg WARN: /Users/jcran/work/intrigue/projects/intrigue-core/entities/base.rb:10:in `set_attribute'
20:41:28 worker.1 | /Users/jcran/work/intrigue/projects/intrigue-core/lib/task/parse.rb:136:in `block in parse_entities_from_content'
20:41:28 worker.1 | /Users/jcran/work/intrigue/projects/intrigue-core/lib/task/parse.rb:134:in `each'
20:41:28 worker.1 | /Users/jcran/work/intrigue/projects/intrigue-core/lib/task/parse.rb:134:in `parse_entities_from_content'
Currently reporting options live in tasks/base.rb - split these out into individual libraries
As a user, it may not be obvious to me that an API key is required for a module. It would be handy if you give me a message instructing me to set one up if i try to run a module that's missing a key.
perform method fails with:
21:13:39 worker.1 | 2015-08-25T04:13:39.235Z 52556 TID-ovotv57xg WARN: NoMethodError: undefined method `process' for false:FalseClass
Implement a module that searches the Scans.io data via an API. For more information, see: https://scans.io/study/sonar.ssl
As a user, i'd like to be able to see the full details of an entity.
web_account_brute currently finds quite a few false positives, double check the regexes.
As a user, it'd be handy if you referred discovered WebApplications so i could better keep track of them.
Today we discover the "Uri" type, which is like a particular page on a web application, and we name it as the full URI instead of processing the URI and trying to figure out the underlying base application that URI belongs to. This simplifies things, but also means we could have many URIs for a single application, which can be confusing.
Link API key configuration in the UI, and make it possible to have some keys that are not editable.
Some specifics:
As a dev, i'd like the ability to register a configuration option (like an API key) with my task. This should be configurable by the user at runtime. It's not the same as an option, the user shouldn't have to verify it every time they run the task.
Not sure if you'd want tasks to register in the global config, but this should be in consideration.
Suggested Levels (increasing in verbosity):
As a user, i know that DnsRecord and IpAddresses are actually just different ways of referring to a Host. Today, these entities are different in Intrigue and this can be confusing. I'd prefer if they were a single entity so the system would be easier to use and integrate with my other asset-management / vulnerability scanning systems.
What this means:
Hi, I am trying to install intrigue-core on Ubuntu 16.04 beta but I got this error:
ddos@PenTester:/pentest/intelligence-gathering/core$ bundle exec rake migrate
Setup initiated!
Copying system config....
File exists: /pentest/intelligence-gathering/core/config/config.json
Copying database config....
File exists: /pentest/intelligence-gathering/core/config/database.yml
Copying sidekiq scan worker config....
File exists: /pentest/intelligence-gathering/core/config/sidekiq-scan.yml
Copying sidekiq task worker config....
File exists: /pentest/intelligence-gathering/core/config/sidekiq-task.yml
Obtaining geolocation database...
File exists: /pentest/intelligence-gathering/core/data/geolitecity/latest.dat
Fatal! Unable to read /pentest/intelligence-gathering/core/config/config.json
Error... Unable to migrate: unexpected return
jcran@masterblaster:~/core$ bundle exec ./core-cli.rb start uri_spider Uri#http://intrigue.io
Got entity: {"type"=>"Uri", "attributes"=>{"name"=>"http://intrigue.io"}}
Payload: {"task"=>"uri_spider", "options"=>[], "entity"=>{"type"=>"Uri", "attributes"=>{"name"=>"http://intrigue.io"}}}
/home/jcran/.rvm/gems/ruby-2.2.0@core/gems/rest-client-1.8.0/lib/restclient/abstract_response.rb:74:in `return!': 404 Resource Not Found (RestClient::ResourceNotFound)
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/rest-client-1.8.0/lib/restclient/request.rb:495:in `process_result'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/rest-client-1.8.0/lib/restclient/request.rb:421:in `block in transmit'
from /home/jcran/.rvm/rubies/ruby-2.2.0/lib/ruby/2.2.0/net/http.rb:853:in `start'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/rest-client-1.8.0/lib/restclient/request.rb:413:in `transmit'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/rest-client-1.8.0/lib/restclient/request.rb:176:in `execute'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/rest-client-1.8.0/lib/restclient/request.rb:41:in `execute'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/rest-client-1.8.0/lib/restclient.rb:69:in `post'
from ./core-cli.rb:377:in `_background'
from ./core-cli.rb:121:in `single'
from ./core-cli.rb:111:in `start'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
from /home/jcran/.rvm/gems/ruby-2.2.0@core/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
from ./core-cli.rb:384:in `<main>'
jcran@masterblaster:~/core$
As a user, i usually want to screenshot an entire netblock at a time. This should be possible.
Currently overwhelming:
[ ] uri_spider_and_gather_metadata: Id: 1333c4e5-744c-4470-b0b9-ad7df4352d4b
[ ] uri_spider_and_gather_metadata: Entity: Uri#http://www.iastate.edu
[ ] uri_spider_and_gather_metadata: Options: [{"name"=>"threads", "value"=>"5"}, {"name"=>"depth_limit", "value"=>"3"}, {"name"=>"obey_robots", "value"=>"false"}, {"name"=>"create_urls", "value"=>"false"}, {"name"=>"show_source_uri", "value"=>"true"}, {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}]
[ ] uri_spider_and_gather_metadata: Calling Setup
[ ] uri_spider_and_gather_metadata: Got user options list: [{"name"=>"threads", "value"=>"5"}, {"name"=>"depth_limit", "value"=>"3"}, {"name"=>"obey_robots", "value"=>"false"}, {"name"=>"create_urls", "value"=>"false"}, {"name"=>"show_source_uri", "value"=>"true"}, {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}]
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"threads", "value"=>"5"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Verifying this user option: threads
[ ] uri_spider_and_gather_metadata: Regex should match an integer
[ ] uri_spider_and_gather_metadata: Converting threads to an integer
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"threads", "value"=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"depth_limit", "value"=>"3"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Verifying this user option: depth_limit
[ ] uri_spider_and_gather_metadata: Regex should match an integer
[ ] uri_spider_and_gather_metadata: Converting depth_limit to an integer
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"depth_limit", "value"=>3}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"obey_robots", "value"=>"false"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Verifying this user option: obey_robots
[ ] uri_spider_and_gather_metadata: Regex should match a boolean
[ ] uri_spider_and_gather_metadata: Converting obey_robots to a bool
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"obey_robots", "value"=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"create_urls", "value"=>"false"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Verifying this user option: create_urls
[ ] uri_spider_and_gather_metadata: Regex should match a boolean
[ ] uri_spider_and_gather_metadata: Converting create_urls to a bool
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"create_urls", "value"=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"show_source_uri", "value"=>"true"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Verifying this user option: show_source_uri
[ ] uri_spider_and_gather_metadata: Regex should match a boolean
[ ] uri_spider_and_gather_metadata: Converting show_source_uri to a bool
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"show_source_uri", "value"=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Processing user option: {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"threads", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"depth_limit", :type=>"Integer", :regex=>"integer", :default=>5}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"obey_robots", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"create_urls", :type=>"Boolean", :regex=>"boolean", :default=>false}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"show_source_uri", :type=>"Boolean", :regex=>"boolean", :default=>true}
[ ] uri_spider_and_gather_metadata: Checking against allowed option list: {:name=>"user_agent", :type=>"String", :regex=>"alpha_numeric", :default=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Verifying this user option: user_agent
[ ] uri_spider_and_gather_metadata: Regex should match an alpha-numeric string
[ ] uri_spider_and_gather_metadata: No need to convert user_agent to a string
[+] uri_spider_and_gather_metadata: Congrats! Allowed this user option: {"name"=>"user_agent", "value"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}
[ ] uri_spider_and_gather_metadata: Task configured with the following options: [{"threads"=>5}, {"depth_limit"=>3}, {"obey_robots"=>false}, {"create_urls"=>false}, {"show_source_uri"=>true}, {"user_agent"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"}]
As a user, i may run a scan that goes down a path i'm not interested in... ala apple, or yahoo. Give me the ability to filter out these entities during the scan.
Several Social search APIs
Person:
https://api.fullcontact.com/v2/[email protected]&apiKey=YOUR_API_KEY_GOES_HERE
Company:
https://api.fullcontact.com/v2/company/lookup.json?domain=fullcontact.com&apiKey=YOUR_API_KEY_GOES_HERE
Location normalizer:
https://api.fullcontact.com/v2/address/locationNormalizer.json?place=denver%20co%20us&apiKey=YOUR_API_KEY_GOES_HERE
Name normalizer:
https://api.fullcontact.com/v2/name/normalizer.json?q=mr%20john%20(johnny)%20michael%20smith%20jr%20mba&apiKey=YOUR_API_KEY_GOES_HERE
As a user, it'd be handy if you could set a global resolver so i don't have to set it in each module.
I'm receiving the following error.
root@kali:~/kalifiles/scanners/intrigue-core# foreman start
11:39:47 web.1 | started with pid 26653
11:39:47 task-worker.1 | started with pid 26654
11:39:47 scan-worker.1 | started with pid 26655
11:39:50 web.1 | Setting puma environment to: development
11:39:50 web.1 | Puma starting in single mode...
11:39:50 web.1 | * Version 3.1.0 (ruby 2.1.5-p273), codename: El Niño Winter Wonderland
11:39:50 web.1 | * Min threads: 0, max threads: 16
11:39:50 web.1 | * Environment: development
11:39:52 scan-worker.1 | Intrigue-core system environment: development
11:39:52 task-worker.1 | Intrigue-core system environment: development
11:39:52 scan-worker.1 | 2016-03-15T00:39:52.261Z 26655 TID-6dl9g INFO: Booting Sidekiq 4.1.1 with redis options {:url=>"redis://localhost:6379/", :namespace=>"intrigue"}
11:39:52 task-worker.1 | 2016-03-15T00:39:52.276Z 26654 TID-2a9yk INFO: Booting Sidekiq 4.1.1 with redis options {:url=>"redis://localhost:6379/", :namespace=>"intrigue"}
11:39:52 web.1 | Intrigue-core system environment: development
11:39:54 scan-worker.1 | /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in require': /var/lib/gems/2.1.0/bundler/gems/censys-83dab46e0534/lib/censys.rb:58: syntax error, unexpected ':', expecting => (SyntaxError) 11:39:54 scan-worker.1 | "query": keyword 11:39:54 scan-worker.1 | ^ 11:39:54 scan-worker.1 | from /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in
require_with_backports'
11:39:54 scan-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/tasks/search_censys.rb:1:in <top (required)>' 11:39:54 scan-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in
require_relative'
11:39:54 scan-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in block in <top (required)>' 11:39:54 scan-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in
each'
11:39:54 scan-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in <top (required)>' 11:39:54 scan-worker.1 | from /root/kalifiles/scanners/intrigue-core/core.rb:132:in
require_relative'
11:39:54 scan-worker.1 | from /root/kalifiles/scanners/intrigue-core/core.rb:132:in <top (required)>' 11:39:54 scan-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/lib/sidekiq/cli.rb:237:in
require'
11:39:54 scan-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/lib/sidekiq/cli.rb:237:in boot_system' 11:39:54 scan-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/lib/sidekiq/cli.rb:49:in
run'
11:39:54 scan-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/bin/sidekiq:12:in <top (required)>' 11:39:54 scan-worker.1 | from /usr/local/bin/sidekiq:23:in
load'
11:39:54 scan-worker.1 | from /usr/local/bin/sidekiq:23:in <main>' 11:39:54 task-worker.1 | /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in
require': /var/lib/gems/2.1.0/bundler/gems/censys-83dab46e0534/lib/censys.rb:58: syntax error, unexpected ':', expecting => (SyntaxError)
11:39:54 task-worker.1 | "query": keyword
11:39:54 task-worker.1 | ^
11:39:54 task-worker.1 | from /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in require_with_backports' 11:39:54 task-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/tasks/search_censys.rb:1:in
<top (required)>'
11:39:54 task-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in require_relative' 11:39:54 task-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in
block in <top (required)>'
11:39:54 task-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in each' 11:39:54 task-worker.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in
<top (required)>'
11:39:54 task-worker.1 | from /root/kalifiles/scanners/intrigue-core/core.rb:132:in require_relative' 11:39:54 task-worker.1 | from /root/kalifiles/scanners/intrigue-core/core.rb:132:in
<top (required)>'
11:39:54 task-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/lib/sidekiq/cli.rb:237:in require' 11:39:54 task-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/lib/sidekiq/cli.rb:237:in
boot_system'
11:39:54 task-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/lib/sidekiq/cli.rb:49:in run' 11:39:54 task-worker.1 | from /var/lib/gems/2.1.0/gems/sidekiq-4.1.1/bin/sidekiq:12:in
<top (required)>'
11:39:54 task-worker.1 | from /usr/local/bin/sidekiq:23:in load' 11:39:54 task-worker.1 | from /usr/local/bin/sidekiq:23:in
stop' for nil:NilClass 11:39:54 web.1 | /var/lib/gems/2.1.0/gems/puma-3.1.0/lib/puma/single.rb:16:in
stop': undefined method stop' for nil:NilClass (NoMethodError) 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/puma-3.1.0/lib/puma/launcher.rb:147:in
stop'block in setup_signals' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in
call'require_with_backports' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/mail-2.6.3/lib/mail/message.rb:2:in
<top (required)>'require' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in
require_with_backports'<module:Mail>' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/mail-2.6.3/lib/mail.rb:2:in
<top (required)>'require' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in
require_with_backports'<top (required)>' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/backports-3.6.8/lib/backports/std_lib.rb:9:in
require'require_with_backports' 11:39:54 web.1 | from /root/kalifiles/scanners/intrigue-core/lib/tasks/email_bounceback.rb:1:in
<top (required)>'require_relative' 11:39:54 web.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in
block in <top (required)>'each' 11:39:54 web.1 | from /root/kalifiles/scanners/intrigue-core/lib/all.rb:20:in
<top (required)>'require_relative' 11:39:54 web.1 | from /root/kalifiles/scanners/intrigue-core/core.rb:132:in
<top (required)>'require' 11:39:54 web.1 | from config.ru:2:in
block in 'instance_eval' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/rack-1.6.4/lib/rack/builder.rb:55:in
initialize'new' 11:39:54 web.1 | from config.ru:in
'eval' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/rack-1.6.4/lib/rack/builder.rb:49:in
new_from_string'parse_file' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/puma-3.1.0/lib/puma/configuration.rb:316:in
load_rackup'app' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/puma-3.1.0/lib/puma/runner.rb:119:in
load_and_bind'run' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/puma-3.1.0/lib/puma/launcher.rb:173:in
run'run' 11:39:54 web.1 | from /var/lib/gems/2.1.0/gems/puma-3.1.0/bin/puma:10:in
<top (required)>'load' 11:39:54 web.1 | from /usr/local/bin/puma:23:in
'Currently, this page does not update attributes as the task runs, please update it so that i can see when a task is started, complete, etc.
Entity name is getting overwritten by the default.
When using nmap in Development VM in Ubuntu, you will run into issues running nmap without root leveraging the command line switches provided by the framework. You may see unhandled exceptions from sidekiq that reference not being root.
Currently in private beta, requires an API key. http://dev.peoplegraph.io/login
The PeopleGraph API allows you to quickly look up anyone using their email address, their social urls or their name and company.
Click one of the examples below to make your first query:
$ GET api.peoplegraph.io/v2/[email protected]&apiKey=<key>
$ GET api.peoplegraph.io/v2/lookup?url=http://github.com/drazvan&apiKey=<key>
We should catch this:
20:49:40 worker.1 | 2015-08-25T03:49:40.048Z 50282 TID-oxsi0g1a0 WARN: RuntimeError: redirection forbidden: http://www.whitehouse.gov/sites/default/files/docs/employing_people_with_disabilities_toolkit_february_3_2015_v4.pdf -> https://www.whitehouse.gov/sites/default/files/docs/employing_people_with_disabilities_toolkit_february_3_2015_v4.pdf
Seems to be skipping all sites, look into it.
[ ] : Id: 2471
[ ] : Entity: WebAccount#ADMIN
[ ] : Calling setup()
[ ] : Options: [{"specific_sites"=>""}]
[ ] : Calling run()
[ ] : Checking target against 141 possible sites
[ ] : Skipping https://about.me/ADMIN, doesn't match our type
[ ] : Skipping http://imcservices.passion.com/profile/ADMIN, doesn't match our type
[ ] : Skipping http://www.adultmatchdoctor.com/profile_ADMIN.html, doesn't match our type
[ ] : Skipping https://angel.co/ADMIN, doesn't match our type
[ ] : Skipping http://www.anobii.com/ADMIN/books, doesn't match our type
[ ] : Skipping http://ask.fm/ADMIN, doesn't match our type
[ ] : Skipping https://ADMIN.atlassian.net/login, doesn't match our type
[ ] : Skipping https://ADMIN.atlassian.net/admin/users/sign-up, doesn't match our type
[ ] : Skipping https://audioboom.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.authorstream.com/ADMIN/, doesn't match our type
[ ] : Skipping http://badoo.com/ADMIN/, doesn't match our type
[ ] : Skipping https://bitbucket.org/api/2.0/users/ADMIN, doesn't match our type
[ ] : Skipping https://bit.ly/u/ADMIN, doesn't match our type
[ ] : Skipping http://blip.fm/ADMIN, doesn't match our type
[ ] : Skipping http://www.blackplanet.com/ADMIN/, doesn't match our type
[ ] : Skipping http://blogmarks.net/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.break.com/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.cardomain.com/member/ADMIN/, doesn't match our type
[ ] : Skipping http://profile.cheezburger.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.codeplex.com/site/users/view/ADMIN, doesn't match our type
[ ] : Skipping http://coderstats.net/github/ADMIN/, doesn't match our type
[ ] : Skipping http://www.colourlovers.com/lover/ADMIN, doesn't match our type
[ ] : Skipping https://www.conferize.com/u/ADMIN/, doesn't match our type
[ ] : Skipping http://copytaste.com/profile/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.deviantart.com/, doesn't match our type
[ ] : Skipping https://www.diigo.com/profile/ADMIN, doesn't match our type
[ ] : Skipping https://diy.org/ADMIN, doesn't match our type
[ ] : Skipping http://www.docstoc.com/profile/ADMIN, doesn't match our type
[ ] : Skipping https://www.dribbble.com/players/ADMIN, doesn't match our type
[ ] : Skipping http://www.ebay.com/usr/ADMIN, doesn't match our type
[ ] : Skipping http://eightbit.me/ADMIN, doesn't match our type
[ ] : Skipping http://www.eporner.com/profile/ADMIN/, doesn't match our type
[ ] : Skipping https://www.etsy.com/people/ADMIN, doesn't match our type
[ ] : Skipping https://www.facebook.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.fanpop.com/fans/ADMIN, doesn't match our type
[ ] : Skipping http://favstar.fm/users/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.flavors.me, doesn't match our type
[ ] : Skipping http://www.foodspotting.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.fotolog.com/ADMIN/, doesn't match our type
[ ] : Skipping http://www.freesound.org/people/ADMIN/, doesn't match our type
[ ] : Skipping http://www.geekgrade.com/geeksheet/ADMIN/blogs, doesn't match our type
[ ] : Skipping http://www.geocaching.com/seek/nearest.aspx?u=ADMIN, doesn't match our type
[ ] : Skipping http://getiton.com/profile/ADMIN, doesn't match our type
[ ] : Skipping https://api.github.com/users/ADMIN, doesn't match our type
[ ] : Skipping http://www.godtube.com/ADMIN/, doesn't match our type
[ ] : Skipping http://www.gogobot.com/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.goodreads.com/ADMIN, doesn't match our type
[ ] : Skipping http://en.gravatar.com/profiles/ADMIN.json, doesn't match our type
[ ] : Skipping https://hackerslist.com/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.howaboutwe.com/users/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.hubpages.com/, doesn't match our type
[ ] : Skipping https://imageshack.com/user/ADMIN, doesn't match our type
[ ] : Skipping http://imgur.com/user/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.insanejournal.com/profile, doesn't match our type
[ ] : Skipping http://instagram.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.instructables.com/member/ADMIN/, doesn't match our type
[ ] : Skipping http://archive.org/search.php?query=ADMIN, doesn't match our type
[ ] : Skipping http://www.interpals.net/ADMIN, doesn't match our type
[ ] : Skipping https://keybase.io/ADMIN, doesn't match our type
[ ] : Skipping https://klout.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.kongregate.com/accounts/ADMIN, doesn't match our type
[ ] : Skipping http://lanyrd.com/profile/ADMIN/, doesn't match our type
[ ] : Skipping http://www.last.fm/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.librarything.com/profile/ADMIN, doesn't match our type
[ ] : Skipping https://www.linkedin.com/in/ADMIN, doesn't match our type
[ ] : Skipping http://www.livejasmin.com/perfinfo.php?performerid=ADMIN, doesn't match our type
[ ] : Skipping http://www.mate1.com/profiles/ADMIN, doesn't match our type
[ ] : Skipping https://medium.com/@ADMIN, doesn't match our type
[ ] : Skipping http://www.mixcloud.com/ADMIN/, doesn't match our type
[ ] : Skipping http://www.mixcrate.com/ADMIN, doesn't match our type
[ ] : Skipping http://mixlr.com/ADMIN/, doesn't match our type
[ ] : Skipping http://www.moddb.com/members/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.muzy.com/, doesn't match our type
[ ] : Skipping http://myanimelist.net/profile/ADMIN, doesn't match our type
[ ] : Skipping http://www.mybuilder.com/profile/view/ADMIN, doesn't match our type
[ ] : Skipping http://www.myfitnesspal.com/user/ADMIN/status, doesn't match our type
[ ] : Skipping http://ADMIN.newsvine.com/_tps/_author/profile, doesn't match our type
[ ] : Skipping https://www.netvibes.com/ADMIN, doesn't match our type
[ ] : Skipping https://ADMIN.okta.com, doesn't match our type
[ ] : Skipping http://www.pandora.com/profile/ADMIN, doesn't match our type
[ ] : Skipping http://smg.photobucket.com/user/ADMIN/profile/, doesn't match our type
[ ] : Skipping https://www.pinterest.com/ADMIN/, doesn't match our type
[ ] : Skipping http://plancast.com/ADMIN/, doesn't match our type
[ ] : Skipping http://playlists.net/members/ADMIN, doesn't match our type
[ ] : Skipping http://www.plurk.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.pof.com/basicusersearch.aspx?usernamet=ADMIN, doesn't match our type
[ ] : Skipping http://www.porn.com/profile/ADMIN, doesn't match our type
[ ] : Skipping http://www.pornhub.com/users/ADMIN, doesn't match our type
[ ] : Skipping http://psnprofiles.com/ADMIN, doesn't match our type
[ ] : Skipping http://raptr.com/ADMIN/about, doesn't match our type
[ ] : Skipping http://rateyourmusic.com/~ADMIN, doesn't match our type
[ ] : Skipping https://www.readability.com/ADMIN/, doesn't match our type
[ ] : Skipping http://www.reddit.com/user/ADMIN/, doesn't match our type
[ ] : Skipping http://www.redtube.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.reunion.com/ADMIN/, doesn't match our type
[ ] : Skipping https://community.rapid7.com/people/ADMIN, doesn't match our type
[ ] : Skipping http://www.setlist.fm/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.shelfari.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.shopcade.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.singlemuslim.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.slideshare.net/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.smugmug.com, doesn't match our type
[ ] : Skipping http://www.smule.com/ADMIN/, doesn't match our type
[ ] : Skipping http://www.snooth.com/profiles/ADMIN/, doesn't match our type
[ ] : Skipping https://www.soldierx.com/hdb/ADMIN, doesn't match our type
[ ] : Skipping https://soundcloud.com/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.soup.io, doesn't match our type
[ ] : Skipping https://speakerdeck.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.sporcle.com/user/ADMIN/connections, doesn't match our type
[ ] : Skipping http://steamcommunity.com/id/ADMIN, doesn't match our type
[ ] : Skipping http://stupidcancer.org/community/profile/ADMIN, doesn't match our type
[ ] : Skipping http://www.tf2items.com/id/ADMIN/, doesn't match our type
[ ] : Skipping https://id.theguardian.com/profile/ADMIN/public, doesn't match our type
[ ] : Skipping http://www.tripadvisor.com/members/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.tumblr.com, doesn't match our type
[ ] : Skipping http://twitpic.com/photos/ADMIN, doesn't match our type
[ ] : Skipping https://twitter.com/ADMIN, doesn't match our type
[ ] : Skipping http://twtrland.com/profile/ADMIN/, doesn't match our type
[ ] : Skipping https://untappd.com/user/ADMIN/, doesn't match our type
[ ] : Skipping http://www.ustream.tv/channel/ADMIN, doesn't match our type
[ ] : Skipping http://www.viddler.com/explore/ADMIN/, doesn't match our type
[ ] : Skipping http://videolike.org/video/ADMIN, doesn't match our type
[ ] : Skipping https://vid.me/u/ADMIN, doesn't match our type
[ ] : Skipping http://vimeo.com/ADMIN, doesn't match our type
[ ] : Skipping http://vi.sualize.us/ADMIN/, doesn't match our type
[ ] : Skipping https://www.voices.com/people/ADMIN, doesn't match our type
[ ] : Skipping http://wanelo.com/ADMIN, doesn't match our type
[ ] : Skipping http://www.wattpad.com/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.weeworld.com/home/ADMIN, doesn't match our type
[ ] : Skipping http://www.wishlistr.com/profile/ADMIN/, doesn't match our type
[ ] : Skipping https://en.wikipedia.org/wiki/ADMIN, doesn't match our type
[ ] : Skipping https://profiles.wordpress.org/ADMIN, doesn't match our type
[ ] : Skipping https://wordpress.org/support/profile/ADMIN, doesn't match our type
[ ] : Skipping http://ADMIN.xanga.com/, doesn't match our type
[ ] : Skipping http://www.xboxgamertag.com/search/ADMIN/, doesn't match our type
[ ] : Skipping http://www.xboxlivescore.com/profile/ADMIN, doesn't match our type
[ ] : Skipping http://xhamster.com/user/ADMIN, doesn't match our type
[ ] : Skipping http://www.xvideos.com/profiles/ADMIN, doesn't match our type
[ ] : Skipping https://na.edit.yahoo.com/reg_json?PartnerName=yahoo_default&RequestVersion=1&[email protected]&ApiName=ValidateFields&intl=us, doesn't match our type
[ ] : Skipping https://www.youtube.com/user/ADMIN/videos, doesn't match our type
[ ] : Skipping http://zooppa.com/en-us/users/ADMIN, doesn't match our type
[+] : Run complete. Ship it!
[ ] : Calling cleanup!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.