haimbr / crawler Goto Github PK
View Code? Open in Web Editor NEWweb crawler using nodejs, express, socket.io, redis and sqs the worker responsible for the crawling. the server is responsible for communication with the client and to send the worker to the worker through sqs