Giter Site home page Giter Site logo

backend-challenge's Introduction

On Heroku

https://blooming-savannah-87377.herokuapp.com/user_profile/n_i_k_o_z

My Fork

https://github.com/Tzikas/backend-challenge

Most of the work is in routes/index.js and instascrape.js

Coding Challenge for a backend.

Build a simple API that scrape a site.

I put up this challenge on CODEWARS (was rated about 2 kata) a year ago and became really popular among the community of hackers. It was a short challenge that took 2 - 3 hours and most developers had fun building it and continue to use it for other everyday purposes. The idea and concept of it applies to real life applications that are relevant in today's data-driven world.

  • Language: Javascript
  • Technology: Node.js, Express
  • Difficulty: Medium
  • Completion time: 2 - 3 hours. Took lead engineer 1.5 hour to complete. Shoudn't take you more than 5 hours. It doesn't count against you, but please explain why and provide evidence to prove your point. A good engineer is also a good lawyer ;)

Things we will look for:

  • Everything is modulized. Comments to explain your reasoning (I don't need a paragraph explaining. Good engineers will explain in a few sentences)
  • Efficiency. code is clean, neat and organized.
  • Memory, CPU handling.
  • Every error is handled.
  • Design and Architecture structure.
  • Knowledge of browserless technology
  • Routing
  • Data manipulation.
  • Automation using kue or other job priority libs.

Detail:

Learning experience 1:
Build an API with scraper.
When you invent new products, sometimes the data you need isn't available for you yet. Therefore, you must scrape the internet to provide the data for the client users.

Learning experience 2:
Understand data process.
In order to be a good developer, you must know the lifecycle of every data/parameter/variable you use and HOW, WHY, WHEN they change. This is just simple data process question. That's why you must parse all the data you scrape into a readible JSON format.

Instructions:

Build two routes: GET:{baseUrl}/user_profile and GET:{baseUrl}/user_profile_continue

Please upload to a server. AWS, Digital Ocean, Heroku, you're own IP Address, etc...it doesn't matter as long as it works somewhere.

Part 1:

Parameter: username
Tools (You don't have to use these. You may use different ones or add more if you'd like):

  • cheerio
  • puppeteer
  • $q

{baseUrl}/user_profile will scrape for user information and returns a JSON formatted response. For example:

{
  data: {
    "username": "unumdesign",
    "fullName": "Jack Beanstalk",
    "imageUrl": "profileimage.orsomething.com",
    "followers": 123,
    "following": 12345
  }
}

You may provide more information if you'd like but those are the most important ones.

Errors: If there are errors such as "no users exist" or "private users" please handle them correctly.

Part 2:

Parameter: username
Tools (You don't have to use these. You may use different ones or add more if you'd like):

  • cheerio
  • puppeteer
  • $q
  • kue

GET:{baseUrl}/user_profile_continue will do the same thing BUT it will continue to scrape and console.log out the response every 2 minutes.

The response won't matter. What matters is the console.log output on the terminal that shows the response every 2 minutes and should end after the 10th output. Please use KUE or any other automation for this step.

For example this is what it should show on the terminal:

After 2 minutes...
{
  data: {
    "username": "unumdesign",
    "fullName": "Jack Beanstalk",
    "imageUrl": "profileimage.orsomething.com",
    "followers": 123,
    "following": 12345
  }
}

After 2 minutes...
{
  data: {
    "username": "unumdesign",
    "fullName": "Jack Beanstalk",
    "imageUrl": "profileimage.orsomething.com",
    "followers": 123,
    "following": 12345
  }
}

After 2 minutes...
{
  data: {
    "username": "unumdesign",
    "fullName": "Jack Beanstalk",
    "imageUrl": "profileimage.orsomething.com",
    "followers": 123,
    "following": 12345
  }
}

and so on...

Errors:* If there are errors such as "no users exist" or "private users" please handle them correctly.

When you are completed, please create a pull request!

If you have any other questions, please put them in the ISSUES.

HAVE FUN!!

backend-challenge's People

Contributors

jarrettc124 avatar tzikas avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.