ryanbae89/data-512-a1

UW DATA512 Fall 2018 assignment 1 repo

data-512-a1

UW DATA 512 Fall 2018 Assignment 1 Repo

Creator: Ryan Bae

Date: October 18th, 2018

Goal

The goal of this project is to perform an exercise in data curation and analysis reproducibility by obtaining data from Wikimedia's API to download monthly Wikipedia page views data and generate a plot. The idea is that any user can look at this repo and reproduce the results from this exercise without difficulty.

Final Data Schema

Final en-wikipedia_traffic_200712-201809.csv file has the following schema:

• year (int)
• month (int)
• pagecount_all_views (int)
• pagecount_desktop_views (int)
• pagecount_mobile_views (int)
• pageview_all_views (int)
• pageview_desktop_views (int)
• pageview_mobile_views (int)

Data Considerations

There are few considerations to take into account when reproducing the results. There are some differenes between the two API's used to obtain the data.

• Legacy Pageview API Data

  • Includes web spiders/crawlers

• Current Pagecounts API Data

  • Does not include web spiders/crawlers
  • Further divides mobile data into mobile-app and mobile-web data

API Documentation

Also look at the following resources for API documentation:

Wikimedia REST API

Legacy Pagecounts API Documentation

Legacy Pagecounts API Endpoint

Pageviews API Documentation

Pageviews API Endpoint

License/Terms of Use and Documentation

When reproducing the results, please take a look at the following link terms of use information:

Wikimedia Terms of Use

The content accessed via Wikimedia's API is licensed under the CC-BY-SA 3.0 and GFDL licenses, and you irrevocably agree to release modifications or additions made through this API under these licenses.