uwaterloo/datasets

Datasets powering the Open Data API

datasets's Issues

We have a new residence

• Add data to buildings

https://uwaterloo.ca/housing/residences/waterloo-central-place

Mega Issue for uPrint

• Update Printers/Printers.csv/ setup cron job/script to update autoupdate branch
• add columns to Printers.csv to support uPrint
• create a new data store to support generic uPrint information for the whole fleet:
  • pricing
  • anything else?

Printers.csv columns will be:

• Boolean: part of uPrint fleet
• Building Location matching /buildings/list
• Canonical Room Number
• Colloquial location description
• Service Hours if part of uPrint fleet
• Available print-job types
• Boolean: Available for use?
• Anything else?

Reference: http://uprint.uwaterloo.ca/locations.html

Show former buildings

The buildings should show former UW buildings. This requires changing the API to add a boolean column which indicates whether or not the building is current.

/Datasets/blob/watcard/Watcard/watcardvendors.json completed; commit to API eom

add https://uwaterloo.ca/human-resources/pay-administration/pay-dates to Open Data

exam schedule before 1115

Is exam schedule before 1131(2013 winter) gonna be available ???

Police service calls missing 2013 data

I can probably take this one, but just adding it here to remind myself.

Sanitize parking lot data

Transfer parking lot data from v1 and make it more readable and parseable.

• https://uwaterloo.ca/parking/lot-information/lot-descriptions
• https://uwaterloo.ca/parking/alternative-commuting/facts/lot-information/rates

Cases

• Visitor parking is available evenings at 3:45pm
• There is currently no evening or weekend parking available
• Visitor parking is available all day/evening
• Visitor parking is available evenings and weekends
• Evening and weekend parking for visitors is available until 10:00 pm each night
• Lot X is open and free from 4:30 pm Friday until 6:00 am Monday.
• Visitor parking $2.00 per hour, daily maximum of $15.00 Evenings after 5:00 pm
• Contractor permit parking only
• Only between the hours of 6:00 am - 6:00 pm.
• No overnight parking from 3:00 - 6:00 am

Response (needs work)

{
  "name" : "A",
  "latitude": 43.474847,
  "longitude" : -80.5446,
  "available_weekends" : true,
  "available_evenings" : true,
  "available_daytime" : true,
  "is_currently_open" : true,
  "current_visitor_cost" : 5,
  "type" : ["Faculty", "Staff", "Visitor", "Contractor"],
  "not_open" : [
    {
      "days" : ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday"],
      "time_start" : "03:00",
      "time_end" : "06:00"
    }
  ],
  "note" : "Permits are sold to faculty and staff if space is available"
}

List of faculties

Would be great to assemble a list of all faculties (grad and undergrad)

/v2/codes/faculties.json

@nathanv can you help out with this?

Do we want transit data in the API?

Do you know if we plan to add transit data? According to http://campusdata.org/rankings/ we don't have it in our api.

My friend has made a transit app and has the GRT data, also would something like the feds bus schedule be helpful?

Infosessions scraper should handle second run

Currently sessions.py only works when you run it the first time. Since it relies on the rsvp links to get the session id, the link to rsvp disappears for sessions before the current date and an exception is raised.

A quick fix would be to keep a dict of the csv from the 1st run and check what ids are present in the dict vs scraped before raising exception.

Rel: uWaterloo/OpenData#104
cc: @jashp

Think we get the data from that UWaterloo Pedestrian App?

In Fall 2012, the University published an app (the name escapes me) that tracked pedestrian paths around campus. While static, I still think this would be a cool dataset (think heatmaps, Zombies vs. Humans uses, maximizing club advertisments, etc.)

Is UWaterloo willing to give that up? (Or do we already have it?)

Buildings/Buildings.csv : UW Place Beck Hall and UW Place Eby Hall don't have an alt name

The name should be UWP

Extended info for Watcard Locations

On V1 there used to be several more properties in a Watcard location, most importantly a Lat/Lng. Can we have these values in V2 as well?

Old - https://api.uwaterloo.ca/#!/watcardvendors
New - https://github.com/uWaterloo/api-documentation/blob/master/v2/foodservices/watcard.md

Add Bike Lockers layer to Open Data

Will attached GeoJSON file in the comment.

Public Sector Salary Disclosure

Just as a note for myself. Thanks, @ndehnashi!

Fix quest data

Fix spelling and expand names

• Organizations
• Subjects

examschedule.json should not include a "day" key

examschedule.json has both "day", which is the day of the week, and "date", which is a full ISO date. Since "day" can be easily calculated from "date", "day" should be deprecated and eventually removed.

Buildings List - Update from BB to EC

The buildings list needs to be updated changing the BlackBerry buildings to East Campus, as mentioned in the announcement: http://lists.uwaterloo.ca/pipermail/registrar/2014-October/000026.html

Dates in holidays.csv should be in ISO format

Dates in holidays.csv are currently in a format like "Monday February 20" with the year stored in a separate column. This file should be just two columns: ISO date and name of holiday.

Subjects and departments are not the same thing

The DepartmentsList service is returning a large number of items which are not UW departments. Department codes and subject codes are mixed together. Departments are more properly known as units and faculties are groups. Subjects are owned by units, which are owned by groups. For example, the AFM subject code is owned by the ACC unit, which is owned by the ART group. There needs to be a service for all three of subjects, units, and groups.

Buildings/Buildings.csv : "Manulife Water Street" should have alt name of Velocity Foundry

Stale data for employer info sessions

Still returns data for Spring 2014

Exam Schedule not up to date

According to https://uwaterloo.ca/registrar/final-examinations/exam-schedule, exam schedule was updated ysterday (February 26, 2014). However, the last update @dbe0f32 was done 3 days ago.

Add IQC to new alternate building name

Since IQC is now part of QNC we can should add the following building names as alternate names for QNC, and RAC

• "Institute for Quantum Computing"
• "IQC"

cc @nathanv @zachomedia

List of shops/restaurants

Suppose I were to provide data for a list of shops, restaurants, and their opening times. Does Open Data concern itself with locations belonging to the university (e.g. food services locations), locations on campus (e.g. Tim Horton's), or locations near campus (e.g. campus plaza)?

examschedule.json should use array for "locations"

In examschedule.json, the key "location" should be deprecated. A key "locations" should be added containing an array of the locations.

Currently, "location" is a quasi comma-separated list of locations.

building_id is being cast to int

In the buildings endpoint, the building_id contains only numbers, not the letters that some ids have. This suggests that these are being cast to int.

Add locker locations to Open Data

Inspired by http://www.reddit.com/r/uwaterloo/comments/2y907m/locker_to_store_guitar_on_campus/

Pros: locations of lockers for students.
Cons: could give thieves a shopping list of where to go to cut locks and steal things.

Make "Course Schedule by Term" data available for older terms (Fall 2013 and earlier)

via email support:

"Can you say why "Course Schedule by Term" data for some older terms is not available?

I'm gathering class sizes for a collection of courses and am finding that no data is available for terms older that fall 2013 (1139).

The UW Registrar's page usually restricts listings to 4 terms worth of course schedule data:
http://www.adm.uwaterloo.ca/infocour/CIR/SA/under.html

The CS dept, however, has a similar page of course schedule data (for any UW
course) for over 10 years of terms. And the data looks like the UW Registrar pages. Their URL is https://cs.uwaterloo.ca/cscf/teaching/schedule/

Here's an example for course CHEM 123 in Winter 2013 (1131):
This URL
https://api.uwaterloo.ca/v2/terms/1131/CHEM/123/schedule.json?key=e4f2a20a4d95c0e0292aa5f14224ca83

returns

{
"meta":{
"requests":78,
"timestamp":1407528285,
"status":204,
"message":"No data returned",
"method_id":1193,
"method":{
}
},
"data":[

]
}

The CS dept course schedule page has this information:

https://info.uwaterloo.ca/cgi-bin/cgiwrap/infocour/salook.pl?sess=1131&level=under&subject=CHEM&cournum=123

just not in json format.
"

Update Subjects

IAP website has more than 300 subjects listed, our csv is ~200

https://uwaterloo.ca/institutional-analysis-planning/university-data-and-statistics/glossaries/code-lookups

building_code is being treated as unique key

Rows with duplicated building_codes are being dropped from output. building_code is not unique and is correctly shared between more than one related building.

Healthy Foods & More missing from /Watcard/watcardvendors.json

Reference: http://page2rss.com/c/f7d49711faf6373a9925bf33f5078f9a/7138813_7223693/healthy-foods-and-more
New vendor added on 01/12/15

Add Publicly Accessible Microwave locations data

Locations are posted to http://www.reddit.com/r/uwaterloo/wiki/microwaves but not georeferenced. Would love if someone out in the community curated this. Format could be something like https://github.com/uWaterloo/Datasets/blob/master/CoffeeAndDonutShops/cndlocations.csv

Buildings/Buildings.csv: CLV Community Centre is missing Lat/Long

Add a blank 1151GooseWatch.csv

Geese are almost here. Need to be ready!

ExamSchedule/1149ExamSchedule.csv has extra blank columns

There is a known issue with extra columns in the csv. Some notes:

• This issue ONLY affects the CSV and not the API.
• Since we wanted to get the data out as fast as possible and since the API works as per usual, we are releasing the data with this known issue.
• In the future, we endeavour to properly normalize the data in conjunction with the partners at the registrar.

edit 13:06 16-10-2014: for clarity

[Datasets / Watcard / watcardvendors.json] Make updates

• Remove Curry in a Hurry (recently closed)
• Add Milestones to Watcard Locations

Add 2014 salary disclosures to /Salaries

https://uwaterloo.ca/about/what-we-stand/accountability/salary-disclosure-2014

Add optional parent_code to https://github.com/uWaterloo/Datasets/blob/master/Buildings/Buildings.csv

This will help fix uWaterloo/OpenData#160

examschedule.json should use course codes as keys in "data"

In examschedule.json the "data" key contains an array of objects. It would be easier to get the information for a specific course if "data" was an object with the keys being the course codes and the values being the arrays of sections currently in the "sections" key.

return pelmorex alerts for Region of Waterloo/Muskoka (Huntsville)/Perth County (Stratford)

Add Scanners to /Scanners/Scanners.csv

As per suggestion in uWaterloo/OpenData#167

@jacktruong
You can use https://github.com/uWaterloo/Datasets/blob/master/Printers/Printers.csv as inspiration for modeling the data-set.

alternate_building_names should have own file

There should be a separate file to store the data now in column alternate_building_names in Buildings.csv. The columns of this new file should be (building_id, alternative_name) and these should together form the primary key. Once this is done, alternate_building_names should be removed from Buildings.csv.

This creates a better relational structure and keeps the data in Buildings.csv objective.

Another possibility would be to have primary key (building_id, name) and another column to define the status of the name, such as colloquial or archaic.

WCMS site listings have html entites in JSON

JSON strings should not contain HTML entities. Preferred format: & or the \u (Unicode) representation.

Example:

"unit_short_name": "Mechanical & Mechatronics Engg",

should be either:

"unit_short_name": "Mechanical & Mechatronics Engg",

or

"unit_short_name": "Mechanical \u0026 Mechatronics Engg",

Add year field to sunshine list

The endpoint is done but we need to revision the data by adding a year field is the csv

https://api.uwaterloo.ca/v2/resources/sunshinelist.json?year=2003

@axyjo can you add that

See uWaterloo/OpenData#71

Dates have been posted up to 2020

HR has posted the dates up until the end of 2020:
https://uwaterloo.ca/human-resources/pay-administration/paid-holidays

examschedule.json should use ISO time

In examschedule.json, "start_time" and "end_time" are in 12-hour time. I suggest that these keys be deprecated and new keys "time_start" and "time_end" be added showing the times in ISO 24 hour format.

Incorrect `lecture` spelling in /codes/instructions

Via email:

https://github.com/uWaterloo/api-documentation/blob/master/v2/codes/instructions.md

In the code/instructions section of the API. You have a spelling mistake for “LEC” it currently returns “Leture”. You have it wrong in the readme and in the API response.

Just thought I'd let you know.

Adam Gierscher
Honours Mathematics

Missing AP location data.

APs in some buildings are missing location data. For example: APs in QNC.