dlab-berkeley/r-data-wrangling

D-Lab's 4 hour two-part workshop on data wrangling in R using tidyverse.

D-Lab Data Wrangling Workshop

This repository contains the materials for D-Lab's Data Wrangling workshop.

Prerequisites

We recommend attending D-Lab's R Fundamentals series prior to this workshop.

Check D-Lab's Learning Pathways to figure out which of our workshops to take!

Workshop Goals

In this workshop, we provide practical guidance on data wrangling using R's tidyverse suite of packages. In Part 1, we cover how to transform tidydata in order to perform data analysis and visualization. In Part 2, we cover how to wrangle messy data into tidy data.

Learning Objectives

After this workshop, you will be able to:

1. Use tidyverse functions for transforming your data to be used for visualization and analysis

2. Identify how an "untidy" dataframe gets in the way of data visualization and analysis

3. Use pivot_longer and pivot_wider to wrangle a dataframe into "tidy" form, and join multiple dataframes

This workshop does not cover the following:

• Data visualization in R (this is covered in D-Lab's R Data Visualization workshop)
• Advanced data analysis in R, such as machine learning (this is covered in D-Lab's R Machine Learning workshop)

⚠️ At the start of the workshop we will install R, RStudio and the workshop materials from this repository. If you want to do that yourself beforehand, please follow the instructions below.

Installation Instructions

We will use RStudio to go through the workshop materials, which requires installation of both the R language and the RStudio software.

1. Download R: Follow the links according to the operating system that you are running. Download the package, and install R onto your compute. You should install the most recent version (at least version 4.3).

2. Download RStudio: Install RStudio Desktop. This should be free. Do this after you have already installed R. The D-Lab strongly recommends an RStudio edition of 2022.02.0+443 "Prairie Trillium" or higher. Check RStudio is working by opening up the program.

3. Download these workshop materials:

   • Click the green "Code" button in the top right of the repository information.

   • Click "Download Zip".

   • Extract this file to a folder on your computer where you can easily access it (we recommend Desktop).

4. Optional: If you are familiar with git, you can instead clone this repository by opening a terminal and entering

Is RStudio not working on your computer?

If R/RStudio is not installing correctly or you do not wish to download these programs to your laptop, we strongly recommend using the UC Berkeley Datahub to run the materials for these lessons. You can access the DataHub by clicking the following button:

The DataHub downloads this repository, along with any necessary packages, and allows you to run the materials in an RStudio instance on UC Berkeley's servers. No installation is necessary from your end - you only need an internet browser and a CalNet ID to log in. By using the DataHub, you can save your work and come back to it at any time. When you want to return to your saved work, just go straight to the D-Lab DataHub, sign in, and you click on the [Workshop-Name] folder.

Additional Resources

Check out the following resources to learn more about data wrangling in R:

• R for Data Science: A comprehensive online textbook by Hadley Wickham, who helped develop tidyverse

About the UC Berkeley D-Lab

D-Lab works with Berkeley faculty, research staff, and students to advance data-intensive social science and humanities research. Our goal at D-Lab is to provide practical training, staff support, resources, and space to enable you to use R for your own research applications. Our services cater to all skill levels and no programming, statistical, or computer science backgrounds are necessary. We offer these services in the form of workshops, one-to-one consulting, and working groups that cover a variety of research topics, digital tools, and programming languages.

Visit the D-Lab homepage to learn more about us. You can view our calendar for upcoming events, learn about how to utilize our consulting and data services, and check out upcoming workshops.

Other D-Lab R Workshops

Here are other R workshops offered by the D-Lab:

Basic R

R Fundamentals

Intermediate/Advanced Competency

R Data Visualization

R Machine Learning (links to the pilot)

R Deep Learning

Contributors

Hero Ashman

r-data-wrangling's Issues

Part 1 - IOKN2K

Remove the first mention of IOKN2K so that instructors don't skip the subsection on it. Also make sure we don't skip over the project directory section.

Part 2 - for the first lengthening data

First pivot_longer() example takes a long time to explain and there is repetition between the text and the diagrams

• consider putting the diagrams before the code and replacing the text with the diagrams
• switch diagram 2 and 1 - want states to be shown first, then worker types to come second

Project file naming

Make sure that the .Rproj file is named the same as the workshop and when people download the github material they should be able to click on the Rproj file to open up RStudio

Part 2 - Challenge 1

Challenge 1 was too hard - no one seemed to be able to finish it

• Supposed to draw on challenge 6 from part 1 so its integral that we get there
• Talk over outside the challenge what are the variables that we would need to answer this question
• • When doing this go through the dataset together
• • This will also give more context for the data and the challenge
• More context for the variables would be helpful (maybe from Part 1)
• Ask people to share their code when they’re stuck
• Prompt people to look at documentation or Google - one participant said “stuck on mutate function…can’t recall the syntax for this function”
• Could split it into two parts - calculating the total and then the share
• For each part show the dataset and ask for validation that we’ve done it right (for instance, rough population estimate of the states)
• Add in hint: the cleanest tidyverse way to do this is to use these functions in this exact order

Part 2 - challenge 3

Challenge 3 - wording is confusing

• Clarify that the college variable is share of people with a college degree and don’t need to do any modifications with summarize() or mutate() before answering this question - just need a pivot_wider()
• More guidance in what needs to happen and then let the participants work out how to code it up
• Point out at the end how the income ranking and the college degree ranking is similar - might be a clue as to why the income difference is large in certain places

Challenge 6 - part 1

• Add in a hint that says that all this code can be found above in different subsections
• Add in the codes for worker_type into the challenge so they don't have to scroll up
• remove the chatGPT tip b/c we haven't actually used it anywhere else

Challenge 3 - Part 1

Challenge 3 was harder and too long - most got first one, 3/6 got two
Maybe instead do the first one all together not as a challenge
Then have conditions 2 and 3 as a challenge

Part 2 - joining data

• start with the diagram before we go over the syntax of the code
• make sure the image file path is correct (images/join.jpg")

Challenge 5 - Part 1 - typo

In the tip should say "using" rather than "use"

Part 2 - line 103

Highlight the word "SAME"

Part 2 - conceptual question around line 114

Remove - its too confusing to ask how they would get us back to tidy data when we haven't talked about wrangling at all

Typo in poll 1-1

"original"

Incorrect name in Challenge 1 Part 1

Replace .csv with .rmd

arrange() in Part 1

Add in a line of mutate() to make worker_type a factor before doing arrange() so that when they take the average education they can see the worker types rather than just numbers.

Also add in description about what mean(educ == "BA" | educ == "graduate") - say why it wouldn't make sense for & and only |

Answer for question in filter() - part 1

How do we know we got the right rows? Goal is to demonstrate table() and unique() - could have them in the code above when making the d_west_coast df

Working directories

Be explicit in challenge 1 that the working directory between a notebook and the console is different. Also if people haven't opened via .RProj then they need to set a working directory if they want to use the console (get them to type pwd)

Exploring data after challenge 1

Add in explicit text to get people to explore the dataframe that's read in and see what variables we have

Part 2 - pivot_longer() section

Have only one pivot_longer() and one pivot_wider() → don’t need to demonstrate everything that could lead us to a pivot_longer() but want to give people the general problem (values in the variables) and one example and challenge