hadley/tidy-data

A paper on data tidying

Tidy data

Files available in this package:

• tidy-data.tex and tidy-data.pdf: the latex source to generate the paper and the resulting pdf

• data/: raw datasets, the code to tidy them, and the results, as used in Section 3. Source individual .R files to recreate the tidied data.

• t-test.r: code used to generate Table 14 (model-1.tex and model-2.tex), comparing data needed for paired t-test vs. a mixed effects model.

• case-study/: the code and data for the case study in Section 5. Run case-study.r to recreate all tables and plots.

tidy-data's Issues

Comments from Stavros

Re "observational unit" and "level", I still think a better definition would be useful, perhaps in 2.3, perhaps in 3.4.

I wonder if the observations about denormalization (parag 3 of 3.4 and last parag of 4.1) might be worth elaborating on? Or maybe not... the examples of join in 5 might be enough.

I really like the "tidy data" framework and the way it highlights the cleanness and unity of the tools you've developed.

On p. 22, line 4, I think the comma after "data" is inappropriate because the "which" starts a restrictive clause.

About the ETL literature, I agree that a lot of it is either commercial or very IT-ey (as opposed to CS-ey), but there does seem to be a fair amount of discussion around data warehouse schemas. Some other papers I've come across (I don't claim to have done a serious literature search!) include Boehnlein, Vassiliadis et al., and Vassiliadis (p. 9ff talks about the "pivoting problem").

Create an updated version of the paper using `dplyr`

In my opinion, the "Tidy Data" paper is a good resource for R students to learn about the concept. However, this means that they learn about the concept of tidy data with examples using plyr, which has, as far as I know, largely been replaced by dplyr. It seems to me that it would be preferable to introduce the concept of tidy data and the tools for its transformation in the same paper. Now, they learn about the plyr functions while reading the paper and then have to learn about dplyr afterwards. I therefore think that an updated version of the paper would make sense for these purposes.

This is certainly not an important issue but I do feel like it would make an introduction to R more efficient. As I realize that this is probably not a priority for you, I would be happy to create such an updated version. If you have concerns, I am, of course, happy to drop the issue, as well.