Summary bucketing to columns

[ebay]

Summary bucketing to columns

[jondegenhardt]

Nice example. I need to read through your proposals in more detail. However, my initial thought is that what you mention in point 2 is the key. Using tsv-summarize -H --group-by 1,2 --count produces "long, narrow" data. What you are asking for is a "wide" data format. Both forms are commonly used in tools for statistics, machine learning, etc. I need to put my own data in those forms quite commonly, so I definitely want to support them.

tsv-append was created for creating long, narrow data. tsv-join is good for creating wide data sets. However, it can be cumbersome to use in certain circumstances and I expect to create a version more tailored to the task. I haven't spent much time thinking about how tsv-summarize fits into this, but figuring this out might help keep the tools self-consistent.

A couple references on long/narrow and wide data formats:

• http://stanford.edu/~ejdemyr/r-tutorials/wide-and-long/
• http://vita.had.co.nz/papers/tidy-data.html

[Llammissar]

Real quick before bed: after posting this, I discovered datamash (exists. Also that it) has cross-tabulation as a separate tool, so there's at least one toolset in the same domain treating it as a separate tool. Idly, I wonder if there's not room for both concepts to coexist?