wireservice/agate

A Python data analysis library that is optimized for humans instead of machines.

wireservice

Name	Badges
agate
agate-charts
agate-dbf
agate-excel
agate-lookup
agate-remote
agate-sql
agate-stats
csvkit
leather
proof

agate's Issues

Cookbook: filter to values within a range

Column.all

BooleanColumn

See https://github.com/onyxfish/csvkit/blob/master/csvkit/typeinference.py#L54

NumberColumn.quintiles

Return a list of quintiles for the given column.

Table.__init__: validate dimensions of rows, names and types

Tests failing due to float precision issues

FAIL: test_median (tests.test_columns.TestFloatColumn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/onyxfish/src/journalism/tests/test_columns.py", line 191, in test_median
    self.assertEqual(self.table.columns['two'].median(), 2.805)
AssertionError: 2.8049999999999997 != 2.805

======================================================================
FAIL: test_sum (tests.test_columns.TestFloatColumn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/onyxfish/src/journalism/tests/test_columns.py", line 177, in test_sum
    self.assertEqual(self.table.columns['two'].sum(), 10.71)
AssertionError: 10.709999999999999 != 10.71

Cookbook: generate a histogram

Tests for IntColumn.stdev and FloatColumn.stdev

Column.frequency

Stubbed as freq, but should be spelled. Returns a dict, I guess? Or a Table?

Table.join

Table.percent_change

Compute percent change between two columns into another

Multi-column sort

Column.variance

Cookbook: emulating Excel formulas

Table.compute

Table.find

Find the first row that satisfies the criteria

Cookbook: filter by glob

NumberColumn.quartiles

Return a list of quartiles for the given column.

Cookbook: emulating vlookup

Cookbook: filter by regex

Table.compute_column(func)

Iterate over the table and compute a new column based existing columns

What to call this?

Table.merge

Like Table.join (#11) but for rows.

DateTimeColumn

See #7

Column.count

Count instances of a value

Column.any

Cookbook: write a table back out to a CSV

Cookbook: loading a csv

Column.some(func)

Returns true if any value in the column passes a truth test

Py3 support

Memoize instantiated Columns

Table.filter and Table.reject should take test funcs, not lists

This should be standard... maybe support both?

Column.__contains__

Support variable-length rows

csvkit does this by return None when a key is outside the row range, which seems sound

NumberColumn.percentiles

Return an list of percentiles for a given column.

Table.remove_column

Should have a way to create a new table from a subset of columns of an existing table. What is the API for this? What is it called? subset? part? subtable?

Cookbook: using with csvkit

TimeColumn

See #7

Append columns to Table

How should this work? Operator overloading? Function call? What sort of validation do we need?

Cookbook section of docs

Cookbook: filter to 90th percentile

DateColumn

See https://github.com/onyxfish/csvkit/blob/master/csvkit/typeinference.py#L126

Tests for Column.cast

Eliminate duplicative column index lookups

Since tables are immutable this can be cached:

i = self._table._column_names.index(self._k)

Memoize data access functions in Column

_data, _data_without_nulls, _data_sorted

Reminder: tables are immutable so we can cache this indefinately

Cookbook: emulating pivot table

Cookbook: levenshtein

Combine Table.where with levenshtein algorithm. Maybe this one? https://pypi.python.org/pypi/python-Levenshtein/0.11.2

Table.rank

Table.rank(self, func)

func returns a value, these values are compared to generate a rank IntColumn.

Optional sort_by_rank param?

Cookbook: round a column to n decimal places

Beg, borrow and steal from latimes-calculate

http://latimes-calculate.readthedocs.org/en/latest/

Most of these probably don't make sense, but a few, like Benford's would be a good fit

Column.map API isn't consistent

Should probably just drop this now that Table.compute is in?

Table.group

Should output an dict of tables

Should take a function name that returns a grouping key