make rbind() work with DFs with non-identical column sets

[juliadata]

make rbind() work with DFs with non-identical column sets

[tshort]

I couldn't figure out how to merge but keep open issue #198, so I'll repeat my comments from there:

Also ddresses issues #180.

I thought this was better as a pull request because it hits a few different things that may need feedback. Here are a few items of note:

• vecbind is added. I hoped to use vcat, but it wasn't good enough (see JuliaLang/julia#2326).
• To get NAtype DataArrays to work, I added a promote_rule.

promote_rule{T}(::Type{T}, ::Type{NAtype} ) = T

• Because of this promote_rule, we now get warnings (see below). What is the promote_rule for AbstractDataArrays used for? I don't see something equivalent for AbstractArrays.

Here are some examples:

julia> using DataFrames
Warning: ignoring conflicting import of Stats.quantile into DataFrames
Warning: New definition promote_rule(Type{AbstractDataArray{T,N}},Type{T}) is ambiguous with promote_rule(Type{T},Type{NAtype}) at /home/tshort/.julia/DataFrames/src/natype.jl:32.
         Make sure promote_rule(Type{AbstractDataArray{T,N}},Type{NAtype}) is defined first.
Warning: New definition promote_rule(Type{AbstractDataArray{S,N}},Type{T}) is ambiguous with promote_rule(Type{T},Type{NAtype}) at /home/tshort/.julia/DataFrames/src/natype.jl:32.
         Make sure promote_rule(Type{AbstractDataArray{S,N}},Type{NAtype}) is defined first.

julia> d1 = @DataFrame(x => NA, y => 1)
1x2 DataFrame:
         x y
[1,]    NA 1


julia> rbind(d1,d1)
2x2 DataFrame:
         x y
[1,]    NA 1
[2,]    NA 1


julia> d1 = DataFrame({[1:3], LETTERS[1:3]}) # arrays
3x2 DataFrame:
        x1  x2
[1,]     1 "A"
[2,]     2 "B"
[3,]     3 "C"


julia> d2 = @DataFrame(x1 => [5:6], y1 => letters[5:6])
2x2 DataFrame:
        x1  y1
[1,]     5 "e"
[2,]     6 "f"


julia> rbind(d1,d1)
6x2 DataFrame:
        x1  x2
[1,]     1 "A"
[2,]     2 "B"
[3,]     3 "C"
[4,]     1 "A"
[5,]     2 "B"
[6,]     3 "C"


julia> rbind(d1,d2)
5x3 DataFrame:
        x1  x2  y1
[1,]     1 "A"  NA
[2,]     2 "B"  NA
[3,]     3 "C"  NA
[4,]     5  NA "e"
[5,]     6  NA "f"


julia> rbind(d2,d1)
5x3 DataFrame:
        x1  y1  x2
[1,]     5 "e"  NA
[2,]     6 "f"  NA
[3,]     1  NA "A"
[4,]     2  NA "B"
[5,]     3  NA "C"


julia> rbind(d2[[2]],d1)
5x3 DataFrame:
         y1 x1  x2
[1,]    "e" NA  NA
[2,]    "f" NA  NA
[3,]     NA  1 "A"
[4,]     NA  2 "B"
[5,]     NA  3 "C"


julia> rbind(stack_df(d1, ["x1","x2"]),d2)
8x4 DataFrame:
        variable value x1  y1
[1,]        "x1"     1 NA  NA
[2,]        "x1"     2 NA  NA
[3,]        "x1"     3 NA  NA
[4,]        "x2"   "A" NA  NA
[5,]        "x2"   "B" NA  NA
[6,]        "x2"   "C" NA  NA
[7,]          NA    NA  5 "e"
[8,]          NA    NA  6 "f"


julia> rbind(stack_df(d1, ["x1","x2"]),d2) | dump
DataFrame  8 observations of 4 variables
  variable: DataArray{Union(ASCIIString,UTF8String),1}(8) ["x1", "x1", "x1", "x2"]
  value: DataArray{Any,1}(8) {1, 2, 3, "A"}
  x1: DataArray{Int64,1}(8) [NA, NA, NA, NA]
  y1: DataArray{ASCIIString,1}(8) [NA, NA, NA, NA]

[johnmyleswhite]

Sorry: I just meant merge, then reopen.

[garborg]

Long done.