BUG: Columns resulting from joining multiIndex dataFrame are incorrect in python 3.12.1

[yuji38kwmt]

BUG: Columns resulting from joining multiIndex dataFrame are incorrect in python 3.12.1

[yuji38kwmt]

@rajnee28

can you pls provide folder path for this issue.

The bug does not depend on folder path.

For example, you can reproduce the bug in IPython.

In [59]: import pandas
    ...:
    ...: df1=pandas.DataFrame({("A",""):[1,2,3]},index=["x","y","z"])
    ...: df2=pandas.DataFrame({("B",""):[11,12,13]},index=["x","y","z"])
    ...:
    ...: df3 = df1.join(df2)
    ...:
    ...: print(f"{df3.columns=}")
df3.columns=Index([slice(None, None, None), slice(None, None, None)], dtype='object')

[rhshadrach]

Confirmed that this still exists on 2.2.1, but not yet on main. Further investigations and PRs to fix are welcome!

[rajnee28]

@yuji38kwmt - can you pls provide folder path for this issue.

[Groni3000]

I don't know what causes the problem, but concat is a little workaround:

df3 = pd.concat([df1, df2], axis=1) #there is join options too

[yuji38kwmt]

The following code behaved expectedly.

In [89]: import pandas
    ...:
    ...: df1=pandas.DataFrame({("A","X"):[1,2,3]},index=["x","y","z"])
    ...: df2=pandas.DataFrame({("B","Y"):[11,12,13]},index=["x","y","z"])
    ...:
    ...: df3 = df1.join(df2)

In [90]: print(f"{df3.columns=}")
df3.columns=MultiIndex([('A', 'X'),
            ('B', 'Y')],
           )

Apparently the problem occurs when the key with label 1 is an empty string.

[yuji38kwmt]

I did a litter investigated the bug.

The following code creates a pd.Series containing index which of name is slice(None, None, None).
https://github.com/pandas-dev/pandas/blob/v2.2.1/pandas/core/reshape/merge.py#L837-L838

Trial1

• Python 3.12.1
• pandas 2.2.1

In [21]: df=pandas.DataFrame({"A":[1,2,3]},index=["x","y","z"])

In [22]: df[:]
Out[22]:
   A
x  1
y  2
z  3

In [24]: df[:].columns
Out[24]: Index(['A'], dtype='object')



In [25]: df11=pandas.DataFrame({("A","x"):[1,2,3]},index=["x","y","z"])

In [26]: df11[:]
Out[26]:
   x
x  1
y  2
z  3

In [27]: df11[:].columns
Out[27]: Index(['x'], dtype='object')

In [28]: df12=pandas.DataFrame({("A",""):[1,2,3]},index=["x","y","z"])

# Look at the code !
In [29]: df12[:]
Out[29]:
x    1
y    2
z    3
Name: slice(None, None, None), dtype: int64

In [30]: type(df12[:])
Out[30]: pandas.core.series.Series

Trial2

• Python 3.11.7
• pandas 2.2.1

In [3]: df11=pandas.DataFrame({("A","x"):[1,2,3]},index=["x","y","z"])

In [4]: df11[:]
Out[4]:
   A
   x
x  1
y  2
z  3

In [5]: df11[:].columns
Out[5]:
MultiIndex([('A', 'x')],
           )

In [6]: df12=pandas.DataFrame({("A",""):[1,2,3]},index=["x","y","z"])

In [7]: df12[:]
Out[7]:
   A

x  1
y  2
z  3

In [8]: df12[:].columns
Out[8]:
MultiIndex([('A', '')],
           )