BUG: Unexpected read_csv parse_dates behavior

[kounoupis]

BUG: Unexpected read_csv parse_dates behavior

[Groni3000]

Same problem here. Though I don't think manual using of dtype and parse_dates together can be considered as something that has "expected behaviour".

Btw, if you use

columns = {"Ticker": str, "Last Update Timestamp": pandas.StringDtype()}

There is no problem at all.

And one more thing: try to specify incorrect date_format like that:

df = pandas.read_csv(
        csv,
        usecols=list(columns.keys()),
        dtype=columns,
        parse_dates=["Last Update Timestamp"],
        date_format="%Y/%m/%d %H:%M:%S",
    )

Kind of magic xD. Definitely a bug. Seems like date format inferring or something like that... because if you set

df = pandas.read_csv(
        csv,
        usecols=list(columns.keys()),
        dtype=columns,
        parse_dates=["Last Update Timestamp"],
        #date_format="%Y/%m/%d %H:%M:%S",
        infer_datetime_format=True,
    )

Still has a bug.

INSTALLED VERSIONS

INSTALLED VERSIONS ------------------ commit : f538741 python : 3.11.4.final.0 python-bits : 64 OS : Windows OS-release : 10 Version : 10.0.19045 machine : AMD64 processor : AMD64 Family 23 Model 113 Stepping 0, AuthenticAMD byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : Russian_Ukraine.1251

pandas : 2.2.0
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.8.2
setuptools : 65.5.0
pip : 24.0
Cython : None
pytest : 8.0.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.8.3
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 15.0.0
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None

[rhshadrach]

Thanks for the report and investigation, confirmed on main. Further investigations and PRs to fix are welcome!

Also, while they appear as integers, I'm seeing that the output in the OP are in fact strings.

[ruimamaral]

take