Exclude directories from expanded glob result

[nameexhaustion]

Exclude directories from expanded glob result

[deanm0000]

I can't find the issue but I like the suggestion someone had of just making it like how pyarrow datasets work where you'd just do pl.scan_parquet('.env/dataset') rather than having to put in all the asterisks.

[kszlim]

The glob pattern is useful for cases where you want to selectively "ignore" legal hive partition parts as well, I don't think it should be removed entirely.

Like:

pl.scan_parquet("s3://my-bucket/*/id=5/*")

[deanm0000]

For backward compatibility I don't think changing it outright should be on the table but, at least off the top of my head, I don't see why it couldn't detect that the input isn't using globbing and then that it is a directory (not a file) and then take up the pyarrow like behavior from there.