alexkuva/the-rougon-macquart-project

the-rougon-macquart-project's Issues

Typo in Medium article

Hi there!
This is about an article. In a section "Define functions" you have the same description "Calculate number of Files and number of Folders for a specific path" for both functions. 1st one should be about the size, I guess.

Notebook returns empty dataframe

Hello there!
I was trying to create/find a tool to calculate the size of our ADLS Gen1. I wrote my own one in Python, but the lack of performance on a local PC is killing me. I've found your project quite promising as you are using parallel execution. I've imported an example notebook to my environment and set it up. As I was able to get a list of folders in a given root, I assume that all the settings are correct. In the pre-last cell I have a df filled with datasource, path and env values. When I run the last cell, I expect it will take each directory path and will dive into it using listdir command, summarizing all file sizes and counts in a recursive way, but it just does nothing. I've added display(df) command after that part in brackets, it shows me the same df as the previous cell. What am I doing wrong? My cluster looks like this:
42.00 GB | 12 Cores | DBR 7.3 LTS | Spark 3.0.1 | Scala 2.12

Thanks in an advance!