marinestrada/dedomena

Database creator

dedomena's Issues

Given file structure read files with spark

###What
given a file system such as
folder1
file1
file2
file3
folder2
file1
file2
file3
....
you can assume that there is only one level of files at this point, how you save the files is your design at this point. But they should be saved using spark.

Hints:
find out what is spark --> quick summary allow you to read files and perform queries on data read using python and sql syntax.
File out how to read file with spark.