Console analysis application for the most frequent matches among the most frequent words for applications with open code
This is a tutorial work for the third task in the OTUS “Python web-developer” course. Links to a task:
- https://gist.github.com/Melevir/5754a1b553eb11839238e43734d0eb79
- https://gist.github.com/Melevir/cd9f3f3935f9dfe34d78ae9b18f8e2b9
The application analyzes, shows and saves to file, how frequently the most common words in code's parts repeat in a projects set.
For example, the result ('take', 5) 2
shows the word take
was found 5
times inside each project for all 2
projects.
Next opportunities are available:
- code language:
python
; - projects locations: local drives(for instance
sda1/projects/1
for the *nix-based OS orC:\\projects\1
for the Windows OS), github.com-repositories(for instance"https://github.com/congor/otus_task_1"
) ; - part of speach:
NN
- nouns andVB
- verbs (for python); - places for searching:
funtions
- functions names,variables
- variables names; - saving files format:
json
,csv
.
- A programming language is Python 3.
- A settings file format is 'json'.
- Due to this application is for console usage, results are outputted to console in the form of column, where every string has a construction has mentioned in
Usage objective
. - Results files are being saved in the path
code_analyzer/results
with a proper extensions. - Due to parts of speech is based on
nltk
-library algorithm, some words having different parts of speech may be recognized as not a required part of speech. - The application considers a context of using every word in required places for searching and that way determines a proper part of speech that has been applied in these names and takes words have been used only in a part of speech.
- Clone or download with extracting this application.
- Install required libraries:
$ pip install requiremets.txt
- If there is a requirement, change settings in the settings file. It is located in the path
code_analyzer/settings/settings.json
.
- A formats for each settings have been mentioned in
Usage objective
. - The default settings file has next structure and settings:
{
"code_language": "python",
"part_of_speech": "VB",
"searching_place": "functions",
"projects": [".", "https://github.com/congor/otus_task_1"],
"saving_format": "json"
}
- Make sure that the folder
test_data
is not located inside the same folder where the main application filecode_analyzer.py
is. Otherwise the internal test mode will be launched. - Open console and from the folder
code_analyzer
launch the application file:
$ pyhon3 code_analyzer.py
- Take results in console and the file with results in
code_analyzer/results
.
- For internal testing put the folder
test_data
to inside the folder where the application filecode_analyzer.py
is and launch this application the same way has pointed above.