This repository recommends sports-climbing routes
I created a route recommender based on the features of the routes. I took the databases scrapped from the website 8a.nu and cleaned them. After cleaning I used K-means modelling to cluster the data and get a table of clustered routes. With that information and the routes ratings I was able to build a recommender based on the user's input features and routes climbed.
I also created an app using this recommender. I deployed it in:
-
Heroku:
- Deployed for free on Heroku platform. You can find it online on this link
-
AWS:
- In this case I dockerized the app and connected to a MongoDB docker container in order to have data persistency.
- Everything done inside an AWS EC2 instance. you can find it on this link. As long as my EC2 instance is up.
Notebooks
src/1.get_routes_table.ipynb
- Where I get routes table by cleaning and playing with the ascents tablesrc/2.clustering_routes.ipynb
- Where I apply K-Means model to cluster the routues by their featuressrc/3.routes_rating.ipynb
- Where I get the routes global rating from different featuressrc/4.route_recommender.ipynb
- Where I define the route recommender
Clases
src/lib/climber_class.py
- User classsrc/lib/location_class.py
- Class for the locationsrc/lib/text_processor_class.py
- It contains sentiment, language recognition, word matching and cleaning.src/lib/grades_class.py
- Used for the class conversor
Guides
- I created these guides to understand how my classes work
GUIDE.climber_class.ipynb
GUIDE.grades_conversor_class.ipynb
GUIDE.location_class.ipynb
GUIDE.text_processor_class.ipynb
Streamlit
app/app.py
- The app in streamlit.
In this section I explain the whole process. You can find the notebook files on the src/
folder numbered, so it will be easier to follow. In the end of this document there is a diagram explaining the workflow.
In order to obtain the routes list features I took an 'ascents table' and converted it into a 'routes table'. For that purpose I had to:
- String cleaning for each column (using class
text_processor_class.py
) - Sentiment (first checks language and then if comments are positive or negative, using class
text_processor_class.py
) - Splitting
- Cleaning and filtering outlayers
- Adding climbers table features (this table was obtained from ascents table as well, how to in this project)
- Clean and group naming (it compares names and replaces if they are alike, using class
text_processor_class.py
)
I separated the features in 3 types:
- Independent features - I created groups or clusters using K-Means modeling and visualized using PCA. Every route was asigned to a cluster.
- Rate features - These features were analyzed using PCA and finally mixed into a single feature that will sort the cluster option
- User features - - These ones are defined by user and are used as a filter.
With the final table I was able to create a recommender based on new user features and climbed routes. I created an app to make it easier and more user friendly. You can input the climber information and the recommender will output the recommended routes per region, cluster and ordered by rating.
Clusters (more or less) explanation
- 0 - Soft routes
- 1 - Routes for some reason preferred by women
- 2 - Famous routes
- 3 - Very hard routes
- 4 - Very repeated routes
- 5 - Chipped routes, with soft rate
- 6 - Traditiona, not chipped routes
- 7 - Easy to On-sight routes, not very repeated
- 8 - Very famouse routes but not so repeated and not so traditional
-
This repository uses 3 different databases. Users, ascents and grades.
-
The data is not stored here but you can find them in this link: https://www.kaggle.com/dcohen21/8anu-climbing-logbook
Thanks to David Cohen for creating the scrapper (https://github.com/mplaine/8a.nu-Scraper) and hosting the data on Kaggle website (https://www.kaggle.com/dcohen21/8anu-climbing-logbook)
- matplotlib==3.3.4
- numpy==1.19.5
- pandas==1.4.1
- polyglot==16.7.4
- regex==2020.11.13
- scikit_learn==1.0.2
- scipy==1.6.0
- seaborn==0.11.2
- sentiment_analysis_spanish==0.0.25
- streamlit==1.8.0
- vaderSentiment==3.3.2