guilhermenoronha/stats_bg

End to end project to collect and analyze data from my board game club.

It's me, Guilherme Noronha

Data Engineer | Python Developer

• 📊 Working as a Data Engineer at the moment.
• 🌱 Learning about Data Quality

Career

📊 Data Engineer - Five Acts (07/2022 - 12/2023)

📊 Data Engineer Professor - IGTI (12/2020 - 11/2022)

📊 Computer Science Professor - IFMG (08/2019 - 12/2019)

📊 Information Science Professor - UFMG (2016; 2017; 2020; 2021)

📊 Software Developer - Sisloc Softwares (10/2011 - 02/2014)

Education

👨‍🎓 Ph.D. in Knowledge Organization and Management - UFMG (2018-2022)

👨‍🎓 Master in Knowledge Organization and Management - UFMG (2015-2017)

👨‍🎓 Bachelor in Computer Science - PUC Minas (2008 - 2012)

Publications

📃 PROCESSAMENTO DE LINGUAGEM NATURAL PARA ANÁLISE DE SENTIMENTO UTILIZANDO PYTHON - Tecnologias utilizadas em pesquisas acadêmicas em Ciências Sociais Aplicadas

📃 ENSINO SUPERIOR DURANTE A PANDEMIA: UMA EXPERIÊNCIA DOCENTE EM SISTEMAS DE RECUPERAÇÃO DA INFORMAÇÃO - Informação & Informação

📃 Representação lógica do conhecimento para ontologias - Fronteiras da Representação do Conhecimento

📃 Análise gramatical de etiquetas em sites colaborativos: estudo de caso Flickr - Comunicação & Informação

📃 DESIDENTIFICAÇÃO DE DOCUMENTOS CLÍNICOS PARA PESQUISA BIOMÉDICA - REPRESENTAÇÃO DO CONHECIMENTO, ONTOLOGIAS E LINGUAGEM: pesquisa aplicada em Ciência da Informação

📃 Using Adaptive Automata as a Phonetic Routine Prototype - IEEE Latin America Transactions

Tools

Languages

   

Databases

     

Frameworks

 

Packages

   

OS

   

Other

   

stats_bg's Issues

Attendances table creation doesn't consider a person if he/she attended late.

The way the sheet is annotated, if a person would arrive at the meeting late, he/she will not be considered to be attended the meeting. To solve this, the code must consider every person per date, not per row (the first one) as it is now.

Replace selenium with scrapy

Selenium is proving too slow for web scrapping. A replacement should improve performance.

Add last_date on persons and games table

Add a last_date on persons table to indicate which was the last time a person attended to a game. Do the same on games table to indicate what was the last time a game was played.

Create boardgame taxonomy tables

Create 4 tables using Ludopedia Scrapper:

• BG_DOMAINS: with game domain
• BG_THEMES: with game themes
• BG_CATEGORIES: with game cateogories
• BG_MECHANICS: with game mechanics

Each game may have multiples taxonomies in each table.

Matches table is created wrongly

The game_id on matches table is, in fact, showing the game_owner_id. The correct would to show the unique id from every game instead.

Create opponent of the year KPI

This KPI measures which player beats more opponents on average.

Add data tests

Use great expectations or similar to guarantee data quality

Create a Ludopedia Scrapper

Create a scrapper for ludopedia to get user's collection and board games metadata to enhance the data analysis

Refactor code to adapt it to ludopediascrapper

Refactor the whole data_wrangler.py to use ludopediascrapper.py

Create KPI hoarder of the year

This KPI measures the player who acquired more games through the year.

Add Boardgamegeek scrapper

Add a scrapper to get "game weight" and "game best played" with features.

Add unittests to the attendances.py file.

Add unittests to cover all project functions

Implement database to support multple sessions in the same day

Actually, the database doesn't support multiple sessions in the same day.

Example:

Player 1 hosted a game in his house.
Player 2 hosted a game in his house.

The attendances and matches databases should distinct all the matches and attendances for this day as separated ones.

Add column game_details in stats_bg sheets

The matches sheet in stats_bg must have a game_details column to provide more details about the matches. Below are some examples:

• Register different abilities used by the players
• Register the main strategy used by the players
• Register the game mode played
• Etc.

Add KPI to see how much (in %) a player plays its own games in each session

The goal of this KPI is to see if a player is able to play his own games. The metric sees if in a session the player was able to play at least one of this game.

For example: I attendend in 10 sessions. In 4 of them I played my games. How much I played my games? 40%.

Added new ranks for end of the year awards

Add the folowing metrics to the stats:

• Vice award (to the player who came most times in second place)
• Loser award (to the player who came most time in last place)

Use weighted average on players' ranking

Players ranking should be an average weight which encompass the game weight and the number of opponents in a match:

$$ R = \frac{ \sum_{i=1}^n (gameWeight_i * numPlayers_i * playerRank) } { \sum_{i=1}^n (gameWeight_i * numPlayers_i) } $$

add game_owner to the stats_bg sheets

The sheet must have a game_owner column. This column is necessary to disambiguate the game stats for the games which have more than one owner.

Implement DBT framework to refine data wrangling

Add dbt framework to refine date from bronze to gold.

Add KPIs

Implement following KPIs to be loaded on dataviz (subject to changes)

KPIs (Board game):

• How many board games we have?
• How distributed the board games are (per member)?
• Which are the mechanic most popular?
• Which are the category most popular?
• Which are the theme most popular?
• What is the average weight of the board games?
• What is the recommended number of players (min, max, avg)?

KPIs (Session)

• How many sessions we had?
• What is the average games per session?
• What is the average players per session?
• How is the interchanging players between sessions?
• How is the interchanging hosts between sessions?
• How long a member showed up last time?
• How long a game was played last time?

Add Awards KPIs

Add the following KPIs on DBT:

• Host of the year
• Addicted of the year
• Group of the year
• Game of the ear
• Explorer of the year
• category of the year
• mechanic of the year
• theme of the year
• strategist of the year
• sommelier of the year

Create Dashboard

Create a dashboard with every KPI and awards metrics created in issues #18 and #22