loganalyzehelper's Introduction

logAnalyzeHelper

工程的目的

该工程是论坛日志分析系统的辅助工程，主要功能模块是帮助日志分析系统清洗数据、以及在Hive中建立相应的数据表。

工程目录结构

/src

------/main

--------------/java

---------------------/cn.edu.hust

---------------------------------------/preprocess

-------------------------------------------------/domain

-------------------------------------------------------/WebLogBean 日志的POJO对象

-------------------------------------------------/ClickStream r日志清洗数据

---------------------------------------/udf

-------------------------------------------bean

---------------------------------------------Pair IP查找的辅助类

-------------------------------------------BrowserUtils 获取浏览器的UDF函数

-------------------------------------------CityUtils 获取城市的UDF函数类

-------------------------------------------IPUtils 获取省份的UDF函数类

-------------------------------------------OSUtils 获取操作系统UDF函数类

-------------/resources

------/test

------/pom,xml

工程使用的数据集

工程使用的数据集可以在作者的百度云中下载日志数据,

工程的流程

原始数据--->数据规整--->ETL--->导入mysql数据库--->可视化

ETL使用的SQL

使用的SQL在本本工程中命名为点击流.sql

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.

Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

TensorFlow

An Open Source Machine Learning Framework for Everyone

Django

The Web framework for perfectionists with deadlines.

Laravel

A PHP framework for web artisans

D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

web

Some thing interesting about web. New door for the world.

server

A server is a program made to process requests and deliver data to clients.

Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

Visualization

Some thing interesting about visualization, use data art

Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.

Microsoft

Open source projects and samples from Microsoft.

Google

Google ❤️ Open Source for everyone.

Alibaba

Alibaba Open Source for everyone

D3

Data-Driven Documents codes.

Tencent

China tencent open source team.

hanmichael / loganalyzehelper Goto Github PK