AMC is an open-source Java package implementing the algorithm proposed in the paper (Chen and Liu, KDD 2014), created by Zhiyuan (Brett) Chen. For more details, please refer to this paper.
If you use this package, please cite the paper: Zhiyuan Chen and Bing Liu. Mining Topics in Documents: Standing on the Shoulders of Big Data. In Proceedings of KDD 2014, pages 1116-1125.
If you have any question or bug report, please send it to Zhiyuan (Brett) Chen ([email protected]).
## Quick StartFirst, Clone the repo: git clone https://github.com/czyuan/AMC.git
.
Then, 2 quick start options are available:
- Import the directory into Eclipse (recommended).
If you get the exception Java.Lang.OutOfMemoryError, please increase the Java heap memory for Eclipse: http://www.mkyong.com/eclipse/eclipse-java-lang-outofmemoryerror-java-heap-space/.
- Use Maven
a. Then, change the current working directory to Src.
cd AMC/Src
b. Build the package.
mvn clean package
c. Increase the Java heap memory for Maven.
export MAVEN_OPTS=-Xmx1024m
d. Run the program.
mvn exec:java -Dexec.mainClass="launch.MainEntry"
- -ismall: the path of input domains directory containing small dataset (e.g., 100 reviews each domain).
- -ibig: the path of input domains directory containing big dataset (e.g., 1000 reviews each domain).
- -o: the path of output model directory.
- -nthreads: the number of threads used in the program. The program runs in parallel supporting multithreading.
- -nTopics: the number of topics used in Topic Model for each domain.
- domain.docs: each line (representing a document) contains a list of word ids. Here, a document is a sentence in a review after preprocessing.
- domain.vocab: mapping from word id (starting from 0) to word, separated by ":".
The output directory contains topic model results for LDA and AMC.
Under each model (LDA or AMC) directory, we have results for different datasets (small or big). Under the sub-folder "DomainModels", there are a list of domain folders where each domain folder contains topic model results for each domain. Under each domain folder, there are 9 files (can be opened by text editors):
- domain.docs: each line (representing a document) contains a list of word ids.
- domain.dtopicdist: document-topic distribution.
- domain.knowl_cannotlinks: record the cannnot-links used in the model (for AMC only).
- domain.knowl_mustlinks: record the must-links used in the model (for AMC only).
- domain.param: parameter settings.
- domain.tassign: topic assignment for each word in each document.
- domain.twdist: topic-word distribution
- domain.twords: top words under each topic. The columns are separated by '\t' where each column corresponds to each topic.
- domain.vocab: mapping from word id (starting from 0) to word.
- Increase the number of threads in the program (specified by -nthreads in file "global/CmdOption.java"). The topic models are execuated in parallel in each domain using multithreading.
- Use a better implementation for Apriori algorithm with muliple supports or use faster frequent itemset algorithm such as FP-growth.