magsilva / dtm-old Goto Github PK
View Code? Open in Web Editor NEWDynamic Topic Model (based upon code released by David Blei at http://www.cs.princeton.edu/~blei/topicmodeling.html)
Dynamic Topic Model (based upon code released by David Blei at http://www.cs.princeton.edu/~blei/topicmodeling.html)
THIS REPOSITORY HAS BEEN ARCHIVED Please, use the new repository/project at: https://github.com/magsilva/dtm Reason for archival: this repository was created before Blei creating his official project at Github (https://github.com/blei-lab/dtm). Hence this project is not shown as its fork (which is not appropriate). With the new project at magsilva/dtm, we how have it as a proper "fork" of blei-lab/dtm. *************************** Dynamic Topic Models and the Document Influence Model *************************** This code is the result of work by David M. Blei blei[at]cs.princeton.edu and Sean M Gerrish sgerrish[at]cs.princeton.edu. (C) Copyright 2006, David M. Blei (blei [at] cs [dot] princeton [dot] edu) (C) Copyright 2011, Sean M. Gerrish (sgerrish [at] cs [dot] princeton [dot] edu) It includes software corresponding to models described in the following papers: [1] D. Blei and J. Lafferty. Dynamic topic models. In Proceedings of the 23rd International Conference on Machine Learning, 2006. [2] S. Gerrish and D. Blei. A Language-based Approach to Measuring Scholarly Impact. In Proceedings of the 27th International Conference on Machine Learning, 2010. These files are part of DIM. DIM is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. DIM is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA ------------------------------------------------------------------------ A. COMPILING You will need to have several libraries installed to compile this package: gsl gflags Depending on your package manager, you may be able to install these with *one* of the following commands: sudo aptitude install libgsl0-dev # Ubuntu 10.04 sudo zypper install gsl-devel # OpenSUSE 11.2 sudo yum install gsl-devel # CentOS 5.5 You can make the main program by changing your working directory to dtm/ and typing: make This software has been compiled on Ubuntu 10.04, OpenSUSE 11.2, and CentOS 5.5. Depending on your environment, you may need to install additional libraries. B. RUNNING Once everything is compiled, you can run this software by typing the command "./main <flags>", where flags is a list of command-line options. An example command and a description of the input and output files is given in dtm/sample.sh. You can see all command-line options by typing ./main --help (although we suggest you start out with the example in dtm/sample.sh). You should also replace 'main' by the appropriate executable (depending on your computer architecture and operating system). We currently provide binaries for Linux (dtm-linux32 and dtm-linux64), MacOS (dtm-darwin64) and Windows (dtm-win32.exe and dtm-win64.exe). C. SUPPORT and QUESTIONS This software is provided as-is, without any warranty or support, WHATSOEVER. If you have any questions about running this software, you can post your question to the topic-models mailing list at [email protected]. You are welcome to submit modifications or bug-fixes of this software to the authors, although not all submissions may be posted. D. USAGE This progam takes as input a collection of text documents and creates as output a list of topics over time, a description of each document as a mixture of these topics, and (possibly) a measure of how "influential" each document is, based on its language. We have provided an example dataset, instructions for formatting input data and processing output files, and example command lines for running this software in the file dtm/sample.sh. E. CHANGES Changes in this version include: - Change the default top_obs_var flag to 0.5 (from -1.0) - Change to use more iterations and a tighter convergence criterion in each doc's E-step. - Change to initialize random topics to be a bit more "flat".
Hi,
As far as I understood from the sample.sh the gammas give us the topic proprtions. So it they are in doc_countxtopic_count format. But when I run a fit on 10k documents with 20 topics i get 20k gamma values, I thought i should get 200k value. Is that correct or am I just missing something? Maybe the vector is sparse?
Thanks.
I am getting an error when compiling. Do you know why?
/dtm-master/dtm$ make
g++ -fPIC -I ../lib -I ../lib/math -I ../gslwrap/include -I ../gslwrap/include/gslwrap -O2 -I /usr/darwinx/usr/include/ ss-lm.o gsl-wrappers.o data.o param.o util.o lda-seq.o lda.o params.o main.o -o dtm -lgsl -lm -lgslcblas -lgflags
//usr/local/lib/libgflags.a(gflags.cc.o): In function gflags_mutex_namespace::Mutex::Mutex()': gflags.cc:(.text._ZN22gflags_mutex_namespace5MutexC2Ev[_ZN22gflags_mutex_namespace5MutexC5Ev]+0x39): undefined reference to
pthread_rwlock_init'
//usr/local/lib/libgflags.a(gflags.cc.o): In function gflags_mutex_namespace::Mutex::Mutex(gflags_mutex_namespace::Mutex::LinkerInitialized)': gflags.cc:(.text._ZN22gflags_mutex_namespace5MutexC2ENS0_17LinkerInitializedE[_ZN22gflags_mutex_namespace5MutexC5ENS0_17LinkerInitializedE]+0x3c): undefined reference to
pthread_rwlock_init'
//usr/local/lib/libgflags.a(gflags.cc.o): In function gflags_mutex_namespace::Mutex::~Mutex()': gflags.cc:(.text._ZN22gflags_mutex_namespace5MutexD2Ev[_ZN22gflags_mutex_namespace5MutexD5Ev]+0x2c): undefined reference to
pthread_rwlock_destroy'
//usr/local/lib/libgflags.a(gflags.cc.o): In function gflags_mutex_namespace::Mutex::Lock()': gflags.cc:(.text._ZN22gflags_mutex_namespace5Mutex4LockEv[_ZN22gflags_mutex_namespace5Mutex4LockEv]+0x20): undefined reference to
pthread_rwlock_wrlock'
//usr/local/lib/libgflags.a(gflags.cc.o): In function gflags_mutex_namespace::Mutex::Unlock()': gflags.cc:(.text._ZN22gflags_mutex_namespace5Mutex6UnlockEv[_ZN22gflags_mutex_namespace5Mutex6UnlockEv]+0x20): undefined reference to
pthread_rwlock_unlock'
collect2: error: ld returned 1 exit status
make: *** [main] Error 1
Looks like this is simple to address user-side, but it looks like the 32bit is actually a 64 bit binary and vice versa. I checked this with SysInternals sigcheck.exe and also by looking in the headers.
Here's the output from sigcheck:
sigcheck.exe c:\dtm\bin\dtm-win32.exe
Sigcheck v2.60 - File version and signature viewer
Copyright (C) 2004-2017 Mark Russinovich
Sysinternals - www.sysinternals.com
c:\dtm\bin\dtm-win32.exe:
Verified: Unsigned
Link date: 5:01 PM 3/23/2014
Publisher: n/a
Company: n/a
Description: n/a
Product: n/a
Prod version: n/a
File version: n/a
MachineType: 64-bit
sigcheck.exe c:\dtm\bin\dtm-win64.exe
Sigcheck v2.60 - File version and signature viewer
Copyright (C) 2004-2017 Mark Russinovich
Sysinternals - www.sysinternals.com
c:\dtm\bin\dtm-win64.exe:
Verified: Unsigned
Link date: 5:01 PM 3/23/2014
Publisher: n/a
Company: n/a
Description: n/a
Product: n/a
Prod version: n/a
File version: n/a
MachineType: 32-bit
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.