alno/batch-learn

Batch Learn

Batch-Learn is an implementation of ML algorithms which may be applied to on-disk data batch-by-batch, without loading full dataset to memory.

Algorithms included:

• FFM
• NN / MLP

It's extracted from the code written during Outbrain Click Prediction competition on Kaggle and now is undergoing some rewrite and refactoring.

Installation

Batch-learn uses CMake as a build tool and depends on following libraries:

• boost-program-options
• boost-iostreams

To compile code you need to install boost libraries and then call:

mkdir build
cd build
cmake ..
make

Usage

First, you need to convert to batch-learn format:

batch-learn convert -f ffm -b 24  ffm_dataset.txt -O bl_dataset

To train ffm model and make predictions on test dataset:

batch-learn ffm --train tr1 --test te1 --pred pred.txt

You also may specify validation dataset:

batch-learn ffm --train tr1 --test te1 --val va1 --pred pred.txt

To get list of available commands just run:

batch-learn help

To get help about some specific command:

batch-learn help ffm

batch-learn's Issues

Error on Installation

Hi Alexey,
I tried to install and make the code; I ran into some errors - see below. I believe I have the prerequisite packages installed; see below for logs. I am working on Red Hat Enterprise Linux Server release 7.2 (Maipo)
Any ideas ? See the error below "error: no matching function for call to âstd::unordered_map<std::basic_string, std::unique_ptr"; I have only pasted in part of the error log.
Darragh.

dhanley2@apsrd8018:/app/dev/dhanley2/recommender/model/batch-learn
$ ll
total 20
drwxr-xr-x. 3 dhanley2 dce 4096 Jan 24 07:57 build
-rw-r--r--. 1 dhanley2 dce  584 Jan 24 07:49 CMakeLists.txt
drwxr-xr-x. 2 dhanley2 dce 4096 Jan 24 07:49 include
-rw-r--r--. 1 dhanley2 dce 1229 Jan 24 07:49 README.md
drwxr-xr-x. 5 dhanley2 dce 4096 Jan 24 07:49 src
dhanley2@apsrd8018:/app/dev/dhanley2/recommender/model/batch-learn
$ cd build
dhanley2@apsrd8018:/app/dev/dhanley2/recommender/model/batch-learn/build
$ cmake ..
-- Boost version: 1.53.0
-- Found the following Boost libraries:
--   program_options
-- Configuring done
-- Generating done
-- Build files have been written to: /app/dev/dhanley2/recommender/model/batch-learn/build
dhanley2@apsrd8018:/app/dev/dhanley2/recommender/model/batch-learn/build
$ make
[ 20%] Building CXX object CMakeFiles/batch-learn.dir/src/batch-learn.cpp.o
/app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp: In function âint main(int, char**)â:
/app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:55:58: error: no matching function for call to âstd::unordered_map<std::basic_string<char>, std::unique_ptr<command> >::insert(std::pair<const char*, help_command*>)â
     commands.insert(make_pair("help", new help_command()));
                                                          ^
/app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:55:58: note: candidates are:
In file included from /usr/include/c++/4.8.2/unordered_map:48:0,
                 from /app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:5:
/usr/include/c++/4.8.2/bits/unordered_map.h:362:7: note: std::pair<typename std::_Hashtable<_Key, std::pair<const _Key, _Tp>, _Alloc, std::__detail::_Select1st, _Pred, _Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<std::__not_<std::__and_<std::__is_fast_hash<_Hash>, std::is_default_constructible<_Hash>, std::is_copy_assignable<_Hash>, std::__detail::__is_noexcept_hash<_Key, _Hash> > >::value, false, true> >::iterator, bool> std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::insert(const value_type&) [with _Key = std::basic_string<char>; _Tp = std::unique_ptr<command>; _Hash = std::hash<std::basic_string<char> >; _Pred = std::equal_to<std::basic_string<char> >; _Alloc = std::allocator<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >; typename std::_Hashtable<_Key, std::pair<const _Key, _Tp>, _Alloc, std::__detail::_Select1st, _Pred, _Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<std::__not_<std::__and_<std::__is_fast_hash<_Hash>, std::is_default_constructible<_Hash>, std::is_copy_assignable<_Hash>, std::__detail::__is_noexcept_hash<_Key, _Hash> > >::value, false, true> >::iterator = std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::unique_ptr<command> >, false, true>; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::value_type = std::pair<const std::basic_string<char>, std::unique_ptr<command> >]
       insert(const value_type& __x)
       ^
/usr/include/c++/4.8.2/bits/unordered_map.h:362:7: note:   no known conversion for argument 1 from âstd::pair<const char*, help_command*>â to âconst value_type& {aka const std::pair<const std::basic_string<char>, std::unique_ptr<command> >&}â
/usr/include/c++/4.8.2/bits/unordered_map.h:369:2: note: template<class _Pair, class> std::pair<typename std::_Hashtable<_Key, std::pair<const _Key, _Tp>, _Alloc, std::__detail::_Select1st, _Pred, _Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<std::__not_<std::__and_<std::__is_fast_hash<_Hash>, std::is_default_constructible<_Hash>, std::is_copy_assignable<_Hash>, std::__detail::__is_noexcept_hash<_Key, _Hash> > >::value, false, true> >::iterator, bool> std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::insert(_Pair&&) [with _Pair = _Pair; <template-parameter-2-2> = <template-parameter-1-2>; _Key = std::basic_string<char>; _Tp = std::unique_ptr<command>; _Hash = std::hash<std::basic_string<char> >; _Pred = std::equal_to<std::basic_string<char> >; _Alloc = std::allocator<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >]
  insert(_Pair&& __x)
  ^
/usr/include/c++/4.8.2/bits/unordered_map.h:369:2: note:   template argument deduction/substitution failed:
/usr/include/c++/4.8.2/bits/unordered_map.h:365:32: error: no type named âtypeâ in âstruct std::enable_if<false, void>â
       template<typename _Pair, typename = typename
                                ^
/usr/include/c++/4.8.2/bits/unordered_map.h:396:7: note: std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::iterator std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::insert(std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::const_iterator, const value_type&) [with _Key = std::basic_string<char>; _Tp = std::unique_ptr<command>; _Hash = std::hash<std::basic_string<char> >; _Pred = std::equal_to<std::basic_string<char> >; _Alloc = std::allocator<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::iterator = std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::unique_ptr<command> >, false, true>; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::const_iterator = std::__detail::_Node_const_iterator<std::pair<const std::basic_string<char>, std::unique_ptr<command> >, false, true>; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::value_type = std::pair<const std::basic_string<char>, std::unique_ptr<command> >]
       insert(const_iterator __hint, const value_type& __x)
       ^
/usr/include/c++/4.8.2/bits/unordered_map.h:396:7: note:   candidate expects 2 arguments, 1 provided
/usr/include/c++/4.8.2/bits/unordered_map.h:403:2: note: template<class _Pair, class> std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::iterator std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::insert(std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::const_iterator, _Pair&&) [with _Pair = _Pair; <template-parameter-2-2> = <template-parameter-1-2>; _Key = std::basic_string<char>; _Tp = std::unique_ptr<command>; _Hash = std::hash<std::basic_string<char> >; _Pred = std::equal_to<std::basic_string<char> >; _Alloc = std::allocator<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >]
  insert(const_iterator __hint, _Pair&& __x)
  ^
/usr/include/c++/4.8.2/bits/unordered_map.h:403:2: note:   template argument deduction/substitution failed:
/app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:55:58: note:   cannot convert âstd::make_pair<const char (&)[5], help_command*>((*"help"), (* &(operator new(312ul), (<statement>, ((help_command*)<anonymous>)))))â (type âstd::pair<const char*, help_command*>â) to type âstd::unordered_map<std::basic_string<char>, std::unique_ptr<command> >::const_iterator {aka std::__detail::_Node_const_iterator<std::pair<const std::basic_string<char>, std::unique_ptr<command> >, false, true>}â
     commands.insert(make_pair("help", new help_command()));
                                                          ^
In file included from /usr/include/c++/4.8.2/unordered_map:48:0,
                 from /app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:5:
/usr/include/c++/4.8.2/bits/unordered_map.h:418:2: note: template<class _InputIterator> void std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::insert(_InputIterator, _InputIterator) [with _InputIterator = _InputIterator; _Key = std::basic_string<char>; _Tp = std::unique_ptr<command>; _Hash = std::hash<std::basic_string<char> >; _Pred = std::equal_to<std::basic_string<char> >; _Alloc = std::allocator<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >]
  insert(_InputIterator __first, _InputIterator __last)
  ^
/usr/include/c++/4.8.2/bits/unordered_map.h:418:2: note:   template argument deduction/substitution failed:
/app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:55:58: note:   candidate expects 2 arguments, 1 provided
     commands.insert(make_pair("help", new help_command()));
                                                          ^
In file included from /usr/include/c++/4.8.2/unordered_map:48:0,
                 from /app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:5:
/usr/include/c++/4.8.2/bits/unordered_map.h:429:7: note: void std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::insert(std::initializer_list<typename std::_Hashtable<_Key, std::pair<const _Key, _Tp>, _Alloc, std::__detail::_Select1st, _Pred, _Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<std::__not_<std::__and_<std::__is_fast_hash<_Hash>, std::is_default_constructible<_Hash>, std::is_copy_assignable<_Hash>, std::__detail::__is_noexcept_hash<_Key, _Hash> > >::value, false, true> >::value_type>) [with _Key = std::basic_string<char>; _Tp = std::unique_ptr<command>; _Hash = std::hash<std::basic_string<char> >; _Pred = std::equal_to<std::basic_string<char> >; _Alloc = std::allocator<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >; typename std::_Hashtable<_Key, std::pair<const _Key, _Tp>, _Alloc, std::__detail::_Select1st, _Pred, _Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<std::__not_<std::__and_<std::__is_fast_hash<_Hash>, std::is_default_constructible<_Hash>, std::is_copy_assignable<_Hash>, std::__detail::__is_noexcept_hash<_Key, _Hash> > >::value, false, true> >::value_type = std::pair<const std::basic_string<char>, std::unique_ptr<command> >]
       insert(initializer_list<value_type> __l)
       ^
/usr/include/c++/4.8.2/bits/unordered_map.h:429:7: note:   no known conversion for argument 1 from âstd::pair<const char*, help_command*>â to âstd::initializer_list<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >â
/app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:56:64: error: no matching function for call to âstd::unordered_map<std::basic_string<char>, std::unique_ptr<command> >::insert(std::pair<const char*, convert_command*>)â
     commands.insert(make_pair("convert", new convert_command()));
                                                                ^
/app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:56:64: note: candidates are:
In file included from /usr/include/c++/4.8.2/unordered_map:48:0,
                 from /app/dev/dhanley2/recommender/model/batch-learn/src/batch-learn.cpp:5:
/usr/include/c++/4.8.2/bits/unordered_map.h:362:7: note: std::pair<typename std::_Hashtable<_Key, std::pair<const _Key, _Tp>, _Alloc, std::__detail::_Select1st, _Pred, _Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<std::__not_<std::__and_<std::__is_fast_hash<_Hash>, std::is_default_constructible<_Hash>, std::is_copy_assignable<_Hash>, std::__detail::__is_noexcept_hash<_Key, _Hash> > >::value, false, true> >::iterator, bool> std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::insert(const value_type&) [with _Key = std::basic_string<char>; _Tp = std::unique_ptr<command>; _Hash = std::hash<std::basic_string<char> >; _Pred = std::equal_to<std::basic_string<char> >; _Alloc = std::allocator<std::pair<const std::basic_string<char>, std::unique_ptr<command> > >; typename std::_Hashtable<_Key, std::pair<const _Key, _Tp>, _Alloc, std::__detail::_Select1st, _Pred, _Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<std::__not_<std::__and_<std::__is_fast_hash<_Hash>, std::is_default_constructible<_Hash>, std::is_copy_assignable<_Hash>, std::__detail::__is_noexcept_hash<_Key, _Hash> > >::value, false, true> >::iterator = std::__detail::_Node_iterator<std::pair<const std::basic_string<char>, std::unique_ptr<command> >, false, true>; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::value_type = std::pair<const std::basic_string<char>, std::unique_ptr<command> >]
       insert(const value_type& __x)
       ^

Did you use some tricks to obtain better accuracy than libffm?

Hey @alno , did you use some special tricks to obtain better accuracy than libffm?

I have to say it's really a amazing tool, thanks a lot.

Segmentation Fault

Hi Alexey,
I am able to convert my libffm formatted file to binary format using batch-learn convert, however when I try to learn on the file I get a Segmentation Fault error. See below, the conversion, the learning error, along with the files I have as input and a header of the original libffm format.
Do I have an error on the input file ?
Thanks,
Darragh.

$ ./batch-learn/build/batch-learn convert -f ffm -b 24  ../data/ctraincvffm.txt -O ../data/bl_dataset
Converting ../data/ctraincvffm.txt to ../data/bl_dataset using 24 index bits... 1M... 2M... 3M... 4M... 5M... 6M... 7M... 8M... 9M... 10M... 11M... 12M... 13M... 14M... Done.

$ ./batch-learn/build/batch-learn ffm --train ../data/bl_dataset --pred pred.txt
Loading ../data/bl_dataset.index... 14164401 examples
Segmentation fault
dhanley2@apsrd8018:/app/dev/dhanley2/recommender/model
$ ll ../data/*
-rw-r--r--. 1 dhanley2 dce 3476555272 Jan 24 08:32 ../data/bl_dataset.data
-rw-r--r--. 1 dhanley2 dce  283288052 Jan 24 08:32 ../data/bl_dataset.index
-rw-r--r--. 1 dhanley2 dce 1563042128 Jan 24 08:34 ../data/bl_datasettst.data
-rw-r--r--. 1 dhanley2 dce  129698792 Jan 24 08:34 ../data/bl_datasettst.index
-rw-r--r--. 1 dhanley2 dce  118723700 Jan 12 11:31 ../data/clicks_testact2.csv
-rw-r--r--. 1 dhanley2 dce  105753816 Jan 12 11:32 ../data/clicks_testcv2.csv
-rw-r--r--. 1 dhanley2 dce  254224064 Jan 12 11:34 ../data/clicks_traincv2.csv
-rw-r--r--. 1 dhanley2 dce 2215620435 Jan 12 09:01 ../data/ctestcvffm.txt
-rw-r--r--. 1 dhanley2 dce 1467777848 Jan 20 06:10 ../data/ctestcvliblin.txt
-rw-r--r--. 1 dhanley2 dce 2360872430 Jan 20 07:36 ../data/ctestcvvw.txt
-rw-r--r--. 1 dhanley2 dce 4929610885 Jan 12 09:41 ../data/ctraincvffm.txt
-rw-r--r--. 1 dhanley2 dce 3269793750 Jan 20 06:07 ../data/ctraincvliblin.txt
-rw-r--r--. 1 dhanley2 dce 5237960398 Jan 20 07:34 ../data/ctraincvvw.txt
-rw-r--r--. 1 dhanley2 dce 3945694669 Jan 16 09:11 ../data/ctraincvvw.txt.cache
dhanley2@apsrd8018:/app/dev/dhanley2/recommender/model
$ head ../data/ctraincvffm.txt
1 2:105766:1  41:1149661:1 4:2:1 5:212:1 6:2060:1 7:2259:1 40:1151028:1 8:2176:1 9:769:1 45:1445187:1  36:1149661:1 32:1149661:1 34:1956223:1 32:1874517:1 34:1158609:1 34:1980677:1 34:1951578:1 34:1143880:1 34:1224890:1 34:1045784:1 25:3420:1 25:8616:1 25:9453:1 25:3453:1 25:7882:1 25:6817:1    1:3:1
0 2:215967:1  41:1149661:1 4:2:1 5:212:1 6:2060:1 7:2259:1 40:1445187:1 8:21495:1 9:1863:1 45:1151028:1  36:1149661:1 32:1149661:1 34:1956223:1 32:1874517:1 34:1158609:1 34:1980677:1 34:1951578:1 34:1143880:1 34:1224890:1 34:1045784:1 25:3420:1 25:8616:1 25:9453:1 25:3453:1 25:7882:1 25:6817:1    1:3:1
0 2:300808:1  41:1149661:1 4:2:1 5:212:1 6:2060:1 7:2259:1 40:9999999:1 8:28257:1 9:1510:1   36:1149661:1 32:1149661:1 34:1956223:1 32:1874517:1 34:1158609:1 34:1980677:1 34:1951578:1 34:1143880:1 34:1224890:1 34:1045784:1 25:3420:1 25:8616:1 25:9453:1 25:3453:1 25:7882:1 25:6817:1    1:3:1
0 2:241662:1  41:157455:1 4:1:1 5:212:1 6:2061:1 7:2030:1 40:1588639:1 8:25873:1 9:1634:1 45:153509:1  36:153509:1 36:1280592:1 36:157455:1 32:153509:1 32:1280592:1 32:157455:1 32:2026217:1 32:2167788:1 32:2461802:1 25:1589:1 25:2592:1 25:2929:1 25:4212:1 25:323:1 25:938:1 25:436:1 25:844:1 39:153509:1  43:5623230:1 1:2:1