Can't install bitextor Ubuntu 16.04

[bitextor]

Can't install bitextor Ubuntu 16.04

[iyadahmad]

also I got just two commands after completing bitextor-train-document-alignment && bitextor-get-html-text only.

[lpla]

Hi, installation commands and dependencies are written in the Wiki ( https://github.com/bitextor/bitextor/wiki#requirements) and README.md ( https://github.com/bitextor/bitextor#dependencies). El mar., 4 dic. 2018 a las 10:23, iyad ahmad (<[email protected]>) escribió:

…

also I got just two commands after completing bitextor-train-document-alignment && bitextor-get-html-text only. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABdl0Uwem6weq9O_SDHuTx0lVkV-2pSIks5u1j8tgaJpZM4ZAMDi> .

[hieuhoang]

@iyadahmad I also have problems with bitextor & Ubuntu 16.04, I now use Ubuntu 18.04. You may want to upgrade, or give more detailed descriptions so that the issue can be debugged

[mespla]

Which version of Bitextor are you trying to install? Master branch? Miquel. Missatge de Hieu Hoang <[email protected]> del dia dt., 4 de des. 2018 a les 12:38:

…

@iyadahmad <https://github.com/iyadahmad> I also have problems with bitextor & Ubuntu 16.04, I now use Ubuntu 18.04. You may want to upgrade, or give more detailed descriptions so that the issue can be debugged — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGOwSFtEe7nXuNbzWmkcL3uXJQilR85bks5u1l6kgaJpZM4ZAMDi> .

[iyadahmad]

yes the master branch , btw I've tried with ubuntu 18.04 and the result the same , I will attach log files.

[iyadahmad]

I followed the instructions in this files https://github.com/bitextor/bitextor/blob/master/README.md
autogen.log
config.log
makeinstalllog.log
makelog.log

[iyadahmad]

CMakeError-kenlm.log
CMakeError-mgiza.log
CMakeError-preprocess.log
CMakeOutput-kenlm.log
CMakeOutput-mgiza.log
CMakeOutput-preprocess.log

[lpla]

Looks like some parts are missing from boost or something is broken with libboost or cmake versions, maybe they are too old. This are the most important errors here (looking at mgiza CMakeError log file):

error: ‘std::tr1’ has not been declared
...
CheckSymbolExists.c:(.text+0x1b): undefined reference to `pthread_create'
...
/usr/bin/ld: cannot find -lpthreads

Check if libboost-all-dev installed

[iyadahmad]

thanks for your quick reply ,

libboost-all-dev already installed and the version is 1.65.1
cmake version 3.10.2

[IT-coach-666]

Yesterday , I try to install the latest version of bitextor. There is no error message in the installing process, but in the end, I can't use the "bitextor" command to crawl data.

The installing process described as following:
run the script 'configure'
./autogen.sh
make
sudo make install

no error message but can't run bitextor command, I am sure that there is no PATH problem.

[IT-coach-666]

no error message but can't run bitextor command.
I install to a specific path ,and find that only has the command of the following graph:

[lpla]

Something is wrong in the screenshot, JY, as lots of bitextor scripts are missing there (only two ‘bitextor-*’ scripts are shown and we have more than 20 scripts with this suffix). Check compilation errors and paste them here, please. And Iyad, which version of libboost-all-dev do you have in Ubuntu 18.04?

…

El 5 dic 2018, a las 15:01, JY ***@***.***> escribió: no error message but can't run bitextor command. I install to a specific path ,and find that only has the command of the following graph: <https://user-images.githubusercontent.com/19731222/49518202-09c07680-f8d9-11e8-9985-2a55f2534029.png> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABdl0X1sxoVBVFyzbVbtoh3lne10NHV1ks5u19HFgaJpZM4ZAMDi>.

[lpla]

Forget my previous message. There is an issue in master branch which is fixed in our development branch. Please, use the released tag Bitextor 6 package meanwhile we fix it: https://github.com/bitextor/bitextor/releases/tag/v6

[IT-coach-666]

Thanks for your reply, lpla.
The above bitextor link is Bitextor 6, and the following links is the Bitextor 7? And I also want to konw when I can formally use it? Thanks.
https://github.com/bitextor/bitextor/

[mespla]

I have just fixed the Makefile.am in the master so it copies all the scripts to $PREFIX/bin Missatge de Leo <[email protected]> del dia dc., 5 de des. 2018 a les 15:11:

…

Forget my previous message. There is an issue in master branch which is fixed in our development branch. Please, use the released tag Bitextor 6 package meanwhile we fix it: https://github.com/bitextor/bitextor/releases/tag/v6 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGOwSMr9DCiyNFGK5pR04eFQVYMzRGVyks5u19QkgaJpZM4ZAMDi> .

[lpla]

Ok, now master is fixed, as @mespla said.

Some notes. Bitextor 7 is the next stable release of Bitextor, which is now in development. We manage our code similarly as Debian does:

• tags are for stable releases that have support until next stable comes, which is Bitextor 6 series (well, 6.0.1 now thanks to your issue with the '\s' in sed, so thanks!). That's why I recommended to use it until we fixed master.
• master is our testing (Debian equivalent) in which we integrate all the new features we finished (or reached a milestone) for next stable release, like full Python 3 compatibility, WARC format support and more. It can be broken sometimes (today, for example, but now it is working again).
• Rest of branches are our development playground for new features and they will be probably not even documented in the Wiki or README.md and, of course, prone to be broken.

So, please, @iyadahmad and @JiaYueHuang , test now your installation pulling updates for master and report any other issue. In case you find more issues and we take long to answer, please, try with Bitextor 6.x series.

[IT-coach-666]

Thanks~

[iyadahmad]

Hello ,
Master , still "bitextor" binary doesn't exist just [/usr/local/bin/bitextor-get-html-text
and /usr/local/bin/bitextor-train-document-alignment]
CMakeError-kenlm.log
CMakeError-mgiza.log
CMakeError-preprocess.log
CMakeOutputkenlm.log
CMakeOutput-mgiza.log
CMakeOutput-preprcess.log

[iyadahmad]

i will test v6.x tag and will write my feedback .

[iyadahmad]

V6.x notes

when run ./autogen.sh >> configure: error: this package requires the following libraries for Python:

• keras: Please, install v1.0.3 or later and try again: sudo pip install keras

but keras is already installed with pip3 [Keras (2.2.4)]

here are the used steps from my side :

1. git submodule update --init --recursive
2. sudo apt install cmake g++ automake pkg-config openjdk-8-jdk python3 python3-pip python3-magic libboost-all-dev maven libbz2-dev liblzma-dev zlib1g-dev libffi-dev
3. sudo pip3 install -e git://github.com/mammadori/magic-python.git#egg=Magic_file_extensions
4. sudo pip3 install --upgrade python-Levenshtein tensorflow keras iso-639 langid nltk regex h5py ftfy warc3-wet
5. sudo apt install httrack
6. sudo pip3 install -r bicleaner/requirements.txt
7. sudo pip3 install --upgrade matplotlib sklearn numpy && sudo apt install python3-tk
8. ./autogen.sh
9. make
10. sudo make install

[lpla]

Hi. Bitextor 6.x is Python 2 based. You need to install the libraries with pip2. Take into account that wiki is updated for Bitextor 7, so only the README.md inside Bitextor-v6.0.1 package is valid for Bitextor 6.

…

El 6 dic 2018, a las 9:32, iyad ahmad ***@***.***> escribió: V6.x notes when run ./autogen.sh >> configure: error: this package requires the following libraries for Python: keras: Please, install v1.0.3 or later and try again: sudo pip install keras but keras is already installed with pip3 [Keras (2.2.4)] here are the used steps from my side : git submodule update --init --recursive sudo apt install cmake g++ automake pkg-config openjdk-8-jdk python3 python3-pip python3-magic libboost-all-dev maven libbz2-dev liblzma-dev zlib1g-dev libffi-dev sudo pip3 install -e git://github.com/mammadori/magic-python.git#egg=Magic_file_extensions sudo pip3 install --upgrade python-Levenshtein tensorflow keras iso-639 langid nltk regex h5py ftfy warc3-wet sudo apt install httrack sudo pip3 install -r bicleaner/requirements.txt sudo pip3 install --upgrade matplotlib sklearn numpy && sudo apt install python3-tk — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABdl0fztgGo525ZQoCAITGNGtNfLErmFks5u2NY5gaJpZM4ZAMDi>.

[iyadahmad]

Thanks a lot, it works now .