Comments (3)
As stated here: https://github.com/castorini/anserini#-getting-started
We're on JDK 11... can you downgrade and try again?
from anserini.
java 11
i changed to java 11:
$ java --version
openjdk 11.0.21 2023-10-17 LTS
OpenJDK Runtime Environment Corretto-11.0.21.9.1 (build 11.0.21+9-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.21.9.1 (build 11.0.21+9-LTS, mixed mode)
and after adding the -Dtests.verbose=true
flag I didn’t have any of the errors related to memory shortage mentioned above.
java 20
i changed back to java 20:
$ java --version
openjdk 20.0.2 2023-07-18
OpenJDK Runtime Environment Corretto-20.0.2.9.1 (build 20.0.2+9-FR)
OpenJDK 64-Bit Server VM Corretto-20.0.2.9.1 (build 20.0.2+9-FR, mixed mode, sharing)
and despite adding the -Dtests.verbose=true
I received an error message:
[**ERROR**] **Failures:**
[**ERROR**] **ExtractDocumentLengthsTest.test:70->Assert.assertEquals:146->Assert.assertEquals:117 expected:<[]Total number of term...> but was:<[MockRandomCodec: reading from seg=_0 formatID=MockRandom_0 seed=8474362640500239571**
**MockRandomCodec: readBufferSize=2255**
**MockRandomCodec: reading BlockTree terms dict**
**]Total number of term...>**
[**INFO**]
[**ERROR**] **Tests run: 701, Failures: 1, Errors: 0, Skipped: 0**
[**INFO**]
[**INFO**] **------------------------------------------------------------------------**
[**INFO**] **BUILD FAILURE**
[**INFO**] **------------------------------------------------------------------------**
[**INFO**] Total time: 04:14 min
[**INFO**] Finished at: 2023-12-03T01:07:23-05:00
[**INFO**] **------------------------------------------------------------------------**
[**ERROR**] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.1.0:test **(default-test)** on project anserini: **There are test failures.**
[**ERROR**]
[**ERROR**] **Please refer to /Users/sueszli/dev/anserini/target/surefire-reports for the individual test results.**
[**ERROR**] **Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.**
[**ERROR**] -> **[Help 1]**
[**ERROR**]
[**ERROR**] To see the full stack trace of the errors, re-run Maven with the **-e** switch.
[**ERROR**] Re-run Maven using the **-X** switch to enable full debug logging.
[**ERROR**]
[**ERROR**] For more information about the errors and possible solutions, please read the following articles:
[**ERROR**] **[Help 1]** http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
docker
additionally, I wrote a dockerfile:
# syntax=docker/dockerfile:1
# run: $ docker build -t anserini .
# access: $ docker run -it anserini
FROM ubuntu:latest AS build
RUN apt-get update
RUN apt-get install -y openjdk-11-jdk
RUN apt-get install -y maven
COPY . /app
WORKDIR /app
RUN mvn clean package appassembler:assemble -Dtests.verbose=true
which (after increasing the memory dedicated to the container in the docker settings) resulted in this error message:
489.3 [**INFO**] Results:
489.3 [**INFO**]
489.3 [**ERROR**] **Failures:**
489.3 [**ERROR**] **JsonEndToEndZhTest>EndToEndTest.checkIndex:212->Assert.assertEquals:146->Assert.assertEquals:117 expected:<...1",**
**489.3 "contents" : "[?????????University of Waterloo?????UWaterloo?UW??????????????????????????????????????1957??????????????????????????????????????????????????Faculty of Mathematics?????????????Faculty of Environment??]"**
**489.3 }> but was:<...1",**
**489.3 "contents" : "[???????????????????????????University of Waterloo???????????????UWaterloo???UW??????????????????????????????????????????????????????????????????????????????????????????????????????????????????1957??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????Faculty of Mathematics???????????????????????????????????????Faculty of Environment??????]"**
**489.3 }>**
489.3 [**ERROR**] **JsonEndToEndZhTest>EndToEndTest.testSearching:307->EndToEndTest.checkRankingResults:329->Assert.assertEquals:633->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89 expected:<0> but was:<1>**
489.3 [**ERROR**] **TopicReaderTest.testDprNq:517 expected:<['Wilhelm Conrad R[?]ntgen']> but was:<['Wilhelm Conrad R[??]ntgen']>**
489.3 [**ERROR**] **TopicReaderTest.testDprTrivia:533 expected:<['[?]', 'Helicopters', 'C...> but was:<['[????]', 'Helicopters', 'C...>**
489.3 [**ERROR**] **TopicReaderTest.testNonEnglishTopics:1187 expected:<[?????????????????]> but was:<[???????????????????????????????????????????????????]>**
489.3 [**ERROR**] **TopicReaderTest.testNonEnglishTopics_TopicIdsAsStrings:1245 expected:<[?????????????????]> but was:<[???????????????????????????????????????????????????]>**
489.3 [**ERROR**] **TopicReaderTest.testNq:614 expected:<['Wilhelm Conrad R[?]ntgen']> but was:<['Wilhelm Conrad R[??]ntgen']>**
489.3 [**INFO**]
489.3 [**ERROR**] **Tests run: 701, Failures: 7, Errors: 0, Skipped: 0**
489.3 [**INFO**]
489.3 [**INFO**] **------------------------------------------------------------------------**
489.3 [**INFO**] **BUILD FAILURE**
489.3 [**INFO**] **------------------------------------------------------------------------**
489.3 [**INFO**] Total time: 08:08 min
489.3 [**INFO**] Finished at: 2023-12-03T06:05:59Z
489.3 [**INFO**] **------------------------------------------------------------------------**
489.3 [**ERROR**] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.1.0:test **(default-test)** on project anserini: **There are test failures.**
489.3 [**ERROR**]
489.3 [**ERROR**] **Please refer to /app/target/surefire-reports for the individual test results.**
489.3 [**ERROR**] **Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.**
489.3 [**ERROR**] -> **[Help 1]**
489.3 [**ERROR**]
489.3 [**ERROR**] To see the full stack trace of the errors, re-run Maven with the **-e** switch.
489.3 [**ERROR**] Re-run Maven using the **-X** switch to enable full debug logging.
489.3 [**ERROR**]
489.3 [**ERROR**] For more information about the errors and possible solutions, please read the following articles:
489.3 [**ERROR**] **[Help 1]** http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
------
Dockerfile:13
--------------------
11 | COPY . /app
12 | WORKDIR /app
13 | >>> RUN mvn clean package appassembler:assemble -Dtests.verbose=true
14 |
--------------------
ERROR: failed to solve: process "/bin/sh -c mvn clean package appassembler:assemble -Dtests.verbose=true" did not complete successfully: exit code: 1
from anserini.
i got it to work
TL;DR:
- you can't build it in docker (see: castorini/pyserini#1733)
- java must be version 11
- mvn must be used with the
-Dtests.verbose=true
flag
from anserini.
Related Issues (20)
- Maven build / test issue HOT 2
- Add DL19/DL20 for Cohere V3 embeddings HOT 2
- Anserini Retrieval latency question - Mono thread/CPU ?
- bge-base-en-v1.5 encoder query length issues HOT 1
- Allow trec_eval to take symbols representing standard qrels (instead of full qrel files) HOT 7
- Upgrade JDK? HOT 4
- Add dl22 docs to Anserini HOT 2
- Change local filename of downloaded pre-built index HOT 4
- Duplicate downloading of ONNX files for test cases?
- Can't run 2CR on pre-built indexes directly on fatjar - can't read YAML files HOT 14
- Building anserini on MacOS HOT 21
- Missing appassembler-maven-plugin:2.1.0:assemble HOT 6
- Instructions for reproducing runs on MS MARCO V2.1 with prebuilt indexes HOT 1
- Align RunMsMarco with Fatjar regression instructions HOT 3
- Errors with new MS MARCO v2.1 and BEIR regressions HOT 6
- REST API design HOT 4
- Implement run fusion directly in Anserini HOT 5
- Aligned doc output with 2CR repro classes HOT 1
- Try out new REST API - connect with RankLLM HOT 1
- Discussion: REST API routes for different corpus/model combinations - how do we name? HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from anserini.