Comments (9)
I am glad that you wrote the examples using the JSON DSL, this is going to help the community migrate to this syntax... I'm contemplating rewriting the entire ref guide to feature the JSON syntax instead of the older one ;-).....
from ai-powered-search.
Will plan to upgrade to latest 8.x version prior to final book release. There were a LOT of changes in Solr 9 that could significantly change the way some things work. Will probably also update to Solr 9, but likely after the book is out.
In general, most version-specific code is encapsulated in helper functions in python (it's mostly around setup / schema / collection management - the actual API usage for searching is typically very stable across Solr versions), so likely there won't be much that's actually in the printed book that changes with the Solr 9 upgrade, hence that could be done in the code repo post-release if needed.
I'll take a shot at the Solr 9 upgrade prior to the final release, but I'm anticipating potentially a lot of changes being needed due to the way some things were modularized / removed in Solr 9. TBD.
from ai-powered-search.
Let try to upgrade to the latest 8.x, we can try 9.x after the release
from ai-powered-search.
Interested bystander here ;-). If there are any "rough spots" that you see in Solr that you've identified through the book writing effort, I'd be very interested in hearing about them and getting them sorted. Help me help you make the migration ;-).
from ai-powered-search.
Thank, Eric. Will do!
from ai-powered-search.
@epugh I attempted an upgrade to 8.11.2, and all initially seem to have gone smoothly.
I attempted to upgrade to 9.4.0, however, and queries are busted. Looks like something whacky happened with the JSON query DSL no longer supporting the defType
parameter and so many of the queries throughout the book (which all post JSON queries and use defType=edismax
) are returning zero results.
I added some debugging lines, and get this:
query = "ipad"
collection = "products"
request = {
"query": query,
"fields": ["upc", "name", "manufacturer", "score"],
"limit": 5,
"params": {
"qf": "name manufacturer longDescription",
"defType": "edismax",
"indent": "true",
"sort": "score desc, upc asc",
"echoParams": "all"
}
}
print(requests.post(f"{SOLR_URL}/{collection}/select", json=request).json())
response:
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":2,
"params":{
"q":"ipad",
"defType":"lucene",
"df":"_text_",
"qf":"name manufacturer longDescription",
"indent":"true",
"echoParams":"all",
"fl":[
"upc",
"name",
"manufacturer",
"score"
],
"json":"{\"query\": \"ipad\", \"fields\": [\"upc\", \"name\", \"manufacturer\", \"score\"], \"limit\": 5, \"params\": {\"qf\": \"name manufacturer longDescription\", \"defType\": \"edismax\", \"indent\": \"true\", \"sort\": \"score desc, upc asc\", \"echoParams\": \"all\", \"useParams\": \"defType\"}}",
"sort":"score desc, upc asc",
"rows":"5",
"rid":"aips-solr-2612"
}
},
"response":{
"numFound":0,
"start":0,
"maxScore":0.0,
"numFoundExact":true,
"docs":[
]
}
}
Seems that the defType
is now forced to use lucene
and so the json query DSL is busted. Looks like it's documented here: https://issues.apache.org/jira/browse/SOLR-16916
I'll try downgrading to Solr 9.1 or so to see if I can get it working, but short of maybe rewriting all the JSON DSL queries throughout the book (which is what most of the book uses to express query syntax) to use a different syntax that overwrites with localparams, it doesn't looks to me like the latest version of Solr 9x is going to work well.
from ai-powered-search.
Reproduction steps:
-
Pull branch with Solr 9.4.0 upgrade:
git checkout "solr-9x-upgrade"
-
Build and start docker container (will take a while):
cd docker
docker-compose up --build
-
Index the dataset:
--Index data: http://localhost:8888/notebooks/ch4/1.ch4-setting-up-the-retrotech-dataset.ipynb (run whole notebook) -
On the first listing (4.5), search fails and returns zero results:
-- http://localhost:8888/notebooks/ch4/2.ch4-signals-boosting.ipynb
You can replace the content of the listing with my python code from the last comment and see the echoParams output demonstrating the problem.
from ai-powered-search.
Looks like this issue has a pending fix here that just needs to be merged and released in a Solr bug fix version: apache/solr#2045
from ai-powered-search.
I was just able to successfully run with both Solr 9.1 and 9.3, so looks like this defType
issue was newly introduced into 9.4.0. I'll focus my upgrade efforts on 9.3.0 for now to test out other functionality and plan to upgrade to a newer version once the issue is fixed.
from ai-powered-search.
Related Issues (20)
- `ch11/a.end-to-end-auto-ltr.ipynb` doesn't work end-to-end HOT 2
- Creating indexes in `ch12/0.ch12.setup.ipynb` generates Solr errors HOT 2
- `Listing 12.12 - Rerun A/B test on new test3 model` fails HOT 2
- `Listing 12.13 - Fully Automated LTR Loop` fails HOT 6
- Discuss: Swtich to Jupyter Lab interface instead of using classic notebooks
- Jupyter Lab interface generates a lot of noise in the logs
- `ch5/1.open-information-extraction.ipynb` - Listing 5.1 `RuntimeError: could not create a primitive descriptor for a matmul primitive` on Apple M2 HOT 1
- Should we remove "Old ALS Code" in the `ch09/1.personalization.ipynb` ? HOT 2
- Indexing errors in `ch10/1.setup-the-movie-db.ipynb`, `ch11/0.setup.ipynb`, `ch12/0.setup.ipynb` HOT 1
- Code in `Listing 13.12` is failing because of the problems with dependencies
- Add a note to `ch14/1.question-answering-visualizer.ipynb` (and other notebooks in Chapter 14) that it requires that `outdoors` data set is already installed HOT 2
- `ch14/3.question-answering-GPU-fine-tuning.ipynb` should have a comment that it's designed to run on Google Collab, not in the Jupyter directly HOT 1
- Problems in `ch07/2.semantic-search.ipynb` HOT 3
- Improvement: refactor `docker-compose.yaml` to avoid using `container_name` HOT 2
- "Loading / logging training set (omitted from book)" in `ch10/2.judgments-and-logging.ipynb` and "Log large training set" in `ch10/3.pairwise-transform.ipynb` fail with error HOT 2
- Listing numbers in `ch13/4.semantic-search.ipynb` doesn't match to book's HOT 1
- Some packages are installed twice, increasing the build time & image size HOT 1
- Future: fix additional warnings from JupyterLab
- "Enable Scrolling for Outputs" for notebooks with long outputs
- `ch7/2.semantic-search.ipynb` - no example of use for defined functions HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ai-powered-search.