A5 extract_query_doc_features

[kbalog]

A5 extract_query_doc_features

[BerntA]

hey. I think there is still an error in the new test you gave us from A5-errata
q2d5_features = extract_query_doc_features(analyze_query(es, QUERY[2], 'body'), 'd5', es, index='toy_index')
assert q2d5_features['unique_query_terms_in_body'] == 2
assert q2d5_features['avg_TF_body'] == 1.0
where
q2d5_features['avg_TF_body'] should be 0.5 and not 1
d5 is
('d5', {'title': 't2', 'body': 't1 t2 t3 t5'})]
and QUERY[2] is
['t5', 't2']
so the sum of query terms frequency in body is 2. and the total number of terms is 4 --> 2/4 = 0.5.
Am I missing something here?

I get 1.0, because I divide on the unique amount of query terms, so 2/2 because we're aggregating over the terms that exist in both the analyzed query and document field?

[Dregster]

Wondering about the same!

feature_vect_q3_d3 is only correct if we consider query length, and not unique as above.

Wondering about the same!

feature_vect_q3_d3 is only correct if we consider query length, and not unique as above.

But I am not sure if we can do that as it is mentioned 'unique query term'

[tlinjordet]

Hello, please look at
https://github.com/kbalog/ir-course/blob/master/assignments/A5_errata.md.

[thek123]

hey. I think there is still an error in the new test you gave us from A5-errata
q2d5_features = extract_query_doc_features(analyze_query(es, QUERY[2], 'body'), 'd5', es, index='toy_index')
assert q2d5_features['unique_query_terms_in_body'] == 2
assert q2d5_features['avg_TF_body'] == 1.0
where
q2d5_features['avg_TF_body'] should be 0.5 and not 1
d5 is
('d5', {'title': 't2', 'body': 't1 t2 t3 t5'})]
and QUERY[2] is
['t5', 't2']
so the sum of query terms frequency in body is 2. and the total number of terms is 4 --> 2/4 = 0.5.
Am I missing something here?

[thek123]

hey. I think there is still an error in the new test you gave us from A5-errata
q2d5_features = extract_query_doc_features(analyze_query(es, QUERY[2], 'body'), 'd5', es, index='toy_index')
assert q2d5_features['unique_query_terms_in_body'] == 2
assert q2d5_features['avg_TF_body'] == 1.0
where
q2d5_features['avg_TF_body'] should be 0.5 and not 1
d5 is
('d5', {'title': 't2', 'body': 't1 t2 t3 t5'})]
and QUERY[2] is
['t5', 't2']
so the sum of query terms frequency in body is 2. and the total number of terms is 4 --> 2/4 = 0.5.
Am I missing something here?

I get 1.0, because I divide on the unique amount of query terms, so 2/2 because we're aggregating over the terms that exist in both the analyzed query and document field?

should it be unique query terms though?

or else an aggregation function (sum, maximum, or average) over the term frequencies of each query term.

or should it be over the frequencies ( length) of query?

[ChristofferHolmesland]

should it be unique query terms though?
or should it be over the frequencies ( length) of query?
@thek123

It's all query terms, check the last line of the errata.