[Bug]: [Bug]:failed to search/query delegator 617 for channel by-dev-rootcoord-dml_4_448959102009636230v0: fail to Search, QueryNode ID=617, reason=Timestamp lag too large

[MonikaVijayakumar25]

[Bug]: [Bug]:failed to search/query delegator 617 for channel by-dev-rootcoord-dml_4_448959102009636230v0: fail to Search, QueryNode ID=617, reason=Timestamp lag too large

[yanliang567]

@MonikaVijayakumar25 I guess the milvus has some downtime before the issue occurs, so Milvus fails to search for the tt lag is too huge. Please try to upgrade to latest milvus 2.3.19 or 2.4.6, which has improve the efficiency to catch up the tt lag.
/assign @MonikaVijayakumar25
/unassign

[xiaofan-luan]

failed to read file/stats_log/448959102008730178/448959102008730179/448959102008730202/100/1:

@yanliang567 looks like there are some file missing file/stats_log/448959102008730178/448959102008730179/448959102008730202/100/

from this collection. We don't know what happened but this is only stats log.

to fix that:

1. use birdwatcher and find what log it actually mssing
2. there should be a command to fix the log missing by remove the log (stats logs are fine to remove, check if there are insert and delta log missing)
3. compact the collection and your old collection should be ok to work with.