Reproduce - doc2query document expansion

[b8zhong]

Update Doc + Scripts for doc2query

Summary of Changes

1. Dropbox Links

• Removed some dropbox links (ex. below, seems to be deleted or maybe these links are private? Let me know and I'll revert them)

  wget https://www.dropbox.com/s/57g2s9vhthoewty/msmarco-passage-pred-test_topk10.tar.gz -P collections/msmarco-passage
  

---

2. Eval Commands

• Changed outdated references to target/appassembler/bin/trec_eval and IndexCollection (to bin/run.sh):

  target/appassembler/bin/trec_eval -c -m map -c -m recip_rank \
   tools/topics-and-qrels/qrels.car17v2.0.benchmarkY1test.txt \
   runs/run.car17v2.0.bm25.expanded-topk10.txt
  

• Updated to:

  tools/eval/trec_eval.9.0.4/trec_eval -c -m map -m recip_rank \
   tools/topics-and-qrels/qrels.car17v2.0.benchmarkY1test.txt \
   runs/run.car17v2.0.bm25.expanded-topk10.txt
  

• Should this be 9.0.4? https://github.com/castorini/anserini-tools has the updated 9.0.8 for trec_eval, the results don't change though

---

3. Retrieval

• tools/scripts/msmarco/retrieve.pyis defunct, so I replaced with pyserini.search.lucene module:

  python tools/scripts/msmarco/retrieve.py --hits 1000 \
   --index indexes/msmarco-passage/lucene-index-msmarco-expanded-topk10 \
   --queries collections/msmarco-passage/queries.dev.small.tsv \
   --output runs/run.msmarco-passage.dev.small.expanded-topk10.tsv
  

  Updated to:

  python -m pyserini.search.lucene \
    --index indexes/msmarco-passage/lucene-index-msmarco-expanded-topk10 \
    --topics collections/msmarco-passage/queries.dev.small.tsv \
    --topics-format default \
    --output runs/run.msmarco-passage.dev.small.expanded-topk10.tsv \
    --output-format msmarco \
    --bm25 --k1 0.82 --b 0.68 --hits 1000
  

• You need the Python environment though, so not sure if you want it like this

---