Causality-driven Adhoc Information Retrieval
Forum for Information Retrieval Evaluation (FIRE) - 2021
December (Online Event)

Call for Participation

Why will you participate?

In contrast to a traditional search system, a Causal Search System seeks to retrieve documents that provide information on the likely causes leading to a query event. In this extended search system, in addition to the topically relevant ranked list of documents, the user will also be presented with a list of causally relevant documents. On submitted queries pertaining to an event (e.g. ‘drop of pound’ or ‘housing crisis’), the system then retrieves adequate information required to construct further analysis for the purpose of automated (or semi-automated with humans-in-loop) decision and policy making. Moreover, information extracted from causally related documents could also serve as the necessary explanations in order to support an automatically generated decision prescribing ways to eradicate a likely cause.

Read More

What to do?

Participants will be given a static test collection of documents and a list of queries related to events that are likely to be caused by a number of other past events. The participants are then required to develop ranking models that can effectively retrieve documents containing information on such past events that are likely candidates to lead to the query event. The officially submitted ranked lists of different participating systems will then be evaluated by comparing them against a set of manually judged relevant documents.



We provide you a static test collection of news articles constituting the official English ad hoc IR collection of FIRE. Access to the data will be provided on receipt of the below access form.

To have an access to the corpus, please email the complete and signed organisational-access form to both and Organisations may use the individual-access form to manage access rights internally and these individual access forms need not be sent to us.

Click here to download corpus

Train Topics

We will release a training set comprising 5 topics (with the relevance assessments) followed by 20 test topics to the participants. Each topic follows the standard TREC format, i.e., is comprised of a 'title' (usually a small number of keywords) and a 'narrative' (a paragraph describing the relevance criteria in details).

Download training topics

Relevance Judgements

For the 5 training topics, we provide you binary relevance judgements following TREC format. This will help to analyze the causal relevance which you need to address (instead of the topical relevance). This will also enable you to tune prototype systems and explore a number of early approaches and with the help of the evaluation (using the provided manual assessments) to see what works and what does not.

Download rel judgments (train topics)

Related Research

If you are sans suitable reading resources, here is the preprint of our SIGIR'20 paper. Might be helpul, we believe. You can also find the paper at : SIGIR'20 Proceedings


Important Dates

Training Data Release 30th April, 2021
Test Data Release 15th June, 2021
Run Submission Deadline 25th June, 2021
Results Declaration 15th July, 2021
Working Note Submission 31th August, 2021 (AOE)
Review Notifications 20th September, 2021 (AOE)
Final Version of Working Note 10th October, 2021 (AOE)

Latest Updates

Training Data Released.. find here


What Do You Submit?

Your proposed system must generate a 6 column .tsv file following the standard TREC format. In order to encourage the investigation of different kind of features, three runs per participating group are allowed. You can send on runs over

How Do You Get Evaluated?

We will employ standard evaluation metrics, such as nDCG and MAP, to take into account both precision and recall (in the graded and binary cases respectively) of the submitted runs. Additionally, we will also rank systems based on precision alone using nDCG and P@5.

Know Where You are?

To Be Published....Good Luck!!!!


May We Help You?

Debasis Ganguly, University of Glasgow
Suchana Datta, University College Dublin
Dwaipayan Roy, Indian Institute of Science Education and Research
Derek Greene, University College Dublin

Reach Out Here

Please reach out to the below in case you have any kind of queries related to the task.

Subscribe below for all latest updates.