Causality-driven Adhoc Information Retrieval
Forum for Information Retrieval Evaluation (FIRE) - 2021
December 13th-17th (Virtual Event)

Call for Participation

Why will you participate?

In contrast to a traditional search system, a Causal Search System seeks to retrieve documents that provide information on the likely causes leading to a query event. In this extended search system, in addition to the topically relevant ranked list of documents, the user will also be presented with a list of causally relevant documents. On submitted queries pertaining to an event (e.g. ‘drop of pound’ or ‘housing crisis’), the system then retrieves adequate information required to construct further analysis for the purpose of automated (or semi-automated with humans-in-loop) decision and policy making. Moreover, information extracted from causally related documents could also serve as the necessary explanations in order to support an automatically generated decision prescribing ways to eradicate a likely cause.

Read More

What to do?

Participants will be given a static test collection of documents and a list of queries related to events that are likely to be caused by a number of other past events. The participants are then required to develop ranking models that can effectively retrieve documents containing information on such past events that are likely candidates to lead to the query event. The officially submitted ranked lists of different participating systems will then be evaluated by comparing them against a set of manually judged relevant documents.



We provide you a static test collection of news articles constituting the official English ad hoc IR collection of FIRE. Access to the data will be provided on receipt of the below access form.

To have an access to the corpus, please email the complete and signed organisational-access form to both and Organisations may use the individual-access form to manage access rights internally and these individual access forms need not be sent to us.

Click here to download corpus

Train Topics

We will release a training set comprising 5 topics (with the relevance assessments) followed by 20 test topics to the participants. Each topic follows the standard TREC format, i.e., is comprised of a 'title' (usually a small number of keywords) and a 'narrative' (a paragraph describing the relevance criteria in details).

Download training topics

Download test topics

Relevance Judgements

For the 5 training topics, we provide you binary relevance judgements following TREC format. This will help to analyze the causal relevance which you need to address (instead of the topical relevance). This will also enable you to tune prototype systems and explore a number of early approaches and with the help of the evaluation (using the provided manual assessments) to see what works and what does not.

Download rel judgments (train topics)

Download complete relevance judgments

Related Research

If you are sans suitable reading resources, here is the preprint of our SIGIR'20 paper. Might be helpul, we believe. You can also find the paper at : SIGIR'20 Proceedings


Important Dates

Training Data Release 30th April, 2021
Test Data Release 15th June, 2021
Run Submission Deadline 25th June, 2021
Results Declaration 15th July, 2021
Working Note Submission 31th August, 2021 (AOE)
Review Notifications 24th September, 2021 (AOE)
Final Version of Working Note 11th October, 2021 (AOE)

Latest Updates

Check results... find here
Test data released... find here
Training data released... find here


What Do You Submit?

Your proposed system must generate a 6 column .tsv file following the standard TREC format. In order to encourage the investigation of different kind of features, three runs per participating group are allowed. You can send on runs over

How Do You Get Evaluated?

We will employ standard evaluation metrics, such as nDCG and MAP, to take into account both precision and recall (in the graded and binary cases respectively) of the submitted runs. Additionally, we will also rank systems based on precision alone using nDCG and P@5.

Know Where You are?

Run Name MAP P@5
(National University of Ireland, Galway)
0.5761 0.7800
(National Institute of Technology, Silchar)
0.0014 0.0100
(National Institute of Technology, Silchar)
0.0001 0.0000
(National Institute of Technology, Silchar)
0.1063 0.4800

You can also evaluate your models using the Relevance Judgements provided here.

Articulate Your Model

We are now accepting submissions. Working notes have to be submitted at Look at the guidelines below before you go -

Submission Guidelines

  • Submissions must describe substantial, original and unpublished work. Wherever appropriate, concrete evaluation and analysis must be included. If the paper being submitted is under review at any other venue, the same should be explicitly mentioned when making the submission. Such papers, if accepted, should be withdrawn from all other places.
  • All submissions should be in Single column CEUR format. Authors should use one of the CEUR Templates below:
  • There is no page limit for submissions, but should be more than 4 pages.
  • We will NOT be allowing multiple working notes from the same set of authors.
  • Each paper should have a copyright clause included in the paper (See the Author agreement variants)
  • Each author should also submit a copyright agreement signed by the authors. (Partially filled agreement will be shared shortly).

Basic Guidelines

  • Check basic sanity of the paper for grammatical errors and reported results.
  • Papers should have sufficient information for reproducing the mentioned results.
  • Check the papers for text reuse/Plagiarism. This includes self-plagiarism as well. Any paper found to have plagiarized content will be rejected without further consideration.
  • Author names should not have any salutations like Dr., Prof., etc.


May We Help You?

Debasis Ganguly, University of Glasgow
Suchana Datta, University College Dublin
Dwaipayan Roy, Indian Institute of Science Education and Research
Derek Greene, University College Dublin

Reach Out Here

Please reach out to the below in case you have any kind of queries related to the task.

Subscribe below for all latest updates.