IL-TUR
A Benchmark of Indian Legal Text Understanding and Reasoning

Abhinav Joshi1*    Shounak Paul2*    Akshat Sharma1   
Pawan Goyal2    Saptarshi Ghosh2    Ashutosh Modi1+   
1IIT Kanpur    2IIT Kharagpur
*Equal Contribution    +Corresponding Author
“ItalianTrulli”

Paper Github Dataset Leaderboard

Tasks in IL-TUR

Legal Named Entity Recognition (L-NER)

L-NER aims to automatically predict Named Entities (e.g. Judge, Appellant, Respondent, etc.) in a legal document.

Rhetorical Role Prediction (RR)

RR aims to segment legal document into topically coherent units such as Facts, Arguments, Rulings, etc.

Court Judgment Prediction with Explanation (CJPE)

CJPE requires, given the facts and other details of a court case, predicting the final outcome, i.e., appeal granted/denied (Prediction), as well as identifying the sailent sentences leading to the decision (Explanation)

Bail Prediction (BAIL)

BAIL requires to automatically predict whether the accused should be granted bail or not, given the case document (including the facts)

Legal Statute Identification (LSI)

LSI involves identifying the relevant statutes (written laws) given the facts of a court case document

Prior Case Retrieval (PCR)

PCR requires identifying relevant prior cases (based on facts and precedents) from a set of candidate case documents, given a query case document

Summarization (SUMM)

SUMM automates the process of generating a gist of a legal case document that captures the critical aspects of the case

Legal Machine Translation (L-MT)

L-MT involves translating a legal snippet (law article / paragraph of a case judgment) in English to Indic languages

BibTeX

Please consider citing the following paper if you use IL-TUR in your research:

@inproceedings{iltur-2024,
      title = "IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning",
      author = "Joshi, Abhinav and Paul, Shounak and Sharma, Akshat and Goyal, Pawan and Ghosh, Saptarshi and Modi, Ashutosh"
      booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
      month = aug,
      year = "2024",
      address = "Bangkok, Thailand",
      publisher = "Association for Computational Linguistics",
  }