Introduction to IL-TUR
IL-TUR is a benchmark of Natural Language Processing (NLP) tasks for the Indian Legal Domain, including Text Classification, Retrieval and Generation. While most existing Legal NLP benchmarks such as LexGLUE and LEXTREME are mainly concerned with text classification, we also include other types of tasks, such as text retrieval and text generation, as well as explanability. IL-TUR introduces several foundational tasks that can be useful for several downstream legal applications.
Task Motivation
Legal systems worldwide are inundated with exponential growth in cases and documents. To streamline the legal system, there is an imminent need to develop NLP and ML techniques for automatically processing and understanding legal documents. However, evaluating and comparing various NLP models (e.g., LLMs) developed specifically for the legal domain is challenging. This paper address this challenge by proposing IL-TUR: Benchmark for Legal Text Understanding and Reasoning. We propose various domain-specific tasks that address different aspects of the legal domain from the point of view of understanding and reasoning. We also present baseline models for each task, outlining the gap between the LLM-based models and ground truth. A public leaderboard has been created where the research community can upload and test legal text understanding systems on various metrics, thus fostering research in the legal domain.
Tasks
IL-TUR includes the following tasks:
Datasets
Datasets for all the tasks present in the benchmark are available on HuggingFace.
Leaderboard
The leaderboard for all the tasks present in the benchmark is available here.