Legal Statute Identification (LSI)

Task Motivation and Description #

Since India also has civil law system, one of the first steps in the judicial process manually performed by police personnel/legal professionals (based on the facts of the current situation) is to decide on which of the statutes/laws are applicable in the given setting. Manually rummaging through multiple legislation and laws to find out the relevant statutes can be a hectic task, and automating this stage of the process would reduce the workload and improve the efficiency of the entire judicial system.

The task of Legal Statute Identification (LSI) is to automatically identify the relevant statutes given the facts of a case.

Dataset #

In one of our recent works, we have created the Indian Legal Statute Identification (ILSI) dataset (Paul et al., 2022), and we use the same for the LSI task. The dataset consists of fact portions of 65k court case documents (derived from criminal court cases from the Supreme Court of India (SCI) and 6 High Courts of India), split into train/dev/test splits. The Indian Penal Code (IPC) comprises most criminal statutes and procedures in India; the 100 most frequently occurring statutes in the IPC are chosen as the target statutes. All texts are in English, and all named entities of the type PERSON and ORGANIZATION in the case documents have been anonymized to remove any bias.

Dataset Format #

Each document (json) has the following format:

Dict{
  'id': string  // case identifier
  'text': List(string)  // sentences of case facts
  'labels': List(ClassLabel)  // list of relevant labels (IPC statutes)
}

Task Evaluation #

LSI can be formulated as a multi-label text classification task. The facts, a functional segment of the entire case document, are provided as input. The expected output is one or more statutes from a list of target statutes relevant to the given fact portion. Standard classification metrics such as macro-averaged precision, recall, and F1 score are used for evaluation.

Baseline Models #

We chose some models from the BERT family as baselines for this task. Since fact descriptions (input for LSI) can be long, they may not fit within the maximum 512-token limit for BERT encoders, necessitating a hierarchical model. Examples from the ILSI dataset are pre-segmented into sentences. We pass each sentence individually through the BERT encoder and gather the [CLS] embeddings for each document. The sequence of [CLS]-embeddings are passed through an upper Bi-LSTM layer coupled with attention, yielding a single representation for the entire fact portion. It then passes through a fully connected layer with sigmoid activation to obtain label probabilities. Labels with a probability score > 0.5 are considered relevant.

We experiment with four different BERT encoders:

(i) bert-base-uncased,

(ii) LegalBERT,

(iii) CaseLawBERT,

(iv) InLegalBERT

Results #