Abstract: A method includes receiving input data including unstructured text representing one or more sequences of terms. For each respective sequence of terms, the method includes generating a corresponding line of therapy (LoT) pseudo-label indicating whether the respective sequence of terms includes LoT information, generating a corresponding LoT indicator predicting whether the respective sequence of terms includes LoT information, and determining a corresponding LoT indication loss based on the corresponding LoT pseudo-label and the corresponding LoT indicator. The method also includes fine-tuning a pre-trained transformer model based on the LoT indication losses determined for the one or more sequences of terms.