1 Introduction

In this practical, we will familiarize ourselves with the basic flow of working with tidyllm; from setting the schema, preparing the input text, processing with LLM, and saving/viewing the resulting JSON file.

We will use Google’s Gemini LLM to parse the free text and structure it into a clean, analyzable dataset.

First, let’s load the required packages.

library(tidyllm)
library(jsonlite)
library(purrr)
library(dplyr)

2 Setting up the API

We need to authenticate our session with Google’s API. Here, we read the API key from a local file namely “api” to keep it secure and out of the script. Make sure you already obtained API from Google’s AI Studio using your Google account.

# API
Sys.setenv(GOOGLE_API_KEY = readLines("api"))

3 Basic Data Extraction Workflow

In this section, we set up a foundational pipeline. We define a simple structured schema where symptoms is defined as a standard character array (character[]), feed an unstructured text string to the LLM, and examine the minified JSON layout returned.

Define the schema

patient_schema <- tidyllm_schema(
  name = "PatientHistory",
  age = "numeric",
  gender = "character",
  symptoms = "character[]"
)

Raw unstructured text

raw_text <- "Mrs. Amy, a 35-year-old lady came to the outpatient 
             clinic with the complaint of fever for three days and cough."

Process using a tidy pipeline

full_prompt <- paste("Extract patient note according to the schema.", raw_text)
result <- llm_message(full_prompt) |> 
  chat(
    .provider = gemini(), # Gemini as LLM provider
    .model = "gemini-3.1-flash-lite", # Gemini model
    .json_schema = patient_schema # Schema
  )
result
#> Message History:
#> system:
#> You are a helpful assistant
#> --------------------------------------------------------------
#> user:
#> Extract patient note according to the schema. Mrs. Amy, a
#> 35-year-old lady came to the outpatient
#> clinic with the complaint of fever for three days and cough.
#> --------------------------------------------------------------
#> assistant:
#> {"age": 35, "gender": "female", "symptoms": ["fever",
#> "cough"]}
#> --------------------------------------------------------------

Structure as JSON

json_result <- get_reply(result) |> minify()
json_result
#> {"age":35,"gender":"female","symptoms":["fever","cough"]}

4 Refining the Extraction Setup

Now, we scale our pipeline to handle a sequence of multiple unstructured medical histories. We use tidyllm’s metadata field helpers to define semantic constraints (such as explicit factors for gender and structural guidelines for extracting variables like patient_name). The outcomes are streamed and saved on disk as a single-line JSON Lines (.jsonl) file, before being read into a relational data frame.

Refine the schema

patient_schema <- tidyllm_schema(
  name = "Patient History",
  patient_name = field_chr(
    "Patient's name"
  ),
  age = field_dbl(
    "Patient's age in years old."
  ),
  gender = field_fct(
    "Gender of the patient.",
    .levels = c("male","female")
  ),
  symptoms = field_chr(
    "List of current clinical symptoms reported by the patient.", 
    .vector = TRUE
  )
)

Raw unstructured texts

raw_notes <- c("Mrs. Amy, a 35-year-old lady came to the outpatient clinic with the complaint of fever for three days and cough.",
               "Danial, a 6-month-old boy was brought to his paediatrician by his mother. His mother noted that he looked pale and 
               was not active like a baby of his age.")

Set output file in JSON Lite format

output_file = "pt_data2.jsonl"
#> [1] TRUE

Process using a tidy pipeline

# Need to iterate as now we have two notes
for(note in raw_notes) {
  
  full_prompt <- paste("Extract patient's note according to the schema.", note)
  
  result <- llm_message(full_prompt) |> 
    chat(
      .provider = gemini(), # Gemini as LLM provider
      .model = "gemini-3.1-flash-lite", # Gemini model
      .json_schema = patient_schema # Schema
    )
  
  # Extract the textual JSON reply from the LLM message object
  json_string <- get_reply(result)
  
  # Minify and append to .jsonl
  compact_json <- minify(json_string)
  cat(compact_json, "\n", file = output_file, append = TRUE, sep = "")
}

5 Final Dataset Inspection

Let’s stream in our newly generated .jsonl document to observe how cleanly R parses the nested array metadata into structural Tidy columns.

# Read the jsonl file into a data frame
patient_data <- stream_in(file("pt_data2.jsonl"))
#> opening file input connection.
#>  Found 2 records... Imported 2 records. Simplifying...
#> closing file input connection.
glimpse(patient_data)
#> Rows: 2
#> Columns: 4
#> $ patient_name <chr> "Amy", "Danial"
#> $ age          <dbl> 35.0, 0.5
#> $ gender       <chr> "female", "male"
#> $ symptoms     <list> <"fever", "cough">, <"pale", "not active">
patient_data
#>   patient_name  age gender         symptoms
#> 1          Amy 35.0 female     fever, cough
#> 2       Danial  0.5   male pale, not active