Skip to main content

Building a document one section at a time

Plan evidence, carry continuity, inspect stopping, and hold for review

Learn a three-section local generation workflow with explicit evidence scopes, token budgets, continuity state, and publication gates.

Kai needs a short public update about a fictional seed-sharing pilot. Asking a small model to “write the whole report” would combine document planning, evidence selection, continuity, and prose generation in one opaque call.

This lesson separates those jobs. A human-written plan assigns evidence to three sections. The local model generates each section. A compact, evidence-checked continuity state moves forward without copying the whole draft back into the prompt.

TipWhat you will learn

This lesson shows how to:

• plan a three-section document before generating prose;
• give each section a bounded evidence ledger and output budget;
• account for prompt and output tokens in a decoder-only context window;
• carry compact, checked continuity state between section calls;
• distinguish end-of-sequence stopping from a token-cap stop;
• inspect repetition, unsupported numbers, and evidence coverage; and
• reject the assembled draft for publication until human review is complete.

Load the local causal model

The course helper loads a prepared local snapshot through huggingfaceR. The model is a decoder-only causal language model, so prompt tokens and newly generated tokens share the same context window.

suppressPackageStartupMessages({
  library(dplyr)
  library(huggingfaceR)
  library(jsonlite)
  library(purrr)
  library(reticulate)
  library(stringr)
  library(tibble)
  library(tidyr)
})

source("R/use-nlg.R")

long_model <- load_nlg_pipeline(
  "qwen_1_5b_instruct",
  "text-generation"
)

model_config_path <- file.path(
  "data-raw",
  ".cache",
  "nlg-models",
  long_model$metadata$local_directory,
  "config.json"
)
model_config <- fromJSON(model_config_path)
context_limit <- as.integer(model_config$max_position_embeddings)

model_record <- long_model$metadata |>
  transmute(
    model = model_id,
    revision,
    license,
    architecture = "decoder-only causal language model",
    context_limit
  )

Pinned model and its shared causal context window

Model	Revision	License	Architecture	Context limit
Qwen/Qwen2.5-1.5B-Instruct	989aa7980e4cf806f80c7fef2b1adb7bc71aa306	Apache-2.0	decoder-only causal language model	32768

For this architecture, enforce prompt_tokens + max_new_tokens <= context_limit.

An encoder-decoder model has separate encoder-input and decoder-output limits. That distinction matters when moving the workflow to another checkpoint.

Plan before writing

The evidence below is fictional demonstration material. It is not a survey estimate, a program evaluation, or a claim about a real seed library.

Each evidence row receives an ID. The plan assigns those IDs to one section and states the section’s job before any prose is generated.

evidence_ledger <- tribble(
  ~evidence_id, ~evidence,
  "E01", "The pilot registered 48 residents.",
  "E02", "The pilot completed 36 pickup appointments in April.",
  "E03", "12 volunteers staffed the collection table.",
  "E04", "Volunteers did not guarantee that any seed would grow.",
  "E05", "29 participants requested an evening pickup.",
  "E06",
  paste(
    "Staff will decide whether to add one Thursday evening;",
    "no decision has been made."
  )
)

document_plan <- tribble(
  ~section_id, ~heading, ~purpose, ~evidence_ids, ~max_new_tokens,
  ~continuity_in, ~continuity_out,
  "SEC-01",
  "Pilot activity",
  "Describe participation, completed pickups, and staffing.",
  c("E01", "E02", "E03"),
  120L,
  "This is the opening section.",
  paste(
    "The pilot registered 48 residents, completed 36 April pickups,",
    "and used 12 volunteers."
  ),
  "SEC-02",
  "Participant request",
  "Describe the request for an evening pickup without claiming a decision.",
  c("E05"),
  100L,
  paste(
    "The pilot registered 48 residents, completed 36 April pickups,",
    "and used 12 volunteers."
  ),
  "Twenty-nine participants requested an evening pickup.",
  "SEC-03",
  "Decision and limit",
  "State the pending Thursday decision and the volunteer guarantee limit.",
  c("E04", "E06"),
  120L,
  "Twenty-nine participants requested an evening pickup.",
  paste(
    "A Thursday evening remains undecided, and volunteers did not",
    "guarantee seed growth."
  )
)

A three-section document plan fixes scope before generation

Section ID	Heading	Section purpose	Evidence IDs	Output budget
SEC-01	Pilot activity	Describe participation, completed pickups, and staffing.	E01, E02, E03	120
SEC-02	Participant request	Describe the request for an evening pickup without claiming a decision.	E05	100
SEC-03	Decision and limit	State the pending Thursday decision and the volunteer guarantee limit.	E04, E06	120

The continuity state is a checked restatement of assigned evidence. It is not a model-generated summary of the previous section. This prevents an unsupported detail in one section from becoming accepted context for the next.

The context-only policy is stricter than simple factual support: incoming continuity may orient a transition, but a section must not restate a continuity fact unless that fact is also assigned to the section’s scoped evidence. The following E05 pattern is declared before generation to check that policy for SEC-03.

continuity_patterns <- tribble(
  ~section_id, ~context_evidence_id, ~pattern,
  "SEC-03",
  "E05",
  paste(
    "\\b(29|twenty-nine|twenty nine)\\b.*\\bevening",
    "|\\bevening\\b.*\\b(29|twenty-nine|twenty nine)\\b"
  )
)

Predeclared cross-section restatement check

Section checked	Context-only evidence
SEC-03	E05

Build section prompts and budgets

Each prompt contains one section purpose, only its assigned evidence, and the compact incoming state. Raw previous prose is not concatenated.

long_system <- paste(
  "Write one section of a public program update in 2 or 3 sentences.",
  "Use only the listed evidence.",
  "Do not add examples, causes, recommendations, people, places, or outcomes.",
  "Do not write a heading."
)

section_requests <- document_plan |>
  mutate(
    scoped_evidence = map(
      evidence_ids,
      \(ids) {
        evidence_ledger |>
          filter(evidence_id %in% ids) |>
          transmute(line = paste0(evidence_id, ": ", evidence)) |>
          pull(line) |>
          paste(collapse = "\n")
      }
    ),
    user_prompt = pmap_chr(
      list(section_id, purpose, continuity_in, scoped_evidence),
      \(section_id, purpose, continuity_in, scoped_evidence) {
        paste0(
          "SECTION ID: ", section_id,
          "\nPURPOSE: ", purpose,
          "\nCONTINUITY STATE: ", continuity_in,
          "\nSCOPED EVIDENCE:\n", scoped_evidence
        )
      }
    ),
    model_prompt = map_chr(
      user_prompt,
      \(user) {
        nlg_chat_prompt(
          long_model$tokenizer,
          long_system,
          user
        )
      }
    ),
    prompt_tokens = map_int(
      model_prompt,
      \(prompt) nlg_token_count(long_model$tokenizer, prompt)
    ),
    planned_total_tokens = prompt_tokens + max_new_tokens,
    fits_context = planned_total_tokens <= context_limit
  )

Decoder-only token accounting for every section call

Section ID	Prompt tokens	Maximum new tokens	Planned total	Fits context
SEC-01	136	120	256	TRUE
SEC-02	127	100	227	TRUE
SEC-03	133	120	253	TRUE

Generate all three sections

The loop below makes three separate neural generation calls. Greedy decoding keeps this demonstration repeatable. Literature documents repetitive degeneration as a risk of likelihood-maximizing decoding in open-ended tasks; it is an empirical tendency, not a guarantee that every greedy output loops.

section_generations <- section_requests |>
  mutate(
    generation = map2(
      model_prompt,
      max_new_tokens,
      \(prompt, budget) {
        nlg_generate(
          long_model,
          prompt,
          max_new_tokens = budget,
          repetition_penalty = 1.05
        )
      }
    ),
    generated_text = map_chr(generation, "text"),
    output_tokens = map_int(generation, "output_tokens"),
    ended_by_eos = map_lgl(generation, "ended_by_eos"),
    hit_token_cap = map_lgl(generation, "hit_token_cap")
  )

Three separately generated sections

Section ID	Heading	Generated section
SEC-01	Pilot activity	The pilot successfully registered 48 residents, completed 36 pickup appointments in April, and was staffed by 12 volunteers at the collection table.
SEC-02	Participant request	Residents have expressed interest in an evening pickup service, with 29 participants requesting this option.
SEC-03	Decision and limit	The upcoming Thursday decision involves a volunteer guarantee limit, with twenty-nine participants requesting an evening pickup. The staff will determine whether to add one Thursday evening, but no decision has been finalized as of now.

Inspect architectural stopping

A final period is a surface character. It does not reveal why generation stopped. The helper checks the final generated token ID against the model’s EOS configuration and separately records whether the output reached the token cap.

stopping_diagnostics <- section_generations |>
  transmute(
    section_id,
    output_tokens,
    output_budget = max_new_tokens,
    ended_by_eos,
    hit_token_cap,
    ends_with_punctuation = str_detect(
      str_trim(generated_text),
      "[.!?]$"
    )
  )

Token-level stopping evidence beside a surface heuristic

Section ID	Output tokens	Output budget	EOS emitted	Hit token cap	Ends with punctuation
SEC-01	33	120	TRUE	FALSE	TRUE
SEC-02	20	100	TRUE	FALSE	TRUE
SEC-03	41	120	TRUE	FALSE	TRUE

EOS says the model selected a termination token. It does not establish that the section is complete, accurate, or ready to publish. EOS and the token-cap flag are independent: both may be true when a model forces EOS on the last permitted new token.

Check repetition and scoped evidence

The distinct trigram ratio divides unique three-token sequences by all three-token sequences after lowercasing and extracting letter and number sequences. Values nearer 1 show fewer exact repeated trigrams. No universal publication threshold is asserted.

The evidence flags are simpler: each planned fact has a surface pattern. A missing pattern is a reason to revise. A present pattern cannot prove that the fact has the correct relation or scope.

distinct_ngram_ratio <- function(text, n = 3L) {
  tokens <- str_extract_all(
    str_to_lower(text, locale = "en"),
    "[\\p{L}\\p{N}]+"
  )[[1]]

  if (length(tokens) < n) {
    return(1)
  }

  starts <- seq_len(length(tokens) - n + 1L)
  ngrams <- map_chr(
    starts,
    \(start) paste(tokens[start:(start + n - 1L)], collapse = " ")
  )
  length(unique(ngrams)) / length(ngrams)
}

evidence_patterns <- tribble(
  ~evidence_id, ~pattern,
  "E01", "\\b48\\b",
  "E02", "\\b36\\b.*\\bApril\\b|\\bApril\\b.*\\b36\\b",
  "E03", "\\b(12|twelve)\\b.*\\bvolunteer",
  "E04", "\\b(not|no|never|without|did not)\\b.*\\bguarantee",
  "E05",
  paste(
    "\\b(29|twenty-nine|twenty nine)\\b.*\\bevening",
    "|\\bevening\\b.*\\b(29|twenty-nine|twenty nine)\\b"
  ),
  "E06", "\\bThursday\\b.*\\b(no|not|undecided|pending)"
)

section_diagnostics <- section_generations |>
  transmute(
    section_id,
    distinct_trigram_ratio = map_dbl(
      generated_text,
      distinct_ngram_ratio
    ),
    source_numbers = map(
      scoped_evidence,
      \(text) sort(str_extract_all(text, "\\b\\d+\\b")[[1]])
    ),
    output_numbers = map(
      generated_text,
      \(text) sort(str_extract_all(text, "\\b\\d+\\b")[[1]])
    ),
    unsupported_number = map2_lgl(
      source_numbers,
      output_numbers,
      \(source, output) !all(output %in% source)
    ),
    unsupported_evaluation = str_detect(
      generated_text,
      regex(
        "\\b(successfully|effective|improved|successful)\\b",
        ignore_case = TRUE
      )
    )
  )

coverage_diagnostics <- document_plan |>
  select(section_id, evidence_ids) |>
  unnest_longer(evidence_ids, values_to = "evidence_id") |>
  left_join(evidence_patterns, by = "evidence_id") |>
  left_join(
    section_generations |>
      select(section_id, generated_text),
    by = "section_id"
  ) |>
  mutate(
    surface_flag = str_detect(
      generated_text,
      regex(pattern, ignore_case = TRUE)
    )
  ) |>
  select(section_id, evidence_id, surface_flag)

continuity_diagnostics <- continuity_patterns |>
  left_join(
    section_generations |>
      select(section_id, generated_text),
    by = "section_id"
  ) |>
  mutate(
    restated_outside_scope = str_detect(
      generated_text,
      regex(pattern, ignore_case = TRUE)
    )
  ) |>
  select(
    section_id,
    context_evidence_id,
    restated_outside_scope
  )

Repetition and numeral diagnostics

Section ID	Distinct trigram ratio	Unsupported numeral present	Unsupported evaluative wording
SEC-01	1	FALSE	TRUE
SEC-02	1	FALSE	FALSE
SEC-03	1	FALSE	FALSE

Evidence coverage flags for each generated section

Section ID	Evidence ID	Surface flag present
SEC-01	E01	TRUE
SEC-01	E02	TRUE
SEC-01	E03	TRUE
SEC-02	E05	TRUE
SEC-03	E04	FALSE
SEC-03	E06	TRUE

Cross-section continuity leakage check

Section ID	Context-only evidence	Restated outside assigned scope
SEC-03	E05	TRUE

These are structural hygiene checks. They do not detect an invented plant example, an unsupported causal explanation, a shifted obligation, or a contradiction written without a new numeral. This render exposes three concrete failures. Successfully adds an evaluation absent from the ledger, and the final section does not state that volunteers made no guarantee about seed growth. SEC-03 also restates the E05 evening request from continuity context even though E05 belongs to SEC-02. The number 29 was supplied to the model, so it is not an invented numeral; its reuse still violates the declared section scope. These findings send the draft back for revision.

Assemble the draft and make a decision

The headings come from the human plan. The prose under each heading comes from the three model calls above.

assembled_document <- section_generations |>
  transmute(
    section_id,
    section = paste0("## ", heading, "\n\n", generated_text)
  ) |>
  pull(section) |>
  paste(collapse = "\n\n")

cat(assembled_document)

## Pilot activity

The pilot successfully registered 48 residents, completed 36 pickup appointments in April, and was staffed by 12 volunteers at the collection table.

## Participant request

Residents have expressed interest in an evening pickup service, with 29 participants requesting this option.

## Decision and limit

The upcoming Thursday decision involves a volunteer guarantee limit, with twenty-nine participants requesting an evening pickup. The staff will determine whether to add one Thursday evening, but no decision has been finalized as of now.

The automatic findings determine whether the draft already needs revision. They still cannot authorize publication. Until a person checks every generated claim, continuity across sections, audience, and tone, the explicit publication decision is reject.

automatic_revision_needed <- any(
  stopping_diagnostics$hit_token_cap,
  section_diagnostics$unsupported_number,
  section_diagnostics$unsupported_evaluation,
  !coverage_diagnostics$surface_flag,
  continuity_diagnostics$restated_outside_scope
)

held_sections <- unique(c(
  stopping_diagnostics$section_id[
    stopping_diagnostics$hit_token_cap
  ],
  section_diagnostics$section_id[
    section_diagnostics$unsupported_number |
      section_diagnostics$unsupported_evaluation
  ],
  coverage_diagnostics$section_id[
    !coverage_diagnostics$surface_flag
  ],
  continuity_diagnostics$section_id[
    continuity_diagnostics$restated_outside_scope
  ]
))

long_review <- tibble(
  review = c(
    "local section generation",
    "prompt plus output budget",
    "token-level stopping diagnostics",
    "automatic evidence and continuity screens",
    "human factual, continuity, tone, and audience review"
  ),
  status = c(
    sprintf(
      "completed: %d sections",
      sum(nzchar(section_generations$generated_text))
    ),
    sprintf(
      "within budget: %d/%d sections",
      sum(section_requests$fits_context),
      nrow(section_requests)
    ),
    sprintf(
      "recorded: %d/%d sections",
      sum(
        stopping_diagnostics$ended_by_eos |
          stopping_diagnostics$hit_token_cap
      ),
      nrow(stopping_diagnostics)
    ),
    sprintf(
      "held: %d/%d sections flagged",
      length(held_sections),
      nrow(section_generations)
    ),
    "pending"
  )
)

document_decision <- tibble(
  automated_recommendation = if_else(
    automatic_revision_needed,
    "revise before human review",
    "send to human review"
  ),
  publication_decision = "reject",
  reason = paste(
    "Human factual, continuity, tone, and audience review is pending;",
    "automatic checks cannot approve the document."
  )
)

Completed scans do not approve the generated sections

Review gate	Status
local section generation	completed: 3 sections
prompt plus output budget	within budget: 3/3 sections
token-level stopping diagnostics	recorded: 3/3 sections
automatic evidence and continuity screens	held: 2/3 sections flagged
human factual, continuity, tone, and audience review	pending

The generated document is not approved for publication

Automated recommendation	Publication decision	Reason
revise before human review	reject	Human factual, continuity, tone, and audience review is pending; automatic checks cannot approve the document.

A future editor may revise and accept the document. This render records no such approval.

What to remember

• Long-text generation starts with document planning and section-scoped evidence.
• Decoder-only prompts and outputs share one context window.
• Carry compact, evidence-checked continuity state instead of unbounded raw history.
• Check EOS and token-cap status directly; punctuation is only a surface clue.
• Repetition, numeral, and evidence flags are preliminary filters.
• Reject generated prose for publication until a human editor reviews the complete document.

Sources

• Reiter and Dale on document planning in applied NLG
• Holtzman et al. on neural text degeneration
• Fan et al. on hierarchical long-form generation
• See et al. on multi-paragraph generation and discourse failures
• Transformers text-generation controls
• Qwen2.5-1.5B-Instruct model card