LLM Summarizer

Summary

This guide explains how to build a reliable LLM summarizer step in ProcessFlow that converts complex input data into concise, structured insights. It focuses on safe prompt design, output normalization, truncation limits, and fallback handling when model responses are imperfect. Use this recipe to summarize API responses, query results, and document extracts in a consistent format.

Design the summarizer step for predictable output

A robust summarizer should accept heterogeneous input but always return one stable response schema. This is important because downstream process steps should not depend on model-specific wording or formatting quirks. If your process expects summary, bullets, and takeaway, enforce those fields even during fallback.

Prompting for JSON output is helpful, but model responses can still include markdown or commentary. Always parse defensively and keep fallback behavior explicit. A predictable fallback is better than a hard failure for non-critical summarization flows.

Input contract and output schema

Use process_input.data as the primary source and fall back to process_input only when necessary. This keeps summarizer behavior aligned with common ProcessFlow step output conventions and reduces ambiguity in multi-step pipelines. Standardizing input shape improves both debugging and reuse.

const inputData = process_input.data ?? process_input ?? null;
if (inputData == null || (typeof inputData === "object" && Object.keys(inputData as object).length === 0)) {
  return {
    success: false,
    error: {
      code: "VALIDATION_ERROR",
      message: "No input data provided",
    },
    data: null,
  };
}

Return output in this stable structure:

{
  "success": true,
  "data": {
    "summary": "Concise summary text",
    "bullets": ["Point 1", "Point 2"],
    "takeaway": "Single-line key takeaway",
    "raw_response": "Original model output"
  },
  "message": "Summary generated successfully"
}

Core implementation pattern

The implementation below follows a production-safe sequence: normalize input, constrain payload size, call the model, parse JSON defensively, and apply output limits. This gives good summaries while keeping token usage and failure modes under control.

function truncateWords(text: string, maxWords: number): string {
  const words = text.trim().split(/\s+/).filter(Boolean);
  if (words.length <= maxWords) {
    return text.trim();
  }
  return `${words.slice(0, maxWords).join(" ")}...`;
}

function extractJsonObject(text: string): string {
  const match = text.match(/\{[\s\S]*\}/);
  return match?.[0] ?? text;
}

try {
  const inputData = process_input.data ?? process_input ?? null;
  if (inputData == null) {
    return {
      success: false,
      error: { code: "VALIDATION_ERROR", message: "No input data provided" },
      data: null,
    };
  }

  const serialized = JSON.stringify(inputData, null, 2);
  const payload =
    serialized.length > 5000 ? `${serialized.slice(0, 5000)}\n... (truncated)` : serialized;

  const prompt = [
    "Analyze the JSON data and create a concise summary. Extract key facts and insights.",
    "Ignore IDs and timestamps unless directly relevant.",
    'Return ONLY valid JSON: {"summary":"~200 words","bullets":["point 1"],"takeaway":"one-line"}',
    "",
    "JSON Data:",
    payload,
  ].join("\n");

  const llmResult = (await llm.callLLM(prompt, { max_tokens: 1000 })) as {
    response?: { response?: string } | string;
  };

  const responseText = String(
    (typeof llmResult.response === "object" ? llmResult.response?.response : llmResult.response) ?? "",
  ).trim();

  if (responseText === "") {
    throw new Error("Empty LLM response");
  }

  const jsonText = extractJsonObject(responseText);
  let summary = responseText;
  let bullets: string[] = [];
  let takeaway = "";

  try {
    const parsed = JSON.parse(jsonText) as {
      summary?: string;
      bullets?: string[];
      takeaway?: string;
    };
    summary = String(parsed.summary ?? "").trim() || summary;
    bullets = (parsed.bullets ?? [])
      .map((item) => String(item).trim())
      .filter(Boolean)
      .slice(0, 5);
    takeaway = String(parsed.takeaway ?? "").trim();
  } catch {
    summary = responseText.trim();
  }

  summary = truncateWords(summary, 200);
  if (takeaway.length > 150) {
    takeaway = `${takeaway.slice(0, 147)}...`;
  }

  return {
    success: true,
    data: {
      summary,
      bullets,
      takeaway,
      raw_response: responseText,
    },
    message: "Summary generated successfully",
  };
} catch (error) {
  const details = error instanceof Error ? error.message : "Failed to generate summary";
  return {
    success: false,
    error: {
      code: "LLM_SUMMARY_ERROR",
      message: "Failed to generate summary",
      details,
    },
    data: null,
  };
}

Prompt and response tuning

Keep prompts explicit about output structure, focus area, and excluded noise fields so summaries stay relevant. Domain-specific wording usually improves summary quality more than simply increasing token limits. Start with a narrow prompt and widen scope only when business requirements demand it.

function buildSummaryPrompt(yamlData: string, focus: string): string {
  return [
    `Analyze this YAML data with focus on: ${focus}.`,
    "Return ONLY valid JSON with keys: summary, bullets, takeaway.",
    "Keep summary concise and remove irrelevant IDs/timestamps.",
    "",
    "YAML Data:",
    yamlData,
  ].join("\n");
}

Use cases and safeguards

This summarizer pattern works well for API payload summaries, query-result overviews, document extraction outputs, and integration event snapshots. In all cases, remove sensitive fields before calling the model so summaries never include secrets or regulated data by accident. Data minimization should be part of your standard pre-summary step.

If summaries are too generic, tighten the focus instructions and reduce noisy input fields instead of only increasing token budgets. If output is frequently truncated, summarize pre-aggregated data rather than full raw payloads. Better source curation usually beats larger prompt windows.

Final checklist

Use this checklist before production rollout:

• Previous step returns data in process_input["data"] shape.
• Sensitive fields are removed before summarization.
• Prompt requires strict JSON-only response structure.
• Response parsing handles markdown/code-fence leakage safely.
• Summary, bullets, and takeaway limits are enforced.
• Fallback behavior is defined for malformed model output.

Related documentation

• LLM integration
• Process communication
• Data transformation and mapping