KB Answer With Quality Check
Generate knowledge base answers for customer support questions and automatically evaluate each response for groundedness, relevance, clarity, and tone before sending.
Knowledge BaseQuality CheckCustomer SupportAnswer Evaluation
Overview
The KB Answer + Quality Check workflow automates answering customer support questions from a specified knowledge base and then rigorously evaluates the quality of those answers before they reach customers. It ensures that replies are accurate, relevant, grounded in your documentation, and professionally phrased.
The workflow does four things:
- Ingests the request and constraints — Takes a support question, the relevant knowledge base collection, and a desired tone (for example, friendly, formal, or concise) as input.
- Generates a grounded, on-brand answer — Uses an AI agent to search the knowledge base thoroughly, retrieve the most relevant content, and compose a concise answer with citations that matches the requested tone.
- Runs a strict quality evaluation — Invokes a second AI evaluator that scores the answer on groundedness, relevance, clarity, and tone, and determines whether it is safe and appropriate to send as-is.
- Emits a verdict and suggestions — Produces a clear verdict (for example, safe to send vs. needs edits) along with structured feedback and suggestions for improvement when the answer does not meet quality standards.
What You Need
- A Needle account with access to the Needle automation platform.
- A knowledge base collection configured in Needle containing your support documentation.
- Permissions to use AI nodes within the platform.
How the Flow Works
| Node | Description |
|---|---|
| Manual Trigger | Starts the flow with the input question, the knowledge base collection ID, and the desired tone for the answer. |
| AI Answer Generator | Performs focused, multi-query retrieval over the knowledge base, selects the most relevant document chunks, and composes a grounded answer with citations in the requested tone. |
| AI Answer Evaluator | Evaluates the generated answer strictly on groundedness, relevance, clarity, and tone, assigning scores and deciding whether it is acceptable to send as-is. |
| Code Node | Processes the evaluation output into a final verdict and a formatted report, including suggestions for edits when the answer needs improvement. |
Inputs
| Field | Description |
|---|---|
question | The customer support question to answer. |
kb_collection | The ID of the knowledge base collection to search. Replace the placeholder value with your own collection ID. |
tone | The desired tone for the response (e.g., "Friendly, clear, and professional."). |
Output
At the end of the workflow, you receive:
- The AI-generated answer, including a "Sources" section citing the underlying knowledge base documents used.
- A detailed evaluation report with scores for groundedness, relevance, clarity, and tone.
- A clear verdict indicating whether the answer is safe to send or should be revised.
- Concrete suggestions for improving the answer when it does not meet your quality bar.
Notes
- The workflow uses strict grounding criteria: if an answer is not fully supported by the retrieved sources, it is flagged for review rather than auto-approved.
- The evaluator node expects precise, well-structured inputs (question, answer, and supporting context) to score responses reliably.
- You can adjust prompts or temperature settings in the AI nodes to tune answer style, verbosity, or creativity while still enforcing the quality checks.
- Keeping the knowledge base collection up to date is essential; stale or incomplete content will limit answer quality and can trigger unnecessary flags.
- Important: The manual trigger contains a placeholder collection ID. Make sure to replace it with your own knowledge base collection ID before running the workflow.
Want to showcase your own workflows?
Become a Needle workflow partner and turn your expertise into recurring revenue.