Evaluating LLM Outputs

Option C is correct. useLayoutEffect was introduced in React 16.8 (February 2019, the same release as all hooks). The model's statement that it was "introduced in React 18" is a hallucination with false precision.

Signs that a response may be hallucinated:
• Specific version numbers, dates, or statistics without a cited source
• API names that "feel right" but don't match documentation
• High-confidence language on verifiable facts ("X was introduced in Y")
• Names of papers, authors, or RFC numbers that might be fabricated

Verification workflow for code-related LLM output:
1. Run the code locally — does it actually work?
2. Check official docs for any API, version, or behaviour claims
3. Search for the specific function signature — does it match?
4. Be especially sceptical of "new in version X" or "deprecated in version Y" claims

Asking the same model again is not verification — it will often give the same incorrect answer, or vary between answers, which tells you it's uncertain.

Option B is the correct professional approach. The statement is actually accurate — flatMap() is equivalent to .map().flat(1) and is widely supported. But Option B illustrates the proper verification workflow even when you suspect the answer is correct:

1. Separate claims — "browser support" and "functional equivalence" are two distinct factual claims
2. Use authoritative sources — MDN for browser compatibility and spec behaviour, caniuse.com for compatibility tables
3. Test the equivalence — especially for code explanations, a quick test in a REPL confirms behaviour

The lesson: don't reject LLM output wholesale, and don't accept it wholesale. Evaluate each factual claim independently.

The danger of Option A ("trust it") is not that this specific claim is wrong — it isn't — but that the habit of unconditional trust will fail you when the claim is subtly wrong in a production-critical context.

Option C is the professional iterative-correction prompt. It works because it:

• Identifies exactly what was wrong — "you misunderstood the schema" and what specifically was misunderstood
• States the correct information — the exact schema structure with column names
• Gives a concrete instruction — "rewrite using a JOIN on user_roles instead of filtering on users.role"

The iterative prompting loop for LLM output evaluation:
1. Identify the error category: (a) factual hallucination; (b) misunderstood context; (c) correct approach, wrong output format; (d) missing edge case
2. Provide the correction: the specific fact, schema, or constraint that was wrong
3. Specify what to redo: "rewrite X", "fix only the query", "keep the explanation, replace the code"

Options A and B ("wrong, try again" / "not what I asked") give the model no information about what to fix and will often produce a different but equally wrong answer.