Checking SQL Generated by a Model

A query returns zero and the database reports no error. That may be correct. The model may also have selected the wrong department code or joined tables on the wrong key.

Successful execution checks only part of the problem. The query also needs to match the request.

Check before and after execution

Before execution, apply access rules and execution restrictions. Permissions, allowed operations, and time limits belong in the application and database.

After execution, examine meaning. Do the filters match the question? Did a join turn a count of people into a count of contracts? Is the time interval correct?

Some checks can be implemented in code. A separate model evaluation step, often called a judge, can help assess the question and query together. Its approval is still not proof of correctness.

Investigate suspicious results

If a complex query returns zero, inspect filters, lookup values, and relationships. Does the department exist? Does the selected code match the requested value? Did a join exclude necessary records?

The existence of women and the existence of a department do not imply that women work in that department. Partial counts are clues for investigation, not grounds to reject an empty result automatically. Probe queries also need scope and time limits.

Give repairs a specific reason

A retry should receive the original question, previous SQL, and the identified problem. Useful feedback includes “the location code is absent from the lookup” or “the join produces multiple rows per person, but the request asks for headcount.”

Bound attempts and total time. If verification fails, say so in the response and preserve the failure as a test case.

Measure the loop on questions with verified answers. Track corrected errors, rejected correct results, and added latency. That shows where an extra check is worth its cost.