Deferring Lookups and Choosing Models for Smaller Tasks

An agent needs one industry code from a list of thousands. Putting the entire list in the SQL-generation prompt makes the model handle query structure and value selection together. Most of that context will never be used.

Split the tasks. Let the generator construct the query with a parameter for the unresolved value. A separate resolver finds the matching record.

Resolve a value rather than guess its code

For example, the generator identifies that the location filter needs Prague's code. The resolver retrieves candidates and returns a verified code. The application binds it as a query parameter.

A small lookup can be inspected in full. A large one needs candidate retrieval first. If several values match, or none does, the resolver must be able to return that state. Forcing a choice hides the uncertainty.

This separation also helps when codes change. It depends on loading current lookup data and invalidating any stale cache.

Choose a model using actual examples

A smaller model may handle selection from a short list. Exact matches and unambiguous mappings can often use ordinary code. Complex query planning may need a different model.

Compare error rates, response times, and costs on examples from the application. Structured output makes the format easier to validate; it does not establish that the chosen ID has the correct meaning.

Parallelize independent work

Independent lookups can run concurrently. The next step needs a clear result from every branch, including failures. Limit total calls and elapsed time so that splitting the work does not add more overhead than it saves.

Record time spent on lookup, generation, and validation separately. That makes it possible to improve a particular slow stage and check whether answer quality changed.