Choosing Context for a Bot over a Large Database

A database may have hundreds of tables while one question needs only a few. Passing the entire schema to the model is easy to implement. It makes it harder to see why the model chose a table and which information shaped its answer.

Separate context preparation from retrieval for a particular question. The hourglass analogy describes this narrowing: a large body of knowledge is filtered through one request into a smaller set of useful material.

Prepare the index and metadata

Index table names, column descriptions, and domain terms. Keep relationships alongside them: foreign keys, possible join paths, and lookup references.

Basic statistics, such as approximate row counts, may also help. Record when they were collected. Refresh frequency should match how quickly the schema and data change; a nightly job is not sufficient for every system.

These metadata help locate and interpret data. They do not replace current values required for the answer.

Expand context in stages

For a headcount question about the Prague office, the agent might first select people and organizational units. It then retrieves their columns, relationships, and filter definitions. Only then does it need to resolve the record representing that office.

The context grows through candidate tables, structure and relationships, relevant lookup values, and material for query generation and validation. If something is missing, a later stage should be able to request it. Narrowing too early can exclude a necessary table.

Watch what consumes the context

One large lookup can dominate a prompt. Bound each source's contribution and search long value lists separately. Preserve references to the original metadata so the agent can retrieve more detail.

Compare context size with answer quality, cost, and processing time. The goal is sufficient information for this question. The shortest prompt is not necessarily the best, and the longest is not necessarily complete.