When a data-backed application feels slow, “latency” can mean several different things.
There is the time it takes to make trusted data current, the time required to prepare that data for production use, and the time needed to return the current published version to a caller.
These are different jobs with different systems and failure modes, so they should not share one synchronous path.
The data refresh pipeline
The data refresh pipeline is the dbt domain.
It is responsible for turning source data into trusted, tested, current models. It includes the work of extracting or selecting source data, applying model logic, running tests, and deciding when a new data state is ready for use.
Refresh latency answers:
How long does it take for the data team’s models to become current?
That answer depends on the source systems, the models, the tests, the warehouse, the schedule, and the freshness the business actually needs. Some models should update continuously. Others are intentionally daily or weekly. There is no single correct refresh interval for every use case.
The important point is that this work concerns data correctness and currency, not responding to an application request.
The publish and release pipeline
The publish and release pipeline is the Qry domain. It takes a trusted result from the data refresh process, prepares a production version for workflows and agents, decides what is available to read, activates that version, and preserves a clear release history when a new version is not ready.
Publish latency answers a different question:
How long does it take for a trusted data result to become the version that production workflows and agents can use?
This separation matters because a successful dbt run does not automatically mean that every workflow or agent should immediately receive the result; the result may still need review, validation, packaging, or release according to the product’s rules. Conversely, a release process should not need to rerun the entire data refresh pipeline merely to make a version available.
Qry gives trusted SQL and dbt models a production delivery path. The dbt-native workflow describes how model execution remains where the data team already works, while Qry handles the publication of the resulting interface.
Read latency is the cost of using the current version
Read latency starts when a workflow or agent asks for data and measures the time from request to response.
The read path for workflows and agents should use the current published version. It should not repeat the refresh and release work for every request. Its job is to apply their permitted inputs and return a predictable result.
Read latency answers:
Once a version is published, how quickly can a workflow or agent use it?
That latency should remain predictable even when the data refresh is still running, a new version is being prepared, or the source system is temporarily busy. A workflow or agent should not turn into an unplanned data refresh simply because it needs to read the same published model.
Why coupling these jobs creates a bad API
The simplest implementation is often a live query proxy: receive an API request, send work to the warehouse, wait for the result, and return it. That looks fresh, but it makes every read depend on the slowest and least predictable part of the system.
The consequences are familiar:
- Data refresh work becomes API latency.
- Repeated reads repeat expensive computation.
- A busy source system makes a user-facing request slow.
- A burst of callers creates a burst of data-system load.
- A failed refresh can take reads down with it.
The opposite extreme is also unhelpful: copying data somewhere else without a clear relationship between the source model, the published version, and the workflow’s or agent’s response. Fast responses are not enough if nobody can explain which version was used or whether it is trusted.
The useful separation is not “fast data” versus “fresh data.” It is a release model in which each stage has an explicit responsibility.
Decoupling does not mean ignoring freshness
Decoupling the paths does not mean accepting stale data without explanation; it means making freshness a product policy instead of an accidental side effect of request timing.
A team can decide how often a model should refresh, when a result is ready to publish, and what workflows and agents should receive while a new version is being prepared. Workflows and agents can use a known published version, while data owners can reason about whether that version remains within the intended freshness policy.
The right question is not “is this query live?” It is:
What freshness does this use case require, and what should the read path do when the next refresh or release is late?
An account-health tool may accept a different freshness policy than a financial reconciliation process. Neither is improved by making every read pay the full cost of rebuilding the data.
Why Qry takes this path
Qry sits after trusted model development and before the software that needs to use those models. The data team owns the logic and trustworthiness of the models, while Qry makes selected models available as secure, read-only production interfaces for workflows and agents, giving them stable access without requiring them to own the refresh process.
That architecture separates two kinds of reliability:
- Data reliability: the model is current enough, tested, and trusted.
- Serving reliability: an authorized caller can use the published version predictably.
Workflows and agents need both, but they do not need them implemented as one synchronous request.
Qry’s role is deliberately focused. It is the deployment and delivery layer that connects trusted data refreshes to predictable production reads.