Repository navigation
Backend query timeout when forwarding long-range queries (24+ hours) to Prometheus/Thanos #353
Description
Environment:
ASAP Query Version: 0.4.0
Deployment: Kubernetes (Helm Chart)
Backend: Thanos Query
Query range: 24+ hours with 5-minute steps
Issue:
When executing long-range queries with forwardUnsupportedQueries: true, ASAP times out while forwarding to Thanos backend.
Error:
2026-05-21T09:50:52.441377Z ERROR query_engine_rust::drivers::query::fallback::prometheus: asap-query-engine/src/drivers/query/fallback/prometheus.rs:151: Failed to forward range query to Prometheus: error sending request for url (http://x-thanos-query.x-thanos.svc.cluster.local:9090/api/v1/query_range?query=sum%28sum_over_time%28x_aggregated_logs%7Blog_type%3D%22access%22%2C+component%3D%7E%22x-lb%22%2C+region%3D%7E%22.*%22%2C+host%3D%7E%22.*%22%2C+environment%3D%7E%22.*%22%2C+server_name%3D%7E%22.*%22%2C+upstream_name%3D%7E%22.*%22%2C+status%3D%7E%22.*%22%7D%5B5m%5D%29%29&start=1779346200&end=1779357000&step=300): operation timed out
Impact:
Blocks long-range queries (24+ hours) through ASAP
Affects high-cardinality metrics
No current workaround without source code modification
Activity
[-]Backend query timeout when forwarding long-range queries (3+ hours) to Prometheus/Thanos[/-][+]Backend query timeout when forwarding long-range queries (24+ hours) to Prometheus/Thanos[/+]milindsrivastava1997 commentedon May 25, 2026
Thanks for reporting this issue @brnmndl . PR #365 makes the timeout configurable. You should now be able to add a longer timeout in the YAML config.
You can build this from source and try it out. Otherwise, I will cut out a new release of ASAPQuery soon, and you can use that.
milindsrivastava1997 commentedon May 26, 2026
Fixed in v0.5.1 https://github.com/ProjectASAP/ASAPQuery/releases/tag/v0.5.1