Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 20 additions & 21 deletions quickwit/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions quickwit/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -91,10 +91,10 @@ fnv = "1"
futures = "0.3"
futures-util = { version = "0.3", default-features = false }
glob = "0.3"
# We can't directly update google-cloud-auth to 1.3 and google-cloud-gax to 1.4, because the latest version
# of google-cloud-pubsub is "0.30" which explicitly depends on: google-cloud-auth ^0.17 and google-cloud-gax ^0.19.
# These are yoshidan's google-cloud-rust crates. Their crates.io names now belong to Google's official SDK,
# with an unrelated API from google-cloud-auth 0.18, google-cloud-gax 0.20 and google-cloud-pubsub 0.31 on,
# so they can't be bumped past google-cloud-pubsub 0.30 (which requires google-cloud-auth ^0.17).
google-cloud-auth = { version = "0.17", default-features = false, features = ["rustls-tls"] }
google-cloud-gax = "0.19"
google-cloud-googleapis = { version = "0.16", features = ["pubsub"] }
google-cloud-pubsub = { version = "0.30", default-features = false, features = ["auth", "rustls-tls"] }
governor = "0.10"
Expand Down
2 changes: 0 additions & 2 deletions quickwit/quickwit-indexing/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ flume = { workspace = true }
fnv = { workspace = true }
futures = { workspace = true }
google-cloud-auth = { workspace = true, optional = true }
google-cloud-gax = { workspace = true, optional = true }
google-cloud-googleapis = { workspace = true, optional = true }
google-cloud-pubsub = { workspace = true, optional = true }
itertools = { workspace = true }
Expand Down Expand Up @@ -68,7 +67,6 @@ quickwit-storage = { workspace = true }
[features]
gcp-pubsub = [
"dep:google-cloud-auth",
"dep:google-cloud-gax",
"dep:google-cloud-googleapis",
"dep:google-cloud-pubsub",
]
Expand Down
4 changes: 2 additions & 2 deletions quickwit/quickwit-indexing/src/source/gcp_pubsub_source.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ use anyhow::Context;
use async_trait::async_trait;
use bytes::Bytes;
use google_cloud_auth::credentials::CredentialsFile;
use google_cloud_gax::retry::RetrySetting;
use google_cloud_pubsub::client::{Client, ClientConfig};
use google_cloud_pubsub::subscription::Subscription;
use quickwit_actors::{ActorExitStatus, Mailbox};
Expand Down Expand Up @@ -133,7 +132,8 @@ impl GcpPubSubSource {
max_messages_per_pull=%max_messages_per_pull,
"starting GCP PubSub source"
);
if !subscription.exists(Some(RetrySetting::default())).await? {
// `None` makes the client use its default retry setting.
if !subscription.exists(None).await? {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve retries for the subscription existence check

When the Pub/Sub API returns a transient transport or UNAVAILABLE error during source startup, None disables the GAX retry wrapper rather than selecting RetrySetting::default(). The previous call retried this idempotent GET in place; now try_new immediately fails the entire pipeline spawn and leaves recovery to the slower supervisor-level respawn cycle. Keep the compatible 0.19 retry type available, such as through a renamed exact dependency, and continue passing the default retry setting.

Useful? React with 👍 / 👎.

anyhow::bail!("GCP PubSub subscription `{subscription_name}` does not exist");
}
Ok(Self {
Expand Down
Loading