feat(android-sqlite): Add SentrySQLiteDriver (JAVA-275)#5466
feat(android-sqlite): Add SentrySQLiteDriver (JAVA-275)#54660xadam-brown wants to merge 2 commits into
Conversation
|
📲 Install BuildsAndroid
|
Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| ad8da22 | 365.86 ms | 427.00 ms | 61.14 ms |
| abfcc92 | 337.38 ms | 427.39 ms | 90.00 ms |
| b8bd880 | 314.56 ms | 336.50 ms | 21.94 ms |
| 44472da | 313.96 ms | 365.35 ms | 51.39 ms |
| b03edbb | 352.20 ms | 423.69 ms | 71.49 ms |
| f6cdbf0 | 314.19 ms | 357.59 ms | 43.40 ms |
| f064536 | 329.00 ms | 395.62 ms | 66.62 ms |
| 806307f | 357.85 ms | 424.64 ms | 66.79 ms |
| d15471f | 343.13 ms | 361.47 ms | 18.34 ms |
| 6edfca2 | 316.43 ms | 398.90 ms | 82.46 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| ad8da22 | 1.58 MiB | 2.29 MiB | 719.83 KiB |
| abfcc92 | 1.58 MiB | 2.13 MiB | 557.31 KiB |
| b8bd880 | 1.58 MiB | 2.29 MiB | 722.92 KiB |
| 44472da | 0 B | 0 B | 0 B |
| b03edbb | 1.58 MiB | 2.13 MiB | 557.32 KiB |
| f6cdbf0 | 0 B | 0 B | 0 B |
| f064536 | 1.58 MiB | 2.20 MiB | 633.90 KiB |
| 806307f | 1.58 MiB | 2.10 MiB | 533.42 KiB |
| d15471f | 1.58 MiB | 2.13 MiB | 559.54 KiB |
| 6edfca2 | 1.58 MiB | 2.13 MiB | 559.07 KiB |
Previous results on branch: feat/support-sqlite-driver
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 2a55b58 | 365.41 ms | 434.64 ms | 69.23 ms |
| f8d2380 | 309.09 ms | 365.52 ms | 56.43 ms |
| 4993a1b | 303.45 ms | 392.65 ms | 89.20 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 2a55b58 | 0 B | 0 B | 0 B |
| f8d2380 | 0 B | 0 B | 0 B |
| 4993a1b | 0 B | 0 B | 0 B |
a590992 to
0084312
Compare
Introduces support for AndroidX's SQLiteDriver via a new SentrySQLiteDriver wrapper. SentrySQLiteDriver automatically creates spans for each SQL statement it executes, and its data scheme closely tracks that of SentrySupportSQLiteOpenHelper, which it's designed to replace. (Span duration is an important exception; see the SentrySQLiteStatement KDoc for more details.) A key motivation for Google's using SQLiteDriver with Room 2.7+ was Kotlin Multiplatform support. We've been careful to keep the SentrySQLiteDriver KMP-compatible as well, should we one day want to lift it into sentry-kotlin-multiplatform. --- Co-authored-by: Angus Holder <7407345+angusholder@users.noreply.github.com>
4c87cb9 to
4c67ea9
Compare
| ) { | ||
| private val stackTraceFactory = SentryStackTraceFactory(scopes.options) | ||
|
|
||
| private val spanHelper = SQLiteSpanHelper(scopes, dbMetadataFromDatabaseName(databaseName)) |
There was a problem hiding this comment.
Helper lets us share span population logic between the old and new worlds. Behavior on the legacy path is unchanged.
| SQLiteDriver { | ||
|
|
||
| init { | ||
| SentryIntegrationPackageStorage.getInstance().addIntegration("SQLiteDriver") |
There was a problem hiding this comment.
There was a problem hiding this comment.
Whilst we surface this information to developers on the issue details page, the main use case is for us (sentry) to get an idea of which integrations are used and to track adoption.
So using a different name here is actually preferred.
| * Span duration is purposefully restricted to accumulated database time, i.e., each [step] call is | ||
| * individually timed and the durations are summed. Time the application spends between steps (e.g., | ||
| * processing rows, sleeping, or doing I/O) is intentionally excluded so the span accurately | ||
| * represents how long SQLite itself was working. |
There was a problem hiding this comment.
Alternative approaches for measuring span duration
This is the most impactful design decision of this PR from a user's perspective. Let's think through our options. I've implemented alternative (3) below; I'm on the fence between (3) and (4).
An example SQLiteStatement call sequence
val statement = sqliteConnection.prepare("SELECT * ...")
while (statement.step()) {
val id = statement.getLong(0)
val sender = statement.getText(1)
val bodyBytes = statement.getBlob(2)
val body = converter.fromEncryptedBytes(bodyBytes)
results.add(Message(id, sender, body))
}A few terms
-
Preparation time: Time the SQLite engine spends parsing and optimizing a statement before executing it (represented by the sqliteConnection.prepare() call in the example above)
-
Execution time: Time the SQLite engine spends executing and advancing a statement, plus loading results into native memory (calls to step() in the example).
-
Consumption time: Time spent bringing statement results into the ART-managed heap and accessing them at the application level (calls to getLong(), getText() and getBlob() in the example). Includes things like type conversion, JNI boundary crossing, copying native bytes to ART's heap, GC sweeps during consumption, etc.
-
Application time: Time the application spends processing statement results while the statement is still ongoing (here, converter.fromEncryptedBytes(bodyBytes)). Can involve arbitrary work, including parsing, decryption, etc.
Alternatives
-
Only track the initial step() call: Includes all execution time for single-step() statements; only captures part of execution time for multi-step() statements. Superficially similar to what we do with
SupportSQLiteOpenHelper(which is why I bothered to include it here), but materially different in that the open helper's initial call can load an entire cursor window, whereas step() amortizes loading across invocations. -
Track the entire duration between the initial and final step() calls: Includes execution time + consumption time + application time. Matches easily with the
ISpanAPI, which expects a start and end timestamp for the span. But application time can make span duration misleading (see dropdown below).
Why we don't want to include application time: Click to expand
Application time can be long and it's not db-related
E.g., app queries 200 rows from a SQLite db, where each row contains an encrypted column that the app decrypts between step() calls. (See converter.fromEncryptedBytes(bodyBytes) from the example query above).
If each step() takes ~0.1ms and each decryption takes ~2ms, cumulative execution time is ~20ms, but execution + application time is ~420ms. Developers will think they have an inefficient query when in fact 95% of the span is unrelated decryption overhead.
-
Track only time spent in each step() call (what this PR implements): Limited to execution time. This or (4) are presumably what most developers would want from a
db.sql.queryspan. I opted for (3) because it's simpler to implement, gives an (arguably) purer look at query + SQLite version performance, and leaves open the possibility of adding a separatedb.sql.consumespan to cover (4) in the future. That said, I'm open to being convinced.... -
Track time spent in all SQLiteStatement methods (step() + getBlob(), getLong(), etc.): Includes execution time + consumption time. My second choice.
Another axis
To any of the above, we could consider adding preparation time, which we'd track via SentrySQLiteConnection.prepare(). I've omitted it because:
- it's computed once per statement construction, but each statement can be executed multiple times, meaning we'd either have to attribute the cost asymmetrically to the first execution, or misleadingly to all; and
- we often exclude statement preparation when computing spans for our legacy open helper, including whenever the statement returns a cursor.
We could add a separate db.sql.preparation span later if we want to track preparation time in isolation.
There was a problem hiding this comment.
Good callout! To be honest I would actually lean towards 2: tracking everything including application time, as this would result in non-synthetic span end timestamps. But your concerns are more than valid, I would even expect Application Time to be the main culprit of many wrongly classified "slow queries". I see a few options, not sure if they're feasible:
-
create a child span for "application time", this will cause our backend to calculate a
span.self_timefor the parent span, which then can be queried and analyzed -
extend https://github.com/getsentry/sentry-java/blob/main/sentry/src/main/java/io/sentry/SpanDataConvention.java to simply add the application timings as a property
-
seems to be the way to go, as it allows all sorts of queries out of the box - but comes with the cost of generating extra spans.
@romtsn any thoughts on this?
| } | ||
|
|
||
| @Test | ||
| fun `all calls are propagated to the delegate`() { |
There was a problem hiding this comment.
Delegation tests (here and below) copy existing tests for the legacy open helper. Let me know if you think they're overkill.
| @@ -0,0 +1,66 @@ | |||
| package io.sentry.sqlite | |||
There was a problem hiding this comment.
As mentioned in the PR description, I've created a separate package for the wrapper without an .android. infix and scrubbed the package of any Android dependencies.
The point of Google's introducing the new driver was to support KMP, so I suspect we may want to lift the wrapper to sentry-kotlin-multiplatform at some point.
There was a problem hiding this comment.
...a new README captures the policy.
| * **Warning:** Do not use [SentrySQLiteDriver] together with | ||
| * [io.sentry.android.sqlite.SentrySupportSQLiteOpenHelper] on the same database file. Both wrappers | ||
| * instrument at different layers, so combining them will produce duplicate spans for every SQL | ||
| * statement. |
There was a problem hiding this comment.
I plan to repeat a similar warning in the Sentry Docs (update forthcoming).
Method reference `System::nanoTime` compiles to FunctionReferenceImpl, which breaks R8 in the SDK size test app.
markushi
left a comment
There was a problem hiding this comment.
Looks very promising already, left a few comments. Once we've decided what execution phases spans should cover, I'll have a second look.
| @@ -0,0 +1,66 @@ | |||
| package io.sentry.sqlite | |||
| SQLiteDriver { | ||
|
|
||
| init { | ||
| SentryIntegrationPackageStorage.getInstance().addIntegration("SQLiteDriver") |
There was a problem hiding this comment.
Whilst we surface this information to developers on the issue details page, the main use case is for us (sentry) to get an idea of which integrations are used and to track adoption.
So using a different name here is actually preferred.
| * | ||
| * @param delegate The [SQLiteDriver] instance to delegate calls to. | ||
| */ | ||
| public class SentrySQLiteDriver private constructor(private val delegate: SQLiteDriver) : |
There was a problem hiding this comment.
Once we lift this up for kmp, we'll probably need to extend this, like e.g. passing a scope / options / logger instance, instead of relying on ScopesAdapter.getInstance() like we do below. But IMHO that's fine for now and won't cause a breaking change.
| * Span duration is purposefully restricted to accumulated database time, i.e., each [step] call is | ||
| * individually timed and the durations are summed. Time the application spends between steps (e.g., | ||
| * processing rows, sleeping, or doing I/O) is intentionally excluded so the span accurately | ||
| * represents how long SQLite itself was working. |
There was a problem hiding this comment.
Good callout! To be honest I would actually lean towards 2: tracking everything including application time, as this would result in non-synthetic span end timestamps. But your concerns are more than valid, I would even expect Application Time to be the main culprit of many wrongly classified "slow queries". I see a few options, not sure if they're feasible:
-
create a child span for "application time", this will cause our backend to calculate a
span.self_timefor the parent span, which then can be queried and analyzed -
extend https://github.com/getsentry/sentry-java/blob/main/sentry/src/main/java/io/sentry/SpanDataConvention.java to simply add the application timings as a property
-
seems to be the way to go, as it allows all sorts of queries out of the box - but comes with the cost of generating extra spans.
@romtsn any thoughts on this?
📜 Description
Introduces
SentrySQLiteDriverfor wrapping and instrumentingandroidx.sqlite.SQLiteDriver. Like our existingSentrySupportSQLiteOpenHelper, the new wrapper produces a span per executed SQL statement.Example use:
💡 Motivation and Context
Room 2.7 introduced the
SQLiteDriverAPI as a replacement forSupportSQLiteOpenHelper; Room 3.0+ makes use ofSQLiteDrivermandatory.A key motivation behind the introduction of
SQLiteDriverwas Kotlin Multiplatform compatibility. This PR makesSentrySQLiteDriveravailable on Android only, but the wrapper has been packaged so that we can lift it into our KMP module in the future without having to break clients.Addresses JAVA-275.
[1] Unlike
SentrySupportSQLiteOpenHelper, theSentrySQLiteDriveris not automatically wrapped via the sentry-android-gradle-plugin. (We can add byte code support later if we want – or do it now if there's a strong interest.)[2] API is not marked as
@Experimental. (Super small surface area: acreate(SQLiteDriver)constructor; future additions are non-breaking; sufficient alignment withSentrySupportSQLiteOpenHelperdata model. That said, chime in if you think we should add it.)[3] Behavior differences vs
SentrySupportSQLiteHelperClick to expand
Behavior differences
SentrySupportSQLiteOpenHelper(old)SentrySQLiteDriver(new)performSqlcall (incl. app time during cursor materialization)step()db time only – app time between steps excludedgetCount/onMove/fillWindowtimed; later rows untimedstep()contributes to cumulative span timestep()of the cycledb.namederivationdatabaseName(for Room, the builder name, e.g.,"tracks"fromdatabaseBuilder(ctx, MyDb, "tracks")).File(fileName).name— the on-disk filename of the path Room passes todriver.open()(e.g.,"tracks.db").db.namevalues during migration. Both paths report data correctly, but will attribute it to different sources.execSQL("CREATE TABLE …; INSERT …; INSERT …;")produces one span whose description is the full script.prepare(...), so multi-statement scripts must be split by Room (or the caller) into separate prepare/step cycles → one span per statement.[4] Risk of duplicate spans with Room's bridge adapter
SentrySQLiteDriverprotects internally against duplicate span creation should a developer try to double-wrap it or any of its components. Room and SQLDelight ensure that use of the driver andSentrySupportSQLiteOpenHelperare mutually exclusive at the API level in virtually all instances, save for one:Room 2.7+ (but not Room 3.0+) exposes a bridge adapter (
SupportSQLiteDriver) that lets users delegate to aSQLiteDriverfrom an existingSupportSQLiteOpenHelper. Double-wrapping both components is possible if folks aren't mindful.I'll be updating the Sentry Docs to warn users against wrapping both adapter components. Another option is to log an error, as mentioned here.
Updates to Sentry Docs: Click to expand
Avoiding duplicate spans with Room 2.7+
AndroidX ships an adapter class,
SupportSQLiteDriver, that lets developers bridge an existingSupportSQLiteOpenHelperto aSQLiteDriverthat Room 2.7+ accepts. Do not wrap both the open helper and the driver. (Remember that the Sentry Android Gradle Plugin will wrap the open helper for you at the byte code level if enabled.) If you double-wrap, you'll produce duplicate spans for every SQL statement:💚 How did you test it?
Unit tests cover:
SentrySQLiteDriverdelegation and wrappingSentrySQLiteConnectiondelegation and wrappingSentrySQLiteStatementspan creation, cancellation, and success/error taggingSQLiteSpanRecorderspan lifecycle (start/finish/cancel)The legacy open helper didn't have the above, so I followed suit. Let me know if you think any is worth it and I'll be happy to address (eg, real Room db test, which would require a test dependency on androidx.sqlite:sqlite-bundled but wouldn't require Robolectric).
📝 Checklist
sendDefaultPIIis enabled.🔮 Next steps
SQLiteDrivervia the sentry-android-gradle-plugin at some point (not currently on the roadmap).