Skip to content

perf: Enhance objects using constant sql queries count to get rid of N+1 queries - #2857

Open
Koc wants to merge 4 commits into
mainfrom
feature/optimize-enhance-tables
Open

perf: Enhance objects using constant sql queries count to get rid of N+1 queries#2857
Koc wants to merge 4 commits into
mainfrom
feature/optimize-enhance-tables

Conversation

@Koc

@Koc Koc commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

In our organization we operate a large amount of tables/views, so there are performance bottleneck here:

PERFORMANCE: Controller OCA\\Tables\\Controller\\Api1Controller::index executed 1883 queries in 20.1 seconds
PERFORMANCE: Controller OCA\\Tables\\Controller\\TableController::index executed 1887 queries in 16.6 seconds
  • Replaced TableService::enhanceTable() in a loop with a single call of the enhanceTables()
  • Replaced ViewService::enhanceView() in a loop with a single call of theenhanceViews()
  • Added batch helpers in mappers/services and UserHelper for row/column counts, share counts and owner display names.

🖼️ Screenshots

Here profiler results for GET /apps/tables/table endpoint for 18 tables and 9 views:

🏚️ Before 🏡 After
121 queries 29 queries
image image

🏁 Checklist

  • ⛑️ Tests (unit and/or integration) are included or not needed
  • 🔙 Backport requests are created or not needed: /backport to stableX.X
  • 📅 Milestone is set
  • 🌸 PR title is meaningful (if it should be in the changelog: is it meaningful to users?)

🤖 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@Koc
Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 9b6bd68 to 4033598 Compare July 29, 2026 00:28
@Koc
Koc marked this pull request as ready for review July 29, 2026 18:08
@Koc
Koc requested review from blizzz and enjeck as code owners July 29, 2026 18:08
@Koc Koc changed the title perf: Call enhanceTables for a multiple tables at once to get rid of N+1 queries perf: Enhance objects for a multiple tables at once to get rid of N+1 queries Jul 29, 2026
@Koc Koc changed the title perf: Enhance objects for a multiple tables at once to get rid of N+1 queries perf: Enhance objects using constant sql queries count to get rid of N+1 queries Jul 29, 2026
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 29cbb47 to 44e315b Compare July 29, 2026 21:57
enjeck
enjeck reviewed Aug 7, 2026
View reviewed changes
Comment thread lib/Db/ShareMapper.php
Comment thread lib/Service/ViewService.php Outdated
Comment thread lib/Db/RowSleeveMapper.php Outdated
Comment thread lib/Controller/Api1Controller.php Outdated
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 2686136 to e9b83fb Compare August 9, 2026 14:30
@Koc
Koc requested a review from enjeck August 9, 2026 14:46
@samin-z
samin-z self-requested a review August 10, 2026 15:48
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch from e9b83fb to b286459 Compare August 24, 2026 00:51
enjeck
enjeck reviewed Aug 25, 2026
View reviewed changes
Comment thread lib/Db/ShareMapper.php Outdated
Comment thread lib/Controller/Api1Controller.php
Comment thread lib/Service/ShareService.php
Comment thread lib/Controller/Api1Controller.php
Comment thread lib/Db/ColumnMapper.php
Comment on lines +176 to +188
$counts = array_fill_keys($tableIds, 0);
foreach (array_chunk($tableIds, 1000 - 1) as $tableIdsChunk) {
$qb = $this->db->getQueryBuilder();
$qb->select('table_id', $qb->func()->count('*', 'counter'))
->from($this->table)
->where($qb->expr()->in('table_id', $qb->createNamedParameter($tableIdsChunk, IQueryBuilder::PARAM_INT_ARRAY)))
->groupBy('table_id');

$result = $qb->executeQuery();
while ($row = $result->fetch()) {
$counts[(int)$row['table_id']] = (int)$row['counter'];
}
$result->closeCursor();

@enjeck enjeck Aug 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without try...catch, a \OCP\DB\Exception now propagates?

@Koc Koc Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need that try/catch here? When exception should be possible here and we need to suppress it?

Comment thread lib/Db/ViewMapper.php Outdated
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from 306f894 to f5cb044 Compare August 27, 2026 23:58
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch 2 times, most recently from a7383f5 to c9e19d1 Compare August 30, 2026 14:27
@Koc

Koc commented Aug 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@enjeck I've fixed conflicts introduced by #2781 and code review comments, please, check one more time.

enjeck
enjeck reviewed Sep 4, 2026
View reviewed changes

@enjeck enjeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a question:

Comment thread lib/Db/RowSleeveMapper.php Outdated
@enjeck
enjeck force-pushed the feature/optimize-enhance-tables branch from 263e65c to ba01275 Compare September 4, 2026 04:44
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch from ba01275 to ecef56b Compare September 4, 2026 08:27
@Koc
Koc requested a review from enjeck September 4, 2026 09:05
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch from ecef56b to 7243f39 Compare September 9, 2026 09:18
Koc added 3 commits September 12, 2026 17:42
@Koc
perf: Enhance objects using constant sql queries count to get rid of …
2cea001 
…N+1 queries

Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>
@Koc
perf: Enhance objects using constant sql queries count to get rid of …
7aedb00 
…N+1 queries (code review)

Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>
@Koc
perf: Enhance objects using constant sql queries count to get rid of …
90f11b6 
…N+1 queries (code review)

Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>
@Koc
perf: Enhance objects using constant sql queries count to get rid of …
e845c15 
…N+1 queries (code review)

Signed-off-by: Kostiantyn Miakshyn <molodchick@gmail.com>
@Koc
Koc force-pushed the feature/optimize-enhance-tables branch from 7243f39 to e845c15 Compare September 12, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@blizzz blizzz Awaiting requested review from blizzz blizzz is a code owner
@samin-z samin-z Awaiting requested review from samin-z
@enjeck enjeck Awaiting requested review from enjeck enjeck is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

AI assisted

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Koc @enjeck @nextcloud-command