Skip to content

Releases: robinNcode/db_craft

feat: implement SeederGenerator and CLI commands for automated database seeding and migration generation

Choose a tag to compare

@robinNcode robinNcode released this 14 Jul 20:01
6b15893

New Features

Added optional table selection and documented usage for migration generation.
Added clearer progress and completion messages for migration and seeder generation.
Bug Fixes

Improved validation for table names and chunk options.
Added clearer handling for missing tables, empty databases, connection failures, and file-writing errors.
Existing generated migration files are now replaced instead of duplicated.
Chores

Updated the package version from 2.0.0 to 2.0.1.

Optimize memory usage with chunked seeder generation and benchmarking

Choose a tag to compare

@robinNcode robinNcode released this 14 Jul 19:05
f893c41

Summary

Fixes issues #15 and #11, adds a user-configurable chunk size for seed generation, and replaces manual CLI progress output with CodeIgniter's native progress bar. Builds on the earlier memory/performance rework of SeederGenerator.php (streaming generator + keyset pagination), whose benchmark results are included below for reference.

Changes

1. Fix #15 — missing default value in migrations

MigrationGenerator.php:186-196 now emits 'default' => ... in the field definition:

  • Numeric defaults emitted unquoted ('default' => 10,); strings via var_export ('default' => 'active',)
  • Nullable columns with no explicit default get 'default' => null,
  • auto_increment columns get no default; current_timestamp() keeps its existing raw-string path

2. Fix #11FileHandler path

HEAD already used APPPATH, so the literal fix was in place — but FileHandler.php:74-84 was hardened further: the path now uses DIRECTORY_SEPARATOR and creates Database/Migrations/ if it doesn't exist. Previously, file_put_contents silently failed into the CLI::error branch when the folder was missing — the likely remaining pain point in renamed-app setups.

3. Configurable chunk size (default 1000)

  • SeederGenerator.php:24-29 — constructor now takes int $chunkSize = self::DEFAULT_CHUNK_SIZE (clamped to ≥1); all pagination paths use $this->chunkSize
  • GetSeedCommand.php — new --chunk option with validation, plus $arguments/$options metadata so spark help get:seed documents it
 
php spark get:seed users --chunk 5000
 

4. CLI progress bar with percent

SeederGenerator.php:94-104 now uses CodeIgniter's native CLI::showProgress($processed, $totalRows), rendering [####......] 40% Complete in place — updated once per chunk and cleared with showProgress(false) when done, followed by a green summary line with the row count.

Test Results

 
tests/test_seeder.php              16/16 PASS  (incl. custom chunk size honored,
                                                output identical across chunk sizes,
                                                progress bar called per-chunk)
tests/test_migration_defaults.php   6/6  PASS  (string/int/decimal/null defaults,
                                                auto_increment excluded, timestamp raw)
 

Note: the migration defaults test drives generateField() with faked DESCRIBE rows, so it covers MySQL-shaped output — worth a quick live spark get:migration against a real DB to confirm end-to-end.


Background: Seeder Memory/Performance Rework

What changed in SeederGenerator.php

  • True streaming via PHP generatoryieldTableRows() yields one row at a time while fetching from the DB in 1000-row chunks, so only one chunk ever lives in memory (SeederGenerator.php:180).
  • Keyset pagination — when the table has a single-column primary key, chunks are fetched with WHERE pk > last ORDER BY pk instead of LIMIT/OFFSET (SeederGenerator.php:198). OFFSET re-scans skipped rows, making it O(n²) overall — the benchmark showed 64s vs 6.5s at 500k rows. Tables without a usable PK fall back to offset pagination automatically.
  • Single file handle + buffered writes — the initial version called file_put_contents(..., FILE_APPEND) per row (open/lock/close × N). Now the file is opened once with fopen and rows are buffered and flushed once per chunk (SeederGenerator.php:63).
  • Throttled progress output — progress prints once per chunk instead of once per row (2500 rows → 5 CLI writes, not 2500).

Benchmark Results

Ran via tests/bench_run.php (SQLite + CI4 stubs in tests/stubs.php, fresh process per run for clean peak-memory numbers). "Legacy" = the original all-rows-at-once approach.

Rows Legacy time Generator time Legacy peak mem Generator peak mem
10k 0.11s 0.11s 14 MB 4 MB
100k 1.32s 1.27s 148 MB 4 MB
500k 18.8s (needed 2G limit) 6.5s 728 MB 4 MB
1M 💥 OOM at default 512M 10.7s fatal error 4 MB

Key takeaways:

  • Memory is flat at 4 MB regardless of table size — the whole point of the generator. Legacy scales linearly and dies at ~700k rows on a default memory_limit.
  • Speed is equal or better — at 500k rows the generator is ~3× faster because the legacy version thrashes memory concatenating a 176 MB string.
  • The no-PK fallback (offset pagination) still holds 4 MB but takes 64s at 500k rows — that's why keyset pagination was worth adding.

Tests — tests/test_seeder.php

Verifies: file created, empty/migrations tables skipped, output passes php -l, row count and data integrity (quotes, nulls, first/last rows) across multi-chunk boundaries, and CLI output throttling. Run with:

php tests/test_seeder.php

test_performance.php in the repo root (the earlier yield-vs-array experiment) is now superseded by the real benchmark in tests/ — consider deleting it before committing.

`fix`: Seeder template and seeder foreign key checks

Choose a tag to compare

@robinNcode robinNcode released this 01 Oct 17:02

The following issues have been fixed with this release.

  1. Skip the migrations table from seed generation.
  2. Foreign key check for seeder.

thanks @Neo2SHYAlien

`fix`: columnName and foreignColumnName fetching

Choose a tag to compare

@robinNcode robinNcode released this 09 Jun 23:54

The following bug is fixed now: #5

fix: default value null exception

Choose a tag to compare

@robinNcode robinNcode released this 23 Jul 06:27

isCustomDefaultValue method bug of "default value null exception"

Default value current_timestamp fix

Choose a tag to compare

@robinNcode robinNcode released this 22 Jul 19:47
  • In our previous version, our migration generator was unable to create a field that has the default value current_timestamp(). The issue has been fixed with this release.
  • Check the issue

First Release

Choose a tag to compare

@robinNcode robinNcode released this 29 May 11:19
  • Automatic Migration Generation: DB-Craft provides a simple command-line interface to generate migration files automatically based on the connected database. This feature allows developers to keep track of database schema changes and easily apply them to different environments.

  • Table-Specific Migrations: Developers can also generate migration files for specific tables, providing granular control over database changes. This flexibility allows for efficient database management and versioning.

  • Seeding Support: DB-Craft includes functionality to generate seeders, enabling developers to populate their database with initial data. This feature is especially useful for setting up sample data or populating reference tables.