Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

BankStream Feeder

BankStream Feeder generates banking transactions and customer updates in Oracle and IBM i (AS/400, DB2) source databases for a Customer 360 demo. TapData captures committed changes through separately configured Change Data Capture (CDC) tasks and updates downstream customer views.

The Feeder changes source data and verifies results. It does not create schemas or configure TapData tasks, FDM, MDM, or pipelines. Source schemas, demo records, and TapData pipelines must already be set up.

1. Clone the Project

git clone https://github.com/tapdata/BankStream_Feeder.git
cd BankStream_Feeder

2. Choose Deployment Mode

  • Remote Mode (recommended for partners): TapData provides the databases, service, and a preconfigured TapData account. Skip database and TapData task setup and follow Remote Mode.
  • Local Mode: you manage your own Oracle and IBM i databases. Complete the database setup below first, then follow Local Mode.

Local Mode only: create databases, schemas, and demo records first

  1. Deploy Oracle and IBM i (AS/400) and make them reachable from the Feeder computer.
  2. Download this repository to access the SQL files. Run the schema scripts using your database SQL tools, then load the matching sample data:
Database First: create schemas and tables Next: load demo records
IBM i 01_db2i_schema.sql 01_db2i_sample_data.sql
Oracle 02_oracle_schema.sql 02_oracle_sample_data.sql
  1. Review setup placeholders and permissions with your database administrator. In the IBM i script, replace username and apply its table grant after creating the table. The Oracle script requires SQL*Plus/SQLcl and prompts for a password, PDB name, and service.
  2. Grant the Feeder user read/write access to the required tables.

The supplied scripts use BANKCORE for IBM i and C##BANKCRM for Oracle. Use the same names in your connection configuration, or adapt the scripts and configuration together. PostgreSQL is not required for the Feeder commands below.

Remote Mode (recommended for partners)

The CLI connects to the TapData-hosted Control Service. The service uses your partner's assigned databases and keeps database and TapData credentials on the server. You do not need your own database installation.

TapData provides a preconfigured TapData account with pipelines already imported and running and the Customer 360 and KYC APIs already published. TapData task setup is not required in Remote Mode.

1. Install

You need Git, Node.js 24 (the validated baseline), npm, and network access to the Hosted Service. Ask Tapdata for the HTTPS URL, partner API token.

npm ci 

Use the release or branch supplied for your demo. On Windows PowerShell, replace npm with npm.cmd if script execution policy blocks it.

2. Configure

Copy .env.example to .env in the project folder.

PowerShell:

Copy-Item .env.example .env

macOS/Linux:

cp .env.example .env

Edit these three settings in .env using your assigned service URL and partner token:

FEEDER_MODE=remote
DEMO_API_URL=https://113.98.206.139:8443
DEMO_API_TOKEN=<partner-token>
DEMO_API_TIMEOUT_MS=150000

3. Check the connection

npm test
npm run dry-run
npm run demo:status

These commands are read-only. dry-run checks Oracle and IBM i connectivity through the service. demo:status shows your assigned customer, linked account, balances, recent transactions, and KYC details. Once they succeed, use the demo commands below.

Local Mode

Complete the database setup above before these steps.

1. Install

Use Git, Node.js 24, and npm. Download the project and install database dependencies. If you already downloaded it for database setup, open that project folder and run only the install command:

npm ci 

2. Configure

Copy .env.example to .env using the PowerShell or macOS/Linux copy command above. Set:

FEEDER_MODE=local

Configure the Local Mode settings in .env.example for Oracle and IBM i connections, actual schema names, and demo IDs. These settings are required when using Local Mode. The default customer is 100002 and account is AC1000021. Remote service URL/token settings are not used in Local Mode.

3. TapData task setup

Partners complete these steps. Import files are in tapdata_tasks/.

  1. In TapData task import, import replication-tasks_batch-20261023.json.gz, then transformation_tasks_batch-20261023.json.gz. For both imports, you can select Import as Copy under Handling for Name Conflicts. If a task or connection name already exists, TapData creates a new copy with the _import suffix.
  2. Update the database connections and run Test Connection for DB2 for i, Oracle, and MongoDB (FDM and MDM).
  3. Start replication in order: AS400 to FDM, Oracle to FDM, then FDM to MDM. Wait for each task initial synchronization before starting the next. In the transformation section, start MDM_ACCOUNT_AND_TRANSACTIONS_CARDS first, then MDM_CUSTOMER360 once its input data is ready. Keep CDC running.
  4. Once MDM data is ready, open Data Services > Api List > Import and import Tapdata_API_module_batch-20261023.json.gz. Check connections and paths, publish the 13 APIs, and confirm they return demo records.

TapData Task Import dialog with Reuse Existing selected for Remote Mode

4. Check the connection and demo records

npm run doctor
npm test
npm run dry-run
npm run demo:status

doctor checks configuration and runtime dependencies without connecting. dry-run checks database connectivity; demo:status reads the customer, account, and KYC records. These checks do not change data. Continue with the demo commands once they pass.

Configure TapData tasks first. Before running npm run demo:transaction, or the other demo commands, complete the initial synchronization, keep CDC tasks running, and publish the Customer 360 and KYC APIs. Set the TapData API settings in .env for demo:verify. Keep all credentials private.

Demo commands

Run commands from the project folder after completing the steps for your deployment mode. All commands below work in both modes.

Command What it does
npm run dry-run Checks both source database connections using read-only SELECT queries.
npm run demo:status Displays customer, account, transaction, and KYC details without changing data.
npm run demo:transaction Previews the account and asks for a GBP/USD/AUD withdrawal amount and confirmation. Inserts a completed withdrawal and reduces balance and available balance.
npm run demo:kyc Shows current KYC and asks for confirmation before setting status/risk to REVIEW_REQUIRED / HIGH, if needed.
npm run demo:reset Offers KYC reset, balance reset, both, or cancel. Reads current values after selection and confirms only necessary changes. Restores KYC to VERIFIED / LOW and/or both balances to 40000. Keeps transaction history.
npm run demo:verify Reads source transactions/KYC and compares source balances, transactions, and KYC with TapData published APIs. Reports PASS/FAIL without changing data.
npm run realtime -- --duration 5m --interval 15s Generates transactions and customer activity for five minutes, with a 15-second delay between cycles.

Transaction and KYC commands require an interactive terminal. Confirmation defaults to No; enter y to proceed. Cancelling makes no changes. Each confirmed withdrawal creates a new transaction. Use writing commands only against your assigned demo environment.

For a simple demo, run demo:transaction, then demo:verify; run demo:kyc, then demo:verify again. Use demo:reset and select Reset KYC to restore the KYC baseline.

demo:reset commits each database separately. If either step fails, it reports both outcomes and exits with an error. Resolve any remote partial failures before further writes. Remote reset requires a Control Service version with the reset endpoints.

Realtime activity

Adjust --duration and --interval as needed, within your service limits. Remote Mode allows one active job per partner and does not support unlimited duration. The CLI reports the job ID, progress, status, and final counts by table. Press Ctrl+C to request cancellation and let the current database operation finish.

Realtime can change matched customers and accounts across the assigned schemas, beyond the interactive demo account. Activities are randomly selected, so short runs may not change every table.

Cancellation does not undo committed changes. Jobs do not resume after a service restart, and changes across both databases are not one atomic transaction.

Data changed by the Feeder

TapData independently captures these committed changes; demo:verify checks source records and the corresponding published API results.

Database Table Changes
IBM i / AS400 (DB2) ACCOUNT_TRANSACTION Inserts transactions. Realtime also updates pending transactions to completed/posted or rejected. Interactive withdrawals are inserted as completed/posted.
IBM i / AS400 (DB2) BANK_ACCOUNT Updates balances for posted transactions or restores both to 40000 through reset.
Oracle CUSTOMER_PROFILE Updates the preferred customer channel during realtime activity.
Oracle CUSTOMER_KYC Updates KYC status and risk through KYC commands or realtime activity.
Oracle CUSTOMER_INTERACTIONS Inserts customer service interactions during realtime activity.
Oracle CUSTOMER_OFFERS Updates existing offer status during realtime activity.

Troubleshooting

Problem What to do
Authentication failed Check DEMO_API_TOKEN. Confirm that the token is valid and your partner access is enabled.
Cannot reach the Hosted Service Check DEMO_API_URL, network access, service availability, and certificate trust. localhost means your own computer. Confirm the service URL and certificate configuration; do not disable TLS verification.
Verification reports FAIL CDC may still be synchronizing. Wait and rerun verification. If it still fails, check the reported source/API values and check your TapData tasks and published APIs. Realtime activity can change records during comparison.
Realtime options rejected Choose duration/interval values allowed by the service. Remote unlimited jobs are not supported.
A write or job request times out It may already have executed. Keep the reported idempotency key or job ID and confirm the request outcome before submitting another writing command. Rerunning a command creates a new request.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages