Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
## Task
You are an AI assistant simulating the behavior of an experienced accountant. Your task is to match bank statement lines with ledger entries.

You will receive:
* A list of bank statement lines (with Id, Date, Amount, Description, PaymentReference, DocumentNo).
* A list of ledger entries (with Id, Date, Amount, Description, PaymentReference, ExtDocNo, DocumentNo).
* Dates on statement lines and ledger entries are specified in the format YYYY-MM-DD

You must return:
* A list of match representations
* Represent each match as (Bank Statement Line ID, [Ledger Entry ID1, Ledger Entry ID2...]). Enclose each match in parentheses. Multiple matches for a single statement line should be in square brackets, separated by commas within that match entry.
* Return **only appropriate matches**, by following the matching instructions.

### Matching Instructions:
* First look for matches on PaymentReference, by using this rule:
** If a bank statement line PaymentReference is equal to a ledger entry Description, match them.
** If a bank statement line PaymentReference is a substring of a ledger entry Description, match them.
* Next, among the statement lines and ledger entries that are left unmatched, look for matches on ledger entry ExtDocNo, by using these rules:
** If a ledger entry ExtDocNo is equal to a bank statement line DocumentNo, match them.
** If a ledger entry ExtDocNo is equal to a bank statement line Description, match them.
** If a ledger entry ExtDocNo is a substring of a bank statement line Description, match them.
* Next, among the statement lines and ledger entries that are left unmatched, look for matches on DocumentNo, by using these rules:
** If DocumentNo fields are equal on a bank statement line and ledger entry, match them.
** If a ledger entry DocumentNo is equal to bank statement line DocumentNo, match them.
** If a ledger entry DocumentNo is a substring of bank statement line Description, match them.
* Next, among the statement lines and ledger entries that are left unmatched, use Amount, Description and Date to match them, by using these rules:
** If both Date and Amount fields are equal on a bank statement line and ledger entry, match them. If there are multiple pairs of statement lines and ledger entries with equal Date and Amount, use the similarity of their Description to decide which ones to match.
** If Amount fields are equal on a bank statement line and a ledger entry, and it is a unique match on Amount, match them, but only if Date is off by less than one month.
** If Amount fields are equal on a bank statement line and a ledger entry, and there are multiple ledger entries with the same Amount, match the ledger entry with the Date closest to the Date of the statement line.
** If Amount fields are equal on a bank statement line and a ledger entry, and there are multiple statement lines with the same Amount, match the statement line with the closest Date to the Date of the ledger entry.
* Next, among the statement lines and ledger entries that are left unmatched, look for one-to-many and many-to-one matches based on Amount, by using these rules:
** A single statement line may match multiple ledger entries if the customer made a combined payment for multiple open ledger entries. The sum of the amounts in the matched ledger entries should bring the net balance to 0, considering the difference between the statement line amount and the combined ledger entry amounts.
** Multiple statement lines may match a single ledger entry if the customer made multiple installments for a single ledger entry. The sum of the amounts in the matched statement lines should bring the net balance to 0, considering the difference between the combined statement line amounts and ledger entry amount.

### General Instructions:
* Prioritize precision over completeness.
* Think long and before making a match. Don't just take a first acceptable match for each line, but provide the best set of matches.
* If a bank statement line can't be matched to any leger entry, do not return any output for that bank statement line.
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Task:
Match the given **statement line descriptions** (which represent expense descriptions) to one of the given **G/L Accounts** (which represent expense categories).

# Instructions:
- Find the G/L Account whose name matches the statement line description best.
- A match is acceptable if the expense described by statement line description matches the expense category described by the G/L Account name.
- Include only acceptable matches in the answer.
- Provide the answer **only for matched lines** by representing each match as a pair of numbers enclosed in brackets, like this: (Statement Line Id, G/L Account Id).
- If you cannot match a statement line, don't return anything for it.
3 changes: 3 additions & 0 deletions src/Apps/W1/BankAccRecWithAI/app/app.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@
"url": "https://go.microsoft.com/fwlink/?LinkId=724011",
"logo": "ExtensionLogo.png",
"dependencies": [],
"resourceFolders": [
".resources"
],
"internalsVisibleTo": [
{
"id": "2932b2a8-7399-4f8c-b1c0-1acfc2014ffb",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -194,19 +194,26 @@ codeunit 7251 "Bank Acc. Rec. Trans. to Acc."
var
BankRecAIMatchingImpl: Codeunit "Bank Rec. AI Matching Impl.";
CompletionTaskTxt: SecretText;
CompletionTaskPartTxt: SecretText;
CompletionTaskBuildingFromKeyVaultFailed: Boolean;
SafetyClauseTxt: SecretText;
TaskPromptTxt: Text;
CompletionTaskBuildingFailed: Boolean;
begin
if BankRecAIMatchingImpl.GetAzureKeyVaultSecret(CompletionTaskPartTxt, 'BankAccRecAITransToGLAccount1') then
CompletionTaskTxt := CompletionTaskPartTxt
if BankRecAIMatchingImpl.GetAzureKeyVaultSecret(SafetyClauseTxt, 'BankAccRecAITransToGLAccountSft') then
CompletionTaskTxt := SafetyClauseTxt
else
CompletionTaskBuildingFromKeyVaultFailed := true;
CompletionTaskBuildingFailed := true;

if CompletionTaskBuildingFromKeyVaultFailed then begin
TaskPromptTxt := NavApp.GetResourceAsText('BankAccRecAITransToGLAccountTask.md', TextEncoding::UTF8);
if TaskPromptTxt = '' then
CompletionTaskBuildingFailed := true;

if CompletionTaskBuildingFailed then begin

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Error\ Handling}$

BuildMostAppropriateGLAccountPromptTask (BankAccRecTransToAcc) merges two distinct failure conditions — (1) GetAzureKeyVaultSecret returning false and (2) NavApp.GetResourceAsText returning '' — into the single CompletionTaskBuildingFailed flag, and both paths emit TelemetryConstructingPromptFailedErr whose text reads 'There was an error with constructing the chat completion prompt from the Key Vault.' When the resource file is missing or empty, telemetry will misleadingly point to the Key Vault, sending incident investigation to the wrong subsystem.

Recommendation:

  • emit a distinct telemetry message for the resource-load failure branch, separate from the Key Vault failure branch.

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why · AL review agent v1.31.4

Session.LogMessage('0000LFI', TelemetryConstructingPromptFailedErr, Verbosity::Error, DataClassification::SystemMetadata, TelemetryScope::All, 'Category', BankRecAIMatchingImpl.FeatureName());
Error(ConstructingPromptFailedErr);
end;

CompletionTaskTxt := BankRecAIMatchingImpl.AddCompletionPromptLine(CompletionTaskTxt, TaskPromptTxt);

exit(CompletionTaskTxt);
end;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,26 @@ codeunit 7250 "Bank Rec. AI Matching Impl."
procedure BuildBankRecCompletionTask(IncludeFewShotExample: Boolean): SecretText
var
CompletionTaskTxt: SecretText;
CompletionTaskPartTxt: SecretText;
CompletionTaskBuildingFromKeyVaultFailed: Boolean;
SafetyClauseTxt: SecretText;
TaskPromptTxt: Text;
CompletionTaskBuildingFailed: Boolean;
begin
if GetAzureKeyVaultSecret(CompletionTaskPartTxt, 'BankAccRecAIMatching1') then
CompletionTaskTxt := CompletionTaskPartTxt
if GetAzureKeyVaultSecret(SafetyClauseTxt, 'BankAccRecAIMatchingSft') then
Comment thread
dcenic marked this conversation as resolved.
CompletionTaskTxt := SafetyClauseTxt
else
CompletionTaskBuildingFromKeyVaultFailed := true;
CompletionTaskBuildingFailed := true;

if CompletionTaskBuildingFromKeyVaultFailed then begin
TaskPromptTxt := NavApp.GetResourceAsText('BankAccRecAIMatchingTask.md', TextEncoding::UTF8);
if TaskPromptTxt = '' then
CompletionTaskBuildingFailed := true;

if CompletionTaskBuildingFailed then begin

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$\textbf{🟡\ Medium\ Severity\ —\ Error\ Handling}$

BuildBankRecCompletionTask merges two distinct failure conditions — (1) GetAzureKeyVaultSecret returning false and (2) NavApp.GetResourceAsText returning '' — into the single CompletionTaskBuildingFailed flag, and both paths emit the same telemetry message TelemetryConstructingPromptFailedErr whose text reads 'There was an error with constructing the chat completion prompt from the Key Vault.' When the resource file fails to load, the telemetry will misleadingly attribute the failure to the Key Vault, causing support engineers to investigate the wrong subsystem.

Recommendation:

  • introduce a separate telemetry message (and ideally a separate telemetry event ID) for the resource-load failure path, e.g. 'There was an error loading the completion task prompt resource file.'

👍 useful · ❤️ especially valuable · 👎 wrong - reply with why · AL review agent v1.31.4

Session.LogMessage('0000LFJ', TelemetryConstructingPromptFailedErr, Verbosity::Error, DataClassification::SystemMetadata, TelemetryScope::All, 'Category', FeatureName());
Error(ConstructingPromptFailedErr);
end;

CompletionTaskTxt := AddCompletionPromptLine(CompletionTaskTxt, TaskPromptTxt);

if (IncludeFewShotExample) then begin
CompletionTaskTxt := AddCompletionPromptLine(CompletionTaskTxt, '\n**Example 1**:\n');
CompletionTaskTxt := AddCompletionPromptLine(CompletionTaskTxt, 'Statement Line: Id: 1, Description: A, Amount: 100, Date: 2023-07-01\n');
Expand Down Expand Up @@ -64,17 +71,14 @@ codeunit 7250 "Bank Rec. AI Matching Impl."
exit(CompletionTaskTxt);
end;

local procedure AddCompletionPromptLine(Completion: SecretText; NewPromptLine: Text): SecretText
var
ConcatSubstrTok: Label '%1%2', Locked = true;
internal procedure AddCompletionPromptLine(Completion: SecretText; NewPromptLine: Text): SecretText
begin
exit(SecretStrSubstNo(ConcatSubstrTok, Completion, NewPromptLine));
end;

procedure BuildBankRecCompletionPrompt(TaskPrompt: SecretText; StatementLines: Text; LedgerLines: Text): SecretText
var
CompletionPrompt: SecretText;
ConcatSubstrTok: Label '%1%2', Locked = true;
begin
LedgerLines += '"""\n**Matches**:'; // close the ledger lines section
StatementLines += '"""\n'; // close the statement lines section
Expand All @@ -87,7 +91,6 @@ codeunit 7250 "Bank Rec. AI Matching Impl."
var
UserMessageTxt: SecretText;
EmptyText: SecretText;
ConcatSubstrTok: Label '%1%2', Locked = true;
begin
LedgerLines += '"""\n**Matches**:'; // close the ledger lines section
StatementLines += '"""\n'; // close the statement lines section
Expand Down Expand Up @@ -727,6 +730,7 @@ codeunit 7250 "Bank Rec. AI Matching Impl."

var
AOAIToken: Codeunit "AOAI Token";
ConcatSubstrTok: Label '%1%2', Locked = true;
MatchedByCopilotTxt: label 'Matched by Copilot based on semantic similarity.', Comment = 'Copilot is a Microsoft service name and must not be translated';
SuccessfullyFilteredBLEListTxt: label 'Successfully filtered bank account ledger entries based on posting date.', Locked = true;
UnableToFilterBLEListUnderTokenLimitTxt: label 'Unable to filter the bank account ledger entry list.', Locked = true;
Expand Down
Loading