diff --git a/data-explorer/kusto/management/alter-merge-table-update-policy-command.md b/data-explorer/kusto/management/alter-merge-table-update-policy-command.md index d58c24f469..6efa91b0d1 100644 --- a/data-explorer/kusto/management/alter-merge-table-update-policy-command.md +++ b/data-explorer/kusto/management/alter-merge-table-update-policy-command.md @@ -3,7 +3,7 @@ title: .alter-merge table policy update command description: Learn how to use the `.alter-merge table policy` command to change the table's update policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 06/01/2026 --- # .alter-merge table policy update command @@ -58,4 +58,54 @@ The following command changes the update policy for a table using [multi-line st } ] ``` -```` \ No newline at end of file +```` + +::: moniker range="azure-data-explorer" + +### Example with an accelerated external table reference + +The following example adds an update policy entry whose query joins data from an accelerated external table. Because the external table uses impersonation authentication, the policy must include a `ManagedIdentity`: + +````kusto +.alter-merge table MyDatabase.MyTargetTable policy update +``` +[ + { + "IsEnabled": true, + "Source": "MySourceTable", + "Query": "MySourceTable | join kind=leftouter (external_table('MyAcceleratedExternalTable')) on CommonKey | project-away CommonKey1", + "IsTransactional": true, + "PropagateIngestionProperties": false, + "ManagedIdentity": "system" + } +] +``` +```` + +::: moniker-end + +::: moniker range="microsoft-fabric" + +### Example with an accelerated external table reference + +The following example adds an update policy entry whose query joins data from an accelerated external table. In Fabric, the system automatically populates the `OwnerPrincipalDetails` property for authorization: + +````kusto +.alter-merge table MyDatabase.MyTargetTable policy update +``` +[ + { + "IsEnabled": true, + "Source": "MySourceTable", + "Query": "MySourceTable | join kind=leftouter (external_table('MyAcceleratedExternalTable')) on CommonKey | project-away CommonKey1", + "IsTransactional": true, + "PropagateIngestionProperties": false + } +] +``` +```` + +> [!NOTE] +> When the policy is read back, the returned JSON may include a system-populated `OwnerPrincipalDetails` property. This property captures the identity of the user who sets or alters the update policy, and is used by the system for authorization when the query references external tables. The property is read-only and set automatically. + +::: moniker-end \ No newline at end of file diff --git a/data-explorer/kusto/management/alter-query-acceleration-policy-command.md b/data-explorer/kusto/management/alter-query-acceleration-policy-command.md index 00c4453610..5ddbb60141 100644 --- a/data-explorer/kusto/management/alter-query-acceleration-policy-command.md +++ b/data-explorer/kusto/management/alter-query-acceleration-policy-command.md @@ -3,7 +3,7 @@ title: .alter query acceleration policy command description: Learn how to use the .alter query acceleration policy command to accelerate queries over external delta tables. ms.reviewer: sharmaanshul ms.topic: reference -ms.date: 11/30/2025 +ms.date: 06/01/2026 --- # `.alter query acceleration policy` command @@ -103,3 +103,13 @@ The command returns a table with one record that includes the modified policy ob - [.delete query acceleration policy command](delete-query-acceleration-policy-command.md) - [.show query acceleration policy command](show-query-acceleration-policy-command.md) - [.show external table operations query_acceleration statistics](show-external-table-operations-query-acceleration-statistics.md) + +## Example: Setting hot period for use with update policy + +To allow an accelerated external table to be referenced from an [update policy](update-policy.md) query, the `Hot` period must cover all data (>= 100 years): + +```Kusto +.alter external table MyExternalTable policy query_acceleration '{"IsEnabled": true, "Hot": "36500.00:00:00"}' +``` + +This enables the external table to be used with `external_table('MyExternalTable')` inside update policy functions. For more information, see [Use with update policies](query-acceleration-policy.md#use-with-update-policies). diff --git a/data-explorer/kusto/management/alter-table-update-policy-command.md b/data-explorer/kusto/management/alter-table-update-policy-command.md index c335254dfb..e98dd96ed3 100644 --- a/data-explorer/kusto/management/alter-table-update-policy-command.md +++ b/data-explorer/kusto/management/alter-table-update-policy-command.md @@ -3,7 +3,7 @@ title: .alter table policy update command description: Learn how to use the `.alter table policy update` command to change the table update policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 06/01/2026 --- # .alter table policy update command @@ -60,4 +60,54 @@ Change the update policy for a table (using [multi-line string literals](../quer } ] ``` -```` \ No newline at end of file +```` + +::: moniker range="azure-data-explorer" + +### Example with an accelerated external table reference + +The following example sets an update policy whose query joins data from an accelerated external table. Because the external table uses impersonation authentication, the policy must include a `ManagedIdentity`: + +````kusto +.alter table MyDatabase.MyTargetTable policy update +``` +[ + { + "IsEnabled": true, + "Source": "MySourceTable", + "Query": "MySourceTable | join kind=leftouter (external_table('MyAcceleratedExternalTable')) on CommonKey | project-away CommonKey1", + "IsTransactional": true, + "PropagateIngestionProperties": false, + "ManagedIdentity": "system" + } +] +``` +```` + +::: moniker-end + +::: moniker range="microsoft-fabric" + +### Example with an accelerated external table reference + +The following example sets an update policy whose query joins data from an accelerated external table. In Fabric, the system automatically populates the `OwnerPrincipalDetails` property for authorization: + +````kusto +.alter table MyDatabase.MyTargetTable policy update +``` +[ + { + "IsEnabled": true, + "Source": "MySourceTable", + "Query": "MySourceTable | join kind=leftouter (external_table('MyAcceleratedExternalTable')) on CommonKey | project-away CommonKey1", + "IsTransactional": true, + "PropagateIngestionProperties": false + } +] +``` +```` + +> [!NOTE] +> When the policy is read back, the returned JSON may include a system-populated `OwnerPrincipalDetails` property. This property captures the identity of the user who sets or alters the update policy, and is used by the system for authorization when the query references external tables. The property is read-only and set automatically. + +::: moniker-end \ No newline at end of file diff --git a/data-explorer/kusto/management/query-acceleration-policy.md b/data-explorer/kusto/management/query-acceleration-policy.md index e68831621f..620d9d3291 100644 --- a/data-explorer/kusto/management/query-acceleration-policy.md +++ b/data-explorer/kusto/management/query-acceleration-policy.md @@ -3,7 +3,7 @@ title: Query acceleration policy description: Learn how to use the query acceleration policy to accelerate queries over external delta tables. ms.reviewer: sharmaanshul ms.topic: reference -ms.date: 09/16/2025 +ms.date: 06/01/2026 --- # Query acceleration policy @@ -50,3 +50,22 @@ To enable query acceleration in the Fabric UI, see [Query acceleration over OneL * [.delete query acceleration policy command](delete-query-acceleration-policy-command.md) * [.show query acceleration policy command](show-query-acceleration-policy-command.md) * [.show external table operations query_acceleration statistics](show-external-table-operations-query-acceleration-statistics.md) + +## Use with update policies + +Accelerated external tables can be referenced from an [update policy](update-policy.md) query using the [`external_table()` function](../query/external-table-function.md). This enables enriching or joining ingested data with external delta table data as part of the update policy transformation. + +The following conditions must be met: + +* The query acceleration policy must be **enabled** on the external table. +* The `Hot` period must cover **all data** in the external table. Currently, this requires setting `Hot` to a value >= 100 years (for example, `"Hot": "36500.00:00:00"`). + +::: moniker range="azure-data-explorer" +* If the external table uses impersonation authentication, the update policy must be configured with a `ManagedIdentity` that has appropriate permissions on the external table's underlying storage. +::: moniker-end + +::: moniker range="microsoft-fabric" +* In Fabric, the system automatically handles authorization through the `OwnerPrincipalDetails` property, which is populated when the update policy is created or altered. +::: moniker-end + +For more information, see [Update policy overview - Query limitations](update-policy.md#query-limitations). diff --git a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md index 600db4c098..03d934cabd 100644 --- a/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md +++ b/data-explorer/kusto/management/query-acceleration-troubleshooting-guide.md @@ -3,7 +3,7 @@ title: Query Acceleration Troubleshooting Guide description: Learn how to troubleshoot for common errors encountered in query acceleration. ms.reviewer: urishapira ms.topic: reference -ms.date: 12/08/2025 +ms.date: 06/01/2026 --- # Troubleshoot query acceleration over external delta tables @@ -235,3 +235,32 @@ Run the following command to view the remaining capacity: > [!NOTE] > Altering the capacity policy might have adverse effects on other operations. Alter the policy as a last resort at your own discretion. +## Troubleshoot update policy failures involving external tables + +When an [update policy](update-policy.md) query references an accelerated external table, additional failure scenarios can occur. Use the [`.show ingestion failures`](ingestion-failures.md) command to identify update policy failures: + +```kusto +.show ingestion failures +| where FailedOn > ago(1h) and OriginatesFromUpdatePolicy == true +``` + +### Common update policy errors with external tables + +::: moniker range="azure-data-explorer" + +| Error | Cause | Resolution | +|-------|-------|------------| +| Update policy query references an external table without query acceleration covering all data | The external table's query acceleration policy has a `Hot` period < 100 years, or the policy is disabled. | Enable the query acceleration policy on the external table with `Hot` >= 100 years (for example, `"Hot": "36500.00:00:00"`). | +| Managed identity is missing or invalid | The update policy references an external table with impersonation authentication but doesn't have a valid `ManagedIdentity` configured. | Add a valid `ManagedIdentity` (object ID or `system`) to the update policy. Ensure the identity has `AutomatedFlows` usage in the managed identity policy and appropriate storage permissions. | +| External table access forbidden during update policy execution | The managed identity configured in the update policy doesn't have permissions on the external table's underlying storage. | Grant the managed identity read permissions on the storage account or container backing the external table. | + +::: moniker-end + +::: moniker range="microsoft-fabric" + +| Error | Cause | Resolution | +|-------|-------|------------| +| Update policy query references an external table without query acceleration covering all data | The external table's query acceleration policy has a `Hot` period < 100 years, or the policy is disabled. | Enable the query acceleration policy on the external table with `Hot` >= 100 years (for example, `"Hot": "36500.00:00:00"`). | +| Owner principal is invalid or missing | The `OwnerPrincipalDetails` on the update policy is invalid or the owner no longer has access to the external table. | Re-alter the update policy to refresh the `OwnerPrincipalDetails`. Ensure the policy owner has access to the external table's underlying data. | + +::: moniker-end diff --git a/data-explorer/kusto/management/show-table-update-policy-command.md b/data-explorer/kusto/management/show-table-update-policy-command.md index d99d7b0388..c868338df0 100644 --- a/data-explorer/kusto/management/show-table-update-policy-command.md +++ b/data-explorer/kusto/management/show-table-update-policy-command.md @@ -3,7 +3,7 @@ title: .show table policy update command description: Learn how to use the `.show table policy update` command to show the table's update policy. ms.reviewer: yonil ms.topic: reference -ms.date: 08/11/2024 +ms.date: 06/01/2026 --- # .show table policy update command @@ -36,6 +36,13 @@ You must have at least Database User, Database Viewer, or Database Monitor permi Returns a JSON representation of the policy. +::: moniker range="microsoft-fabric" + +> [!NOTE] +> The returned policy JSON may include system-populated properties such as `OwnerPrincipalDetails`. This property is read-only and automatically set by the system for authorization when the update policy query references external tables. + +::: moniker-end + ## Example The following example shows the update policies for all tables: diff --git a/data-explorer/kusto/management/update-policy-with-managed-identity.md b/data-explorer/kusto/management/update-policy-with-managed-identity.md index 4da4ab6f9d..3fe00934dd 100644 --- a/data-explorer/kusto/management/update-policy-with-managed-identity.md +++ b/data-explorer/kusto/management/update-policy-with-managed-identity.md @@ -3,7 +3,7 @@ title: Run an update policy with a managed identity description: This article describes how to use a managed identity for update policy. ms.reviewer: atefsawaed ms.topic: reference -ms.date: 11/13/2024 +ms.date: 06/01/2026 monikerRange: "azure-data-explorer" ms.custom: sfi-image-nochange --- @@ -15,6 +15,7 @@ The update policy must be configured with a [managed identity](/azure/data-explo * When the update policy query references tables in other databases * When the update policy query references tables with an enabled [row level security policy](row-level-security-policy.md) +* When the update policy query references an accelerated external table that uses impersonation authentication An update policy configured with a managed identity is performed on behalf of the managed identity. @@ -142,3 +143,32 @@ For example, the following command alters the update policy of the table `MyTabl ] ``` ```` + +--- + +## Create an update policy that references an accelerated external table + +When the update policy query references an accelerated external table that uses impersonation authentication, you must configure the policy with a `ManagedIdentity`. The managed identity must have the appropriate permissions on the external table's underlying storage. + +> [!NOTE] +> The external table must have a [query acceleration policy](query-acceleration-policy.md) enabled with a `Hot` period that covers all data (currently >= 100 years). + +### Example + +The following example creates an update policy whose query joins data from an accelerated external table using `external_table()`: + +````kusto +.alter table MyDatabase.MyTargetTable policy update +``` +[ + { + "IsEnabled": true, + "Source": "MySourceTable", + "Query": "MySourceTable | join kind=leftouter (external_table('MyAcceleratedExternalTable')) on CommonKey | project-away CommonKey1", + "IsTransactional": true, + "PropagateIngestionProperties": false, + "ManagedIdentity": "system" + } +] +``` +```` diff --git a/data-explorer/kusto/management/update-policy.md b/data-explorer/kusto/management/update-policy.md index fb9f43d774..b297d3b4e3 100644 --- a/data-explorer/kusto/management/update-policy.md +++ b/data-explorer/kusto/management/update-policy.md @@ -3,7 +3,7 @@ title: Update policy overview description: Learn how to trigger an update policy to add data to a source table. ms.reviewer: orspodek ms.topic: reference -ms.date: 02/04/2026 +ms.date: 06/01/2026 --- # Update policy overview @@ -53,7 +53,10 @@ If the update policy is defined on the target table, multiple queries can run on * The policy-related query can invoke stored functions, but: * It can't perform cross-cluster queries. - * It can't access external data or external tables. + * It can't access external data or external tables, with the following exception: + * The query *can* reference an accelerated external table using the [`external_table()` function](../query/external-table-function.md), provided that: + * The external table has a [query acceleration policy](query-acceleration-policy.md) enabled with a `Hot` period that covers all data (currently `Hot` >= 100 years). + * The update policy is configured with a `ManagedIdentity` property if the external table uses impersonation authentication. * It can't make callouts (by using a plugin). * The query doesn't have read access to tables that have the [RestrictedViewAccess policy](restricted-view-access-policy.md) enabled. @@ -67,7 +70,9 @@ If the update policy is defined on the target table, multiple queries can run on * The policy-related query can invoke stored functions, but: * It can't perform cross-eventhouse queries. - * It can't access external data or external tables. + * It can't access external data or external tables, with the following exception: + * The query *can* reference an accelerated external table using the [`external_table()` function](../query/external-table-function.md), provided that: + * The external table has a [query acceleration policy](query-acceleration-policy.md) enabled with a `Hot` period that covers all data (currently `Hot` >= 100 years). * It can't make callouts (by using a plugin). * The query doesn't have read access to tables that have the [RestrictedViewAccess policy](restricted-view-access-policy.md) enabled. @@ -115,7 +120,7 @@ Each such object is represented as a JSON property bag, with the following prope |Query |`string` |A query used to produce data for the update. | |IsTransactional |`bool` |States if the update policy is transactional or not, default is *false*. If the policy is transactional and the update policy fails, the source table isn't updated. | |PropagateIngestionProperties |`bool`|States if properties specified during ingestion to the source table, such as [extent tags](extent-tags.md) and creation time, apply to the target table. | -|ManagedIdentity | `string` | The managed identity on behalf of which the update policy runs. The managed identity can be an object ID, or the `system` reserved word. The update policy must be configured with a managed identity when the query references tables in other databases or tables with an enabled [row level security policy](row-level-security-policy.md). For more information, see [Use a managed identity to run a update policy](update-policy-with-managed-identity.md). | +|ManagedIdentity | `string` | The managed identity on behalf of which the update policy runs. The managed identity can be an object ID, or the `system` reserved word. The update policy must be configured with a managed identity when the query references tables in other databases, tables with an enabled [row level security policy](row-level-security-policy.md), or accelerated external tables that use impersonation authentication. For more information, see [Use a managed identity to run a update policy](update-policy-with-managed-identity.md). | ::: moniker-end ::: moniker range="microsoft-fabric" @@ -128,6 +133,7 @@ Each such object is represented as a JSON property bag, with the following prope |Query |`string` |A query used to produce data for the update | |IsTransactional |`bool` |States if the update policy is transactional or not, default is *false*. If the policy is transactional and the update policy fails, the source table isn't updated. | |PropagateIngestionProperties |`bool`|States if properties specified during ingestion to the source table, such as [extent tags](extent-tags.md) and creation time, apply to the target table. | +|OwnerPrincipalDetails | `object` | A system-populated, read-only property. Contains the principal details of the user who sets or alters the update policy, used for authorization when the update policy query references external tables. This property is automatically set by the system and can't be modified manually. | ::: moniker-end diff --git a/data-explorer/kusto/query/external-table-function.md b/data-explorer/kusto/query/external-table-function.md index 17888c7ab8..4f1933d787 100644 --- a/data-explorer/kusto/query/external-table-function.md +++ b/data-explorer/kusto/query/external-table-function.md @@ -3,7 +3,7 @@ title: external_table() description: Learn how to use the external_table() function to reference an external table by name. ms.reviewer: alexans ms.topic: reference -ms.date: 08/11/2024 +ms.date: 06/01/2026 --- # external_table() @@ -37,6 +37,23 @@ To accelerate queries over external delta tables, see [Query acceleration policy The authentication method to access an external table is based on the connection string provided during its creation, and the permissions required to access the table vary depending on the authentication method. For more information, see [Azure Storage external table](../management/external-tables-azure-storage.md#authentication-and-authorization) or [SQL Server external table](../management/external-sql-tables.md). +## Use in update policy queries + +The `external_table()` function can be used inside [update policy](../management/update-policy.md) queries under restricted conditions: + +* The external table must have a [query acceleration policy](../management/query-acceleration-policy.md) **enabled**. +* The query acceleration `Hot` period must cover all data in the external table (currently >= 100 years). + +::: moniker range="azure-data-explorer" +* If the external table uses impersonation authentication, the update policy must be configured with a `ManagedIdentity`. +::: moniker-end + +::: moniker range="microsoft-fabric" +* In Fabric, the system automatically handles authorization through the `OwnerPrincipalDetails` property on the update policy, which captures the identity of the user who sets or alters the policy. +::: moniker-end + +For more information, see [Update policy overview - Query limitations](../management/update-policy.md#query-limitations). + ## Related content * [External tables overview](schema-entities/external-tables.md) diff --git a/data-explorer/kusto/query/inline-external-table-operator.md b/data-explorer/kusto/query/inline-external-table-operator.md new file mode 100644 index 0000000000..3c6a45fd94 --- /dev/null +++ b/data-explorer/kusto/query/inline-external-table-operator.md @@ -0,0 +1,208 @@ +--- +title: inline_external_table operator +description: Learn how to use the inline_external_table operator to return a data table whose data is retrieved from the specified external storage artifact. +ms.reviewer: TBD +ms.topic: reference +ms.date: 06/16/2026 +--- +# inline_external_table operator + +> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)] + +The `inline_external_table` operator returns a table whose schema is defined in the query itself or inferred from Delta Lake metadata, and whose data is read from an external storage artifact, such as a blob in Azure Blob Storage or a file in Azure Data Lake Storage. + +::: moniker range="azure-data-explorer" +> [!NOTE] +> The `inline_external_table` operator supports: +> * a specific set of storage services, as listed under [Storage connection strings](../api/connection-strings/storage-connection-strings.md). +> * shared Access Signature (SAS) key, Access key, Microsoft Entra Token, and Managed Identity authentication methods. For more information, see [Storage authentication methods](../api/connection-strings/storage-connection-strings.md#storage-authentication-methods). + +::: moniker-end + +::: moniker range="microsoft-fabric" +> [!NOTE] +> The `inline_external_table` operator supports: +> * a specific set of storage services, as listed under [Storage connection strings](../api/connection-strings/storage-connection-strings.md). +> * shared Access Signature (SAS) key, Access key, and Microsoft Entra Token authentication methods. For more information, see [Storage authentication methods](../api/connection-strings/storage-connection-strings.md#storage-authentication-methods). + +::: moniker-end + +## Syntax + +### Storage mode + +`inline_external_table` `(`*columnName*`:`*columnType* [`,` ...] `)` +`kind` `=` `storage` +[`partition` `by` `(`*partitionName*`:`*partitionType* [`=` *expression*] [`,` ...]`)`] +[`pathformat` `=` `(`*pathFormatElement* [...]`)`] +`dataformat` `=` *dataFormat* +`(` *storageConnectionString* [`,` ...] `)` +[`with` `(` *propertyName* `=` *propertyValue* [`,` ...]`)`] + +### Delta mode + +`inline_external_table` [`(`*columnName*`:`*columnType* [`,` ...] `)`] +`kind` `=` `delta` +`(` *storageConnectionString* `)` + +[!INCLUDE [syntax-conventions-note](../includes/syntax-conventions-note.md)] + +## Parameters + +| Name | Type | Required | Description | +|--|--|--|--| +| *columnName*, *columnType* | `string` | :heavy_check_mark: for `kind=storage` | A list of column names and their types. This list defines the schema of the table. In `kind=delta`, this list is optional. If the schema isn't specified, it's inferred from the Delta log. If a schema is specified, the columns must exist in the Delta table and must use compatible types. | +| *kind* | `string` | :heavy_check_mark: | The inline external table type. Supported values are `storage` and `delta`. | +| *partitionName*, *partitionType*, *expression* | `string` | | A list of optional partition columns for `kind=storage`. Supported partition types are `string`, `datetime`, and `long`. The optional expression maps a partition column to a source column expression. | +| *pathFormatElement* | `string` | | An optional path format definition for `kind=storage`. The path format can include literal path segments, partition column references, and `datetime_pattern()` expressions. | +| *dataFormat* | `string` | :heavy_check_mark: for `kind=storage` | The data format of the external storage files. This parameter isn't supported for `kind=delta`. | +| *storageConnectionString* | `string` | :heavy_check_mark: | A [storage connection string](../api/connection-strings/storage-connection-strings.md) of the storage artifact to query. `kind=storage` accepts one or more storage connection strings. `kind=delta` requires exactly one Delta table root URI. | +| *propertyName*, *propertyValue* | `string` | | A list of optional [supported properties](#supported-properties) that determines how to interpret the data retrieved from storage. These properties apply to `kind=storage`. | + +> [!NOTE] +> For security reasons, make sure that no credential is specified by the *storageConnectionString* property. +> If the query needs to specify credentials, use [query parameters](./query-parameters-statement.md) to specify the whole connection string. +> +> For example, assuming that the query includes a query parameter called `URI` whose value points at external storage, the query would look like this: +> ```kusto +> declare query_parameters(URI:string); +> inline_external_table(x:string) kind=storage dataformat=txt (URI) +> ``` +> +> If this is not possible (for example, you're using a client that does not support setting query parameters), +> be sure to use [obfuscated string literals](./scalar-data-types/string.md#obfuscated-string-literals). + +### Supported properties + +The following properties apply only to `kind=storage`. + +| Property | Type | Description | +|--|--|--| +| includeHeaders | `string` | Header handling for text formats. For example, use `All` to skip the first record in every file when CSV files contain headers. | +| encoding | `string` | Specifies how text is encoded, such as `UTF8NoBOM` or `UTF8BOM`. | +| namePrefix | `string` | If set, only files with this name prefix are read. | +| fileExtension | `string` | If set, only files with this file extension are read. | + +> [!NOTE] +> +> This operator doesn't accept any pipeline input. +> +> Standard [query limits](../concepts/query-limits.md) apply to external data queries as well. +> +> In `kind=delta`, `dataformat`, `partition by`, and `pathformat` aren't supported. The data format, partitioning information, and path format are inferred from Delta metadata. Delta mode requires exactly one Delta table root URI. + +## Returns + +The `inline_external_table` operator returns a data table whose data is retrieved from the specified storage artifact, indicated by the storage connection string. + +In `kind=storage`, the table schema is the schema specified in the query. In `kind=delta`, the table schema is either specified in the query or inferred from the Delta log. Partition columns inferred from Delta metadata are returned as virtual columns. + +## Examples + +The following example queries a Parquet file stored in external storage. + +```kusto +inline_external_table(Timestamp:datetime, DeviceId:string, Value:real) + kind=storage + dataformat=parquet + (h@"https://mycompanystorage.blob.core.windows.net/iot/2026/06/01/data.parquet?...SAS...") +| summarize AvgValue = avg(Value) by DeviceId +``` + +**Output** + +|DeviceId|AvgValue| +|---|---| +|Device-001|23.45| +|Device-002|18.92| +|Device-003|31.67| + +The following example queries multiple CSV files stored in external storage. + +```kusto +inline_external_table(Timestamp:datetime, ProductId:string, Quantity:long) + kind=storage + dataformat=csv + ( + h@"https://mycompanystorage.blob.core.windows.net/archivedproducts/2026/06/01/part-00000.csv.gz?...SAS...", + h@"https://mycompanystorage.blob.core.windows.net/archivedproducts/2026/06/02/part-00000.csv.gz?...SAS..." + ) + with(includeHeaders='All') +| summarize TotalQuantity = sum(Quantity) by ProductId +``` + +**Output** + +|ProductId|TotalQuantity| +|---|---| +|PROD-123|1500| +|PROD-456|2300| +|PROD-789|870| + +The following example queries partitioned Parquet files stored in external storage. + +```kusto +inline_external_table(Timestamp:datetime, DeviceId:string, Region:string, Value:real) + kind=storage + partition by( + Day:datetime = startofday(Timestamp), + RegionPartition:string = Region + ) + pathformat=('day=' datetime_pattern('yyyy/MM/dd', Day) '/region=' RegionPartition) + dataformat=parquet + (h@"https://mycompanystorage.blob.core.windows.net/iotpartitioned;managed_identity=...") +| where Day between (datetime(2026-01-01) .. datetime(2026-01-07)) +| summarize AvgValue = avg(Value) by Region +``` + +**Output** + +|Region|AvgValue| +|---|---| +|East|27.8| +|West|22.1| +|North|19.5| +|South|31.2| + +The following example queries a Delta Lake table root. The schema is inferred from the Delta log, and the inferred `Amount` column is used for filtering. + +```kusto +inline_external_table + kind=delta + (h@"abfss://filesystem@storageaccount.dfs.core.windows.net/delta/customer-events;sharedkey=...") +| where Amount > 0 +| take 10 +``` + +**Output** + +|Timestamp|CustomerId|Amount|EventType| +|---|---|---|---| +|2026-06-01T10:23:45Z|CUST-001|150.00|Purchase| +|2026-06-01T11:15:30Z|CUST-002|75.50|Purchase| +|2026-06-01T12:45:00Z|CUST-003|200.00|Purchase| +|2026-06-01T13:20:15Z|CUST-001|50.25|Purchase| +|2026-06-01T14:30:00Z|CUST-004|125.75|Purchase| + +The following example queries a Delta Lake table root with an explicit schema. + +```kusto +inline_external_table(Timestamp:datetime, CustomerId:string, Amount:real) + kind=delta + (h@"abfss://filesystem@storageaccount.dfs.core.windows.net/delta/customer-events;sharedkey=...") +| summarize TotalAmount = sum(Amount) by CustomerId +``` + +**Output** + +|CustomerId|TotalAmount| +|---|---| +|CUST-001|1250.75| +|CUST-002|890.50| +|CUST-003|2100.00| +|CUST-004|675.25| + +## Related content + +* For more info on storage connection strings, see [Storage connection strings](../api/connection-strings/storage-connection-strings.md). +* For more info on external tables, see [external tables](schema-entities/external-tables.md). diff --git a/data-explorer/kusto/query/toc.yml b/data-explorer/kusto/query/toc.yml index 0c4ef2a575..160b6f08ce 100644 --- a/data-explorer/kusto/query/toc.yml +++ b/data-explorer/kusto/query/toc.yml @@ -128,6 +128,8 @@ items: href: fork-operator.md - name: getschema operator href: getschema-operator.md + - name: inline external table operator + href: inline-external-table-operator.md - name: invoke operator href: invoke-operator.md - name: join operator