feat: report each provider's deletion rules, stop S3 dropping other rules

CLAUDE.md

@@ -85,18 +85,22 @@ Configuration is read from the `"CloudFiles"` section in `appsettings.json` (`Cl
 
 ### Automatic Lifecycle Rules
 
-Three providers automatically create delete rules for temp-prefix objects at DI registration time. All three check for existing rules and only add missing ones.
+Every provider creates delete rules for temp-prefix objects at DI registration time, checks for existing rules and only adds missing (or disabled) ones, and writes every other rule back unchanged. Each also registers `ICloudFilesLifecycle` (PrimitiveTypes), which reads the rules back; the per-provider mapping lives in an internal static `*LifecycleRules` class with pure unit tests.
 
 | Provider | Mechanism | Opt-out |
 |----------|-----------|---------|
 | S3 | S3 Lifecycle Configuration API | `S3CloudFilesOptions.DisableAutoLifecycle = true` |
 | GCS | GCS Bucket Lifecycle API (`PatchBucket`) | `GoogleCloudFilesOptions.DisableAutoLifecycle = true` |
 | Oracle | OCI Object Lifecycle Policy API (`PutObjectLifecyclePolicy`) | `OracleCloudFilesOptions.DisableAutoLifecycle = true` |
-| Azure | **Not automatic** — ARM plane required | N/A (property exists but is no-op) |
+| Azure | Account lifecycle management policy via ARM (`Azure.ResourceManager.Storage` 1.6.2), only when `SubscriptionId` + `ResourceGroupName` are set | `AzureCloudFilesOptions.DisableAutoLifecycle = true` |
 | LocalTests | N/A — call `CloudFilesService.Cleanup()` in test teardown | N/A |
 
 Lifecycle prefixes: `temp1/` (1 day), `temp7/` (7 days), `temp30/` (30 days), `temp365/` (365 days).
 
+The S3 lifecycle API replaces the whole configuration on every write; before 10.0.1 `AddS3CloudFiles` sent only the missing temp rules and so deleted every other rule on the bucket. `S3LifecycleRules.WithTempRules` now returns the full merged set.
+
+`Azure.ResourceManager.Storage` is pinned to 1.6.2: 1.7+ needs an Azure.Core in which the Identity types moved into Core, which clashes with `Azure.Identity` 1.13 (CS0433) here and in every consuming app.
+
 S3 and GCS also auto-create the bucket if it does not exist.
 
 ### GCS: `StorageClient` and `UrlSigner` via DI

README.md

@@ -267,7 +267,7 @@ public class FilesController : ControllerBase
 
 ## Automatic Lifecycle Management
 
-Three of the four providers automatically create **delete lifecycle rules** for temp-prefix objects on startup. This is useful for objects that are only needed temporarily.
+Every provider creates **delete lifecycle rules** for temp-prefix objects on startup — Azure once it can reach the storage account's policy (see below). This is useful for objects that are only needed temporarily.
 
 | Prefix | Expiry |
 |--------|--------|
@@ -276,7 +276,18 @@ Three of the four providers automatically create **delete lifecycle rules** for
 | `temp30/`  | 30 days |
 | `temp365/` | 365 days |
 
-Rules are only added if they don't already exist — existing rules are never modified or removed.
+Rules are only added if they don't already exist (a disabled temp rule is switched back on). Every other rule on the bucket is kept as it is. Before 10.0.1 the S3 provider wrote only the missing temp rules, which replaced the bucket's whole configuration and deleted any other rules on it.
+
+### Reading the rules back
+
+Each provider also registers `ICloudFilesLifecycle` (from `SimplyWorks.PrimitiveTypes`), which reports the bucket's deletion rules as the storage service has them — prefix, days and whether each is enabled — so an app can say how long a file under a given key is kept:
+
+```csharp
+var lifecycle = await serviceProvider.GetRequiredService<ICloudFilesLifecycle>().GetLifecycleAsync();
+var rule = lifecycle.RuleFor("temp30/invoices/123.pdf");   // null when nothing deletes it
+```
+
+Only rules that delete by age are reported; rules that move objects to another tier, delete on a date, or only reach tagged objects are left out. `Unavailable` explains when the rules can't be known without more configuration (Azure without its policy settings).
 
 ### Disabling Lifecycle Management
 
@@ -297,6 +308,11 @@ services.AddOracleCloudFiles(o => {
     // ... other config
     o.DisableAutoLifecycle = true;
 });
+
+services.AddAsCloudFiles(o => {
+    // ... other config
+    o.DisableAutoLifecycle = true;
+});
 ```
 
 ### Provider Support
@@ -306,7 +322,7 @@ services.AddOracleCloudFiles(o => {
 | S3 | ✅ Automatic | Rules applied via S3 Lifecycle Configuration API |
 | Google Cloud | ✅ Automatic | Rules applied via GCS Bucket Lifecycle API; bucket is created if it doesn't exist |
 | Oracle Cloud | ✅ Automatic | Rules applied via OCI Object Lifecycle Policy API |
-| Azure Blob | ⚠️ Manual | Azure lifecycle management requires the Azure Resource Manager (ARM) plane, which is separate from the data-plane SDK used by this library. Configure lifecycle rules via the [Azure Portal](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview), Azure CLI (`az storage account management-policy create`), or ARM templates. |
+| Azure Blob | ✅ Automatic, once configured | The rules live in the storage account's [lifecycle management policy](https://learn.microsoft.com/en-us/azure/storage/blobs/lifecycle-management-overview), on the Azure Resource Manager plane. Set `SubscriptionId` and `ResourceGroupName` (and `StorageAccountName` when the service URL is a custom domain or private link). The managed identity, or a service principal in `AZURE_TENANT_ID` / `AZURE_CLIENT_ID` / `AZURE_CLIENT_SECRET`, needs `Microsoft.Storage/storageAccounts/managementPolicies/read` and `write` — Storage Account Contributor has them; Storage Blob Data Contributor doesn't. Rules are named `{container}-temp30` and filtered to `{container}/temp30/`, so other containers' rules are untouched. Changes can take up to 24 hours to apply. Without the two settings nothing is created and nothing deletes temp files. |
 
 ## Signed URLs
 

SW.CloudFiles.AS.Extensions/IServiceCollectionExtensions.cs

@@ -1,4 +1,6 @@
 using System;
+using Azure;
+using Azure.ResourceManager.Storage;
 using Microsoft.Extensions.Configuration;
 using Microsoft.Extensions.DependencyInjection;
 using SW.CloudFiles.AS;
@@ -10,13 +12,16 @@ namespace SW.CloudFiles.AS.Extensions;
 public static class IServiceCollectionExtensions
 {
     /// <summary>
-    /// Registers Azure Blob Storage as the <see cref="ICloudFilesService"/> implementation.
+    /// Registers Azure Blob Storage as the <see cref="ICloudFilesService"/> implementation, and
+    /// <see cref="ICloudFilesLifecycle"/>, which reads the storage account's deletion rules back.
     /// On startup, creates the container if it does not exist.
     /// <para>
-    /// Note: Azure lifecycle management policies require the Azure Resource Manager (ARM) plane and
-    /// are not configured automatically. Set them via the Azure Portal, Azure CLI
-    /// (<c>az storage account management-policy create</c>), or ARM templates.
-    /// <see cref="AzureCloudFilesOptions.DisableAutoLifecycle"/> is accepted but has no effect.
+    /// Once <see cref="AzureCloudFilesOptions.SubscriptionId"/> and
+    /// <see cref="AzureCloudFilesOptions.ResourceGroupName"/> are set, and unless
+    /// <see cref="AzureCloudFilesOptions.DisableAutoLifecycle"/> is true, also ensures the account's lifecycle
+    /// management policy deletes this container's blobs under temp1/, temp7/, temp30/ and temp365/ after 1, 7,
+    /// 30 and 365 days, keeping every other rule in the policy. The policy lives on the Azure Resource Manager
+    /// plane, so the identity needs a role that can manage it; see <see cref="AzureCloudFilesOptions.SubscriptionId"/>.
     /// </para>
     /// </summary>
     public static IServiceCollection AddAsCloudFiles(this IServiceCollection serviceCollection, Action<AzureCloudFilesOptions> configure = null)
@@ -30,10 +35,34 @@ public static IServiceCollection AddAsCloudFiles(this IServiceCollection service
 
         var blobContainerClient = cloudFilesOptions.CreateClient();
 
+        if (!cloudFilesOptions.DisableAutoLifecycle && cloudFilesOptions.CanManage())
+            EnsureLifecycleRules(cloudFilesOptions);
+
         serviceCollection.AddSingleton(blobContainerClient);
         serviceCollection.AddSingleton(cloudFilesOptions);
         serviceCollection.AddSingleton<CloudFilesOptions>(cloudFilesOptions);
-        serviceCollection.AddSingleton<ICloudFilesService, CloudFilesService>();
+        serviceCollection.AddSingleton<CloudFilesService>();
+        serviceCollection.AddSingleton<ICloudFilesService>(sp => sp.GetRequiredService<CloudFilesService>());
+        serviceCollection.AddSingleton<ICloudFilesLifecycle>(sp => sp.GetRequiredService<CloudFilesService>());
         return serviceCollection;
     }
+
+    private static void EnsureLifecycleRules(AzureCloudFilesOptions options)
+    {
+        var policy = options.ManagementPolicy();
+
+        StorageAccountManagementPolicyData data;
+        try
+        {
+            data = policy.Get().Value.Data;
+        }
+        catch (RequestFailedException ex) when (ex.Status == 404 && ex.ErrorCode == "ManagementPolicyNotFound")
+        {
+            data = new StorageAccountManagementPolicyData();
+        }
+
+        // The policy is written whole, so everything already in it goes back with the additions.
+        if (AzureLifecycleRules.AddTempRules(data, options.BucketName))
+            policy.CreateOrUpdate(WaitUntil.Completed, data);
+    }
 }

SW.CloudFiles.AS.UnitTest/AzureLifecycleRulesTests.cs

@@ -0,0 +1,101 @@
+using System.Collections.Generic;
+using System.Linq;
+using Azure.ResourceManager.Storage;
+using Azure.ResourceManager.Storage.Models;
+using Microsoft.VisualStudio.TestTools.UnitTesting;
+using SW.CloudFiles.AS;
+
+namespace SW.CloudFiles.AS.UnitTest;
+
+/// <summary>Pure policy logic, no storage account needed — unlike the other tests in this project.</summary>
+[TestClass]
+public class AzureLifecycleRulesTests
+{
+    private static ManagementPolicyRule DeleteRule(string name, float days, params string[] prefixes)
+    {
+        var filter = new ManagementPolicyFilter(["blockBlob"]);
+        foreach (var prefix in prefixes) filter.PrefixMatch.Add(prefix);
+        var actions = new ManagementPolicyAction
+        {
+            BaseBlob = new ManagementPolicyBaseBlob { Delete = new DateAfterModification { DaysAfterModificationGreaterThan = days } }
+        };
+        return new ManagementPolicyRule(name, ManagementPolicyRuleType.Lifecycle,
+            new ManagementPolicyDefinition(actions) { Filters = filter });
+    }
+
+    [TestMethod]
+    public void Adds_this_containers_temp_rules_and_keeps_the_rest_of_the_policy()
+    {
+        var data = new StorageAccountManagementPolicyData();
+        data.Rules.Add(DeleteRule("someone-elses", 14, "logs/"));
+        data.Rules.Add(DeleteRule("other-temp30", 30, "other/temp30/"));
+
+        Assert.IsTrue(AzureLifecycleRules.AddTempRules(data, "docs"));
+
+        CollectionAssert.AreEquivalent(
+            new[] { "someone-elses", "other-temp30", "docs-temp1", "docs-temp7", "docs-temp30", "docs-temp365" },
+            data.Rules.Select(r => r.Name).ToArray());
+        Assert.AreEqual("docs/temp30/", data.Rules.Single(r => r.Name == "docs-temp30").Definition.Filters.PrefixMatch.Single());
+    }
+
+    [TestMethod]
+    public void Nothing_to_write_when_the_rules_are_already_there()
+    {
+        var data = new StorageAccountManagementPolicyData();
+        AzureLifecycleRules.AddTempRules(data, "docs");
+
+        Assert.IsFalse(AzureLifecycleRules.AddTempRules(data, "docs"));
+    }
+
+    [TestMethod]
+    public void A_disabled_temp_rule_is_replaced_not_duplicated()
+    {
+        var data = new StorageAccountManagementPolicyData();
+        var disabled = DeleteRule("docs-temp30", 30, "docs/temp30/");
+        disabled.IsEnabled = false;
+        data.Rules.Add(disabled);
+
+        AzureLifecycleRules.AddTempRules(data, "docs");
+
+        var temp30 = data.Rules.Where(r => r.Name == "docs-temp30").ToList();
+        Assert.AreEqual(1, temp30.Count);
+        Assert.IsTrue(temp30[0].IsEnabled);
+    }
+
+    [TestMethod]
+    public void Reads_only_this_containers_prefixes_relative_to_the_container()
+    {
+        var rules = AzureLifecycleRules.ToDeletionRules([
+            DeleteRule("docs-temp30", 30, "docs/temp30/"),
+            DeleteRule("other-temp30", 30, "other/temp30/")
+        ], "docs");
+
+        var rule = rules.Single();
+        Assert.AreEqual("temp30/", rule.Prefix);
+        Assert.AreEqual(30, rule.Days);
+        Assert.IsTrue(rule.Enabled);
+    }
+
+    [TestMethod]
+    public void A_rule_without_prefixes_covers_every_container()
+    {
+        Assert.AreEqual(string.Empty, AzureLifecycleRules.ToDeletionRules([DeleteRule("all", 90)], "docs").Single().Prefix);
+    }
+
+    [TestMethod]
+    public void Tag_filtered_rules_and_rules_without_a_delete_are_left_out()
+    {
+        var tagged = DeleteRule("tagged", 7, "docs/temp7/");
+        tagged.Definition.Filters.BlobIndexMatch.Add(new ManagementPolicyTagFilter("kind", "==", "scratch"));
+        var tiering = new ManagementPolicyRule("tiering", ManagementPolicyRuleType.Lifecycle,
+            new ManagementPolicyDefinition(new ManagementPolicyAction
+            {
+                BaseBlob = new ManagementPolicyBaseBlob
+                {
+                    TierToCool = new DateAfterModification { DaysAfterModificationGreaterThan = 30 }
+                }
+            }));
+
+        Assert.AreEqual(0, AzureLifecycleRules.ToDeletionRules([tagged, tiering], "docs").Count);
+    }
+}

SW.CloudFiles.AS/AzureCloudFilesOptions.cs

@@ -27,10 +27,29 @@ public class AzureCloudFilesOptions : CloudFilesOptions
     public string PublicServiceUrl { get; set; }
 
     /// <summary>
-    /// Reserved for future use. Azure Blob Storage lifecycle management policies require the
-    /// Azure Resource Manager (ARM) plane and cannot be configured automatically through
-    /// the data-plane SDK. Configure lifecycle rules via the Azure Portal, Azure CLI
-    /// (<c>az storage account management-policy create</c>), or ARM templates.
+    /// When true, skips creating the temp-prefix delete rules (temp1/, temp7/, temp30/, temp365/) in the
+    /// storage account's lifecycle management policy. Only applies once <see cref="SubscriptionId"/> and
+    /// <see cref="ResourceGroupName"/> are set: the policy lives on the Azure Resource Manager plane, not
+    /// the blob endpoint, so without them nothing can be created and nothing deletes temp files.
     /// </summary>
     public bool DisableAutoLifecycle { get; set; }
+
+    /// <summary>
+    /// Subscription of the storage account, for its lifecycle management policy. Leave unset and the
+    /// policy is neither created nor read. The identity used — the managed identity, or a service principal
+    /// from <c>AZURE_TENANT_ID</c> / <c>AZURE_CLIENT_ID</c> / <c>AZURE_CLIENT_SECRET</c> — needs
+    /// <c>Microsoft.Storage/storageAccounts/managementPolicies/read</c> and <c>/write</c>, for example the
+    /// Storage Account Contributor role; Storage Blob Data Contributor alone isn't enough.
+    /// </summary>
+    public string SubscriptionId { get; set; }
+
+    /// <summary>Resource group of the storage account. Required alongside <see cref="SubscriptionId"/>.</summary>
+    public string ResourceGroupName { get; set; }
+
+    /// <summary>
+    /// The storage account's name. Optional: taken from <see cref="CloudFilesOptions.AccessKeyId"/> with a
+    /// shared key, or from the first part of the service URL's host with a managed identity. Set it when
+    /// the service URL is a custom domain or a private link.
+    /// </summary>
+    public string StorageAccountName { get; set; }
 }