-
Notifications
You must be signed in to change notification settings - Fork 1.4k
feat(seaweedfs): add SeaweedFS object storage provider #14160
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
chrislusf
wants to merge
71
commits into
apache:main
Choose a base branch
from
chrislusf:feat/seaweedfs-object-store-provider
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
71 commits
Select commit
Hold shift + click to select a range
5691884
feat(seaweedfs): add SeaweedFS object storage provider
chrislusf 007e3cc
fix(seaweedfs): default iamUrl to s3Url, not <s3Url>/iam
chrislusf d6eaca0
fix(seaweedfs): use AWSS3V4Signer not legacy S3Signer; remove dead po…
chrislusf 73bae93
fix(seaweedfs): address PR review comments
chrislusf 5109b1c
Merge branch 'main' into feat/seaweedfs-object-store-provider
chrislusf e6d4dad
fix(seaweedfs): namespace IAM credential keys by store ID
chrislusf 11dd3a9
fix(seaweedfs): update existing bucket credentials on IAM key rotation
chrislusf dcbecc5
fix(seaweedfs): prefer current store URL over stale persisted s3Url d…
chrislusf c651665
fix(seaweedfs): omit bucket from usage result on S3 listing failure
chrislusf 7b22718
fix(seaweedfs): add bounded timeouts to S3 extension HTTP requests
chrislusf 51eaef9
fix(seaweedfs): only accept 2xx as success for S3 extension requests
chrislusf e4b4ea8
feat(seaweedfs): add SeaweedFS to the Add Object Storage UI provider …
chrislusf 7eb6938
test(seaweedfs): fix testSetBucketQuotaNoS3ConfigThrows to clear stor…
chrislusf e353f6f
test(seaweedfs): add deterministic SigV4 signature-verification test
chrislusf 3fae9a4
fix(seaweedfs): read configured s3Url detail first, fall back to stor…
chrislusf 250e085
fix(seaweedfs): fall back to current S3 endpoint when iamUrl detail i…
chrislusf 3d27797
fix(seaweedfs): reject negative bucket quota values
chrislusf 4b949b5
fix(seaweedfs): validate IAM key status and require both credentials …
chrislusf f09d74e
fix(seaweedfs): preserve endpoint path prefix when building quota req…
chrislusf face0a8
fix(seaweedfs): clean up remote bucket on post-create DB update failure
chrislusf 9e73cb3
fix(seaweedfs): scope IAM user policy to account bucket ARNs
chrislusf 07e1142
test(seaweedfs): fix SigV4 signature test timestamp determinism
chrislusf 649bee6
fix(seaweedfs): namespace IAM username by store ID
chrislusf 6aab4da
fix(seaweedfs): deny s3:PutBucketQuota in tenant IAM policy
chrislusf 99e7b25
fix(seaweedfs): make deleteBucket IAM policy refresh best-effort
chrislusf eb02755
fix(seaweedfs): only persist explicitly-supplied endpoint overrides i…
chrislusf 1144c18
fix(seaweedfs): include endpoint path prefix in signed resource path
chrislusf 6f6ca3f
fix(seaweedfs): require IAM credentials for bucket creation, fail on …
chrislusf 6449ad9
fix(seaweedfs): validate non-negative quota before reservation in Buc…
chrislusf 684e907
fix(seaweedfs): use DB-backed GlobalLock for IAM provisioning across …
chrislusf 67aa7af
test(seaweedfs): add buildAccountIAMPolicy and GlobalLock stub tests
chrislusf 0872a51
test(seaweedfs): add lifecycle tests for endpoint default/override pe…
chrislusf f6ce879
fix(seaweedfs): release GlobalLock reference on failure and in finall…
chrislusf e58042e
fix(seaweedfs): validate negative quota before remote side effects in…
chrislusf a813234
test(seaweedfs): fix duplicate anyLong import and update deleteBucket…
chrislusf d185416
fix(seaweedfs): harden createBucket post-create section with IAM lock…
chrislusf 8157660
fix(seaweedfs): revoke IAM grant before S3 delete in deleteBucket
chrislusf 53da699
fix(seaweedfs): revert deleteBucket to delete-then-refresh order with…
chrislusf 6e4f270
fix(seaweedfs): clean up remote bucket when IAM lock acquisition fail…
chrislusf 2c28610
fix(seaweedfs): tolerate 404 on quota disable for deployments without…
chrislusf 88c8574
test(seaweedfs): add quota 404 tolerance test and BucketApiServiceImp…
chrislusf bd882af
fix(seaweedfs): return updated BucketVO from createBucket instead of …
chrislusf b424170
fix(seaweedfs): remove BucketVO before IAM policy refresh in deleteBu…
chrislusf e30593c
fix(seaweedfs): do not swallow NoSuchBucket 404 during quota disable
chrislusf 0349369
feat(seaweedfs): use Prometheus metrics for scalable bucket usage rep…
chrislusf ea0674d
test(seaweedfs): add Prometheus metrics usage and fallback tests
chrislusf 104fc02
fix(seaweedfs): correct Prometheus metric name and SigV4 resource path
chrislusf f0950d1
fix(seaweedfs): split IAM policy lock, restructure deleteBucket, prop…
chrislusf e94c2c6
test(seaweedfs): update metrics test for correct Prometheus metric name
chrislusf 17db4fd
feat(ui): add SeaweedFS endpoint override fields to Add Object Storage
chrislusf 7913fad
fix(seaweedfs): only tolerate quota extension 404 on initial create
chrislusf 068987f
fix(seaweedfs): harden metrics parser so bad scrapes fall back to S3 …
chrislusf 1746859
test(seaweedfs): add quota-clear and metrics fallback regression tests
chrislusf b22207f
fix(ui): localize SeaweedFS object storage labels and placeholders
chrislusf e75ecf6
fix(seaweedfs): reject sample-less and malformed metrics scrapes
chrislusf 3d51df4
fix(seaweedfs): remove BucketVO inside the IAM lock after policy refresh
chrislusf 9986d1c
fix: preserve provider-persisted bucket fields in createBucket response
chrislusf 656da92
test(seaweedfs): cover metrics scrape validation and deleteBucket row…
chrislusf d179988
fix(seaweedfs): mark bucket Destroyed instead of removing the row in …
chrislusf e960c41
fix(seaweedfs): preserve the metricsUrl store detail during initializ…
chrislusf 415d163
fix(seaweedfs): verify the configured admin credentials during initia…
chrislusf 3cc05ca
fix: update stored bucket URLs when an object store URL changes
chrislusf aa5caa8
test(seaweedfs): cover Destroyed-state policy exclusion, metricsUrl a…
chrislusf e7f8be4
fix(seaweedfs): write account credentials per key to avoid clobbering…
chrislusf e443379
fix: reserve before mutating the remote quota in updateBucketQuota
chrislusf 6ae6fbf
fix: skip bucket URL rewrite when the object store has an explicit s3Url
chrislusf c9eaf62
fix(seaweedfs): add a store-wide bucket name lock shared by create an…
chrislusf e76aef2
fix: normalize trailing slashes when composing bucket URLs
chrislusf 932cc3c
fix(seaweedfs): make credential and quota updates idempotent
chrislusf 5c13f88
fix(seaweedfs): propagate DAO failures and roll back partial updates
chrislusf c8a267c
ci: skip coverage-grade PR comment on fork pull requests
chrislusf File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| <!-- | ||
| Licensed to the Apache Software Foundation (ASF) under one | ||
| or more contributor license agreements. See the NOTICE file | ||
| distributed with this work for additional information | ||
| regarding copyright ownership. The ASF licenses this file | ||
| to you under the Apache License, Version 2.0 (the | ||
| "License"); you may not use this file except in compliance | ||
| with the License. You may obtain a copy of the License at | ||
|
|
||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
|
|
||
| Unless required by applicable law or agreed to in writing, | ||
| software distributed under the License is distributed on an | ||
| "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| KIND, either express or implied. See the License for the | ||
| specific language governing permissions and limitations | ||
| under the License. | ||
| --> | ||
| <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | ||
| <modelVersion>4.0.0</modelVersion> | ||
| <artifactId>cloud-plugin-storage-object-seaweedfs</artifactId> | ||
| <name>Apache CloudStack Plugin - SeaweedFS object storage provider</name> | ||
| <parent> | ||
| <groupId>org.apache.cloudstack</groupId> | ||
| <artifactId>cloudstack-plugins</artifactId> | ||
| <version>24.0.0-SNAPSHOT</version> | ||
| <relativePath>../../../pom.xml</relativePath> | ||
| </parent> | ||
| <dependencies> | ||
| <dependency> | ||
| <groupId>org.apache.cloudstack</groupId> | ||
| <artifactId>cloud-engine-storage</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.cloudstack</groupId> | ||
| <artifactId>cloud-engine-storage-object</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.cloudstack</groupId> | ||
| <artifactId>cloud-engine-schema</artifactId> | ||
| <version>${project.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.amazonaws</groupId> | ||
| <artifactId>aws-java-sdk-core</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.amazonaws</groupId> | ||
| <artifactId>aws-java-sdk-iam</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.amazonaws</groupId> | ||
| <artifactId>aws-java-sdk-s3</artifactId> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.fasterxml.jackson.core</groupId> | ||
| <artifactId>jackson-databind</artifactId> | ||
| <version>${cs.jackson.version}</version> | ||
| </dependency> | ||
| <dependency> | ||
| <groupId>com.github.tomakehurst</groupId> | ||
| <artifactId>wiremock-standalone</artifactId> | ||
| <version>${cs.wiremock.version}</version> | ||
| <scope>test</scope> | ||
| </dependency> | ||
| </dependencies> | ||
| </project> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 73bae93 — client/pom.xml now includes the cloud-plugin-storage-object-seaweedfs dependency alongside the other object-storage providers, so the module is packaged into the management-server artifact.