Replace shell-based context with profile-based configuration - #300
Open
alexiswl wants to merge 21 commits into
Open
Replace shell-based context with profile-based configuration#300alexiswl wants to merge 21 commits into
alexiswl wants to merge 21 commits into
Conversation
Create hotfix for projectdata path error
Update import paths across config_helpers, test_config_helpers, bundle_helpers, bundles_deprecate, and bundles_release to use libica.openapi.v3 Configuration and ApiException instead of v2. Also replace wrapica.libica_exceptions.ApiException with the direct libica.openapi.v3.ApiException import.
- Replace ProjectPipeline with ProjectPipelineType in subcommands/__init__.py - Add separate handling for ProjectPipelineType vs PipelineType coercion - Update analysis storage coercion to use keyword arguments - Add 'return value' after analysis coercion to fix early-return bug - Cast pipeline status to PipelineStatusType in bundles_add_pipeline - Use str() for UUID comparisons across bundle, pipeline, and project operations to handle model objects correctly
The clean_multi_args workaround for docopt issue #134 is no longer used after the commented-out call was removed from Command.__init__.
- libica >= 3, < 4 (was >= 2.4.0, < 3) - wrapica >= 2.40.1, < 2.41.0 (was >= 2.27.1) - cwl_utils >= 0.38 (was >= 0.32) - Add pip, pandas-stubs, boto3-stubs, mypy-boto3-s3 dependencies
- Add --include-linked flag to show linked data from other projects - Add --case-insensitive flag for case-insensitive path filtering - Display '@' suffix for linked files in short listing mode - Use parent_folder_id when available for more precise listing - Update sort parameter to use ProjectDataSortParameterType literals - Fix projectdata_helpers to use typed AwsTempCredentials attributes instead of dict.get() calls - Fix s3 sync script trailing slash and arg indentation
- s3-sync-download/upload: use AwsTempCredentials typed attributes, rename CLI args to local_download_path/local_upload_path, initialize s3_sync_args to empty list when not provided - view: fix BROWSER env var check logic (was inverted) - find: fix creator_id comparison to use str() for UUID objects
- cp: copy files/directories within a project using the ICAv2 API, with job completion waiting and optional JSON output - mkdir: create directories in ICAv2 project data
projectpipelines: - launch_wes: support both CWL and Nextflow analysis types, add --cache option, comment out deprecated samplesheet_input logic, use keyword args for analysis_storage coercion - pipelines_update: add icav2.config auto-injection for Nextflow, filter hidden/test files from new file detection, use ProjectPipelineV4 model, fix owner_id comparison - create_cwl_workflow_from_zip: sanitize dots in pipeline code - create_nextflow_pipeline_from_nf_core: fix analysis_storage naming - create_nextflow_pipeline_from_zip: make --workflow-description required - create_wes_input_template: rename samplesheet_input to samplesheet, stringify pipeline id in engine parameters - download_pipeline: make output_dir optional with cwd default projectanalyses: - get_step_logs: update step name default, use typed AnalysisStep, fix log_obj filtering to use .name attribute, use string log names - list_v2: keep time_modified column in non-long listing
- Rename subcommand from 'start-cwl-wes' to 'start-wes' in autocompletion specs (bash, zsh, yaml) and README - Add ICAV2_SERVER_URL export to tenants enter shell function - Update _icav2 shell function command list: add download, rename nfcore variant, add nextflow-pipeline-from-zip - Remove stale FIXME comments in tenants enter
- Add jq version check (>= 1.6) - Add icav2 CLI version check (>= 2.30.0) - Fix shell detection in source.sh to use /proc/65994/exe readlink instead of $SHELL env var (handles cases where login shell differs from running shell) - Use prefixed variable names to avoid polluting user's namespace - Unset temporary variables after use in generated source.sh - Add readlink/greadlink availability check in source.sh
Add product overview, project structure, and tech stack documentation as steering files to guide AI-assisted development.
Add requirements, design, and implementation task documents for the profile-based configuration system that replaces shell-function-based tenant/project context management with an AWS CLI-style profile approach. Key design decisions: - INI-style config file at ~/.icav2-cli-plugins/config - Named profiles with precedence chain (env vars > --profile > ICAV2_PROFILE > default) - Token caching per profile with JWT freshness checks - Lazy imports for fast CLI startup (<500ms for help/version) - Bundled _icav2 binary for standalone operation
- Add hypothesis and pytest to [project.optional-dependencies] test group - Add profile-based configuration constants to globals.py: CONFIG_FILE_PATH, CACHE_DIR, BIN_DIR, LOCAL_BINARY_PATH, PROFILE_NAME_REGEX, VALID_OUTPUT_FORMATS, TOKEN_REFRESH_THRESHOLD - Defer wrapica.enums import in globals.py via get_default_analysis_storage_size() - Implement ConfigParser with AWS CLI-style INI format support: - [default] and [profile <name>] section headers - Comment handling (# and ;), duplicate key resolution (last wins) - ConfigParseError with line numbers for malformed content - Round-trip serialize/parse with file I/O (mode 0600)
Add ProfileResolver that determines active configuration by applying: 1. Explicit env vars (ICAV2_ACCESS_TOKEN, ICAV2_PROJECT_ID, ICAV2_BASE_URL) 2. --profile CLI flag 3. ICAV2_PROFILE env var 4. ICAV2_TENANT_NAME env var (backward compat, treated as profile name) 5. [default] profile in config file Includes ResolvedConfig dataclass with server_url, base_url, access_token, project_id, api_key, and output_format. Provides actionable error messages listing available profiles when a requested profile is not found.
Add TokenManager that handles access token lifecycle per profile: - Cache tokens in ~/.icav2-cli-plugins/cache/<profile>/session.yaml - Check JWT exp claim with 3600s freshness threshold - Generate new tokens from API key via ICAv2 API when stale - Validate ICAV2_ACCESS_TOKEN env var (reject expired/malformed, no fallback) - Graceful handling of malformed cache files (treat as absent) - File permissions set to 0600 for security
- Rewrite cli.py main() as standalone entry point: - Parse --profile and --debug global args - Handle help/version early without heavy imports - Integrate ProfileResolver and TokenManager - Dispatch to plugin subcommands or delegate to _icav2 binary - Use os.execve for binary delegation (replaces subprocess.run) - Add COMMAND_MODULE_MAP with lazy_import_command() dispatcher - Add 'configure' command group to CLI docstring - Defer wrapica/libica/pandas imports in subcommands/__init__.py: - Move heavy imports to point of first use in coerce_magical_value() - Command and DocOptArg classes importable without triggering library loads - Fix type cast in projectanalyses/list_steps.py for project_id
- configure set: Interactive profile configuration with prompts for server_url, x_api_key, and project_name. Validates API key before saving. Creates config file with mode 0600 if absent. - configure list: Displays table of all configured profiles showing name, server_url, and project info. Shows guidance message when no profiles are configured. - Command registration in configure/__init__.py with docopt dispatch
- Add --skip-binary flag for environments with pre-installed icav2 - Add platform/architecture detection (linux/darwin, amd64/arm64) - Download icav2 binary to ~/.icav2-cli-plugins/bin/_icav2 - Create cache/ directory for per-profile token storage - Create empty config file with mode 0600 if not present - Remove 'command icav2' prerequisite check (now self-contained) - Add migration prompt for existing tenants/ directory - Simplify source.sh: set ICAV2_CLI_PLUGINS_HOME and prepend bin/ to PATH - Abort setup if binary download fails
Property-based tests (hypothesis): - Config parser: round-trip, comment invariance, duplicate key resolution, malformed line detection, output format validation - Profile resolver: default values, profile selection, precedence resolution, invalid profile error - Token manager: freshness decision, expired JWT rejection Unit tests: - CLI: lazy imports, startup performance, --profile parsing, help/version - Configure commands: set (with mocked stdin/API), list (format/empty state) - Token manager: caching, refresh, generation, env var validation - Delegation: _icav2 binary execution with env vars - Deferred imports: heavy libraries not loaded at import time - Installer: platform detection, directory structure, migration prompt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the shell-function-based tenant/project context management system with an AWS CLI-style profile configuration approach. The CLI becomes a standalone executable that reads configuration from disk and manages token lifecycle internally — no shell function sourcing required.
Key Changes
New: Profile-based configuration system
~/.icav2-cli-plugins/configwith[default]and[profile <name>]sections--profileflag →ICAV2_PROFILE→ICAV2_TENANT_NAME→[default]~/.icav2-cli-plugins/cache/<profile>/session.yamlNew:
configurecommand groupicav2 configure set [<profile_name>]— interactive profile setup with API key validationicav2 configure list— display configured profilesRefactored: CLI entry point (
utils/cli.py)--profileglobal flag_icav2binary viaos.execvesubcommands/__init__.pyUpdated: Installer (
install.sh)~/.icav2-cli-plugins/bin/_icav2command icav2prerequisite (now self-contained)tenants/configurations--skip-binaryflag for pre-installed environmentsTesting
Backward Compatibility
ICAV2_ACCESS_TOKEN,ICAV2_PROJECT_ID,ICAV2_BASE_URLenv vars continue to work (highest precedence)ICAV2_TENANT_NAMEtreated as profile name for backward compattenants/directory preserved; migration is opt-inCommits