Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 1 addition & 13 deletions app/controllers/internal/download_droplets_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
module VCAP::CloudController
class DownloadDropletsController < RestController::BaseController
def self.dependencies
%i[droplet_blobstore blobstore_url_generator missing_blob_handler blob_sender droplet_url_generator]
%i[droplet_blobstore blobstore_url_generator missing_blob_handler blob_sender]
end

include CloudController::Errors
Expand All @@ -13,17 +13,6 @@ def self.dependencies

attr_reader :blobstore

get '/internal/v2/droplets/:guid/:droplet_checksum/download', :download_droplet_http

def download_droplet_http(guid, droplet_checksum)
if @droplet_url_generator.mtls
url = @droplet_url_generator.perma_droplet_download_url(guid, droplet_checksum)
redirect url
else
download_droplet(guid, droplet_checksum)
end
end

get '/internal/v4/droplets/:guid/:droplet_checksum/download', :download_droplet_mtls

def download_droplet_mtls(guid, droplet_checksum)
Expand All @@ -38,7 +27,6 @@ def inject_dependencies(dependencies)
@blobstore_url_generator = dependencies.fetch(:blobstore_url_generator)
@missing_blob_handler = dependencies.fetch(:missing_blob_handler)
@blob_sender = dependencies.fetch(:blob_sender)
@droplet_url_generator = dependencies.fetch(:droplet_url_generator)
end

def check_app_exists(app, guid)
Expand Down
4 changes: 2 additions & 2 deletions app/controllers/runtime/stagings_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def self.dependencies
get '/internal/v4/staging_jobs/:guid', :find_job_mtls
def find_job_mtls(guid)
job = Delayed::Job[guid:]
StagingJobPresenter.new(job, 'https').to_json
StagingJobPresenter.new(job).to_json
end

get '/staging/packages/:guid', :download_package
Expand Down Expand Up @@ -85,7 +85,7 @@ def download_v3_droplet(guid)
post '/internal/v4/droplets/:guid/upload', :upload_package_droplet_mtls
def upload_package_droplet_mtls(guid)
job = upload_droplet(guid)
[HTTP::OK, StagingJobPresenter.new(job, 'https').to_json]
[HTTP::OK, StagingJobPresenter.new(job).to_json]
end

post '/internal/v4/buildpack_cache/:stack_name/:guid/upload', :upload_v3_app_buildpack_cache
Expand Down
25 changes: 5 additions & 20 deletions app/presenters/api/staging_job_presenter.rb
Original file line number Diff line number Diff line change
@@ -1,28 +1,13 @@
require_relative 'job_presenter'

class StagingJobPresenter < JobPresenter
def initialize(job, scheme)
@scheme = scheme
super(job)
end

def status_url
config = VCAP::CloudController::Config.config

if @scheme == 'https'
URI::HTTPS.build(
host: config.get(:internal_service_hostname),
port: config.get(:tls_port),
path: "/internal/v4/staging_jobs/#{@object.guid}"
).to_s
else
uri = URI::HTTP.build(
host: config.get(:internal_service_hostname),
port: config.get(:external_port),
path: "/staging/jobs/#{@object.guid}"
)
uri.userinfo = [config.get(:staging, :auth, :user), CGI.escape(config.get(:staging, :auth, :password))]
uri.to_s
end
URI::HTTPS.build(
host: config.get(:internal_service_hostname),
port: config.get(:tls_port),
path: "/internal/v4/staging_jobs/#{@object.guid}"
).to_s
end
end
26 changes: 6 additions & 20 deletions docs/internal/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,15 @@ In capi-release, nginx (`jobs/cloud_controller_ng/templates/nginx.conf.erb`) fro
|----------|--------------------|------|--------|
| mTLS internal | `cc.tls_port` (9023) | mTLS (`ssl_verify_client on`) | `/internal/v4/`, `/internal/v5/`, staging completion + upload regexes |
| Public TLS | `cc.public_tls.port` (9024) | in-app (OAuth) | All public API routes; `/internal/v...` is 403'd here |
| Public non-TLS | `cc.external_port` (9022) | in-app (OAuth) | Same route set as the public TLS listener |
| Metrics mTLS | `cc.prom_metrics_server_tls_port` (9025) | mTLS (separate cert) | `/internal/v4/metrics` only |
| Status | `9021` (hardcoded) | none (loopback) | `/internal/v4/status` health check only |

The two public listeners (9024 and 9022) serve identical routes, differing only in TLS. Gorouter registers both for the `api` route but **prefers `tls_port` (9024)** ([route_registrar spec](https://github.com/cloudfoundry/routing-release/blob/develop/jobs/route_registrar/spec)), so external traffic lands on 9024; 9022 exists for on-VM plain-HTTP callers (e.g. BBR probes) and can be disabled via `temporary_disable_non_tls_endpoints`.
The public TLS listener (9024) serves all public API routes. Gorouter registers it for the `api` route.

Auth mechanisms below:

- **mTLS** — Served on the mTLS internal listener (or the metrics listener for `/internal/v4/metrics`). An `/internal/*` path does not by itself imply mTLS — `ssh_access` is OAuth on the public listener, and `log_access` is mTLS **plus** an in-app OAuth check.
- **OAuth** — CC validates the UAA token's scope in-app; served on the public listeners.
- **OAuth** — CC validates the UAA token's scope in-app; served on the public TLS listener.

## Endpoints

Expand All @@ -41,9 +40,9 @@ Auth mechanisms below:

**Intended Consumer:** SSH Proxy

**Auth Mechanism:** mTLS and OAuth. The endpoint is served on the mutual-TLS listener (`cc.tls_port`, default `9023`), where the SSH Proxy authenticates with a client certificate. For backwards compatibility it also remains available over OAuth on the public listeners (`cc.external_port`, default `9022`, and `cc.public_tls.port`, default `9024`).
**Auth Mechanism:** mTLS and OAuth. The SSH Proxy authenticates with a client certificate on the mTLS listener (`cc.tls_port`, default `9023`). The path `/internal/apps/...` does not match the public listener's `/internal/v` forbid rule, so it is also reachable on the public TLS listener (`cc.public_tls.port`, default `9024`) via OAuth.

**Routing:** Public listeners (`cc.public_tls.port` 9024; also `cc.external_port` 9022). The path `/internal/apps/...` does not match the public listener's `/internal/v` forbid rule, so it is proxied to CC and authenticated in-app.
**Routing:** mTLS internal listener (`cc.tls_port`, 9023) and public TLS listener (`cc.public_tls.port`, 9024).

### GET /internal/v4/droplets/:guid/:droplet_checksum/download
**Description:** Permalink for downloading a droplet.
Expand Down Expand Up @@ -153,26 +152,13 @@ Auth mechanisms below:

**Routing:** mTLS internal listener (`cc.tls_port`, 9023)

## Deprecated endpoints

These endpoints are deprecated and have no known live consumer. They are removal candidates.

### GET /internal/v2/droplets/:guid/:droplet_checksum/download
**Description:** (Deprecated) Non-mTLS (HTTP) permalink for downloading a droplet. Use `GET /internal/v4/droplets/:guid/:droplet_checksum/download` instead. **No live consumer, and effectively unreachable through nginx** — CC only generates this URL when `tls_port` is blank (never in a standard deploy, so it always emits the v4/mTLS URL instead). Even if generated, the path is `/internal/v2/droplets/...` on `external_port` (plain HTTP), which every nginx listener rejects: the public listeners 403 any `/internal/v` path, and the mTLS listener has no matching `location` (404).

**Intended Consumer:** Diego running task or process (historical)

**Auth Mechanism:** None enforced at the nginx layer — this is the plain-HTTP (non-mTLS) branch of the droplet URL generator (built with `external_port`). Contrast the v4 permalink, which is the HTTPS/mTLS path.

**Routing:** No nginx listener serves this path (public listeners 403 `/internal/v`; mTLS listener has no matching location). Reachable only by bypassing nginx and hitting the CC app socket directly.

## Endpoints for local dev

These endpoints are only used when CC is configured with `blobstore_type: local` or `local-temp-storage`, i.e. local development setups. They are **not routed in BOSH deployments**: capi-release does not emit a `location /staging/` nginx routing block, so `/staging/*` paths return 404 from the nginx catch-all. In any standard BOSH deployment, `storage-cli` is used and Diego fetches blobs via signed URLs directly from the blobstore, never calling these endpoints.

### Auth: Staging basic auth

Enforced in-app by a Sinatra `before '/staging/*'` filter on the public listeners. The credentials are configured via `staging.auth.user` / `staging.auth.password` in the CC config (populated from `cc.staging_upload_user` / `cc.staging_upload_password` BOSH properties in local dev). In BOSH deployments, capi-release no longer renders these properties and the endpoints are not nginx-routed.
Enforced in-app by a Sinatra `before '/staging/*'` filter on the public TLS listener. The credentials are configured via `staging.auth.user` / `staging.auth.password` in the CC config (populated from `cc.staging_upload_user` / `cc.staging_upload_password` BOSH properties in local dev). In BOSH deployments, capi-release no longer renders these properties and the endpoints are not nginx-routed.

### GET /v2/buildpacks/:guid/download
**Description:** Download a buildpack file. This is a `/v2` public-namespace path, its `download` action is exempt from user OAuth and gated by staging basic auth instead.
Expand All @@ -181,7 +167,7 @@ Enforced in-app by a Sinatra `before '/staging/*'` filter on the public listener

**Auth Mechanism:** Staging basic auth

**Routing:** Public listeners (`cc.public_tls.port` 9024; also `cc.external_port` 9022) via `location /v2/` (when `cc.temporary_enable_v2` is true). Not called in BOSH deployments.
**Routing:** Public TLS listener (`cc.public_tls.port`, 9024) via `location /v2/` (when `cc.temporary_enable_v2` is true). Not called in BOSH deployments.

### GET /staging/packages/:guid
**Description:** Download a package from the local blobstore.
Expand Down
4 changes: 1 addition & 3 deletions lib/cloud_controller/blobstore/url_generator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,7 @@ def droplet_download_url(droplet)
def_delegators :@upload_url_generator,
:droplet_upload_url,
:package_droplet_upload_url,
:buildpack_cache_upload_url,
:http_droplet_upload_url,
:http_buildpack_cache_upload_url
:buildpack_cache_upload_url
end
end
end
Original file line number Diff line number Diff line change
Expand Up @@ -17,19 +17,19 @@ def initialize(blobstore_options, package_blobstore, buildpack_cache_blobstore,
def package_download_url(package)
return nil unless @package_blobstore.exists?(package.guid)

http_basic_auth_uri("/staging/packages/#{package.guid}")
https_basic_auth_uri("/staging/packages/#{package.guid}")
end

def buildpack_cache_download_url(app_guid, stack)
return nil unless @buildpack_cache_blobstore.exists?("#{app_guid}/#{stack}")

http_basic_auth_uri("/staging/v3/buildpack_cache/#{stack}/#{app_guid}/download")
https_basic_auth_uri("/staging/v3/buildpack_cache/#{stack}/#{app_guid}/download")
end

def admin_buildpack_download_url(buildpack)
return nil unless @admin_buildpack_blobstore.exists?(buildpack.key)

http_basic_auth_uri("/v2/buildpacks/#{buildpack.guid}/download")
https_basic_auth_uri("/v2/buildpacks/#{buildpack.guid}/download")
end

def droplet_download_url(droplet)
Expand All @@ -43,7 +43,7 @@ def droplet_download_url(droplet)
return nil
end

http_basic_auth_uri("/staging/v3/droplets/#{droplet.guid}/download")
https_basic_auth_uri("/staging/v3/droplets/#{droplet.guid}/download")
end

private
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@ def droplet_upload_url(droplet_guid)
def buildpack_cache_upload_url(app_guid, stack)
https_no_auth_uri("/internal/v4/buildpack_cache/#{stack}/#{app_guid}/upload")
end

def http_droplet_upload_url(droplet_guid)
http_basic_auth_uri("/staging/v3/droplets/#{droplet_guid}/upload")
end
end
end
end
Original file line number Diff line number Diff line change
@@ -1,30 +1,18 @@
module CloudController
module Blobstore
module UrlGeneratorHelpers
def http_basic_auth_uri(path)
uri = build_http_uri(path)
def https_basic_auth_uri(path)
uri = build_https_uri(path)
uri.userinfo = [@blobstore_options[:user], @blobstore_options[:password]]
uri.to_s
end

def http_no_auth_uri(path)
build_http_uri(path).to_s
end

def https_no_auth_uri(path)
build_https_uri(path).to_s
end

private

def build_http_uri(path)
URI::HTTP.build(
host: @blobstore_options[:blobstore_host],
port: @blobstore_options[:blobstore_external_port],
path: path
)
end

def build_https_uri(path)
URI::HTTPS.build(
host: @blobstore_options[:blobstore_host],
Expand Down
2 changes: 1 addition & 1 deletion lib/cloud_controller/config_schemas/clock_schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ class ClockSchema < VCAP::Config
# rubocop:disable Metrics/BlockLength
define_schema do
{
external_port: Integer,
optional(:external_port) => Integer,
external_domain: String,
tls_port: Integer,
external_protocol: String,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ class DeploymentUpdaterSchema < VCAP::Config
index: Integer, # Component index (cc-0, cc-1, etc)
name: String, # Component name (api_z1, api_z2)

external_port: Integer,
optional(:external_port) => Integer,
tls_port: Integer,
internal_service_hostname: String,
external_domain: String,
Expand Down
2 changes: 1 addition & 1 deletion lib/cloud_controller/config_schemas/worker_schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ class WorkerSchema < VCAP::Config
# rubocop:disable Metrics/BlockLength
define_schema do
{
external_port: Integer,
optional(:external_port) => Integer,
external_domain: String,
system_hostnames: [String],
system_domain: String,
Expand Down
8 changes: 2 additions & 6 deletions lib/cloud_controller/dependency_locator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -207,11 +207,9 @@ def buildpack_blobstore
def blobstore_url_generator
connection_options = {
blobstore_host: config.get(:internal_service_hostname),
blobstore_external_port: config.get(:external_port),
blobstore_tls_port: config.get(:tls_port),
user: config.get(:staging, :auth, :user),
password: config.get(:staging, :auth, :password),
mtls: !!config.get(:tls_port)
password: config.get(:staging, :auth, :password)
}

Blobstore::UrlGenerator.new(
Expand All @@ -226,9 +224,7 @@ def blobstore_url_generator
def droplet_url_generator
VCAP::CloudController::Diego::DropletUrlGenerator.new(
internal_service_hostname: config.get(:internal_service_hostname),
external_port: config.get(:external_port),
tls_port: config.get(:tls_port),
mtls: !!config.get(:tls_port)
tls_port: config.get(:tls_port)
)
end

Expand Down
26 changes: 3 additions & 23 deletions lib/cloud_controller/diego/droplet_url_generator.rb
Original file line number Diff line number Diff line change
@@ -1,40 +1,20 @@
module VCAP::CloudController
module Diego
class DropletUrlGenerator
def initialize(internal_service_hostname:, external_port:, tls_port:, mtls:)
def initialize(internal_service_hostname:, tls_port:)
@internal_service_hostname = internal_service_hostname
@external_port = external_port
@tls_port = tls_port
@mtls = mtls
end

attr_reader :internal_service_hostname, :external_port, :tls_port, :mtls
attr_reader :internal_service_hostname, :tls_port

def perma_droplet_download_url(app_guid, droplet_checksum)
return nil unless droplet_checksum

if mtls
build_https_url(app_guid, droplet_checksum)
else
build_http_url(app_guid, droplet_checksum)
end
end

private

def build_https_url(guid, droplet_checksum)
URI::HTTPS.build(
host: internal_service_hostname,
port: tls_port,
path: "/internal/v4/droplets/#{guid}/#{droplet_checksum}/download"
).to_s
end

def build_http_url(guid, droplet_checksum)
URI::HTTP.build(
host: internal_service_hostname,
port: external_port,
path: "/internal/v2/droplets/#{guid}/#{droplet_checksum}/download"
path: "/internal/v4/droplets/#{app_guid}/#{droplet_checksum}/download"
).to_s
end
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,16 +47,3 @@
end
end
end

RSpec.describe 'DropletsController download endpoint with checksum exists:' do
describe 'GET /internal/v2/droplets/:guid/:droplet_hash/download' do
let(:droplet) { create(:droplet_model) }

context 'when an lrp tries to download a droplet' do
it 'does not return a 404 because the endpoint is still present, and redirects to the droplet-url' do
get "/internal/v2/droplets/#{droplet.guid}/#{droplet.sha256_checksum}/download", nil, {}
expect(last_response.status).to eq(302)
end
end
end
end
Loading
Loading