Add LinkedDataHub docker-compose deployment

.env

@@ -0,0 +1,19 @@
+COMPOSE_PROJECT_NAME=ltlod
+
+PROTOCOL=https
+HTTP_PORT=81
+HTTPS_PORT=4443
+HTTPS_CLIENT_CERT_PORT=5443
+HOST=localhost
+ABS_PATH=/
+
+OWNER_MBOX=martynas@atomgraph.com
+OWNER_GIVEN_NAME=Martynas
+OWNER_FAMILY_NAME=Jusevičius
+OWNER_ORG_UNIT=LTLOD
+OWNER_ORGANIZATION=AtomGraph
+OWNER_LOCALITY=Copenhagen
+OWNER_STATE_OR_PROVINCE=Denmark
+OWNER_COUNTRY_NAME=DK
+
+MAX_CONTENT_LENGTH=4194304

.gitignore

@@ -6,3 +6,11 @@ etl/tools/.venv/
 datasets/current/admin-units/settlements.trig
 datasets/current/admin-units/streets.trig
 __pycache__/
+
+# LinkedDataHub runtime state (bootstrapped by `make up`)
+/ssl/
+/secrets/
+/fuseki/
+/uploads/
+/datasets/owner/
+/datasets/secretary/

CLAUDE.md

@@ -9,12 +9,22 @@ RDF datasets (`datasets/current/`) from live open-data APIs on every run.
 cd etl && make                 # everything: taxonomies → admin-units → seimas → legal-entities
 make -C etl/<domain> all       # one domain (fetch is always fresh — FORCE prerequisite)
 make -C etl/seimas photos      # opt-in: scrape official portraits from lrs.lt
-make BASE=https://host/        # override base URI (default https://linkeddata.lt/, see etl/config.mk)
+make BASE=https://linkeddata.lt/   # prod base URI (default https://localhost:4443/, see etl/config.mk);
+                               # committed datasets/current/ are generated with the prod base
 
 etl/queries/run.sh <q.rq>      # SPARQL over ALL datasets loaded in-memory (~1M quads, -Xmx4g)
 python3 etl/queries/render-examples.py   # regenerate etl/queries/EXAMPLES.md result tables
 
 uv run --project etl/tools ltlod-reconcile <admin-units|persons> --input … --output …
+
+make up                        # deploy LinkedDataHub at https://localhost:4443/ (root Makefile;
+                               # bootstraps secrets + server cert, then docker compose up -d)
+make install                   # one-time: PUT app/ scaffolding (root + containers + taxonomy
+                               # schemes) via LDH CLI; needs ../LinkedDataHub checkout (LDH_HOME=…)
+make load                      # bulk-load datasets/current/*/*.trig into fuseki-end-user TDB2;
+                               # regenerate with `make -C etl` first (committed data has prod base);
+                               # ends with `make public` (anonymous read, LDH make-public.sh equivalent)
+make down / make drop          # stop stack / wipe LDH runtime state (never datasets/current/)
 ```
 
 Prerequisites: Docker (only for `atomgraph/csv2rdf`; no docker-compose), Apache Jena
@@ -50,6 +60,12 @@ merge on load). Unmatched entities go to `cache/unmatched*.csv`, never force-mat
   keys** from the source registry (AR codes, JAR codes, Seimas `asmens_id`) — any
   pipeline mints cross-links from bare foreign keys. Single source of truth:
   `etl/URI-SCHEME.md` — update it when adding containers.
+- **LDH document hierarchy**: ETL outputs are *only* `dh:Item` docs with
+  `sioc:has_container <its-container>` — add both triples in any new mapping.
+  Containers and taxonomy scheme docs come from `app/` (one Turtle file per
+  container, PUT via LDH CLI by `make install`); each carries an
+  `rdf:_1 <#select-children>` → `ldh:Object`/`ldh:ChildrenView` block, without
+  which LDH renders no children listing at all.
 - **Vocabulary cascade**: W3C specs first → domain-specific third-party vocabs
   (EU SEMIC, OP authority tables, FOAF) → schema.org as general fallback → custom
   (`http://linkeddata.lt/ns#`) last. Rationale per domain: `etl/ONTOLOGY-NOTES.md`.
@@ -87,6 +103,33 @@ merge on load). Unmatched entities go to `cache/unmatched*.csv`, never force-mat
 - Seimas API params: `kadencijos_id` (not `p_kade_id`); position rows may reference
   units absent from current feeds (dissolved commissions, the Board) — org-units
   mapping derives those from the members feed.
+- **Switching `BASE` auto-invalidates static-CSV taxonomies** via the
+  `cache/.base` stamp in `etl/taxonomies/Makefile` (normalize outputs embed
+  the base URI; fetched domains are immune — fetch is FORCE'd). Note `make
+  clean` alone does NOT force the rebuild: missing `cache/*.nt` are
+  intermediate files, which make skips when the `.trig` looks up to date.
+- **`make load` bypasses LDH's HTTP API**: it runs `tdb2.tdbloader` directly against
+  the end-user TDB2 store via the one-off `tdb-loader` compose service (the
+  `atomgraph/fuseki` image bundles the full Jena CLI inside the fuseki-server jar).
+  Load is append-only — clean rebuild: `make down && rm -rf fuseki/end-user &&
+  make up && make load`. It stops fuseki-end-user first and removes the stale
+  `tdb.lock` (lock PIDs are container-relative), then restarts the Varnish caches.
+- **Fuseki ports are never published to the host** — query via
+  `https://localhost:4443/sparql` or from inside the network:
+  `docker compose exec linkeddatahub curl http://varnish-end-user/ds/`.
+- **LDH strips client-sent `sioc:has_parent`/`sioc:has_container` on PUT** and
+  manages the hierarchy itself (re-adds `sioc:has_parent` for `dh:Container`,
+  adds `dh:Item` + `sioc:has_container` otherwise, plus `dct:created`/
+  `acl:owner`) — never put sioc triples in `app/*.ttl`. `make install` is
+  idempotent: PUT replaces the whole named graph.
+- **Public read access is class-based**: `make public` grants
+  `acl:accessToClass def:Root, dh:Container, dh:Item, nfo:FileDataObject` —
+  ETL documents match because mappings type them `dh:Item`/`dh:Container`.
+  (Untyped docs would also pass: LDH's ACL query leaves `$Type` unbound when
+  a document has no `rdf:type`, matching any `acl:accessToClass` — see
+  `AuthorizationFilter` + `aclQuery` in LDH web.xml.)
+- `COMPOSE_PROJECT_NAME=ltlod` isolates container/volume names from other local
+  LDH stacks, but ports 81/4443/5443 still clash — one stack at a time.
 
 ## Verification
 

Makefile

@@ -0,0 +1,88 @@
+SHELL := /bin/bash
+
+.PHONY: up down stop logs cert secrets install load public drop
+
+# LDH CLI checkout (provides put.sh etc.); Jena provides the `turtle` command
+LDH_HOME ?= ../LinkedDataHub
+include etl/config.mk          # for JENA_HOME (its BASE is unused here)
+include .env                   # PROTOCOL/HOST/HTTPS_PORT/HTTPS_CLIENT_CERT_PORT/ABS_PATH
+
+ifeq ($(HTTPS_PORT),443)
+BASE_URI  := $(PROTOCOL)://$(HOST)$(ABS_PATH)
+else
+BASE_URI  := $(PROTOCOL)://$(HOST):$(HTTPS_PORT)$(ABS_PATH)
+endif
+PROXY_URI := $(PROTOCOL)://$(HOST):$(HTTPS_CLIENT_CERT_PORT)$(ABS_PATH)
+
+SECRET_FILES := secrets/owner_cert_password.txt \
+                secrets/secretary_cert_password.txt \
+                secrets/client_truststore_password.txt
+
+secrets/%.txt:
+	@mkdir -p secrets
+	openssl rand -base64 24 > $@
+
+secrets: $(SECRET_FILES)
+
+ssl/server/server.crt:
+	./bin/server-cert-gen.sh .env nginx ssl
+
+cert: ssl/server/server.crt
+
+up: secrets cert
+	mkdir -p datasets/owner datasets/secretary uploads fuseki/admin fuseki/end-user
+	docker compose up -d
+	@echo "LinkedDataHub starting — first boot takes ~1-2 min (self-signed cert)."
+	@echo "URL: https://localhost:4443/"
+
+down:
+	docker compose down
+
+stop:
+	docker compose stop
+
+logs:
+	docker compose logs -f linkeddatahub
+
+# Create/update the container scaffolding (root + containers + taxonomy scheme
+# docs from app/) via LDH CLI PUTs. Run once after `make up`; re-running is
+# safe (PUT replaces). Order: make up -> make install -> make load.
+install:
+	@[ -d "$(LDH_HOME)/bin" ] || \
+		{ echo "ERROR: LDH CLI not found — clone https://github.com/AtomGraph/LinkedDataHub to $(LDH_HOME) or pass LDH_HOME=…"; exit 1; }
+	@[ -n "$$(docker compose ps -q linkeddatahub)" ] || \
+		{ echo "ERROR: linkeddatahub container not found — run 'make up' first."; exit 1; }
+	@echo "Waiting for LinkedDataHub health (first-boot seeding must finish)..."
+	@until [ "$$(docker inspect -f '{{.State.Health.Status}}' $$(docker compose ps -q linkeddatahub))" = "healthy" ]; do \
+		sleep 5; echo "  ...waiting"; \
+	done
+	PATH="$$(find "$$(cd $(LDH_HOME) && pwd)/bin" -type d | tr '\n' ':')$(JENA_HOME)/bin:$$PATH" \
+		./app/install.sh "$(BASE_URI)" ssl/owner/cert.pem \
+		"$$(cat secrets/owner_cert_password.txt)" "$(PROXY_URI)"
+
+# Bulk-load datasets/current/*/*.trig into the end-user TDB2 store. APPEND-ONLY:
+# clean rebuild = `make down && rm -rf fuseki/end-user && make up && make load`.
+load:
+	@ls datasets/current/*/*.trig >/dev/null 2>&1 || \
+		{ echo "ERROR: no TriG files under datasets/current/ — run 'make -C etl' first."; exit 1; }
+	@[ -n "$$(docker compose ps -q fuseki-end-user)" ] || \
+		{ echo "ERROR: fuseki-end-user container not found — run 'make up' first."; exit 1; }
+	@echo "Waiting for LinkedDataHub health (first-boot seeding must finish)..."
+	@until [ "$$(docker inspect -f '{{.State.Health.Status}}' $$(docker compose ps -q linkeddatahub))" = "healthy" ]; do \
+		sleep 5; echo "  ...waiting"; \
+	done
+	docker compose stop fuseki-end-user
+	rm -f fuseki/end-user/DB2/tdb.lock
+	docker compose run --rm tdb-loader
+	docker compose up -d fuseki-end-user
+	docker compose restart varnish-end-user varnish-frontend
+	$(MAKE) public
+
+# Grant anonymous read access (idempotent; equivalent of LDH CLI make-public.sh)
+public:
+	./bin/make-public.sh .env
+
+# Wipes LDH runtime state. NEVER touches datasets/current/.
+drop:
+	@read -p "Delete fuseki/, ssl/, secrets/, uploads/, datasets/{owner,secretary}? [y/N] " ans && \
+	[ "$$ans" = "y" ] && { docker compose down -v; sudo rm -rf datasets/owner datasets/secretary fuseki ssl secrets uploads; } || echo "Aborted."

README.md

@@ -123,7 +123,7 @@ cd etl
 make                                  # viskas: taxonomies → admin-units → seimas → legal-entities
 make -C seimas all                    # tik vienas domenas
 make -C seimas photos                 # papildomai: oficialūs portretai iš lrs.lt
-make BASE=https://mano-serveris.lt/   # kita bazinė URI
+make BASE=https://linkeddata.lt/      # produkcinė bazinė URI (numatytoji — https://localhost:4443/)
 ```
 
 Kiekvienas rinkinys pereina tas pačias keturias stadijas, tad naujo šaltinio pridėjimas —
@@ -140,6 +140,70 @@ graph LR
 
 Rezultatas visada atspindi šaltinių būseną paleidimo metu — jokių rankinių žingsnių.
 
+## Publikavimas su LinkedDataHub
+
+Rinkiniai publikuojami kaip naršomi Linked Data dokumentai per
+[LinkedDataHub](https://github.com/AtomGraph/LinkedDataHub) (toliau — LDH). Reikia tik
+Docker Desktop (Compose ≥ 2.23); visa infrastruktūra aprašyta `docker-compose.yml`
+(tas pats išdėstymas kaip `linkeddatahub.com` / `Homepage` projektuose — nginx, LDH,
+du Fuseki, Varnish kešai).
+
+```shell
+make up      # sugeneruoja slaptažodžius + serverio sertifikatą ir paleidžia LDH
+make install # vienkartinis: sukuria konteinerių dokumentus per LDH CLI (reikia ../LinkedDataHub)
+make -C etl  # perkuria rinkinius su numatytąja baze https://localhost:4443/
+make load    # užkrauna datasets/current/*/*.trig tiesiai į triplestore
+```
+
+Po `make up` LDH pasiekiamas adresu **<https://localhost:4443/>** (savo pasirašytas
+sertifikatas — naršyklė įspės; pirmas paleidimas trunka ~1–2 min.). Administravimo
+aplinka — <https://admin.localhost:4443/>.
+
+**Svarbu dėl bazinės URI:** repozitorijoje užfiksuoti `datasets/current/` failai
+sugeneruoti su produkcine baze `https://linkeddata.lt/`, tad prieš `make load` rinkinius
+reikia perkurti su numatytąja lokalia baze (`make -C etl`) — kitaip dokumentų URI
+nesutaps su LDH adresu ir jie nebus pasiekiami.
+
+Duomenų struktūra kuriama dviem lygiais:
+
+- **Karkasas** (`make install`): šakninis dokumentas, konteineriai ir taksonomijų
+  schemos iš `app/` katalogo dokumentas po dokumento **per LDH CLI** (`put.sh`,
+  kaip [LinkedDataHub-Apps](https://github.com/AtomGraph/LinkedDataHub-Apps)
+  projektuose) — taip dokumentai gauna `ldh:ChildrenView` bloką, dėl kurio
+  konteinerių puslapiai rodo vaikų sąrašus. Reikia šalia išklonintos
+  [LinkedDataHub](https://github.com/AtomGraph/LinkedDataHub) repozitorijos
+  (`../LinkedDataHub`, keičiama per `make install LDH_HOME=…`).
+- **Duomenys** (`make load`): ETL rinkiniai — vien `dh:Item` dokumentai su
+  `sioc:has_container` nuorodomis į karkasą — rašomi **tiesiogiai į
+  `fuseki-end-user` TDB2 saugyklą** (`tdb2.tdbloader` per vienkartinį
+  `tdb-loader` konteinerį), ne po vieną dokumentą per HTTP: ~1 mln. ketvertų
+  užsikrauna per kelias minutes. Pabaigoje suteikiama vieša skaitymo prieiga
+  (`make public` — LDH CLI `make-public.sh` atitikmuo, vykdomas tiesiogiai per
+  `fuseki-admin` konteinerių tinkle).
+Triplestore prievadai **neatveriami į host'ą** — SPARQL užklausos teikiamos per LDH:
+<https://localhost:4443/sparql>. Krovimas yra *append-only*: pakartotinis `make load`
+tik papildo saugyklą; švariam perkrovimui:
+
+```shell
+make down && rm -rf fuseki/end-user && make up && make load
+```
+
+Dokumento patikrinimas (Birštono savivaldybė):
+
+```shell
+curl -k -H "Accept: text/turtle" https://localhost:4443/admin-units/12/
+```
+
+Pastabos:
+
+- `make drop` ištrina tik LDH vykdymo būseną (`fuseki/`, `ssl/`, `secrets/`, `uploads/`,
+  `datasets/owner`, `datasets/secretary`) — `datasets/current/` niekada neliečiamas.
+- Prievadai 81/4443/5443 sutampa su kitų lokalių LDH diegimų (pvz., `LinkedDataHub`
+  repozitorijos) prievadais — vienu metu gali veikti tik vienas stack'as.
+- Dokumentai naršomi per LDH konteinerius: kiekvienas ETL dokumentas yra `dh:Item`
+  su `sioc:has_container`, o konteineriai — LDH CLI sukurti `dh:Container`
+  dokumentai iš `app/` (pvz., <https://localhost:4443/admin-units/>).
+
 ## Ką jau galima atsakyti?
 
 Visi rinkiniai užkraunami į atmintį viena komanda ir užklausiami SPARQL'u:
@@ -204,10 +268,10 @@ Daugiau klausimų, į kuriuos duomenys jau atsako (visi su užklausomis ir pilna
 
 ## Ateities darbai
 
-**Publikavimas.** Rinkiniai paruošti [LinkedDataHub](https://github.com/AtomGraph/LinkedDataHub)
-konvencijai (objektas = named graph = dokumentas, `$base` parametrizuotos URI), tad kitas
-žingsnis — LinkedDataHub diegimas ir duomenų publikavimas kaip naršomi Linked Data dokumentai.
-*Tai bus atskira šaka.*
+**Publikavimas.** Lokalus LinkedDataHub diegimas jau yra — žr.
+[Publikavimas su LinkedDataHub](#publikavimas-su-linkeddatahub). Lieka produkcinis
+diegimas `https://linkeddata.lt/` adresu (rinkiniai jau generuojami su `$base`
+parametrizuotomis URI, tad tereikia `make -C etl BASE=https://linkeddata.lt/`).
 
 **Nauji rinkiniai** (integracijos taškai jau paruošti — žr. „kaip pridėti“ žemiau):
 

app/.ldhignore

@@ -0,0 +1 @@
+root.ttl

app/admin-units.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Administraciniai vienetai"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/install.sh

@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Creates/updates the LTLOD container scaffolding (root document + containers +
+# taxonomy scheme containers) on a running LinkedDataHub instance via the LDH
+# CLI. Requires LinkedDataHub's bin/ subdirs and Jena's bin/ (for `turtle`) on
+# $PATH — `make install` in the root Makefile sets this up.
+# Deliberately does NOT call make-public.sh: `make load` ends with `make public`.
+set -euo pipefail
+
+if [ "$#" -ne 3 ] && [ "$#" -ne 4 ]; then
+  echo "Usage:   $0" '$base $cert_pem_file $cert_password [$proxy]' >&2
+  echo "Example: $0" 'https://localhost:4443/ ./ssl/owner/cert.pem Password https://localhost:5443/' >&2
+  echo "Note: special characters such as $ need to be escaped in passwords!" >&2
+  exit 1
+fi
+
+base="$1"
+cert_pem_file=$(realpath "$2")
+cert_password="$3"
+proxy="${4:-$base}"
+
+app_dir="$(cd "$(dirname "$0")" && pwd)"
+
+printf "\n### Updating root document: %s\n" "$base"
+turtle --base="$base" < "$app_dir/root.ttl" | put.sh \
+    -f "$cert_pem_file" \
+    -p "$cert_password" \
+    --proxy "$proxy" \
+    -t "application/n-triples" \
+    "$base"
+
+printf "\n### Updating container documents\n"
+"$app_dir/update-folder.sh" "$base" "$cert_pem_file" "$cert_password" "$app_dir" "$app_dir" "$proxy"

app/legal-entities.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Juridiniai asmenys"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/org-units.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Seimo padaliniai"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/organizations.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Organizacijos"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/parliamentary-groups.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Parlamentinės grupės"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .

app/parties.ttl

@@ -0,0 +1,11 @@
+@prefix dh:  <https://www.w3.org/ns/ldt/document-hierarchy#> .
+@prefix ldh: <https://w3id.org/atomgraph/linkeddatahub#> .
+@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
+@prefix dct: <http://purl.org/dc/terms/> .
+
+<> a dh:Container ;
+    dct:title "Partijos"@lt ;
+    rdf:_1 <#select-children> .
+
+<#select-children> a ldh:Object ;
+    rdf:value ldh:ChildrenView .