# Deploy โ€” docs site (playground) & npm package Two **independent** delivery paths. Confusing them wastes debugging time. | What | Where it lands | Trigger | Owner-of-record | |---|---|---|---| | **npm package** (`dist/`, the library) | **Team Gitea registry** โ€” `https://product-demo.tvustream.com/gitea/api/packages/ux-team/npm/` (was GitHub Packages until INFRA-F73; became https on the standard port with INFRA-F74) | `git push` a `v*.*.*` **tag** โ†’ `.gitea/workflows/publish.yml` | CI (**Gitea**, self-hosted) | | **docs site** (`playground-dist/`, the Vue/React docs) | `https://product-demo.tvustream.com/tvu-design-system/playground-dist/` | `git push` to **Gitea master** โ†’ deploy-hook (see below) | internal host | Publishing npm does **NOT** deploy the docs site, and vice versa. A release must do **both** (see [RELEASING.md](./RELEASING.md) Step 3 = npm, Step 4 = docs). > ๐ŸŒ **The docs site is reachable from the public internet โ€” owner-confirmed 2026-08-07.** โš ๏ธ Note the `Owner-of-record` column above says `internal host`; that names **who operates the delivery** (vs CI on the npm row), **not** network reach โ€” don't read it as "internal-only". Measured the same day: the URL serves with **no login** (index 200, plus a subresource the index itself references at 200 / 4 366 932 B, and a bogus path 404 โ€” de-prefixed negative control, not just a front-page 200). That measurement alone could **not** rule out VPN (the measuring machine had 6 live `utun` interfaces); the owner closed that gap by confirming public reach. > > **Practical consequence, and the reason this line exists:** committing `playground-dist/` is not merely "a deploy" โ€” **it publishes to the open internet.** Whatever is baked into the site (page content, internal hostnames, Gitea URLs, workflow detail) becomes world-readable. So the question "may this content be public?" is part of **RELEASING.md Step 4's owner decision**, not a separate afterthought. The npm registry row is the opposite: it sits behind a Gitea `read:package` token. > **Both paths now run on the internal host** (INFRA-F71, 2026-07-28). npm publishing moved off GitHub Actions because private-repo billing quota kept blocking releases; `.github/workflows/publish.yml` is demoted to a `workflow_dispatch`-only fallback that runs gates and never publishes (reason in that file's header). They are still **independent** โ€” same host, different triggers, different artifacts. The one gate that cannot run on the Gitea runner is `render-gate` (unprivileged user, no `playwright install --with-deps` โ€” INFRA-F40); it is enforced at tag-cut time by `scripts/release.mjs` instead. --- ## Docs-site deploy chain (set up 2026-05-19) ``` git push โ”€โ”€โ–ถ GitHub (mirror) โ”€โ”€โ–ถ Gitea (product-demo:3001, team truth) โ”‚ push webhook (HMAC) โ–ผ deploy-hook http://172.239.57.41:9999/ (internal host) โ”‚ git pull (self-healing) + build (opted out here) โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ :443 nginx โ€” **root-owned, NOT ours** โ”‚ โ”‚ holds the real Let's Encrypt cert for the domain, โ”‚ โ”‚ redirects :80 โ†’ https, forwards the WHOLE domain onward โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ–ผ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ Caddy (`caddy:2-alpine`) โ€” ours โ”‚ โ”‚ host 8080โ†’:80, 8443โ†’:443 (its own TLS is `tls internal` โ”‚ โ”‚ = self-signed, so it is NOT what serves the public cert) โ”‚ โ”‚ serves /srv = /data/nancy_zeng/web/sites โ”‚ โ”‚ + since INFRA-F74: handle_path /gitea/* โ†’ :3001, which โ”‚ โ”‚ since INFRA-F89 is gitea-proxy, not Gitea directly โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ–ผ https://product-demo.tvustream.com/tvu-design-system/playground-dist/ ``` > โš ๏ธ **Correction (INFRA-F74, 2026-07-30).** This diagram used to say the public web server *was* Caddy. It is not: `:80`/`:443` are held by a **host-level nginx running as root** (`nginx: master process /data/nginx/sbin/nginx`), which reverse-proxies the entire `product-demo.tvustream.com` domain to our Caddy container. Measured: Caddy's published ports are `8080->80` / `8443->443`, its Caddyfile sets `local_certs` + `tls internal`, `/etc/nginx` and `/etc/caddy` do not exist on the host, and `/etc/letsencrypt/live` is unreadable to `nancy_zeng`. The earlier "the server is Caddy" note came from checking only `/etc/nginx` โ€” absence of a config dir is not absence of the server. This matters practically: **anything on `:443` itself (certs, vhosts, `client_max_body_size`) is outside our control**; what we can change is the Caddyfile, and everything under the domain reaches it. Proof that forwarding is domain-wide, not path-scoped: `/microapps/`, `/Mediahub/`, `/email-template/`, `/Claude-Design/` all return 200, and a miss returns a **0-byte** 404 (Caddy `file_server`'s shape, not nginx's ~162-byte error page). - **Gitea is the deploy source.** `origin` pushes to **both** Gitea and GitHub; Gitea also pull-mirrors from GitHub. Either way, `git push origin master` lands on Gitea master. - **`:3001` is no longer Gitea's own port โ€” a proxy of ours sits there** (INFRA-F89, 2026-07-31). Both entrances work again and `ROOT_URL` was not touched: ``` main Caddy (unchanged, still reverse_proxy 172.239.57.41:3001) โ†“ :3001 gitea-proxy โ† caddy:2-alpine, joined to the gitea_default network โ”œ handle_path /gitea/* โ†’ strip prefix โ†’ gitea:3000 (over the docker network) โ”” handle โ†’ pass through โ†’ gitea:3000 127.0.0.1:3003 Gitea itself โ€” bound to loopback only, kept as a diagnostic port ``` **Why both entrances can hold at once**: Gitea still renders links with the `/gitea/โ€ฆ` prefix from `ROOT_URL`. Arriving on `:3001`, `gitea-proxy` strips it. Arriving on the domain, the **main Caddy** strips it and what reaches `gitea-proxy` therefore matches the `handle` branch and passes straight through โ€” **it is not stripped twice**. Nothing about `ROOT_URL` or the TLS registry changed. **The `:3001` contract is unchanged**, so git remotes, the registry API, `deploy-hook` and `act_runner` all kept working without reconfiguration. **Its config is version-controlled** in [`infra/gitea-proxy/`](../infra/gitea-proxy/) โ€” the repo is the source of truth, the host is the deployment. Check for drift with `pnpm run verify:host-infra` (exit `0` consistent / `1` drift / `2` **could not check** โ€” never treat `2` as a pass). Reproduced here too, for reading without leaving this page: ```caddyfile # /data/nancy_zeng/gitea-proxy/Caddyfile { auto_https off } :3001 { handle_path /gitea/* { reverse_proxy gitea:3000 } handle { reverse_proxy gitea:3000 } } ``` ```yaml # /data/nancy_zeng/gitea-proxy/docker-compose.yml services: gitea-proxy: image: caddy:2-alpine container_name: gitea-proxy restart: unless-stopped ports: ['3001:3001'] volumes: ['./Caddyfile:/etc/caddy/Caddyfile:ro'] networks: [gitea_net] networks: gitea_net: { name: gitea_default, external: true } ``` โš ๏ธ **A failed first attempt worth remembering**: pointing the **main Caddy** at a new host port (`reverse_proxy 172.239.57.41:3003`) looked fine โ€” `caddy reload` printed `using config from file` and `adapted config to JSON` with no error โ€” yet `/gitea/` then would not serve. The Caddy container reaches the host's `:3001` but **not** a newly published `:3003` (the two containers sit on different networks: `web_default`/172.20 vs `gitea_default`/172.19). **"Reload succeeded" is not "the route works" โ€” verify with a real request.** Hence the final design talks over the docker network by container name. - **Trigger**: the Gitea push webhook calls the deploy-hook; there is also an **~8h cron** fallback on the host. Both are host-side and not visible in this repo. - **The host does not build this repo at all** (since INFRA-F72, 2026-07-29). The root `.deploy-hook-no-build` marker file tells the hook to skip `install && build` after pulling; the host serves the **committed** `playground-dist/`. This is why `playground-dist/` (and `react-pilot/dist/`) are **committed to the repo** (normally a build artifact would be gitignored). **Do not un-track them, and do not delete the marker** โ€” see its header comment for why, and the third failure mode below for what the host build used to break. - Before that marker, the host build was "best-effort": it usually **failed** (it can't `pnpm install` the pilot's React deps โ€” INFRA-F53, work-log 2026-07-08) and fell back to `git checkout -- playground-dist`. The failure was harmless; it was the occasional **success** that broke the chain. - **Practical consequence (unchanged, now explicit)**: nothing on the host will rebuild the site for you. You rebuild and commit `playground-dist/` yourself โ€” [RELEASING.md](./RELEASING.md) Step 4. ### Consequence for releases Because the deployed docs are the **committed** `playground-dist/`, and the in-site version menu / changelog bake `package.json` + `CHANGELOG.md` at **build time**, you MUST rebuild + commit `playground-dist/` from the release commit or the site shows the old version. This is [RELEASING.md](./RELEASING.md) **Step 4**. --- ## Troubleshooting โ€” "the site is stale / shows an old version" Symptom: the site doesn't reflect a merged docs change or a release (e.g. 2026-07-22: showed 0.10.1 after v1.0.0 shipped). ### 1. Is it a stale artifact, or a stale deploy? (decide first) ```bash # What the LIVE site was last deployed โ€” the decisive clue: curl -sI https://product-demo.tvustream.com/tvu-design-system/playground-dist/index.html | grep -i last-modified # Which JS bundle it serves: curl -s https://product-demo.tvustream.com/tvu-design-system/playground-dist/index.html | grep -oE 'assets/index-[^"]+\.js' ``` - **Last-Modified is recent but content is old** โ†’ stale **artifact**: `playground-dist/` was never rebuilt for the change. Fix = rebuild (below) + push. - **Last-Modified is old / frozen at some past date** โ†’ stale **deploy**: the deploy chain stopped running. The rebuild+push won't help until the chain is un-stuck (steps 3โ€“4). - **โš ๏ธ Third mode โ€” the deploy runs, reports failure, and *rolls the site back*** (INFRA-F72, first seen 2026-07-28). Signature: the notification says `:x: [ux_server] deploy ๅคฑ่ดฅ`, `Last-Modified` **is** fresh, yet `index.html` points at an **older** bundle โ€” while the *new* bundle is already present on the host (`curl -I โ€ฆ/assets/index-.js` โ†’ 200) and byte-identical to yours (compare `shasum -a 256`). **Rebuilding and pushing again does nothing** โ€” do not burn time on it (this doc's two-way split above would send you there). **Cause**: the host build occasionally *succeeds*, and its output lands as **untracked** files under the tracked `playground-dist/`. Neither restore path clears those โ€” `build:playground`'s `git checkout --` and the hook's own `git reset --hard HEAD` (read out of `server.py` 2026-07-29) both touch **tracked** files only โ€” so the next `git pull` aborts โ€” reproduced locally 2026-07-28 by checking out the host's stuck commit, dropping the leftovers in, and pulling: ``` error: Your local changes to the following files would be overwritten by merge: playground-dist/index.html error: The following untracked working tree files would be overwritten by merge: playground-dist/assets/index-DcOrz5Uw.js Aborting ``` With the pull aborted, HEAD stays on the old commit, whose **committed** `playground-dist/` is the pre-release artifact โ€” and the fallback faithfully restores *that*. Every subsequent deploy re-pins it (re-firing the webhook via `POST /api/v1/repos/ux-team/tvu-design-system/hooks//tests` โ†’ `204` changed nothing across 10 polls). **Status: fixed on both sides 2026-07-29 โ€” this mode should not recur.** If you are reading this because it *did*, the recovery is step 4 below (`git clean` variant), and one of the two defences has regressed: 1. **Cause removed** โ€” the host no longer builds this repo (root `.deploy-hook-no-build` marker, honoured by `find_buildable_dirs()` in the hook). No build โ†’ no untracked output โ†’ nothing for a merge to collide with. Check the marker still exists on master and in the host checkout. 2. **Self-heal in the hook** โ€” `_sync_repo()` in `/data/nancy_zeng/deploy-hook/server.py` now detects *exactly* this abort signature after `pull --ff-only` and answers it with `git clean -fd` + one retry, logging `[F72] pull aborted by build leftovers`. It is deliberately **not** an unconditional pre-pull clean: sibling sites on that host legitimately depend on untracked build output (`apply_singlefile_alias` writes `build_dir/index.html`), which a later failed build would not recreate โ€” an unconditional clean would trade this bug for that one. `-fd` without `-x` keeps ignored paths, so `node_modules/` is not re-installed every deploy. - Earlier repo-side mitigation (also still in place): `build:playground`'s failure branch runs `git clean -fdq playground-dist react-pilot/dist`, covering the *failed*-build case for any host that still builds. - Host-side edits were made **without SSH**, via the Gitea runner (it is a `hostexecutor`, so a workflow is just a shell on the host as `nancy_zeng`); that technique โ€” including its three pitfalls, which cost ~40 minutes the first time โ€” is written up in [`retrospection/2026-07-29-registry-migration-and-monitoring.md`](./internal/retrospection/2026-07-29-registry-migration-and-monitoring.md) ยงA1 (moved there 2026-08-14 when the INFRA-F72 entry was closed out). Note the hook's `server.py` is a **`:ro` bind-mount into a container** โ€” editing it requires `docker restart deploy-hook` to take effect on the webhook path (the 8h cron path re-reads the file each run, so it picks changes up immediately). Cross-check the source is actually correct: ```bash node -e "console.log(require('./package.json').version)" # expected current version grep -m1 '^## ' CHANGELOG.md # expected top entry git ls-remote http://product-demo.tvustream.com:3001/ux-team/tvu-design-system.git -h refs/heads/master ``` ### 2. Rebuild the artifact (if step 1 says stale artifact) ```bash pnpm build # regenerates playground-dist/ + react-pilot/dist from source # verify the new bundle bakes the right version: MAIN=$(grep -oE 'assets/index-[^"]+\.js' playground-dist/index.html | head -1) grep -oE '' "playground-dist/$MAIN" | head # optional local preview: (cd playground-dist && python3 -m http.server 4290) โ†’ http://127.0.0.1:4290/#/changelog VISUAL_COMMIT_APPROVED=1 git commit -m "chore(docs): rebuild playground-dist for v" playground-dist react-pilot/dist git push origin master ``` > `VISUAL_COMMIT_APPROVED=1` is required โ€” the pre-commit hook treats `playground-dist/` as a visual change; have the owner eyeball the rebuilt site first. ### 3. Un-stick the deploy chain (if step 1 says stale deploy โ€” HOST-SIDE, not in this repo) `172.239.57.41:9999` (the actual hook target โ€” read it live via `GET /api/v1/repos/ux-team/tvu-design-system/hooks`, don't trust this doc) is unreachable from a dev machine / Claude Code โ€” verified 2026-07-28: `curl` โ†’ `http=000` after 8s, `nc -z` โ†’ port filtered. These need host access: 1. **deploy-hook service** โ€” is the process/container at `172.239.57.41:9999` alive? (`docker ps` / `systemctl status โ€ฆ` / port listening). Check its logs for when/why it stopped; restart it. 2. **Gitea webhook** โ€” repo โ†’ Settings โ†’ Webhooks โ†’ the deploy-hook entry โ†’ **Recent Deliveries**: are pushes delivering? Any failures (connection refused / HMAC mismatch)? Use **Redeliver** to re-fire a recent push. 3. **8h cron** โ€” on the host, `crontab -l` / `systemctl list-timers`: is the periodic pull still scheduled and firing? Check its last run + logs. ### 4. Emergency manual deploy (bypass the hook) On the host, in the docs repo checkout: ```bash git pull && git checkout -- playground-dist # serve the committed dist # or copy playground-dist/ into the nginx docroot directly ``` **If `git pull` aborts** (the third mode in step 1 โ€” untracked build leftovers): the plain `git pull` above is exactly what fails, so clear the leftovers first: ```bash git status --short playground-dist | head # expect `??` build leftovers + `M index.html` git log --oneline -1 # expect an OLD commit โ†’ proves the checkout is stuck git clean -fd playground-dist react-pilot/dist # drop untracked build output (the blocker) git fetch --all && git reset --hard origin/master git log --oneline -1 # now the current master ``` ### 5. Verify the fix ```bash curl -sI https://product-demo.tvustream.com/tvu-design-system/playground-dist/index.html | grep -i last-modified # should be today ``` Then open `โ€ฆ/playground-dist/#/component/select` โ€” the version menu should show the current version; check the changelog page (`#/changelog`) version entries open correctly. --- ## Known history - **2026-05-19** โ€” deploy chain built (GitHub โ†’ Gitea โ†’ deploy-hook โ†’ Caddy). - **2026-07-02** โ€” react-pilot-demo 404 fixed by committing `react-pilot/dist/`. - **2026-07-08 (INFRA-F53)** โ€” host build froze the site (can't install React deps); fixed by committing `playground-dist/` + best-effort build with `git checkout` fallback. - **2026-07-29 (INFRA-F72)** โ€” the "deploy fails *and* rolls the site back" mode (third mode above) root-caused in the hook's source and fixed host-side: `_sync_repo()` did `reset --hard` (tracked files only) but never `git clean`, so a *successful* host build's untracked output aborted the next `pull --ff-only` and pinned the checkout to an old commit. Fixed by (a) removing the cause โ€” this repo now carries `.deploy-hook-no-build` and the host skips building it โ€” and (b) a targeted self-heal (clean + retry once) in the hook. Reached without SSH via the Gitea runner; hook container restarted; verified by planting colliding leftovers and watching a real deploy heal itself. - **2026-07-22** โ€” site found frozen at 2026-07-08 (showing 0.10.1) after v1.0.0: the deploy chain (webhook + cron) had stopped running host-side; the v1.0.0 release had also skipped the docs rebuild (now RELEASING Step 4). Artifact rebuilt + pushed (Gitea `9bd13a34`); deploy-chain restoration is host-side.