Skip to content

End-to-end testing — post-deploy smoke against csd-test.wagen.io

Owner: Atlas (keystone Platform Engineer) for the CI shape; Riley (QA Engineer) for the Playwright suite itself.

Architecture

The test-e2e job in .gitlab-ci.yml runs Playwright against the already deployed https://csd-test.wagen.io after deploy:test succeeds. There is no in-job Next.js server, no pnpm install of the full workspace, and no direct DB connection from the test container.

install -> check -> test -> build -> db-migrate -> deploy:test -> test-e2e
                                                  \-> deploy:prod (manual)

This shape is candidate 3 from keystone#61, signed off by the architect on 2026-05-01.

Why this shape

  • The Playwright suite (apps/web/e2e/) is already URL-parameterised via process.env.PLAYWRIGHT_BASE_URL in playwright.config.ts. Pointing it at csd-test.wagen.io is a one-line change.
  • The deployed app already has DB connectivity (via db_net), realtime, and storage. Test environment == production stack — no second class.
  • The pre-cutover model booted a local next start inside the Playwright container, which broke after the keystone cutover because the Playwright container is not a member of db_net and therefore could not resolve keystone-pgbouncer:6432. See courtsidedesk#165 for the regression evidence.

One-time setup

Provision the dedicated E2E test user in the csd_test gotrue tenant:

export E2E_TEST_EMAIL="$(glab variable get --scope test E2E_TEST_EMAIL)"
export E2E_TEST_PASSWORD="$(glab variable get --scope test E2E_TEST_PASSWORD)"
./scripts/provision-e2e-user.sh

The script wraps keystone's manage-user.sh (merged via keystone!95). It is idempotent: re-run it any time. It will create the user if missing and rotate the password to whatever is currently in E2E_TEST_PASSWORD.

Re-run the script after: - Any GOTRUE_JWT_SECRET rotation that wipes the gotrue auth schema. - Any rotation of the E2E_TEST_PASSWORD CI variable.

CI variables (env-scoped to test)

Variable Masked Purpose
E2E_TEST_EMAIL yes Pre-provisioned gotrue user that auth.setup.ts logs in as.
E2E_TEST_PASSWORD yes Password for that user.

These are NOT used at build time — only at runtime by Playwright's auth.setup.ts.

Concurrency

The job sets resource_group: csd-test-e2e. GitLab serialises runs that share a resource group, so two pipelines pushing to main in quick succession will queue rather than race against the shared csd_test DB.

The Playwright suite creates tournaments with names like E2E Test Tournament <ts> and cleans them up via tRPC in beforeAll / afterAll. A second concurrent run would step on the first run's in-flight tournaments before cleanup; serialisation eliminates that.

Pass / fail behaviour

allow_failure: true. Post-deploy e2e is informational while the new shape bakes. Flip to false once a few green runs land — track via the pipeline history filter ?ref=main&status=success.

The suite uses test.skip() defensively: if the deployed app's tRPC endpoint returns a DB error, dependent tests skip rather than fail. So "e2e ran and skipped everything" is not a green signal — check the JUnit report (apps/web/e2e-results.xml) for the actual pass / skip / fail breakdown.

Rollback

If the new shape misbehaves, revert the MR. The pre-cutover gate returns:

rules:
  - if: '$CI_COMMIT_BRANCH == "main"'
    when: manual
    allow_failure: true

That keeps the job definition in the pipeline (operator can click Play manually) without auto-running on every deploy.