- ProductSync: bsc_work.wc_product_id could go stale if a product was ever
deleted out-of-band (wp-admin, a cleanup script). wc_get_product() then
correctly detects "no product" and creates a new one, but the repoint was
gated behind `if (!$existing_id)` — which was already true, so the stale
ID never got corrected. Every future sync repeated this, one duplicate
product per run. Fixed by making the repoint unconditional (cheap,
idempotent UPDATE either way). Reproduced the exact scenario against dev
(deleted a product out-of-band, ran sync-products twice) and confirmed:
one repoint, zero duplicates, product count and per-work product count
both correct across repeated runs.
- Commands.php (sync-hardcover-tags): wp_set_object_terms() with an empty
array clears the taxonomy rather than leaving it alone — verified
directly. Hardcover legitimately returns no moods/content-warnings for
plenty of books, so a --force re-sync could silently wipe existing tags,
including anything hand-tagged. Fixed by skipping the call per-category
when that category's array is empty. Verified via a direct eval test:
pre-existing genre tag survives a sync where genre comes back empty,
mood still gets set normally.
- docker-compose.yml: two stray root-owned secrets/db_password and
secrets/db_root_password directories were already sitting on disk —
Docker auto-creating a bind-mount source as a directory from an earlier
manual `docker compose` call that ran without ENVIRONMENT exported
(reproducing the exact bug already fixed once this session). Removed
the stray dirs and changed every `${ENVIRONMENT}` in a volume mount to
`${ENVIRONMENT:?ENVIRONMENT must be set}` so Compose now hard-fails
instead of silently defaulting to empty. Verified: unset ENVIRONMENT now
fails config validation with a clear error; set, it still works.
- HARDCOVER_API_TOKEN moved to the same _FILE secrets pattern already used
for DB_PASSWORD/DB_ROOT_PASSWORD (docker-compose.yml, deploy.sh,
HardcoverAdapter.php) — it was a live, consumed secret still going
through Compose's ${VAR} interpolation, exposed to the same
mangling bug already fixed for the DB passwords, plus visible via
`docker inspect`. deploy.sh now writes secrets/<env>/hardcover_api_token
(optionally empty). Verified via deploy.sh dev + wp eval: empty file ->
is_configured() false, a real token value -> true.
- backup.sh: mariadb-dump had no --single-transaction, so a dump against a
live site would either table-lock for its duration or produce a
non-atomic/inconsistent dump. Added; verified a real dump still runs
clean and produces a valid, restorable-looking .sql.gz.
Ahead of running this against a live site (not just staging bulk-imports),
addressed the two risks that only matter under concurrent traffic:
- One bad product no longer wastes a whole batch: each product save is
wrapped in its own SAVEPOINT within the batch transaction, so a failure
rolls back just that row (counted in the new failed[] result, surfaced
via WP_CLI::warning like import-gutenberg already does) instead of
discarding up to 200 already-good rows. Verified SAVEPOINT/ROLLBACK TO
SAVEPOINT actually works on this MariaDB instance via a direct $wpdb test
before relying on it.
- Cache invalidation suspension is now scoped per-batch instead of the
whole run, with clean_post_cache() called explicitly per product once
each batch commits, replacing the single end-of-run wp_cache_flush().
This site runs a shared Redis object cache across PHP-FPM workers, so a
whole-run suspension + full flush would let a concurrent visitor see
stale cached product data for the run's duration, and the flush itself
would evict sessions and everything else site-wide. Scoping bounds
staleness to one batch and targets only the products actually touched.
Also moved term resolution (term_exists()/wp_insert_term()) out of the
per-product savepoint entirely: warm_term_cache() now resolves every
distinct subject for a batch before its transaction opens, so a term
created for one product can never get silently rolled back by a *different*
product's savepoint failure while a stale term_id sits cached in memory
and gets attached to a later product (an orphaned term_relationships row,
checked for directly post-fix: 0 found).
Verified after: 2000-item sync still ~90s (savepoints add no measurable
overhead), 0 lookup-table mismatches, 0 term-count mismatches, 0 orphaned
term_relationships, idempotent re-run with 0 duplicates.
Measured 2000-product sync_all() at ~2.5 products/sec baseline (2-core/8GB
staging box). Confirmed via source inspection that WooCommerce's
wc_product_meta_lookup sync is synchronous inside WC_Product::save(), so
every product save was its own implicit autocommit transaction -> its own
fsync. Batching each 200-item chunk into one explicit transaction, deferring
product_cat term counting, suspending post cache invalidation for the
duration, and caching subject->term_id lookups in memory brings the same
2000-item sync to ~21 products/sec (~95s), with the update-path re-run at
~82/sec.
Verified correctness after, not just speed: wc_product_meta_lookup price/
stock matched postmeta exactly (0 mismatches across all 2000), product_cat
term counts matched actual term_relationships (0 mismatches), no orphaned
category assignments, no duplicate products on re-run.
Cover sync (OpenLibraryAdapter + CoverSync + sync-covers command): fetches
a real cover from Open Library's free, keyless, explicitly-licensed-for-
this-use Covers API and attaches it as a genuine Media Library attachment
(not a hotlinked <img> — WooCommerce's shop loop/gallery/structured data
all need a real _thumbnail_id). ISBN-first, title/author-search fallback
via the confirmed cover_i field, matching the pattern already established
in HardcoverAdapter.
Two real bugs found and fixed while verifying this against the actual
catalog, not assumed:
- A Range-header HEAD-equivalent probe (added to avoid double-fetching)
caused Open Library's server to redirect with a misleading content-type,
producing a false positive on a known-fake ISBN. Removed — fetch once,
verify the real bytes.
- Their search endpoint has genuine transient failures under repeated
querying (same request, same input, failed then succeeded seconds
later) — added retry-with-backoff on network errors/5xx, matching how
the rest of this codebase already treats transient failures as
retriable rather than fatal.
Also found chasing what looked like a third cover-sync bug, but wasn't
one: uploads/2026/08 was owned by root, silently blocking www-data-run
wp-cli from writing new files. Root cause was a gap in the earlier
root-hardening pass (docker-compose.yml, Commands.php) — it fixed our own
deploy.sh/backup.sh/Makefile invocations but missed the cron sidecar's
own internal process, which was still running its wp-cli loop as root via
a leftover --allow-root. Fixed at the container level (`user: www-data`
on the cron service) since it's a plain shell loop with none of
php-fpm's master-process-needs-root-to-drop-privileges concern.
Gutenberg importer: filters to actual literature via LoCC (Library of
Congress Classification) — verified directly against the real catalog
that novels consistently get a P* code while government documents/
speeches/law get E/JK/KF/DA and never a P code. Removes the Declaration
of Independence, Bill of Rights, etc. from what was importing as "books."
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Four flat taxonomies attached to product (bsc_genre, bsc_mood,
bsc_content_warning, bsc_tag) reuse WordPress's native taxonomy archive
system for the "click a tag, see everything with it" browsing AO3 is known
for — no custom archive templates or query logic needed. Verified: all
four register correctly, render as clickable chips on the product page
(content warnings get a distinct notice instead of just another chip),
and the archive page + term count both work end-to-end with real test
data.
HardcoverAdapter + `wp bookstore sync-hardcover-tags` pull genre/mood/
content-warning/freeform tags from Hardcover's API to populate these.
This part is explicitly NOT verified against a live response — Hardcover's
API is in beta with informal docs, and there was no API token available
to confirm the exact query/response shape. Flagged clearly in the adapter
itself; needs a real token + introspection query before trusting the
field-parsing logic in production. ISBN-first-then-title/author matching
handles both real future ISBNs and the current Gutenberg-synthetic
catalog's fake ones.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
--allow-root was routing around wp-cli's own safety check rather than
addressing why root was there in the first place: docker exec defaults to
the container's root user because the image never sets a non-root user
for exec sessions — it was never about the actual web-facing attack
surface, which already runs as www-data (verified: php-fpm's worker
processes, the ones executing plugin/theme code for real requests, run as
uid 33, not root; only our own deliberate admin commands were root).
wp-config.php and the rest of wp-core are already owned by www-data (the
official entrypoint sets this up), so there's no actual reason for our
own commands to run as anything else. Verified: a full clean deploy and
a bookstore-core wp-cli command both work identically running as
www-data, no functional change, just removed an unnecessary privilege.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Builds bookstore-core's first real business logic (design doc §02-§05,§07),
replacing the plugin stub with:
- Schema: bsc_work/edition/isbn/cover_variant/supplier_offer/member tables
via dbDelta, with a versioned upgrade path.
- GutenbergImporter: imports Project Gutenberg's live bulk catalog as
synthetic staging data. Deterministic checksum-valid ISBN-13s (there are
no real ISBNs in PG data) keep re-imports idempotent.
- SyntheticOfferGenerator: the GutenbergTestAdapter of §04 — realistic
condition/stock/price distribution with deliberate out-of-stock/stale/
discontinued cases for later exception-path testing.
- PricingEngine: the §05 margin formula, verified against the design
doc's own worked example ($6.00 + $3.99 -> $14.99 exactly).
- ProductSync: one WooCommerce product per Work, never per ISBN (§03),
idempotent (verified: re-sync of 2000 works produces 0 duplicates).
- BSC_Free_Shipping: a real WC_Shipping_Method (WooCommerce's built-in
Free Shipping only supports one static threshold, not membership-
conditional), verified through actual browser-session cart flows in
both directions, not just unit calls.
Found and fixed one real data-integrity bug while scale-testing: PG's
Authors field lists every contributor for anthology works, overflowing
primary_author's column — Work::insert() was failing silently (Catalog
insert methods never checked $wpdb->insert()'s return value), while the
following Edition/Isbn inserts for that same row went ahead anyway and
attached to whichever work_id was last successful. Fixed at the root:
inserts now throw on failure, the importer takes just the first author
(matching the column's actual semantic intent), and the import loop
catches per-row failures so one bad title can't abort a bulk run.
Verified end-to-end against a real 2000-title import: exact work/edition/
isbn/product count parity, no duplicates on re-run, real HTTP cart tests
for both shipping thresholds, live storefront search.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two real, verified findings from actually testing the "let Caddy issue a
self-signed cert automatically" approach against a literal IP:
1. TLS SNI is not sent for literal IP connections (out of spec — SNI's
HostName type explicitly excludes IPs). Confirmed via openssl s_client:
the handshake fails with a TLS-layer internal_error, even though Caddy
logs "certificate obtained successfully" — Caddy has no way to select a
cert without SNI. No Caddyfile config fixes this; the address has to be
a hostname.
2. Caddy's automatic-HTTPS heuristic only treats bare IPs and "localhost"
as obviously-private. Any other name — including a made-up LAN hostname
like "bookstore-staging.test" — it assumes might be real and tries
Let's Encrypt, which fails the same way staging.example.com did.
Fixed by forcing `tls internal` explicitly in Caddyfile.staging,
removing the guesswork entirely.
Verified end-to-end with curl --resolve (proper SNI, no real DNS/hosts
change needed to test): direct HTTPS 200, HTTP->HTTPS redirect chain 200.
README now documents the /etc/hosts requirement and how to trust Caddy's
root cert to skip the one-time browser warning.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two separate leaks were causing the same cosmetic-but-annoying warning to
survive every previous fix attempt:
1. Compose's own `secrets:` block reads the referenced file's *content*
as part of its config model, and applies the same interpolation
warning to it — even with DB_PASSWORD fully removed from every ${VAR}
and --env-file path. Switched from Compose's native `secrets:` to plain
bind mounts at the same /run/secrets/* paths: a bind mount only ever
touches the file's path, never its content, so it's immune. (Verified
this precisely with an isolated repro before rolling it out — the two
mechanisms behave differently even though they look equivalent.)
2. caddy's `env_file: .env.staging` (a leftover from the since-removed
basic-auth setup) loaded the *raw*, unfiltered env file directly,
bypassing deploy.sh/backup.sh/restore.sh's filtered-copy mechanism
entirely. Caddy only ever needed SITE_DOMAIN; switched to passing that
one value directly instead of the whole file.
Also fixed Caddyfile.staging: the site address had no explicit scheme, so
Caddy's automatic-HTTPS logic still applied to a private IP (registering
its own internal CA and redirecting HTTP->HTTPS) — not the "plain HTTP
only" behavior I'd assumed and told the user earlier. Prefixed with
`http://` to genuinely disable automatic HTTPS for this LAN-only box.
Verified end-to-end: fresh deploy with dollar-sign DB passwords produces
zero warnings, serves HTTP 200 on plain http:// with no redirect, and the
DB connection genuinely authenticates.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The bring-up wait only checked that php-cli runs, not that the official
image had finished copying WordPress's core files into the (possibly
freshly-emptied) wp_core volume — a step that takes a few real seconds on
first boot. On a fast pass through that window, wp-cli would run against
an empty /var/www/html and fail with "This does not seem to be a
WordPress installation," even though the container was otherwise healthy.
Now waits on wp-settings.php actually existing instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The reported bug (a generated password containing "$BRjTx3tlmSpz" got
silently blanked, breaking the DB connection) is the same Compose
interpolation issue as the earlier bcrypt hash, but this time in fields
that are genuinely user-chosen and can't just be avoided by convention.
Switched DB_PASSWORD/DB_ROOT_PASSWORD to Docker's official `_FILE`
secrets convention (MARIADB_PASSWORD_FILE / WORDPRESS_DB_PASSWORD_FILE),
backed by Compose's native `secrets:` mechanism: deploy.sh writes the raw
value to secrets/<env>/db_password, and the container reads that file
directly — the value never passes through Compose's ${VAR} interpolation
at all. Verified end-to-end with an actual `$`-containing password,
including a full deploy → backup → restore → still-serving round trip.
Also fixed along the way (found while actually testing, not assumed):
- Makefile never exported ENVIRONMENT, so `make up` alone (bypassing
deploy.sh) would have left the new secrets path unresolved.
- deploy.sh chmod'd the secret files 600, unreadable by the container's
own UID (www-data) — fixed to 644, relying on the containing directory
(700) to keep other host users out instead.
- backup.sh/restore.sh still called `mysqldump`/`mysql`, which don't
exist in the mariadb:11 image under those names — renamed to
mariadb-dump/mariadb. (This means neither script had actually
succeeded before now; both are verified working end-to-end here.)
The supplier/payment API keys remain passed the old way — nothing reads
them yet (bookstore-core is still a stub), so there's no live bug to fix
there; noted in .env.example that the same _FILE pattern should be used
once that code exists.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Staging runs on a private-network VM, not reachable from outside, so the
HTTP basic-auth layer was unnecessary defense-in-depth. Removing it also
lets the Caddyfile drop the render-around-Compose workaround entirely
(that workaround existed specifically because Compose's interpolation
mangles a bcrypt hash) — the noindex header and blog_public=0 stay, since
those guard against search-engine indexing, a separate concern from
network-level access.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Replace the bare-repo post-receive hook with deploy/poll-deploy.sh: Gitea
and the Docker hosts are separate machines, so each box polls its branch
via host crontab instead of needing an exposed webhook receiver.
- Add Blocksy theme + Blocksy Companion auto-install to deploy.sh (free
tier; the paid Book Store starter site still needs a manual license step).
- Fix deploy.sh/backup.sh/restore.sh sourcing .env files as bash: a bcrypt
hash's `$2a$14$...` shape breaks under `set -u`. Replaced with
deploy/lib/env.sh, a literal (non-executing) KEY=VALUE reader.
- Fix docker compose itself mangling the same kind of value: both
`environment: ${VAR}` and `env_file:` run values through Compose's
interpolation, which silently blanks `$identifier`-shaped substrings.
The staging basic-auth hash is now rendered directly into the Caddyfile
by deploy.sh, bypassing Compose's variable system entirely.
- Fix dev/staging/production silently sharing one Compose project (and
therefore one db_data volume) by pinning an explicit -p per environment.
- cron and wordpress now share one environment anchor so they can't drift
apart again (cron was silently missing WORDPRESS_CONFIG_EXTRA before).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
twooey
