🤞 Sync 2026-09-27 22:16:29

luci-app-overview-manager/docs/ARCHITECTURE.md

@@ -0,0 +1,212 @@
+# Implementation notes
+
+Load-bearing facts about LuCI and OpenWrt that this package depends on, why the
+code is shaped the way it is, and how to re-verify each claim. None of it is
+derivable from this repository alone: the upstream behaviour lives in
+`luci-base`, `luci-mod-status` and the active theme.
+
+Upstream references are paths inside `openwrt/luci`.
+
+## The Overview page rebuilds itself every poll cycle
+
+`luci-mod-status/htdocs/luci-static/resources/view/status/index.js` builds one
+`.cbi-section` per include **once**, in `render()`, and then on every poll cycle
+calls each include's `load()` and `render()` again and replaces the section
+*content* through `dom.content`. The interval is `luci.main.pollinterval`,
+default 5 seconds.
+
+Two consequences drive the design of `luci/overview.js`:
+
+1. **Never move a section unless the order actually changed.** Re-appending a
+   node re-inserts it, and re-insertion restarts its CSS animations. The theme
+   puts `fade-in` on every revealed section — `.fade-in { animation: fade-in
+   .4s ease }` with `opacity` `0 → 1` in
+   `themes/luci-theme-bootstrap/htdocs/luci-static/bootstrap/cascade.css` — so
+   re-appending all sections on each cycle makes the whole page flash every
+   five seconds. `applyLayout()` therefore compares the wanted order against
+   the live DOM order and touches nothing when they match.
+
+2. **Never clear a visibility class and re-add it.** Dropping
+   `overview-manager-hidden` from every section and then re-adding it exposes
+   every hidden widget for a frame. Use `classList.toggle(name, force)`, which
+   only mutates when the state really changes.
+
+`scripts/test-overview-ui.js` pins both: it counts `appendChild` calls and class
+mutations across a second identical render and fails if either is non-zero.
+Reverting either property makes it fail, which is worth re-checking if the test
+is ever refactored.
+
+`dom.content` (in `luci.js`) removes all children and appends the new ones in
+one synchronous task, so replacing our own container each cycle cannot paint an
+intermediate state. Only node *moves* and class churn are visible.
+
+`load()` caches its result for the lifetime of the page. The saved layout cannot
+change while the Overview page stays open, and the alternative is two helper
+process spawns every five seconds forever. A failed read is deliberately not
+cached, so a transient rpcd error retries on the next cycle instead of pinning
+an empty widget list for the session.
+
+## Widget-to-section mapping is positional and order-sensitive
+
+There is no attribute tying a rendered section to the include that produced it,
+so the mapping is by index: the *n*-th `.cbi-section` belongs to the *n*-th
+include. `data-overview-widget` is stamped on first use and reused afterwards.
+
+`index.js` requires its includes as `view.status.include.<name>` and sorts
+**those module names**, so the `.js` suffix takes no part in the ordering. A
+file-name sort is not the same sort: `-` is 0x2D and `.` is 0x2E, so plain file
+names order `20_x-y.js` before `20_x.js`, while module names order `20_x` before
+`20_x-y`. One such pair shifts every later widget onto the wrong section, and
+the visible effect is that the wrong card is hidden or moved.
+
+`common.compareWidgets` exists for exactly this and must stay code-unit based.
+`localeCompare` would reorder per browser locale and reintroduce the bug in a
+locale-dependent way.
+
+Two further constraints follow from the same mechanism:
+
+- The include count must equal the section count, or `applyLayout()` gives up
+  after eight retries and silently does nothing. `index.js` filters
+  `fs.list` entries by `type == 'file'`, so an include installed as a symlink
+  would be skipped there while the shell helper's `[ -f ]` follows it and counts
+  it. Nothing in the tree creates such a symlink today; if the layout ever stops
+  applying on a specific device, check this first.
+- `title` must be non-empty and stable. `index.js` uses the include title as the
+  `localStorage` key for its own per-browser Hide button (`includes[i].id =
+  title`), so an empty title claims the empty key and collides with any other
+  untitled include. It is deliberately not wrapped in `_()`: a translated title
+  would change that key whenever the interface language changes.
+
+## Client-side translation
+
+The chain is not obvious and is easy to get subtly wrong.
+
+- The global `_()` is defined in `luci-base/htdocs/luci-static/resources/cbi.js`,
+  not in `luci.js`. It is `TR[sfh(trimws(s))] || s`.
+- `window.TR` is emitted by `admin/translations/<lang>`
+  (`luci-base/ucode/controller/admin/index.uc`), which the theme header loads on
+  every page, including the login page. It is a flat map of eight-hex-digit
+  hashes to strings, merged from **every** `*.<lang>.lmo` in
+  `/usr/lib/lua/luci/i18n`.
+- The lookup key is the hash of the *whitespace-normalised* msgid: `trimws` in
+  `cbi.js` trims and collapses runs of space, tab and newline. `po2lmo` hashes
+  the **raw** msgid from the catalog. **Therefore msgids must already be
+  normalised** — one line, single spaces, no leading or trailing whitespace — or
+  the runtime lookup misses and the string silently stays English.
+- Because all catalogs share one namespace in an unspecified merge order, a
+  string that `luci-base` also translates must either be left out or given the
+  identical translation. `Show`, `Move up` and `Move down` are in that category
+  and use the exact `luci-base` wording so the collision cannot matter.
+
+`scripts/test-po2lmo.py` enforces the normalisation rule, the catalog-to-source
+correspondence in both directions, and round-trips every translation through the
+real hash and a minimal LMO reader.
+
+### Registering the language is mandatory
+
+Shipping a catalog is not enough. `determine_request_language()` in
+`luci-base/ucode/dispatcher.uc` only accepts a language that exists in
+`luci.languages`; with `luci.main.lang=auto` and no matching entry it falls
+through to `en`. A router with no `luci-i18n-*` package installed has an empty
+`luci.languages`, so the catalog would never be selected.
+
+`openwrt/files/etc/uci-defaults/luci-app-overview-manager` adds the entry, the
+same way `luci.mk` generates it for stock `luci-i18n-*` packages. Because this
+package defines its own `postinst`, the default handler that drains
+`/etc/uci-defaults` does not run, so `postinst` invokes the script itself and
+removes it on success; otherwise the language would only appear after the next
+reboot.
+
+Verify on a device:
+
+```sh
+wget -qO- 'http://127.0.0.1/cgi-bin/luci/admin/translations/ru' | head -c 400
+uci show luci.languages
+```
+
+and that content negotiation resolves as expected:
+
+```sh
+curl -s -H 'Accept-Language: ru' http://<router>/cgi-bin/luci/ | grep -o '<html lang="[^"]*"'
+```
+
+### `scripts/po2lmo.py` must stay byte-compatible
+
+It reimplements `luci-base/src/po2lmo.c`. Format details that are easy to lose:
+values are written first, each padded to a four-byte boundary; the index of
+16-byte big-endian records follows, sorted by key hash; the file ends with the
+total size of the value blob. The `val_id` field does not hold a hash, it holds
+the plural count. Entries whose key and value hash identically are omitted so
+the reader falls back to the source string. `extract_string` unescapes only `\"`
+and `\\` — `\n` stays two literal characters, which is what makes the header's
+`Plural-Forms` parsing work. An empty catalog produces no file at all.
+
+To re-verify after any change, build the reference tool and diff its output. The
+reader needs a generated parser header that is not required for writing, so an
+empty stub is enough:
+
+```sh
+git clone --depth 1 --filter=blob:none --no-checkout https://github.com/openwrt/luci
+cd luci && git sparse-checkout set modules/luci-base && git checkout master
+cd .. && mkdir -p oracle/lib
+cp luci/modules/luci-base/src/po2lmo.c oracle/
+sed -n '1,72p' luci/modules/luci-base/src/lib/lmo.c > oracle/lib/sfh.c
+cp luci/modules/luci-base/src/lib/lmo.h oracle/lib/
+: > oracle/lib/plural_formula.h
+cc -O1 -Ioracle/lib -o oracle/po2lmo oracle/po2lmo.c oracle/lib/sfh.c
+
+PO2LMO_REFERENCE=$PWD/oracle/po2lmo python3 scripts/test-po2lmo.py
+```
+
+The same binary can be diffed against every catalog in `luci-base/po/*/*.po` as
+a broader check; all of them currently match byte for byte.
+
+## Static resources are cached without a cache-buster
+
+`L.require` builds module URLs as
+`<base>/<path>.js` plus `?v=<env.resource_version>` **only when that field is
+set**, and the OpenWrt 25.12 page template does not set it — it versions
+`luci.js` alone. uhttpd serves the files with `ETag` and `Last-Modified` but no
+`Cache-Control`, so browsers apply heuristic freshness and can reuse a stale
+copy for a long time.
+
+Practical effect: after changing anything under
+`/www/luci-static/resources/`, a normal reload may keep running the old code. A
+reload that bypasses the cache is required — in Safari that is
+`Cmd+Option+R` ("Reload Page From Origin"); `Cmd+Shift+R` does nothing there.
+Confirm which code is live by reading the section titles, since a stale include
+renders its section heading differently:
+
+```js
+[...document.querySelectorAll('.cbi-section')].map(s => s.querySelector('h3')?.textContent.trim())
+```
+
+## Shell that ships to the router runs under BusyBox
+
+The test suite runs on GNU coreutils, which silently accepts constructs a stock
+OpenWrt image does not have: `sort -o`, `base64`, `grep -P` and `find -printf`
+among them. `scripts/check-busybox-compat.sh` scans the **staged package tree**
+rather than the sources, so the checked set cannot drift from what is installed;
+it also runs `busybox ash -n` on each script when a `busybox` binary is present,
+which no pattern list can replace.
+
+Four files reach the router: `CONTROL/postinst`, `CONTROL/prerm`,
+`etc/uci-defaults/luci-app-overview-manager` and `usr/libexec/overview-manager`.
+
+## Working against a real device
+
+- OpenWrt has no `sftp-server`, so `scp` fails. Stream instead:
+  `ssh <host> 'cat > /path' < localfile`.
+- `apk-tools` on the device has no `mkpkg`, and OpenWrt 25.12 cannot install an
+  `.ipk`. Building a package needs the SDK; for a quick check, deploy the payload
+  files and compare checksums.
+- ACL changes need `/etc/init.d/rpcd reload`, which drops active LuCI sessions.
+- Name the package in every transaction. A blanket upgrade is never appropriate
+  on a live router; `scripts/check-apk-trust.sh` fails the build if one appears
+  in a script or in the documentation.
+- Verify a published package with the device's own `apk` and the tracked public
+  key, and confirm it is rejected without it:
+
+  ```sh
+  apk verify --keys-dir <dir-with-public-key> <package>.apk
+  ```