--- title: Known Limitations icon: phosphor-duotone:warning summary: The honest gaps - what is proven against a real running app and what is not. description: The honest gaps - what is proven against a real running app and what is not. tags: [reference, limitations] --- # Known Limitations BxAgents is under active development. This page tracks the honest gaps - what's tested against a real running app, what still only runs against bx-ai's `"mock"` provider, and real upstream quirks this project ran into. ## Most testing runs against the `mock` provider - one real LLM round trip now runs in CI Every fast-lane spec (build pipeline, generators, CLI verbs) and the [ColdBox integration suite](#real-coldbox-integration-testing) exercise bx-ai's built-in `"mock"` provider - never a real network call to an LLM. This is deliberate (fast, free, deterministic CI). It means no automated test proves any of OpenAI/Anthropic/Gemini/etc.'s own provider-specific request/response shaping actually works against their real APIs - do at least one manual `chat`/`serve` run against the real provider you intend to use and a throwaway API key before depending on this in production. One narrow but real gap this used to leave is now closed: `tests/specs/integration/LiveLlmSpec.bx` runs a genuine LLM round trip - a plain completion, and a real tool-call turn (the model actually deciding to invoke a BoxLang closure, not a scripted mock response) - against [Novita AI](https://novita.ai)'s OpenAI-compatible endpoint, via bx-ai's own `"openai-compatible"` provider (`OpenAICompatibleService.bx` - a real, existing bx-ai provider this required no changes to). Gated on the `NOVITA_API_KEY` repository secret via each `it()`'s own `skip` argument (TestBox's real conditional-skip mechanism): CI (which has the secret) runs it for real; a local `./gradlew testBx`/fork PR without it reports the two specs as cleanly "Skipped," never a failure. This proves bx-ai's HTTP call/response-parsing/tool-calling machinery is sound against a real model - it does not extend to any provider besides the OpenAI-compatible surface Novita implements. ## Real ColdBox integration testing `./gradlew testColdBoxIntegration` boots a **real** `boxlang-miniserver` process against a generated app's own `Application.bx`/`Bootstrap`, and makes a genuine HTTP request through a `toAi()`-registered route. This is the strongest proof point in the suite - it caught three real bugs in `ColdBoxAppGenerator.bx` during development (a missing WireBox `Binder` `extends`, wrong `Bootstrap` constructor argument order, and a bare `getInstance()` call that doesn't exist on `Binder`). ### The `toAi()` first-request race The **very first** HTTP request to a freshly booted app's `toAi()` route can transiently fail with "Function [getInstance] not found" - a genuine ColdBox/WireBox lazy-injection race on the Router's own `getInstance` delegate, not a BxAgents bug. It succeeds reliably on every request after something else has already forced WireBox to build the `GeneratedAgent` singleton once (a health check, a `chat` session, another route). **Send a warm-up request** before relying on a freshly deployed `toAi()` route under load. ## Three integration checks - closed via a different route than originally planned `tests/specs/integration/RuntimeStartupSmokeSpec.bx` still has three `xit()`s, since this file runs via the CLI runner (`runTests.bxs`/`testBx`), and BoxLang's CLI mode never gives a `cgi` scope - which ColdBox's RoutingService needs even to load the router at startup. That's a structural fact about CLI mode, not a gap: all three checks are now proven for real elsewhere, in `tests/specs/integration/coldbox/ColdBoxRuntimeSpec.bx` (which runs inside a real HTTP request served by a real `boxlang-miniserver` process): - A real ColdBox-routed HTTP request reaching the generated agent end-to-end - proven by `ColdBoxRuntimeSpec.bx` plus `runColdBoxIntegrationTests.bxs`'s own `POST /api/chat/invoke` assertion. - `schedules/*` actually registering with a **live** ColdBox `Scheduler` - proven via `SchedulerService.getSchedulers()["appScheduler@coldbox"].hasTask(...)` against a real boot, without waiting out an actual cron fire (the coarsest supported granularity is 1 minute, which would tax every CI run for marginal extra proof once the task is confirmed live and registered). - `chat` and a `serve`d HTTP route never diverging - corrected from its original wording (`chat` deliberately never boots WireBox at all, so it can never share WireBox's singleton *object* by design) to what actually matters: instantiating `GeneratedAgentFactory` outside of WireBox produces a behaviorally-equivalent agent to WireBox's own singleton. ## Fixed: real OS-process CLI testing caught a real module-resolution bug `ModuleCliProcessTest.java` spawns genuine `java -jar module:bxagents ...` child processes against a copy of the actual installable module structure (`build/modules/bxagents`), pointed at a real `modulesDirectory` - exactly how a real BoxLang installation loads this module. Every other CLI spec calls `ModuleConfig.main()`/each verb's `run()` in-process instead, which is faster but never proves the module resolves correctly once genuinely installed. This test caught every CLI verb failing with "class not located" the moment it ran through an actual installed-module process, because internal cross-references used a bare `bxagents.models....` dotted path that only resolved thanks to this repo's own dev/test `boxlang.json` (a hand-declared `/bxagents` mapping) - a genuinely installed module never gets that mapping. Fixed by switching every nested class's internal references to the `Class@bxagents` module-relative suffix form (see `BuildPipeline.bx`'s `init()` docblock for the full explanation); `build.gradle`'s module-structure output moved from `build/module` to `build/modules/bxagents` (folder name must equal the module name for `modulesDirectory` discovery to find it at all), and the dev/test `boxlang.json` now also loads it as a real module. ## Fixed: a built `.bxa` was not actually portable Every runtime path the generated `agent/GeneratedAgentFactory.bx` needed - its own `tools/` directory, each agent node's `instructions.md`, and each class-based node's copied `Agent.bx` - was baked in as the **absolute path it had on the machine that built it**. Building on CI at `/home/runner/work/x` and unzipping the `.bxa` at `/opt/x` therefore threw on the first `getInstance( "GeneratedAgent" )`, and simply *renaming* a project directory broke a local build the same way - despite `package` being documented as producing "a portable `.bxa`". `instructions.md` was worse still: its baked path pointed back into the **source project**, a file `package` never even put in the archive. Fixed in `ColdBoxAppGenerator.bx`, in two parts: - Every agent node's `instructions.md` is now **copied into the app**, mirroring the source layout under `agent/`: the root project's at `.build/app/agent/instructions.md`, each subagent's at `.build/app/agent/subagents//instructions.md` (see [Build Pipeline](build-pipeline.md#4-generate)). - The generated factory resolves all three paths at runtime through its own emitted `resolveAppPath( relative, fallback )` helper, which locates them relative to the factory's own file via `getCurrentTemplatePath()` - the same self-location `WebUiGenerator` already relies on. The build-time absolute path is still emitted, as that helper's **fallback**: self-location is proven for a module-loaded class but not independently verified for the `DynamicClassLoader` load path `chat`/`invoke` use, so if it ever fails to resolve, a built app behaves exactly as it did before - never worse. One consequence worth knowing: a `.bxaignore` pattern broad enough to match `*.md` will strip the agent's own copied `instructions.md` out of the `.bxa`. ## Fixed: a cluster of real bugs found building the testing framework Building `BaseAgentSpec`'s `toHaveCalledTool` matcher and the ColdBox integration harness surfaced - and fixed - a run of real, previously-undetected bugs. Recorded here as a changelog of lessons learned, not as open gaps: - **Agents never received their own tools.** `ColdBoxAppGenerator`'s generated `aiAgent()` call never passed a `tools:` argument at all - confirmed against real bx-ai source (`AiAgent.bx` never references `aiToolRegistry()` internally). No agent BxAgents ever built, in any context, actually received its declared tools; undetected because no test asserted on a real tool invocation, only a non-empty response. Fixed in `ColdBoxAppGenerator.renderAgentFactory()`: every generated `GeneratedAgentFactory.bx` now loads its own `tools/` directory via `ToolRegistryLoader.bx` (a real filesystem path - `aiToolRegistry().scan("tools")`'s mapping-relative resolution silently fails from a `DynamicClassLoader`-loaded context like `chat`, and that rejection still stands), then passes `tools: aiToolRegistry().getAll()` to every `aiAgent()` call. That path started out as the absolute path of the machine that built the app; it is now resolved at runtime relative to the generated factory's own location, with the build-time absolute kept only as a fallback - see the portability entry below. - **`var` cannot be used at the top level of a `.bxs` script.** It declares into the `local` scope, which only exists inside a function, so a top-level `var x = ...` throws `Scope [local] is not available in this context` at runtime, not parse time - it survives review and only fires on the code path that reaches it. Never write `var` outside a function in a `.bxs`, and wrap diagnostic output in its own try/catch so it can never mask the failure it's describing. - **The Application setting for a default datasource is `this.datasource`, NOT `this.defaultDatasource`.** The latter is silently accepted and does nothing - `queryExecute()` still fails with `No default datasource defined...`, naming the registered datasource you're trying to select, which reads as though selection is broken rather than the setting being misspelled. - **qb's `moduleSettings.qb.defaultOptions` did not reach `QueryBuilder` in a real ColdBox boot**, even though qb's own `onLoad()` wiring reads as correct - every query failed with "no default datasource" despite the datasource being registered. `ChatDb.query()` now calls `.mergeDefaultOptions( { datasource : static.DATASOURCE } )` on every builder explicitly rather than depending on that plumbing. - **A generated app now also declares `this.defaultDatasource`**, so anything that runs a query without naming one (ColdBox itself, another module) reaches the one datasource instead of failing. `ColdBoxAppGeneratorSpec` covers both the web-UI and no-web-UI cases. - **The generated `models/ChatDb.bx` did not compile, and no unit spec noticed.** A BoxLang string-literal quoting bug (`?: ""` emitted as `?: "`, opening an unterminated string) shipped in every generated web-UI project; every `WebUiGeneratorSpec` case only grepped generated text, never tried to compile it. There's now a case asserting every generated line has an even number of double quotes - the wider lesson being that a generator spec that only greps its own output doesn't test that the output is valid code. - **`GET /chat/api/health` proves far less than a stale comment claimed.** The generated `health()` action renders a literal 200 and touches nothing else - it says nothing about WireBox, `ChatDb`, or SQLite. A comment in `runColdBoxIntegrationTests.bxs` used to claim otherwise; corrected in place. - **TestBox's `JSONReporter` StackOverflow'd on specs holding live framework singletons** (real `controller`/WireBox/qb objects reflected recursively during serialization) - a passing suite reported as a failed run. `tests/runner-coldbox.bxm` now assembles its own report via a `toScalar()` helper instead, and records pass/fail counts before the report is built. - **In a BoxLang time mask, `nn` is NANOSECONDS, not minutes - use `mm`.** (Note the inverse trap in a *date* mask, where `mm` is the month.) - **Two `handlers/ChatUi.bx` actions had an authorization hole despite deriving identity correctly elsewhere.** `/pending` and `/resume` were addressed by `threadId` alone, so a visitor with someone else's `threadId` could read their pending tool calls and answer their approvals - deriving identity server-side buys attribution, not access control. Both now compare the caller against the `userId` the agent checkpointed in the run options. - **`ProjectValidator`'s path check missed `..` traversal**, catching only a leading separator - `../../var/lib/chat.db` passed cleanly. Now rejected on both, matching whole path segments so legitimate names like `..hidden` still pass. - **`chr()` does not exist in BoxLang - the BIF is `char()`.** Cost several CI cycles because the failure is a runtime one that survives review (`tests/runner-coldbox.bxm` called it in its very first progress marker, so every request 500'd on entry). - **Instrumentation that a retry can overwrite records nothing.** A progress file was being truncated on every retry, hiding the real evidence and turning a plain 500 into an opaque "HTTP 408, never responded"; the retries themselves were also actively harmful, stacking concurrent full test runs onto a small worker pool. Fixed: one request only (the health probe ahead of it already proves the server is up), append-only progress log tagged per-request. - **`request` is a reserved built-in scope name.** A loop variable named `request` silently shadows it with no error at all - fixed by renaming to `recordedRequest` in `BaseAgentSpec.bx`'s matchers. ## `serve`'s miniserver lookup is PATH-only `serve` looks for `boxlang-miniserver` only on `PATH` (`MiniServerLauncher.findExecutable()`). There's no fallback to a configured path or a bundled binary - if it isn't installed and on `PATH`, `serve` fails with a clear, actionable error, but there's no alternate way to point it at one. `invoke --server` reuses `serve` internally, so it inherits the same PATH-only lookup - its own `InvokeSpec.bx` test for the real HTTP round-trip checks `MiniServerLauncher.findExecutable()` first and skips (rather than fails) when no real binary is on `PATH`, the same idiom `RuntimeStartupSmokeSpec.bx` already uses for its own jar-presence check. Do at least one manual `bxAgents invoke --message=... --server` run on a machine with `boxlang-miniserver` installed before depending on the real HTTP path in production - the same honest framing already used elsewhere in this page for gaps this suite can't close itself in every environment. ## The scoped BoxLang runtime home reaches `serve`/`invoke --server` unconditionally; the in-process verbs only if your BoxLang install loads `.env` `serve` scopes the miniserver's own BoxLang runtime home to `.build/runtime` via `serverHome` (a real, confirmed `MiniServerConfig` field - `boxlang-web`'s own `MiniServer` CLI help text: `-s, --serverHome BoxLang server home directory (default: ~/.boxlang)`), so each project's compiled-class cache and any config overrides are isolated per-project rather than shared globally at `~/.boxlang`. `invoke --server` inherits this since it reuses `serve` internally. This part is unconditional - we write the launch config for that process ourselves. `chat`, `build`, `test`, and default (in-process) `invoke` are different: they run inside the already-started `bxAgents` process, whose own BoxLang runtime - and therefore its home - was resolved before any of our BoxLang code, including `ModuleConfig.bx`'s own `main()`, ever got a chance to run (the engine has to exist to interpret that code at all). `BoxRuntime` is a JVM-wide singleton; its home is fixed at first initialization, so nothing a verb class does from inside that already-running process can change it retroactively. There IS a real lever for this, though: `BoxRunner` (BoxLang's own core CLI entry point, not just the miniserver) reads a genuine `BOXLANG_HOME` environment variable before `BoxRuntime` initializes - confirmed directly against the real runtime jar (not just documentation): running it with `BOXLANG_HOME=` set as a real OS environment variable (relative paths resolve against the CWD, same as an absolute path) populates the full home structure at that path instead of `~/.boxlang`, every time. `new` scaffolds a `.env` declaring `BOXLANG_HOME=.build/runtime` - the same path `serve` uses - for exactly this reason (mirroring `ortus-boxlang/bx-ai-intro`'s own `.env`/`BOXLANG_HOME` convention), at the **project root** - i.e. the directory a user actually runs `bxAgents ` from for the in-process verbs, which is the right place for a CWD-based `.env` loader to find it. **What we confirmed, and what we couldn't.** `boxlang-miniserver` (what `serve` launches) does have real, built-in `.env` auto-loading - confirmed by decompiling and then actually running `ortus.boxlang.web.MiniServer` directly: with no `envFile` configured, it resolves `.env` relative to the server's **webRoot** (not the project root) and, if found, loads it as Java `Properties`, applying each key via `System.setProperty()` - real values loaded this way ARE visible to BoxLang code via `getSystemSetting()`. The raw core runtime jar (`BoxRunner`, used by `chat`/`build`/`test`/default `invoke`) has no equivalent logic anywhere in it (confirmed by grepping every class in the jar for `.env`) - so those verbs only pick up `.env` if the actual `boxlang` CLI you have installed (a BVM-provided native launcher, not this raw jar) does its own `.env` loading before the JVM starts, the way `ortus-boxlang/bx-ai-intro` relies on. That's plausible and consistent with real-world usage of that project, but isn't something this sandbox has the real binary to verify directly. One specific, confirmed gotcha even where `.env` loading does happen: **`BOXLANG_HOME` itself does not take effect unless it's a real OS environment variable** - not a JVM system property, and not a `-D` flag. Verified twice directly against the real jars: (1) running `boxlang-miniserver` against a webRoot whose `.env` declared `BOXLANG_HOME=customhome` loaded the file (confirmed via its own "Loaded environment variables from:" log line and via `getSystemSetting()` correctly returning other `.env` values), yet the server still logged `Logs Directory: /root/.boxlang/logs` - the default home, not `customhome`; (2) launching `BoxRunner` directly with `-DBOXLANG_HOME=` (a JVM system property, no `.env` involved) also had no effect on the resolved home, even though `getSystemSetting( "BOXLANG_HOME" )` happily returned the flag's value from BoxLang code. So `BOXLANG_HOME` resolution specifically reads only the true OS environment variable - unlike most settings, `getSystemSetting()` returning a value for it doesn't mean the runtime home actually moved. If your `boxlang` CLI's `.env` loader works the same way MiniServer's does internally (`System.setProperty` after the JVM has already started) rather than exporting a real env var before launching the JVM, `BOXLANG_HOME` in `.env` won't reach the runtime home even though `.env` itself loaded successfully - everything else in it will still work. If your install doesn't give you this at all, source it yourself before running commands (e.g. `set -a; source .env; set +a`) to get the same isolation `serve` already gets unconditionally. ## `new`'s `box install` convenience step isn't exercised by an automated test `new` runs `box install` inside the scaffolded `tests/` folder by default, so `bxAgents test` works immediately (see [CLI Reference](cli-reference.md)). This is a real network operation (CommandBox resolves `testbox` against ForgeBox) - confirmed to take ~25 seconds and fail with a certificate error in this development sandbox specifically, the same ForgeBox-unreachable constraint already noted elsewhere in this project's own tooling. `NewSpec.bx` only exercises the fast, deterministic `--skipInstall` path (and the OS-process `ModuleCliProcessTest.java`/in-process `ModuleConfigCliSpec.bx` invocations of `new` also pass `--skipInstall` for the same reason) - the default install-attempt path itself is verified by manual testing only, not CI. ## `new --repo`'s real clone path is covered by CI, network permitting `RepoCloner.bx` clones via a bundled JGit dependency rather than shelling out to a `git` binary (see [CLI Reference](cli-reference.md)), so unlike the `box install` gap above there is no "is the binary installed" failure mode to skip around - only the network call itself can fail. `RepoClonerSpec.bx`'s live end-to-end test attempts a real clone of a small, stable public GitHub repo and only self-skips if that specific attempt doesn't produce the expected "cloned, but no Agent.bx" outcome. Confirmed empirically in this development sandbox: a real `git clone` to `github.com` over HTTPS DOES succeed here (through the same proxy this sandbox's other outbound HTTPS calls use), so this test exercises the real path rather than skipping - a genuinely better position than `box install`'s, whose own separate certificate/proxy configuration fails here. ## `new --forgebox`'s real install path is covered by CI, network permitting `ForgeBoxClient.bx` talks to ForgeBox's own REST API (`https://www.forgebox.io/api/v1`) directly via `bx:http`, rather than shelling out to the `box` CLI the way `new`'s own `tests/` install step above does - confirmed empirically that this development sandbox CAN reach `forgebox.io` this way, even though the separate `box` CLI process fails here with a certificate error. `ForgeBoxClientSpec.bx`/`ForgeBoxInstallerSpec.bx` cover every not-found/download-failure/missing-Agent.bx code path deterministically via `ForgeBoxClient.bx`'s injectable `entryCaller`/`downloadCaller` seams (the same test-seam idiom `GitHubGateway.bx`'s `apiCaller` already uses) - a real improvement over the old `box install`-based design, which had no way to unit-test any of those paths at all. `ForgeBoxInstallerSpec.bx`'s one live end-to-end test (installing a real, small ForgeBox package) only self-skips if that specific attempt doesn't produce the expected "installed, but no Agent.bx" outcome - confirmed to actually run, not skip, in this sandbox as of this writing. Real ForgeBox entries' exact JSON shape (`{ data, error, pagination, messages }`, an entry's `latestVersion.downloadURL`) was confirmed against the live API (`GET .../entry/coldbox`), not guessed at. ## `--repo` strips the cloned `.git` folder - a deliberate choice, not a bug `new --repo` always removes the freshly-cloned `.git` directory before returning, with no flag to keep it. This is intentional: a template repo's own git history and `origin` remote have no legitimate reason to survive into a fresh project, and leaving them in place is a real foot-gun - a new project's first `git init`/commit/push could otherwise interact with the template's own upstream repo instead of the user's. Someone who genuinely wants the template's git history can clone it manually instead of using `bxAgents new`. ## `chat` needs a real TTY `chat` uses BoxLang's own `MiniConsole`, which shells out to `stty` to set up raw terminal mode - it can only run against a genuine interactive terminal. It doesn't work piped, redirected, or from a non-interactive process (a CI job, a script). There's no non-interactive fallback mode. ## `chat`'s streaming has no cancel, and no history beyond the live session `chat --server`/`--connect` stream the reply over the route's `/stream` (SSE) sub-route, but two things ColdBox's `toAi()` surface does not offer are therefore missing: - **No cancel.** There is no way to interrupt a turn once it starts. `toAi()` registers no cancel endpoint, and the SSE stream is server-driven - a client can stop *reading*, but the agent keeps generating to completion on the server. Ctrl-C ends the whole `chat` process rather than the turn. - **No history.** A session's `threadId` keeps one conversation coherent while `chat` is open, but nothing replays a prior conversation into a new session: reconnecting with `--connect` starts fresh. The agent's own [`memory`](conventions/agent-bx.md) may still give it recall server-side - what is missing is the client showing you what was said before. Both are gaps in what the HTTP surface exposes, not in the client - closing either needs a route that does not exist yet. ## SSE streaming depends on `whitespaceCompressionEnabled` being off `chat --server`/`--connect` stream through BoxLang's native SSE: ColdBox's `toAi()` `/stream` sub-route emits with the `SSE()` BIF, and `AgentClient.stream()` consumes with `http( url ).sse( true ).onChunk( ... )`. There is no frame parser in this codebase - `ortus.boxlang.runtime.net.SSEParser` does that. That works only when the web runtime's **whitespace compression is off**, and it is **on by default**. SSE frames are terminated by a blank line; compression collapses consecutive newlines and eats exactly those terminators. Measured against a live `/__bxagents/stream`: | `whitespaceCompressionEnabled` | Body | `\n\n` in body | `totalEvents` | | --- | --- | --- | --- | | `true` (default) | 916 bytes | **0** | **0** | | `false` | 923 bytes | 6 | 2 | It is not specific to ColdBox, to `toAi()`, or to this client. A bare two-line `SSE()` endpoint loses its terminators the same way (79 bytes, zero `\n\n`), and so does a handler that writes `\n\n` itself with `writeOutput`. A browser `EventSource` against any of them fires no events either. Where it is set: - **`Serve.bx`** writes `whitespaceCompressionEnabled: false` into the served project's own runtime config, under the `.build/runtime` server home it already scopes per project - so it never touches the machine's `~/.boxlang`, and `clean` sweeps it. - **This repo's `boxlang.json`** sets it for the test suite. A project served some other way - a different container, a servlet deployment, someone else's `boxlang.json` - needs the same directive, or its SSE endpoints emit unterminated frames and every SSE client sees nothing. The ColdBox integration run asserts the **delta count** on a real stream, so a regression here fails the build rather than silently degrading to a non-streaming reply. ## Fixed: `chat` and default (non-`--server`) `invoke` used to fail for a class-based `Agent.bx` Previously, both `chat` and default `invoke` threw `The requested class [agent.classes.agentClass] has not been located in any class resolver.` before ever reaching the agent. Root cause: both verbs load the generated `GeneratedAgentFactory.bx` in-process via `DynamicClassLoader.instantiate()` (no ColdBox container involved), and the generated factory used to instantiate a class-based `Agent.bx` via a **relative** dotted-path lookup, which only resolves once something has registered a mapping making the app root resolvable - and nothing did outside a real ColdBox boot. Registering one mid-script doesn't fix it either (confirmed empirically). **Fix:** `ColdBoxAppGenerator.copyAgentClass()` now returns the copied class's own absolute file path, and `renderClassBasedAgentStatement()` instantiates it via `DynamicClassLoader.instantiate( absolutePath, context )` - the same primitive `chat`/`invoke` already use to load `GeneratedAgentFactory.bx` itself - rather than a relative `new "..."()`. Confirmed against `examples/class-based-agent/`: `chat`, default `invoke`, `invoke --server`, and `serve` all build and run the same agent correctly. ## No box.json `executable` install smoke test `box.json` declares `"boxlang": { "executable": "bxAgents" }` so a real module install produces a native `bxAgents` command (see [Installation](getting-started/installation.md)). This wiring itself isn't exercised by an automated test - it relies on the BoxLang module installer's own documented behavior for generating executable wrappers, verified by reading its source, not by an install-and-run test in this repo's own CI. ## `schedules/Scheduler.bx` isn't validated at build time Since it's real, hand-written ColdBox code passed through untouched (see [schedules/](conventions/schedules.md)), `build` can't meaningfully check it the way `{ cron, action }` config used to be checked - a syntax error, a typo in a `getInstance( "..." )` call, or referencing an agent name that doesn't exist all pass `build` cleanly and only surface when the generated app actually boots (`serve`), same as any other real BoxLang class this project doesn't own the contents of. `build` does catch one narrower, adjacent mistake: two agents (root or subagent, any depth) declaring the same `name` - that's a `config/WireBox.bx` binding collision this project generates itself, so it's checked at validation time same as everything else it generates. ## Fixed: a real bug in the `test` verb - resolving "the current BoxLang jar" was ambiguous `TestRunnerService.bx` spawns a child process to run a project's `tests/specs` and needs to know which jar to launch it with. The original "which jar was this class loaded from" trick broke once `boxlang-miniserver-*.jar` (a fat jar that bundles its own copy of `BoxRuntime`) was also on the classpath - the classloader could resolve to the miniserver jar instead, silently launching `MiniServer.main` (which exits 1 immediately) instead of `BoxRunner.main`. Reproducible only through the full `testBx` suite, not in isolation, which is what made it easy to miss. Fixed by resolving the jar from `java.class.path` instead - scanning for a `boxlang-*.jar` entry that does **not** contain `miniserver`, falling back to the old codeSource trick only if no such entry is found. ## Fixed: no CLI verb's non-zero exit code ever reached the actual OS process - every failure looked like success to a shell script A real, previously-shipped bug affecting every verb (`build`, `clean`, `deploy`, `package`, everything): `ModuleConfig.bx`'s `main( args )` computed the right `exitCode` internally and returned it, but confirmed directly against the official BoxLang_Docs MCP source, a module's `main()` return value is never mapped to the real OS process exit code by the runtime - only an explicit `CLIExit( exitCode )` call (a genuine `System.exit()`) does that, and nothing here ever called it. So `boxlang module:bxAgents build && boxlang module:bxAgents deploy ...` in a CI/deploy script would run `deploy` immediately after a **failed** `build`, silently, because the shell always saw exit code `0` regardless of what actually happened. Every in-process test (`ModuleConfigCliSpec.bx`, each verb's own spec) exercises the *computed* exit code directly and would never catch this - only a real OS-process invocation exposes the gap, and nothing here was checking a real process's exit code for a *failing* verb before now (`ModuleCliProcessTest.java` only ever asserted the success path). Fixed by splitting dispatch from exit: `main()` is now a thin wrapper - `return CLIExit( dispatch( args ) )` - and all the actual argv-parsing/verb-dispatch logic moved into a new `dispatch()` method that still just returns an int, exactly as `main()` used to. `CLIExit()` can only live in the outermost `main()`, never in `dispatch()` itself: `dispatch()` is called directly, in-process, by every existing CLI test, and a real `CLIExit()` there would `System.exit()` the whole TestBox JVM on the first assertion. `ModuleCliProcessTest.java` gained a new real-OS-process test - `package` against a project with no prior build - asserting the child process's actual exit code is non-zero, the one check able to tell "genuinely exits non-zero" apart from "never calls `System.exit()` at all, which also happens to exit 0." ## Fixed: `.github/workflows/release.yml`'s `prep_next_release` job had no JDK, so it would have failed the moment a real release ran `prep_next_release` runs `./gradlew bumpMinorVersion` against a freshly-checked-out `development` - a plain `actions/checkout@v7` with no `Setup Java` step anywhere in the job. Gradle itself needs a working JDK to run at all; the sibling `build` job (in the same workflow) has its own `Setup Java` step for exactly this reason, but `prep_next_release` never did. Never actually triggered a real failure yet - this repo has no `main` branch cut and no real release has run through this job - but it was an unconditional break waiting for the first one: `gradlew` would have failed immediately with no JDK on `PATH`/`JAVA_HOME`, before it ever reached `bumpVersion()`. Fixed by adding the identical `actions/setup-java@v5.6.0` step (same `temurin`/`${{ env.JDK }}` as the `build` job) right after checkout. Verified locally: `./gradlew bumpMinorVersion --stacktrace --console=plain` against a copy of `gradle.properties` correctly bumps `1.0.0` → `1.1.0` with a JDK present - the exact command this job runs. ## `deploy`'s `ssh`/`docker`/`digitalocean` targets aren't exercised by an automated real-process test `SshTargetSpec.bx`/`DockerTargetSpec.bx` assert on the exact `scp`/`ssh`/`docker` command each target builds (a real `ProcessBuilder` argument array) without ever invoking the real binary - the same "capture, don't execute" approach used elsewhere in this suite for anything that needs a binary that might not be on `PATH` in CI (see `MiniServerLauncherTest`'s `assumeTrue` skips). `local`'s copy logic IS exercised for real (`LocalTargetSpec.bx`), including a regression test for a real latent bug this refactor fixed: the original `Deploy.bx` picked the "newest" `.bxa` via a lexical filename sort, which silently mis-picks once a project reaches double-digit versions (`v9.0.0` sorts after `v10.0.0`) - fixed by sorting on actual file modification time (`DistArtifactLocator`). `DigitalOceanTargetSpec.bx` similarly only unit-tests the pure `buildAppSpec()`/`findExistingAppId()` logic - the real `GET/POST /v2/apps` calls are never exercised in CI, since that needs a live DigitalOcean account and API token. Do at least one manual `deploy --name=` against a real disposable VM, and one `deploy --name=` against a real DO account with a throwaway app, before depending on either in production - the same honest framing already used above for the mock-provider-only testing gap. ## `deploy`'s `ftp`/`sftp` targets: real connection handling proven, a real successful upload isn't Unlike `ssh`/`docker` (which shell out to external binaries and so can only have their *command construction* tested without a real server), `ftp`/`sftp` call the real [`bx-ftp`](https://github.com/ortus-boxlang/bx-ftp) module's `bx:ftp` component **in-process** - there's no "capture the command, don't execute" option. `BaseFtpTargetSpec.bx` instead makes a genuine connection attempt against `127.0.0.1` on a port nothing listens on, and asserts that the real refused-connection error is caught and re-thrown as a clear `BxAgents.DeployFailed` (confirmed via bx-ftp's own source that every action throws on failure rather than returning a soft `succeeded: false`). This proves the real connect/error-wrap/cleanup path end-to-end - what it can't prove is a real successful upload. That gap is specifically because **this development sandbox has no outbound raw TCP egress at all** - only HTTPS through the environment's proxy (confirmed directly: `curl ftp://test.rebex.net` and a raw `/dev/tcp` connect to a public FTP host both hung and timed out, and `docker info` shows no daemon running, so bx-ftp's own bundled Docker FTP/SFTP test servers couldn't be started here either). This is a sandbox constraint, not a code limitation - do at least one manual `deploy --name=` and one `deploy --name=` against a real reachable server (or bx-ftp's own `docker-compose up` test servers, from a machine that has Docker/network access) before depending on either in production, the same honest framing already used above for `ssh`/`digitalocean`. ## `build` doesn't roll back a partially-written `.build/app` on a mid-generation crash `BuildPipeline.build()` deletes and recreates `.build/app` up front, then runs Phase 5's generators in sequence. Every input that Phase 3 (`ProjectValidator`) can check is checked before any of that happens, so a genuine mid-generation crash should be rare in practice - but if one still occurs (e.g. a `models/`/`schedules/`/`mcp/` entry that fails to load, or an environment/filesystem problem), `.build/app` is left on disk in a partially-written state rather than restored to its prior contents or cleaned up. A subsequent successful `build` overwrites it cleanly, so this isn't sticky, but anything that inspects `.build/app` between a failed build and the next one (a CI step, a manual `package` retry) can see a broken half-generated app. There is no rollback/temp-directory-then-swap step yet. ## An intermittent `StackOverflowError` was observed in `testBx`, unrelated to this project's own code While investigating this milestone, `./gradlew testBx` occasionally (not every run) crashed the whole JVM with a `StackOverflowError` inside the BoxLang engine's own generic-object JSON serialization (`DynamicObjectSerializer`/`BoxStructSerializer` calling each other, alternating, until the stack is exhausted - confirmed to still happen with `-Xss16m`, so it's a genuine cycle, not merely a deep-but-finite structure). Isolated via `git stash` (the crash reproduced identically with none of this session's changes applied, against the exact same commit already on `development`) and via bisecting `tests/specs/**` into every subdirectory both individually and combined (every subset, and even every subset-but-one, ran clean) - only the single full run occasionally reproduces it, and re-running the identical full suite immediately after a crash sometimes passes clean. This points to a timing-dependent race (plausibly interacting with `ExampleScheduler`'s real background `everySecond()` task, which prints from its own thread pool concurrently with whichever spec happens to be running at that moment) rather than a bug in any one spec or in this project's own generated code. If `testBx` fails with a `StackOverflowError` and no other explanation, retry it before assuming a real regression - it is not currently reproducible on demand, so no automated regression test exists for it, and running it down further to a single root cause was out of scope for this round. ## Push-style gateways (Telegram) are tested against a mocked API/scheduler seam only - no live platform integration runs in CI `TelegramGatewaySpec.bx` exercises `TelegramGateway`'s own logic (inbound normalization, outbound chunking at the 4096-char limit, HITL inline-keyboard building, scheduler task registration/removal) entirely against an injectable `apiCaller`/`setScheduler()` test seam - never a real Telegram Bot API call, and never a real ColdBox scheduler boot. This proves the gateway's own code is correct, not that it actually works end-to-end against Telegram's real API or a real running scheduler. Do at least one manual `bxAgents serve` against a project with a real `botTokenEnvVar`-backed Telegram bot before depending on it in production - the same honest framing already used above for the mock-provider-only/no-live-connection testing gaps elsewhere in this file. The same caveat will apply to every future push-style gateway (Slack, Discord, Email, WhatsApp) built the same way. `SlackGatewaySpec.bx` carries the same gap, one level deeper: `SlackGateway`'s persistent websocket connection is tested via an injectable `setSocketOpener()` seam (a fake object standing in for the real `java.net.http.WebSocket`), so every frame-handling/reconnect-logic assertion in the spec runs with zero real network I/O. What genuinely was verified directly (not mocked): a standalone smoke test instantiated the real `SlackSocketListener(gateway)` (which `implements="java:java.net.http.WebSocket$Listener"` directly - BoxLang compiles it as a genuine JVM implementer, no proxy needed) and called the real `HttpClient.newWebSocketBuilder().buildAsync(...)` against an unreachable address, confirming the BoxLang-to-Java interop itself works correctly all the way to the network boundary (it failed with a plain `java.net.ConnectException`, not a casting/interop error) - but no test here has ever completed a real Socket Mode handshake against Slack's actual servers. Do at least one manual `bxAgents serve` against a project with real `botTokenEnvVar`/`appTokenEnvVar`-backed Slack app credentials before depending on it in production. `DiscordGatewaySpec.bx` carries the identical gap, for the identical reason: `DiscordGateway`'s frame-handling/heartbeat/reconnect logic is exercised entirely against injectable `setApiCaller()`/`setSocketOpener()` seams, zero real network I/O. The same standalone smoke-test discipline was applied here too - `gateway.onConnect()` driven against a real `HttpClient.newWebSocketBuilder().buildAsync(...)` call to an unreachable address failed with a plain `java.net.ConnectException`, not a casting/interop error, confirming the interop chain works. What was NOT verified: a real Gateway handshake (`Hello` → `Identify` → `READY`) against Discord's actual servers, real heartbeat timing under Discord's own tolerance, or that the default `intents` value (`GUILDS`+`GUILD_MESSAGES`+`DIRECT_MESSAGES`+`MESSAGE_CONTENT` = `37377`) is actually enough to receive message content once `MESSAGE_CONTENT` is enabled/approved for a real bot in the Discord Developer Portal. Do at least one manual `bxAgents serve` against a project with a real `botTokenEnvVar`-backed Discord bot (with `MESSAGE_CONTENT` enabled) before depending on it in production. `EmailGatewaySpec.bx` carries a larger version of the same gap. Inbound IMAP is tested entirely via an injectable `setImapPoller()` seam (canned normalized message structs, no real mailbox), and outbound is tested entirely via an injectable `setMailService()` seam (`FakeMailService`/`FakeMail`, standing in for `MailService@cbmailservices` - `EmailGateway` never goes through real WireBox at all in these specs, since there's no real ColdBox boot). What genuinely was verified directly this session (not mocked, not assumed): the real `jakarta.mail` API surface `fetchInboundMessages()` depends on (`Session.getDefaultInstance()`, `Flags`/`Flags.Flag`/`FlagTerm`, `Store.getStore("imaps")`, `Folder.READ_WRITE`, `MimeMultipart`, `InternetAddress`) against the real `jakarta.mail-api`/Angus Mail jars (downloaded standalone for this - they aren't vendored in this repo's own test classpath), confirming every class/method name used actually exists and resolves; a real `Store.connect()` against an unreachable address, driven through `EmailGateway.pollInbox()` itself (not a bypassed helper), failed with a plain connection-timeout error, not an interop/casting error, confirming the interop chain reaches the real network boundary correctly, same discipline as Slack/Discord's websocket smoke tests. What was explicitly NOT verified, and is a strictly bigger gap than the chat-platform gateways: no real IMAP handshake against a real mailbox, no real `cbmailservices`/`bx-mail` module installed anywhere in this repo or its test harness (neither is vendored the way `bx-ai`/TestBox are), so the WireBox resolution path (`MailService@cbmailservices` really existing, `BXMail`'s `bx:mail` call really sending) has never been exercised at all, mocked or real, in this codebase. Do at least one manual `bxAgents serve` against a project with real IMAP credentials AND a real `cbmailservices`/`bx-mail` install (confirm `box install` succeeded and `moduleSettings.cbmailservices` resolves) before depending on this gateway in production - this is the least-verified of the four push-style gateways shipped so far. `WhatsAppCloudGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: the signature-verification path is exercised with genuinely computed HMAC-SHA256 signatures (`javax.crypto.Mac`/`SecretKeySpec`, cross-checked independently this session against both `openssl dgst -hmac` and Python's own `hmac` module before trusting the BoxLang computation - a real reference-vector mismatch was caught and turned out to be a typo in the hand-copied expected value, not a bug, but only cross-verification caught that), the verify handshake, webhook dispatch/dedup, outbound send, and interactive button/list rendering are all driven through the gateway's real public methods with only the outbound Graph API HTTP call itself stubbed (`setApiCaller()`). What was NOT verified: the generated `handlers/WhatsAppCloud.bx`'s own ColdBox request-context calls (`event.getHTTPContent()`/`event.getHTTPHeader()`/`event.renderData()`, `rc`'s URL-scope-merged dotted-key query param access for the GET handshake) against a real ColdBox boot - these ARE the documented, standard ColdBox REST-handler idioms (confirmed against ColdBox's own "Building REST APIs" recipe docs, not guessed), a meaningfully more trustworthy starting point than the undocumented `aiGatewayRegistry()` key assumption that turned out to be wrong elsewhere in this file - but "documented" isn't the same as "proven to work in this generated context," and extending the project's own real `runColdBoxIntegrationTests.bxs`/miniserver harness to cover it would have required either passing fake env vars into a separately-spawned miniserver subprocess (no existing mechanism for that) or risking real config-related boot failures in the shared `e2e-coldbox-route` fixture other passing tests depend on - deferred rather than risking that blast radius for what's a lower-confidence-of-actually-being-wrong gap than the registry-key bug was. Do at least one real `bxAgents serve` + a real Meta webhook test (or `curl`) against `/webhooks/whatsapp-cloud` before depending on this route in production. No real Graph API call has ever been made either - `deliver()`/`requestHumanInteraction()`'s HTTP layer is exercised only via the `apiCaller` test seam. `TeamsGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: JWT verification is exercised against a genuinely generated 2048-bit RSA keypair (`java.security.KeyPairGenerator`) and hand-signed test JWTs built entirely inside the spec (no pre-computed fixtures, no external `openssl` dependency at test time) - a valid signature is accepted and dispatches, while a tampered signature, a wrong `aud`, a wrong `iss`, and an expired `exp` are each independently confirmed to reject with 401. The invoke-activity (Adaptive Card button click) path, message dispatch, personal-scope-only filtering, threading via `replyToId`, chunking, and Adaptive Card rendering are all driven through the gateway's real public methods with only the outbound Connector REST call itself stubbed (`setApiCaller()`) and the JWKS/OAuth2 token fetches stubbed (`setJwksFetcher()`/`setTokenFetcher()`). What was NOT verified: the generated `handlers/Teams.bx`'s own ColdBox request-context calls against a real ColdBox boot (same category of gap as WhatsApp Cloud's own handler, and deferred for the identical reason - see that entry above); no real OAuth2 token fetch or Connector REST call has ever been made against Microsoft's actual endpoints; and the JWKS-cached-for-instance-lifetime tradeoff (see `docs/conventions/gateways/teams.md`) means a real key-rotation scenario has never been exercised either. Do at least one real `bxAgents serve` + a real Teams app registration (App ID/password from the Azure/Bot Framework portal) + a real Teams client sending a DM before depending on this gateway in production. `TwilioGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: the `X-Twilio-Signature` HMAC-SHA1/base64 verification path is exercised with a genuinely computed signature built inline in the spec, cross-checked independently this session against Python's own `hmac`/`hashlib` modules before trusting the BoxLang implementation (same discipline as WhatsApp Cloud's own HMAC-SHA256 cross-check) - a real reference signature was computed in Python for a known auth token/URL/params combination and confirmed to match BoxLang's output exactly. Form-body parsing (including a literal `+` correctly round-tripping through `%2B` percent-encoding), the `publicUrl` override for proxy/tunnel deployments, the dual-path TwiML-ack-then-async-REST-reply model, outbound chunking, and the phone-number-keyed HITL reply correlation are all driven through the gateway's real public methods with only the outbound Messages API HTTP call itself stubbed (`setApiCaller()`). What was NOT verified: the generated `handlers/Twilio.bx`'s own `event.getUrl()` call against a real ColdBox boot (same category of gap as WhatsApp Cloud/Teams's own handlers, deferred for the identical reason) - `event.getUrl()` is a documented ColdBox Routable/Request Context method (confirmed via the ColdBox docs MCP), not a guess, but "documented" isn't "proven in this generated context." No real Twilio Messages API call has ever been made either. Do at least one real `bxAgents serve` + a real Twilio phone number webhook test before depending on this route in production - and note the phone-number-keyed HITL correlation (see `docs/conventions/gateways/twilio.md`) has a real, documented limitation: a second HITL request to the same phone number before the first is answered would overwrite the first's `pendingApprovals` entry, silently orphaning it. No allowlist/rate-limiting is built for inbound SMS either - unlike Eve, which documents (but doesn't enforce in code) that its own `allowFrom` config is "mandatory," this port has no equivalent gate at all; any phone number can message a deployed Twilio number and reach the agent. `GitHubGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked, and this gateway's core logic was additionally driven through a standalone real-BoxLang smoke test during development (not just the permanent spec) - which is how a genuine bug was caught before it ever reached the test suite: the mention-extraction helper's substring logic called `left( body, 0 )` whenever the `@mention` occurred at the very start of a comment (a very common case), and BoxLang's `left()` throws `"Count cannot be zero"` rather than returning an empty string for a zero count - confirmed by triggering it with a real comment body, then fixed by explicitly branching around the zero-length case rather than assuming `left()`/`mid()` tolerate it. The `X-Hub-Signature-256` verification, `@mention` regex-lookahead gating (confirmed via a dedicated smoke test that a bot named `mybot` does NOT fire on `@mybot2`), bot-loop guards, delivery-id dedup, issue-vs-review-thread conversation identity, `deliver()`, and the `@mention`-to-reply HITL correlation are all driven through the gateway's real public methods with only the outbound GitHub REST call itself stubbed (`setApiCaller()`). What was NOT verified: the generated `handlers/GitHub.bx` against a real ColdBox boot (same category of gap as every other webhook gateway's own handler in this project, deferred for the identical reason). No real GitHub API call has ever been made, and no real GitHub App/PAT has ever been used against a real repository - do at least one real `bxAgents serve` + a real GitHub webhook configured against a test repository before depending on this gateway in production. `SignalGatewaySpec.bx` covers the gateway's own logic thoroughly and for real, not mocked: `handleSseEvent()`'s JSON-RPC/SSE parsing (blank-line/invalid-JSON handling, group-message filtering, quote-threading, `sourceUuid` fallback when a display name is absent), `deliver()`'s send shape and chunking, and HITL decision correlation/matching are all driven through the gateway's real public methods with only the outbound `rpcCaller`/`connector` I/O calls stubbed, the same seam-testing discipline as every other gateway in this project. Two BoxLang-level findings surfaced during development, both resolved and worth recording as general landmines rather than gateway-specific bugs: (1) a throwaway smoke-test script that named a local variable `request` was silently interacting with BoxLang's own reserved `request` scope instead of creating a plain variable, producing misleading "method not found"/"argument type mismatch" errors from `HttpClient.send()` that looked exactly like a genuine Java-interop limitation but disappeared entirely once the variable was renamed - `SignalGateway.bx` itself never had a bug; (2) a `try/catch` placed directly at the top level of a standalone `.bxs` smoke-test script (not inside a function) triggered a `java.lang.VerifyError: Inconsistent stackmap frames`, a real bytecode-verification limitation of BoxLang's top-level-script compiler, traced via the stack trace to the test script's own generated class, not `SignalGateway.bx` - fixed by wrapping the try/catch inside a named function instead. What was NOT verified, and is the largest gap among all the push-style gateways shipped so far: no real `signal-cli` daemon has ever been available in this environment, so the entire async SSE connection lifecycle - opening the stream via `HttpClient.sendAsync()`+`BodyHandlers.ofLines()`, the exponential reconnect-backoff loop against a genuinely flaky connection, the 30s/120s idle watchdog forcing a reconnect, and a live JSON-RPC round trip - has never been exercised end-to-end, only smoke-tested at the interop-plumbing level (a standalone test reached a real `java.net.ConnectException` against an unreachable address, proving the chain is sound, not that it works against a live daemon). Do at least one manual `bxAgents serve` against a project with a real, running `signal-cli` daemon and a real linked Signal account before depending on this gateway in production - this is a genuinely new transport architecture in this codebase (the only SSE-based gateway among Telegram/Slack/Discord/Email/Signal), not just a new platform on an already-proven transport shape. ## WhatsApp Personal (unofficial personal-account bridge) - researched, not built The original plan (matching Hermes Agent's own architecture) called for a `WhatsAppPersonalGateway` built by spawning a Node.js subprocess running `@whiskeysockets/baileys` (the multi-device WhatsApp Web protocol client Hermes itself uses - MIT licensed, its full `bridge.js` was read directly from Hermes's real source, not summarized). That approach was set aside mid-session on direct instruction to prefer a native BoxLang/JVM integration over a subprocess bridge, and to use a native Java library only if it's open source and neither GPL nor LGPL. That search found **Cobalt** (`com.github.auties00:cobalt`, formerly WhatsappWeb4j) - a real, MIT-licensed, actively maintained (900+ stars) Java implementation of WhatsApp's multi-device "Linked client" protocol, with a documented fluent API (`WhatsAppClient.builder().linkedApi().webClient()...`, `addNewMessageListener()`, `sendMessage()`) that would coerce cleanly from a BoxLang closure via BoxLang's own documented Java-SAM coercion (confirmed via the BoxLang docs MCP - no `createDynamicProxy()` needed for a single-abstract-method listener). Two real blockers surfaced during verification, not guessed: 1. **The first pom.xml read (Cobalt's `master` branch, an in-progress multi-module rewrite) requires Java 25** - two majors ahead of BoxLang's own documented baseline (Java 21+, confirmed via the BoxLang docs MCP and matching this project's own `21.0.10` JDK). Re-checking against the artifact actually published to Maven Central (`cobalt:0.0.10`, the real thing a `` would resolve today, not the unreleased rewrite) showed `21` - so the Java-25 finding was a false alarm caused by reading the wrong branch, not a real blocker. Worth recording as a caution: a GitHub repo's default-branch `pom.xml` is not necessarily what's on Maven Central. 2. **The real, published `cobalt:0.0.10` pulls in `com.aspose:aspose-words` as a hard compile-time dependency** (used internally for generating link-preview thumbnails from Word documents) - Aspose.Words for Java is commercial/proprietary-licensed, not open source, so bundling it would violate the same license constraint Cobalt itself was chosen to satisfy. The full dependency graph (~15 jars: zxing, qr-terminal, curve25519, protobuf-base, jackson or fastjson2 depending on version, libphonenumber, dd-plist, apk-parser, link-preview, jaffree, ez-vcard, slf4j, plus Aspose) would all need to be manually downloaded and bundled under this module's `libs/` folder - BoxLang modules have no Maven-style dependency resolution of their own (confirmed via the BoxLang docs MCP: third-party jars are bundled directly into a module's `libs/` folder, loaded by a per-module classloader - there's no `javaLibraries` key in `box.json` that would resolve a dependency tree automatically). Given the license contamination via Aspose and the manual-fat-jar-assembly effort with no dependency-resolution tooling available to verify the result actually loads, **WhatsApp Personal was descoped** rather than shipped as either a real gateway or a stub. `ProjectValidator`'s `validGatewayTypes` and `GatewayGenerator`'s `TYPE_CLASS_MAP` do not include a `whatsapp-personal` entry - a project that tries to declare one gets the existing "unknown gateway type" validation error, same as any other unsupported type, rather than a misleading half-built stub. Revisiting this is reasonable if Cobalt ever drops the Aspose dependency (it's a narrow feature - link-preview thumbnailing of Word docs - not core to messaging), or if a future session decides the Node/Baileys subprocess-bridge approach (rejected this round on architecture preference, not on a technical blocker) is preferable after all. ## `GatewaySession` is project-wide and root-agent-only (v1) A project with at least one push-style gateway entry gets exactly one generated `GatewaySession`, bundling every push-style gateway and always bound to the project's root agent - matching the existing precedent that `exposes: "agent"` HTTP exposure is also always root-agent-only (see [gateways/](conventions/gateways/index.md#3-push-style-gateways-type-telegram--slack--discord--email--whatsapp-cloud--teams--twilio--github--signal-and-friends)). **This is the architecture, not a gap.** A project has one root agent, and [`subagents/`](conventions/subagents.md) are delegated to *by that root* - the root decides what goes to whom. Every entry point (gateways, the HTTP agent route, the web UI, schedules) therefore lands on the root by design. Binding a gateway straight to a subagent would route around the agent that is supposed to be orchestrating it, which is why there is no per-gateway `targetAgent` key: to make "Telegram talks to support, Slack talks to research" work, give the root that routing logic and let it delegate, rather than splitting the front door. ## The docs site's nav *group* headings stay English in a translated locale Every nav entry backed by a real page takes its label from that page's own frontmatter `title`, so a translated tree gets translated nav labels for free - `Instalación`, `Inicio rápido`, `Introducción a BxAgents` and so on all render correctly under `/es/`. The four synthetic grouping nodes in `docs/nav.json` (`Getting Started`, `Conventions`, `Guides`, `Reference`, plus the `Example Agents` sub-group) have no page behind them, so there is nothing to read a translated title from. bx-sites v1 also hands every locale tree the *default* tree's own `navConfig` verbatim - by design, so a locale's nav always mirrors its default tree's structure - which means there's no per-locale `nav.json` to override them in either. They therefore render in English on every locale. Measured on a real build: the `/es/` sidebar renders `["Getting Started", "Instalación", "Inicio rápido", "Introducción a BxAgents", "Conventions", "Agent.bx", ...]`. Page labels localized, group headings not. Closing this needs an upstream bx-sites change (a per-locale nav override, or localizable group titles), not a change here. ## Fixed: `config/ColdBox.bx`'s own `interceptors = [...]` array never actually registered anything on a real boot - the web UI's login gate and API-key gate were both silently wide open A real, serious, previously-undetected bug, found only by driving the built web UI over genuine HTTP with real `cbauth`/ColdBox module dependencies for the first time (see the `bx-ai 3.4.0` entry below - `retest everything` after that update is what surfaced it). `config/ColdBox.bx`'s `interceptors = [{ class: "interceptors.X" }]` array - the documented ColdBox convention this project used to emit, and the ONLY place it registered `WebUiLoginGate`/`WebUiAuthGate`/`GatewaySessionBootstrap`/any project-declared interceptor - never reached ColdBox's `InterceptorService` at all on this stack. Confirmed directly: `ModuleService.cfc`'s own module-settings merge (a separate but related bug, see the `cbauth` entry below) and `ApplicationLoader.cfc`'s interceptor parsing both read the app's config CFC instance via `getPropertyMixin()`, a reflection helper that reaches into a CFC's own `variables` scope from outside it - and a direct test this session (`new config.ColdBox(); cfgClass.configure(); cfgClass.moduleSettings` immediately after) proved that scope is simply not reachable that way on this BoxLang/ColdBox combination, `variables.`-prefixed or not. `interceptors = [...]` in `config/ColdBox.bx` is dead code on this stack, full stop - not a web-UI-specific bug. The practical consequence: a `webui` entry configuring `users` or `apiKeyEnvVar` documented (see `docs/conventions/web-ui.md`) that doing so "turns on a real sign-in gate." It did not. Every `preProcess` interceptor silently never ran, so **every `/chat/api/*` route answered normally with zero session and zero API key**, on every project that had ever configured either gate - the exact opposite of the documented behavior, and undetected because nothing in the test suite drove a real login/API-key-gated request over actual HTTP; `WebUiRuntimeSpec.bx` only ever called `UserService.isValidCredentials()` directly (bypassing `authenticate()`, the interceptor, and the whole registration path), and `ColdBoxAppGeneratorSpec.bx`/`WebUiGeneratorSpec.bx` only ever asserted on the generated TEXT of the (dead) array, never that it worked. The same root cause meant the web UI's `afterConfigurationLoad`-based schema warm-up (`WebUiSchema.bx`) and the push-gateway starter (`GatewaySessionBootstrap.bx`, see the entry above) were ALSO dead - the former was harmless (`ChatDb.onDIComplete()` migrates on first use regardless, which is exactly what hid it), the latter meant **no push-style gateway (Telegram, Slack, Discord, Teams, WhatsApp Cloud, Twilio, GitHub, Signal) ever actually started** in a real boot, a second, independently serious consequence of the identical bug. Fixed by abandoning the declarative array entirely, following the precedent BuildPipeline.bx had already set for the account seeder (see its own comment: "as an interceptor it silently never fired"): - `WebUiLoginGate`/`WebUiAuthGate`/every project-declared interceptor are now registered for REAL from `Application.bx`'s `onApplicationStart()`, via `interceptorService.register( target:, name: )` - a plain function call against already-built objects, using `getMetadata()` reflection (which works), never `getPropertyMixin()` (which doesn't). See `ColdBoxAppGenerator.renderInterceptorRegistration()`. - `WebUiSchema`'s one-shot `afterConfigurationLoad` action and `GatewaySessionBootstrap`'s one-shot gateway-starting action are no longer interceptor files at all - both are now bare `onApplicationStart()` statements, the same category as the seed statement (`WebUiGenerator`'s `webUiWarmStatement`, `GatewayGenerator`'s `renderSessionStartupStatement()`). Verified for real, not just re-generated: rebuilt a throwaway project declaring `users:` through the actual (fixed) build pipeline, booted it with a real `boxlang-miniserver`, and drove it over genuine HTTP end to end - `GET /chat/api/health` unauthenticated → 200 (the health probe stays exempt, see below), `GET /chat/api/history` unauthenticated → 401 `{"error":true,"message":"Not signed in"}`, a wrong password → a clean 302 to `/chat/login.html?failed=1`, the correct password → a clean 302 to `/chat/` with a session cookie, and that cookie → 200 with real data on the same previously-open route. `WebUiRuntimeSpec.bx` now also calls `authenticate()` itself (not just `isValidCredentials()`) against both a wrong and a correct real password, and confirms `WebUiLoginGate` is genuinely registered as a `preProcess` listener via `InterceptorService.getStateContainer( "preProcess" ).exists( "WebUiLoginGate" )` - not `getInterceptor()`, which only resolves interceptors registered by class path and doesn't apply to `register()`'s object form used here, even though that form genuinely registers and fires (confirmed separately, over real HTTP). One more, smaller bug surfaced by the same real-HTTP pass, fixed alongside it: `event.noExecution()` alone does not stop ColdBox from still trying to render a view/layout - confirmed against real ColdBox 8.1.0 source, `Bootstrap.cfc` checks `getIsNoExecution()` (skips the handler) and `isNoRender()` (skips rendering) as two entirely separate flags. `ChatAuth.bx`'s hand-rolled `redirectTo()` and `WebUiLoginGate`'s own raw-redirect branch called only the former, so a plain redirect (no `renderData()` set) 500'd trying to render a default view that was never asked for, clobbering the intended 302 - invisible before this fix because the interceptor calling it never ran, and `ChatAuth.doLogin()`'s own `noExecution()`-only bug was masked by `authenticate()` throwing first for an unrelated reason (see below). Both now call `event.noRender()` too. ## Fixed: `moduleSettings.cbauth.userServiceClass` in `config/ColdBox.bx` never reached cbauth either - and the test harness's own pinned cbauth version was separately broken against real ColdBox 8.1.0 Two distinct, compounding bugs, both found only by calling `authenticationService@cbauth.authenticate()` for the first time against a real boot (previously, coverage stopped at `UserService.isValidCredentials()` - see the interceptor-registration entry above for why that leaves a real gap). **First bug (a version pin):** `tests/box.json` pinned `cbauth: "^5.0.3"` (resolving to `5.0.7`), whose `AuthenticationService.cfc` calls `interceptorService.processState( ... )` - a method ColdBox removed; real ColdBox 8.1.0's `InterceptorService` exposes only `announce()`. Every call to `authenticate()`/`login()`/`logout()` threw `Method 'processState' not found` before ever reaching credential checking. Confirmed directly against cbauth's real published source: cbauth `7.0.0` (ForgeBox's real "latest", confirmed reachable and resolvable) already calls `announce()` throughout - the fix is version currency, not a code change. `tests/box.json` now pins `cbauth: "^7.0.0"` (with its own `cbstorages: "^3.0.0"` dependency, `3.5.0`). `IUserService`/`IAuthUser` - the two interfaces this project's generated `UserService.bx`/`User.bx` implement - are byte-for-byte identical between cbauth `5.0.7` and `7.0.0`, confirmed directly, so no generator changes were needed for this half. A real end user running a bare `box install cbauth` today gets `7.0.0` and was never exposed to this specific bug; it was a test-harness-only gap, but a real one, since it meant the harness's own `authenticate()` path had never genuinely run end-to-end even once. **Second bug (config, not a version issue - see the interceptor entry above for the same root cause):** even with `announce()` working, `authenticate()` still threw `IncompleteConfiguration - No [userServiceClass] provided`, because `moduleSettings.cbauth` in `config/ColdBox.bx` - the ColdBox-documented way to override a module's settings, and what this generator used to emit - never reaches the module on this stack, for the identical `getPropertyMixin()` reflection gap described above. Fixed by using ColdBox's OTHER, genuinely-working per-module settings mechanism instead: a `config/modules/cbauth.bx` file, which `ModuleService.cfc`'s `loadModuleSettingsOverride()` invokes with a plain function call (`oConfig.configure( mConfig.settings )`, returning the merged struct) rather than reflection. See `ColdBoxAppGenerator.renderCbauthModuleOverride()`. Confirmed for real: `settings.modules.cbauth.settings.userServiceClass` resolves to `"UserService"` in a live boot, and a real sign-in against a real seeded account succeeds end-to-end (see the interceptor-registration entry's own real-HTTP verification, which exercises both fixes together). `qb`'s own `moduleSettings.qb.defaultOptions`/`defaultGrammar` carries the exact same root cause and is already documented, separately, below (`ChatDb.query()` was already worked around by never depending on it). `bxai`'s own `moduleSettings.bxai` (`skillsDirectory`, the `security` passthrough) was NOT specifically re-verified against a real boot this session - bx-ai is a BoxLang-native module, not a ColdBox one, so it is plausible (not confirmed either way) that it reads its settings through an entirely different, unaffected mechanism. Treat it as unverified, not as known-working, until someone checks. ## Fixed: `GatewaySessionBootstrap.bx` looked gateways up by the wrong `aiGatewayRegistry()` key A real, previously-shipped bug: the generated interceptor looked gateways up by the discovered `gateways/*` entry's own filename (e.g. `"telegramChannel"`), but bx-ai's `GatewayRegistry.register()` always keys by the gateway class's own fixed `getName()` (e.g. `"telegram"`) - never anything caller-supplied. This meant `GatewaySession` construction threw at ColdBox boot for every generated project with a push-style gateway; it shipped undetected because the only prior coverage asserted on generated file text, never against a live registry. Fixed in `GatewayGenerator.generate()`: the interceptor now looks gateways up by their TYPE string, deduplicated, with a permanent regression test (`GatewayGeneratorSpec.bx`) that registers a real gateway and proves the emitted key resolves it via a live `aiGatewayRegistry()`. **A real, permanent consequence this fix surfaces (still true today, not a bug):** because the registry is keyed by type, not by entry, two `gateways/*` entries of the same push-style type collide on the same registry slot project-wide - e.g. two `type: "telegram"` entries (two different bot tokens) would silently have the second registration overwrite the first, and `GatewaySession` would only ever see one of them. There's no per-entry alias/registration-name override today. One instance per push-style type, per project, is the actual v1 ceiling. ## Fixed: bx-ai's published snapshot used to lag its own `development` branch While bx-ai's `aiGatewayRegistry()`/`GatewaySession`/`BaseGateway` API and its userId/conversationId-scoped `IAiMemory.summarize()` (needed by `/compact`) were still unreleased, this project pinned `bx-ai@3.4.0-snapshot` - a continuously-republished ForgeBox artifact that repeatedly lagged bx-ai's real `development` HEAD by some amount of time. Three separate times, a fresh `downloadModules` fetched a snapshot predating the API this project's own generated code already called, breaking gateway boot (`Function 'aiGatewayRegistry' not found`), the whole gateway layer (missing `BaseGateway.bx`), or `/compact` (silently summarizing the wrong scope). CI worked around it by cloning and building bx-ai's `development` branch directly and overlaying it over whatever `downloadModules` fetched. bx-ai has since cut a real, stable `3.4.0` release carrying all of it - confirmed directly against the published ForgeBox artifact (`BaseGateway.bx`, `GatewaySession.bx`, `aiGatewayRegistry`/`aiGatewaySession` BIFs, `MockService.bx`, and `summarize( config, userId, conversationId )` are all present). `build.gradle` now pins that stable version instead of the snapshot, and the CI overlay step is removed - `downloadModules` alone is enough. ## The v1 web chat UI (`exposes: "webui"`) - what's real, and what could only be checked against docs, not a live server `WebUiGeneratorSpec.bx` and a `BuildPipelineSpec.bx` end-to-end test both drive the real `WebUiGenerator`/`BuildPipeline` classes against real fixtures: the static `/index.html` shell is confirmed to be written and correctly templated (`__API_BASE__`/`__APP_TITLE__` placeholders substituted, never left in the output), and the optional `interceptors/WebUiAuthGate.bx` is confirmed to be generated only when `apiKeyEnvVar` is configured, to gate exactly `/api/*` and never the bare `` shell itself. Registration is no longer a `config/ColdBox.bx` `interceptors:[...]` entry at all - that mechanism never reached ColdBox's `InterceptorService` on a real boot (see the dedicated "Fixed" entry below; this file used to claim the opposite here, which was itself part of the bug going undetected). `WebUiAuthGate`/`WebUiLoginGate` are now registered for real from `Application.bx`'s `onApplicationStart()`, and this **was** re-verified end-to-end over genuine HTTP this session (see that entry) - not just re-generated and re-asserted on as text. What was NOT verified - and couldn't be, in this dev environment: a real `bxAgents serve` + real browser test of the page actually loading, streaming a reply, and the `X-API-Key` gate actually rejecting/accepting requests over real HTTP. This project's own `runColdBoxIntegrationTests.bxs`/`tests/coldbox` harness (the same one that proved `toAi()`'s `/invoke` route works end-to-end for `http-gateway-agent`) requires `tests/coldbox` to be present via a real `box install` inside `tests/` - CommandBox and ForgeBox network access were both unavailable in this session's sandbox, so that harness could not be exercised for the web UI (or re-confirmed for anything else) this session. Two direct consequences: - The exact `/invoke` JSON response shape used by this project's own generated pages/docs (`{"input": "..."}` in, a response containing `"success": true`) is empirically confirmed **only** via `runColdBoxIntegrationTests.bxs`'s own prior, already-passing assertion (a substring check, not a full shape assertion) - not re-verified this session. - The `/stream` SSE wire format the web UI's own JS parses (`data: {"token":"..."}` lines, terminated by `data: [DONE]`) is taken directly from ColdBox's own official "AI Routing" documentation, not independently re-confirmed against a live server this session - unlike almost every other wire-format claim in this project's docs, which were cross-checked against real running code wherever possible (see e.g. the HMAC-SHA256/SHA1 signature schemes' independent Python/openssl cross-checks elsewhere in this file). Do at least one real `bxAgents serve` + a real browser test (message sent, reply streams in, the `X-API-Key` gate actually 401s a request missing the key) before depending on this feature in production - the same standing advice already given for every generated webhook handler's own untested-against-a-real-boot gap elsewhere in this file. ## The web UI's SQLite store - verified at the library level, not through a real ColdBox boot The qb + bx-sqlite stack underneath `models/ChatDb.bx` was verified directly this session against the real jars, not inferred from docs: qb 13.1.0's `.cfc` sources compile and run natively on BoxLang 1.16 with no `bx-compat-cfml`; `SQLiteGrammar` + `SchemaBuilder` really do create the v1 tables and indexes against a real SQLite file; a second migration pass is a clean no-op; `QueryBuilder` round-trips inserts, filtered/ordered reads and deletes; and the `preferences` composite primary key really does reject a duplicate `(userId, prefKey)`. Two constraints were found this way rather than assumed - qb requires a **named** datasource (its own `appendSqlComments()` types that argument as `string`, so an inline struct throws before any SQL runs), and `SchemaBuilder@qb` is mapped with its `grammar` argument only, never receiving `moduleSettings.qb.defaultOptions` - and the generated code is shaped around both. The three wiring assumptions this depends on - that `getInstance( "ChatDb" )` resolves through the generated app's own WireBox, that `SchemaBuilder@qb`/`QueryBuilder@qb` resolve once qb is installed as a real ColdBox module (qb is a `box.json` dependency, not vendored here - the same honest gap `cbmailservices` already carries), and that `this.datasources` in the generated `Application.bx` is picked up as expected - are now exercised end to end for the shared-preferences path: `runColdBoxIntegrationTests.bxs` writes and reads a preference through `POST`/`GET /chat/api/preferences` from outside the process (see below), which only works if all three hold. What is **not** yet exercised the same way is conversation CRUD, the cross-user guard, and the migration pass - those are proven at the library level and via `WebUiRuntimeSpec.bx` reading real WireBox objects inside a boot, but not driven from a second, independent HTTP caller the way the preferences path now is. Do one `bxAgents serve` against a webui project with `qb` and `bx-sqlite` genuinely installed before relying on the store in production; the failure mode if any of the three wiring assumptions is wrong is loud at boot (an unresolvable WireBox mapping or an unknown JDBC driver), not silent. ## The web UI over real HTTP: proven for `/health` and the shared-workspace preference routes `runColdBoxIntegrationTests.bxs` fetches `GET /chat/api/health` from outside the server, against the generated fixture app booted by a real `boxlang-miniserver`, and fails the build if it is not a 200. On CI it returns: ``` + App probe GET /chat/api/health -> status=200 body: {"success":true,"status":"ok"} ``` That request is the baseline end-to-end proof for the web UI's server side: ColdBox routing reaching `handlers/ChatUi.bx`, WireBox resolving the handler and `ChatDb`, `this.datasources` giving bx-sqlite a usable SQLite file, and qb being a genuinely activated ColdBox module - none of which any source-text assertion can establish. The same runner script goes further and drives two more routes over real HTTP against that same boot: `POST /chat/api/preferences/set` and `GET /chat/api/preferences`, called as two separate, cookie-less HTTP requests specifically to prove the accountless web UI's "one shared workspace" behavior - that a preference written by one caller is read back by a completely independent second caller, through `ChatDb`/SQLite, not just echoed from a session. `WebUiRuntimeSpec.bx` then covers the rest of the store's behaviour (migrations, scoped conversation CRUD, the cross-user guard) by reading real objects out of WireBox inside that same boot. **What is still not covered over real HTTP:** conversation list/history and SSE streaming. Driving those from `WebUiRuntimeSpec` is not possible as structured, because that spec runs *inside* a request served by the same single miniserver, so a loopback call would compete for the worker pool the runner itself occupies. Covering them needs either a second server process or a larger miniserver thread pool, and is a natural follow-up rather than something to fake. The full `/chat/login` round trip was ALSO driven over genuine HTTP this session (see the "interceptor registration" and "cbauth" `Fixed:` entries above for what it caught) - a wrong password redirecting cleanly to `/chat/login.html?failed=1`, a correct one to `/chat/` with a working session cookie, and that cookie then authorizing a previously-open route while a cookie-less request to the same route still 401s. That pass was manual and ad hoc (a throwaway project outside this repo, not `e2e-coldbox-route`), because the fixture's own `secureChat.bx` account uses a `passwordHash` with no known plaintext (deliberately, so CI needs no secret) - there is nothing to log in WITH over real HTTP without adding one. `WebUiRuntimeSpec.bx` now covers the same ground a different way instead: it calls `authenticationService@cbauth.authenticate()` directly (both a wrong and a real, freshly-seeded correct password) and confirms `WebUiLoginGate` is genuinely registered with `InterceptorService` - real coverage, permanently in the suite, just not over the wire. Extending `runColdBoxIntegrationTests.bxs` itself to drive `/chat/login` over HTTP (an env-var-backed account on the fixture, the same pattern `UserSeeder` already uses) is a reasonable follow-up, not done this session. Note that this paragraph previously cited `runner-coldbox.bxm never responded: HTTP 408` as evidence of that starvation. **That attribution was wrong**, and is worth recording because it misled this project for several CI cycles: the 408 had nothing to do with loopback calls. The runner page called `chr( 10 )` - a BIF that does not exist - so it returned 500 on entry to every request, and the orchestrator's retry loop ground through 90 seconds of those 500s before reporting the final timeout. Both are fixed. The loopback-starvation concern above is a genuine structural reason not to drive more routes from inside that spec, but it remains a well-founded expectation rather than something that has actually been observed failing here. ## The web UI's own front-end: driven in a real browser, but against a mocked API The shipped page's JavaScript was exercised for real this session, not just asserted against as source text: the generated `index.html` was loaded in headless Chromium and driven end to end with every `/api/*` route intercepted and answered with realistic payloads. Confirmed working that way - the conversation sidebar rendering from `GET /conversations` and switching/renaming/deleting through its own routes; `GET /info` shaping the toolbar (Compact appearing only when `capabilities.compact` is true, the model name landing in the header); the theme arriving from server-side `preferences` and applying; `/history` rehydrating the transcript; a real SSE turn streaming content, reasoning and tool-call chips out of the bx-ai envelope; markdown rendering; and **New chat** creating a conversation server-side and opening it. Zero JavaScript errors, zero unreplaced `__TOKEN__` placeholders, and the narrow-screen layout was screenshotted at 390px. What that does **not** prove: the API was a Playwright mock, not the generated `handlers/ChatUi.bx` running under a real ColdBox boot against a real SQLite store. The request and response shapes were taken from the generated handler's own source, so a drift between the two would not be caught by this. Everything in the known-limitations entry above about the store applies here too - one manual `bxAgents serve` against a webui project with `qb` and `bx-sqlite` genuinely installed remains the honest gate before production use.