Truth over time
Observed failures, typed diagnostics, and every number that can stop you.
Failure codes with their phases, the provider-effect rules that decide retry safety, and the hard limits of the public contract — sourced from the client README and the installed CLI.
Start with doctor#
Most local surprises are installation surprises. Run weles doctor and read the JSON report: ok, version, node, the bin map, and whether CHROMIUM_PATH or WELES_USE_STOCK_CHROMIUM are set. A wrong or missing managed browser build shows up here before any navigation fails.
managedRuntime distinguishes an absent local deployment from an unreadable revision marker. A file read or malformed revision produces ok:false with the marker path and actual cause in detail, and doctor exits nonzero. If WELES_API_DECLARED_REVISION names a deployment but its marker is missing, verification also fails. A caller with neither a marker nor a declared revision does not need a local managed browser. A matching marker verifies only recorded revision metadata, not browser or provider readiness.
Run node tests/diagnostics/managed-runtime.mjs with WELES_BIN naming the built CLI and WELES_CLI_SHA256 its expected artifact digest. The journey invokes the actual doctor command with an isolated HOME under build/real-tests, exercises missing, malformed, unreadable and matching revision markers, and verifies refusal does not change the file. It retains exact checkout revision, binary digest, commands, exits and reports, then removes only its own fixture home. Other doctor dependency failures remain in the report and are not qualified by this marker-specific journey. It starts no browser and changes no deployment.
Running without Stado, Brama or Skarbiec#
Before a browser starts, Weles asks Stado whether this host is the one the Weles service is placed on. Without Stado that check fails with "BLOCKED: WSession.start cannot verify Weles placement through Stado: <error>. On a machine without Stado set WELES_STANDALONE=1."
Set WELES_STANDALONE=1 on a machine that runs Weles on its own: the placement check is skipped and Weles says so on stderr ("WELES_STANDALONE=1, so this host runs Weles without Stado and its placement is not checked"). It is never assumed from a missing Stado, because a placed host that lost Stado must stop rather than start browsers elsewhere.
Model decisions and page questions go to Brama. Without Brama they fail with "missing required STADO_MODEL_ROUTER_URL; without Brama set WELES_MODEL_ENDPOINT and WELES_MODEL to an OpenAI-compatible provider". Set WELES_MODEL_ENDPOINT to that provider's origin (HTTPS, or HTTP on loopback), WELES_MODEL to the provider's model id, and WELES_MODEL_KEY when it needs a key. Weles then sends /v1/chat/completions there unsigned, uses that one model for text and screenshots (so pick an image-capable one), and runs native Jeden tasks with JEDEN_MODEL_ENDPOINT and JEDEN_MODEL_KEY. The keyword planner's model calls follow the same settings.
Every account password Weles generates (registration, GitHub reset, Microsoft and Entra rotation, Oxylabs proxy users) takes one stated length: the vault policy's min_generated_length, set with skarbiec policy-set min_generated_length <length>. Without it generation fails with "Weles generates account passwords at the length the vault policy states, and it states none". Without Skarbiec (WELES_CREDENTIALS_FILE set) the length is WELES_GENERATED_PASSWORD_LENGTH, a whole number above zero, and a missing or malformed value is named ("WELES_GENERATED_PASSWORD_LENGTH is not set: without Skarbiec declare how long a generated account password is"). A length shorter than the four character classes is refused. A provider narrows only which symbols are used; Oxylabs takes _ ~ + = only.
Account logins, proxy credentials and API keys come from Skarbiec. Without it a read fails with "WC_SKARBIEC_URL is required from the Stado service directory; without Skarbiec set WELES_CREDENTIALS_FILE to an owner-only credentials file". Set WELES_CREDENTIALS_FILE to a JSON file of the form {"<item>": {"<field>": "<value>"}}, using the same item and field names Skarbiec uses (for example weles-gmail-login with username and password). The file must be a regular file owned by the user running Weles with mode 600, or every read is refused. Weles then reads and writes credentials there and asks Skarbiec nothing; a credential the file does not hold is absent, and the browser trajectories name the missing item and field ("WELES_CREDENTIALS_FILE <path> has no <item>/<field>"). A tenant's credentials still come only from Skarbiec: a tenant read with the file set is refused.
With Skarbiec, a trajectory reads its service credential with the workload identity of the run: a process without SKARBIEC_WORKLOAD_ID and SKARBIEC_WORKLOAD_SIGNING_KEY_FILE is refused with "Skarbiec workload identity is required to read <service>/<field>", and a refused acquisition repeats Skarbiec's own words with the record [skarbiec_acquisition item=role:<role> field=<field> consumer=<consumer> reason=<reason>], which the run's failure reports as skarbiec_acquisition_failed. A write to a service that declares no writer identity is refused before anything is sent; a write Skarbiec refuses names how stado secrets put ended and what it said.
WC_SKARBIEC_URL must be an HTTPS origin, or plain HTTP to this machine's own loopback (localhost or an address of one of its internal network interfaces), with no credentials, query, fragment or path. A tenant's Skarbiec binding is read from ~/.stado/weles-skarbiec-tenants/<tenant id>, where Stado lays it out; a tenant id that is not a UUID, and a binding directory or file anyone but its owner could use, is refused naming the path. A reader or acquisition helper that could not run, exited with a status or was ended by a signal is refused with how it ended and what it wrote to standard error.
Worker API refusals#
POST /weles-builder starts its agent on the neutral page the deployment declares in WELES_BUILDER_BOOTSTRAP_URL; without it the route answers Internal Server Error with missing_WELES_BUILDER_BOOTSTRAP_URL and starts nothing. Its body is the goal as text, or JSON whose string member instructions holds it; a body that begins like JSON but does not parse, or carries no string instructions, is refused with instructions_unreadable and the reason. POST /reauth and POST /reauth/enrol-authenticator share one admission: Brama's reauth token, a provider Weles runs, an exact Skarbiec subscription id, the Skarbiec account it resolves to (a lookup refusal keeps its own code and detail) and, when the request names account_revision, the same revision Skarbiec holds now (skarbiec_identity_changed otherwise). An enrolment that is not confirmed names what the run stated blocked it, or authenticator_enrolment_unconfirmed with how the run ended. A failure of any route that is not one of its own refusals is answered as Internal Server Error with that failure's message, public task operations included.
How many steps a browser task may take#
A generic browser task ends on the model's own done or fail. How many model steps it may take before Weles stops it is the deployment's to declare in WELES_AGENT_MAX_STEPS, a whole number above zero; Weles chooses none. Without it the task fails before the browser is driven with "WELES_AGENT_MAX_STEPS is not set: declare how many model steps one browser task may take", and a value that is not a whole number above zero is named the same way. A task that reaches the budget fails with "browser agent exceeded <n> steps" and keeps its action history.
Client-side validation refusals#
The client refuses requests before any network traffic when input keys match password, secret, token, cookie, authorization, or proxy-auth patterns. The single exception is resumeToken / resume_token, the approval continuation handle, which is still redacted from every error detail. Key-name matching is not semantic inspection: keep credential material out of input even under innocuous names.
Secret-bearing provider response diagnostics#
Slack bot creation logs the HTTP status, provider ok/error fields and app identifier rather than serializing the complete manifest response. The same summary accompanies a refused creation or a response missing app_id. The page-response observer logs the origin, path and HTTP status, not the response body or URL query. This keeps returned app credentials out of those diagnostics while retaining the operation and provider refusal.
Slack token diagnostics describe lengths. Tencent captcha solution diagnostics describe both ticket and randstr by length without printing either value. These log summaries do not prove that the Slack installation, message delivery or captcha challenge succeeded; inspect the corresponding operation result.
Capture diagnosis evidence that could not be read#
A capture's diagnosis asks for four evidence sources: video frames cut from the recording by ffmpeg, the console log, the recorded network responses and the DOM snapshot. Every source it was given and could not read is kept, not dropped: unread_evidence lists the source name, the path it was looked for at and the cause — ffmpeg's exit and standard error for the recording, the operating system's read error for a file, 'no file at this path' for a named file that is absent, and the JSON type the responses file holds when it is not an array of response records. The list travels in the model request's input, so the model knows what it did not see, and in the returned diagnosis.
A diagnosis that could not be produced says which step failed. 'Diagnosis unavailable: the Jeden run failed: <cause>' is the Jeden route's own failure. 'Diagnosis unavailable: model output failed schema validation: <check>; reply: <part>' names the first output_schema check the reply failed (no JSON object, an object that does not parse, summary not a string, errors, anomalies or next_steps not an array or holding a value that is not a string) and the part of the reply that failed it. Both are followed by one 'unread evidence: <source> at <path>: <cause>' line per unreadable source.
Run node tests/capture/diagnosis-unread-evidence.mjs after building the Weles revision. It drives the built diagnosis with an empty recording, a directory in place of the console log, a responses file holding an object and an absent DOM path, really calls Jeden, and checks that whichever outcome comes back carries all four sources with their paths and causes; it then checks every parser refusal against literal replies. The report is retained under build/real-tests/capture-diagnosis.
Large instrumentation artifacts#
Activity and final instrumentation dumps stream their containers directly to disk, without first constructing a complete JSON string. Individual strings are escaped in chunks sized from the runtime string limit. Supplementary Unicode characters remain intact, sparse array positions become null, and each member's toJSON conversion is called once. A circular value remains a serialization error, not a successful dump.
Run node --test tests/instrumentation/json-file.mjs after building the corresponding Weles revision. The journey writes actual artifact files, reads back their content, checks Unicode and array preservation and verifies the circular-value refusal. It retains the source revision, command, exit status and files under build/real-tests/instrumentation. A passing small artifact journey does not independently qualify a long browser run or the browser's final close.
Typed failure codes#
| Code | Phase | Meaning |
|---|---|---|
transport-failed | transport | Ambiguous transport outcome; the library does not retry. Reconcile with an approved status-read channel using the original idempotency key. |
WELES_ENDPOINT_UNRESOLVED | admission | The Stado forward file is missing, unsafe, or holds an unusable URL; the bridge never falls back to an environment variable. |
DIRECTORY_EXPECTATION_MISMATCH | admission | The directory cross-check flags disagree with the sealed identity; refusal happens before submit. |
ENTRA_IDENTITY_CONTRACT_MISMATCH | admission | The directory block does not match the pinned Entra identity for the item, field, or provider; nothing is sent to Weles. |
ENTRA_IDENTITY_MISMATCH | identity_verification | The signed-in identity does not carry the expected tenant tid, principal oid, and UPN; the change stops before it happens. |
needs_human_approval | varies | MFA/passkey challenges and unknown-current-password resets pause with an approval resource; resume with skarbiec credential resume --approval <id> --resume-token <token>, never a repeated submit. |
A worker refuses its native runtime#
The startup refusal names native/jeden/bin/jeden or native/jeden/bin/jeden-sandbox-helper and the failed --version operation. A missing file is reported by path; execution failures retain the operating-system error or observed exit status and signal. The worker does not acquire credentials or start its API after this refusal.
The release launcher derives a cache with missing or non-executable native files again from its immutable payload. If the payload is incomplete, it refuses instead of marking the runtime ready. Repair the product release input and publish a complete worker; a different Jeden on the host or a manual service restart does not satisfy this dependency.
The launcher derives runtime files under $HOME/.stado/var/weles/runtime/<payload-sha256>, outside Stado's immutable installation. First and repeated startup must leave the installed archive unchanged; do not weaken Stado's archive comparison or delete an installed version to force a retry.
Both publishers derive the public surface from the source they package. The worker payload carries released-surface.json; weles release adopt-baseline records that published surface and version for the next release decision.
A page question fails after worker startup#
A PageQuestionError names the unanswered question and the failed image request. Transport refusals name the endpoint that was asked (Brama, or WELES_MODEL_ENDPOINT when it is set), the chat-completions POST, the model and the underlying cause; Weles sets no limit on how long the request may take. The browser loop stops on this error, including during cached replay; it must not accept a later done action or save the failed attempt as a successful flow.
Since Weles 0.7.3, screenshot bytes go through Brama's best selector using the existing Weles bearer and signed identity. Brama selects an image-capable model from that caller's authorized subscription pool; text-only decisions still use the weles alias. Weles infers PNG, JPEG, GIF or WebP from the bytes rather than the artifact filename and does not re-encode the image. A page question does not start a native agent to open a local file. Unreadable or unsupported image input is refused.
Since Weles 0.7.6 a page question carries only the actual screenshot, not the complete HTML document, which could overwhelm the request and expose content outside the visible page. If the screenshot does not show the requested detail, the answer must say so. A read action cannot click, scroll or navigate; those operations need their own browser actions and recorded outcomes.
A screenshot question states no output-token allocation of its own: the routed model's own default applies. The vision record retains the model, the finish reason and usage when returned, the function the model called, and the answer or error. An empty answer still fails even when the provider reports finish_reason=stop; the error names the model and quotes the provider response.
A provider_failure saying chat-primary is not a multimodal model was the Weles 0.7.2 failure: it sent images to the text-only route behind weles. This was not a local-file permission problem and no OCR process or worker restart repaired it. Current image requests use the already authorized best selector, not a hardcoded provider or a new account. If that caller has no active image-capable model, Brama refuses the request and the browser run remains failed. Weles does not substitute another caller identity or add a subscription.
Successful flow caches use weles.successful-flow.v2 and contain only successful steps. Flows containing fill_credential are not saved or replayed: their opaque capabilities are single-use, not reusable login instructions. Older caches are ignored because they could include discarded failures or spent capabilities. The next authorized run discovers its path again. Use generic_keeper_task when the objective requires fresh observation rather than cached replay.
In Weles 0.7.0, TypeScript callers of replayFlow must first narrow result.success. Success can contain value; failure names failedAtStep and can contain error. Both value and error are unknown and require type checks before use. An unfinished replay is a failure, not an empty success.
An observed control cannot be clicked#
Weles 0.7.5 accepts the complete control description from its current page observation as click(target), with or without the leading [index]. Preserve the tag and every reported field. Include the index when otherwise identical controls need distinguishing; an index alone is not a target. Weles checks the current description and binds the matching DOM node before moving the pointer. A later DOM replacement cannot redirect that handle. This path does not ask an image model to rediscover an already identified control.
The action history reports observed_target_stale when the complete description no longer matches, observed_target_ambiguous when an unindexed description matches multiple controls or a target matches multiple frames, and observed_target_invalid for an invalid index. These targets are refused before pointer input; Weles does not try another control or an image guess. Observe again and choose an unambiguous current target. A failure to read a page or frame is included in the observation with its cause.
Weles 0.7.7 refuses an empty or whitespace-only click or fill target with target_empty before input, so an empty target never selects an unrelated control such as the site logo. A missing required string argument is an invalid-argument error, not an empty target or permission to select the first control.
An explicit selector must match exactly one visible control across the current frames. selector_target_absent, selector_target_ambiguous and selector_target_invalid refuse before pointer input; Weles does not substitute an image guess or a shortened selector.
The shared rowShowingSource table reader accepts a complete cell value or the page's ellipsis-terminated prefix, after whitespace normalization. It returns a row only when that display matches uniquely. row_source_ambiguous includes the requested value and every matching row's displayed text; it refuses before the NCBR result-indicator editor opens a row menu. A missing row keeps the caller's existing not-found refusal. Read a more specific offered identifier or repair the reviewed selector; never resolve ambiguity by choosing the first row.
Checking table-row selection#
On the Stado-selected dedicated Weles host, run node --test tests/trajectories/row-selection.mjs from the Weles checkout after building its managed session library. The test refuses standalone mode, an account-bound process, an existing worker action-log ID and uncommitted reader or test changes. It opens its own browser profile and never attaches to an operator tab.
The journey evaluates the real shared reader against tables rendered in the managed browser. It covers full and whitespace-normalized names, both ellipsis forms, unmarked prefixes, repeated cells in one row and ambiguous rows. It retains the source revision, session-module hash, host, command, actual exit status, observations, screenshots and managed recordings under build/real-tests/row-selection. It exercises the table reader only; it neither edits an NCBR application nor proves the complete result-indicator workflow.
A browser decision or outcome cannot be verified#
Weles 0.7.7 provides the actual browser functions, including click, read and fill_credential, through Brama's existing tools interface and requires one declared function call. Each function declares its own argument names and types. Arguments belong directly in that function call, not in a nested tool or args wrapper. Missing required argument names, unknown names, malformed JSON, undeclared functions and multiple calls are refused. Router records retain the returned function name, finish reason and token usage. Weles does not repair truncated JSON by adding missing text.
Generic browser drafts are passed to the executor, not merely recorded as metadata. They remain subordinate to the supplied objective, constraints and current page. Existing-account sign-in uses only supplied scoped credentials; it does not authorize a generated replacement or invented login details. Completed navigation and prefills seed the executor's actual action history before its first decision. Normal results retain these non-secret receipts in initialization_history separately from subsequent history; consumed capability references are not included. Prefilled runs neither replay nor save a flow cache. Fields absent at prefill time retain their unspent capability. Navigation can clear a filled field; if no usable capability remains, stop rather than replaying a spent one.
Normal generic_keeper_task results retain the action history on success as well as failure. Privacy-limited results still use their restricted payload. Check the recorded actions, their actual results or errors, and the final browser URL. A model's thought or done text is not evidence that a click, scroll or navigation happened.
A credential field is refused#
Each API launcher owns a private broker socket below $HOME/.stado/run/weles-api-*/ and passes its exact path to its browser children. A second launcher or an older unit cannot unlink that socket while losing the API-port race. The capability file and origin routes remain shared with the normal Stado issuer. A broker transport refusal names the socket that the failed fill actually tried; an answering HTTP API alone does not prove credential execution works.
CAPABILITY_TRANSPORT_FAILED preserves the failed redeem or cancel operation, the exact socket path and the operating-system cause. The browser loop stops on this error, including cached replay, rather than trying different field descriptions until its step limit. A connection refusal does not establish whether an earlier request consumed a capability; reconcile the retained operation before retrying.
CAPABILITY_DENIED means the broker answered but refused the exact capability. The run stops at that refusal; it does not try the same capability again or accept a later done action. The refusal alone does not distinguish an expired capability from an already consumed one. Reconcile the preceding operation and its retained result before requesting another authorized run.
fill_credential resolves one visible, editable field on the capability's exact origin before redemption and keeps that same page and element handle through input. It verifies the resulting value inside the executor without returning the credential. CREDENTIAL_FILL_FAILED stops the run when redeemed material is empty, the bound page or field changes, input fails, or the field does not retain the supplied value. A missing field before redemption leaves the capability unspent; it is not a successful fill.
When an OAuth popup or navigation arrives while Weles is choosing an action, Weles discards the old-page decision and observes the new page before further input. It does not repeat the login button against a stale observation. A completed prefill is removed from the available inputs; navigation that clears it does not make its single-use capability reusable.
credential_target_mismatch means the field fill_credential resolved does not declare the requested class in its own markup. It names the field's input type and autocomplete tokens and the ones the class needs: a password, token or API key goes only into a masked input (type password), a token also into an input declaring autocomplete one-time-code, and an email or username into an input of type email, a WHATWG autocomplete of email, username or nickname, or a plain text input. The check reads the page's markup, not the words of its label, so a label in any language passes and a comment box labelled "password" does not. It is made before the capability is redeemed.
credential_target_unusable means the resolved field is hidden or not editable. Point the target at the field the page actually shows. Supply target, field_class and the complete issued capability object; never alter its id, purpose, resource or target to work around a refusal. fill_identity applies the same markup check to the generated password and fills the persona's other details by description.
Literal type_text, fill and set_control inspect the actual focused or resolved field, including its type, autocomplete and label, before writing. A selector such as #identifierId does not bypass the credential-field check. The refusal says credential fields require fill_credential; use the already authorized opaque credential or generated-identity operation rather than an invented value. The field's existing value is not read into the diagnostic or changed by the refused operation.
The existing origin, field-presence and scoped-capability checks still apply. Credentials remain inside the executor's secret boundary. A successful fill does not prove that sign-in completed; verify the resulting account and page through the authorized workflow.
Provider effect decides retry safety#
"We did not observe a change" is not "nothing changed": a failed operation whose effect cannot be read reports unknown, never none. Rollback status is reported separately as none, completed, failed, or unknown.
| Value | Retry policy |
|---|---|
none | Nothing changed provider-side; automatic retry is permitted. |
changed | A change was observed; run verify or confirm a rollback before any retry. |
unknown | Effect absent or malformed; never retried automatically — the item is quarantined rather than risked against a password that may already have changed. |
Limits#
| Limit | Value |
|---|---|
approval_id length | at most 64 characters of [A-Za-z0-9._-] |
resume_token length | at most 128 characters of [A-Za-z0-9._-] |
Diagnostic message length | at most 512 characters, no control characters |
execution_host length | at most 128 characters of [A-Za-z0-9._-] |
action_log_id length | at most 200 characters of [A-Za-z0-9._-] |
request_id / evidence_digest | 64 hexadecimal characters |
Bridge request schema | skarbiec.credential-operation.v3 only; v1 and v2 rejected with no alias |
Bearer scopes | task create, read, and cancel only |
Plaintext HTTP endpoints | loopback hosts only (the IPv4 and IPv6 loopback addresses and localhost) |
Evidence retrieval | not exposed by the current API |
Automatic retry | intentionally absent everywhere in the contract |
Reporting a failed run#
Open an operational defect on the repository issue tracker with the action-log row ID, worker instance ID, and release coordinate. Never attach recordings or credentials to an issue — see Support.