fix(codemode): align promise combinators with Test262

This commit is contained in:
Aiden Cline 2026-07-08 17:51:41 -05:00
commit 314388f46e
5 changed files with 397 additions and 52 deletions

View file

@ -121,9 +121,14 @@ closing. Timeout and external interruption cancel immediately instead.
### Confirmed defects ### Confirmed defects
- [ ] Return rejected promises for invalid `Promise.all`/`allSettled`/`race` inputs instead of throwing during the call.
- [ ] Align handler callability with the values CodeMode reports as functions, or document the narrower callback - [ ] Align handler callability with the values CodeMode reports as functions, or document the narrower callback
allowlist. For example, unsupported constructor-like callables are currently treated as absent handlers. allowlist. For example, unsupported constructor-like callables are currently treated as absent handlers.
- [ ] Run synchronous Promise reactions to completion. Reactions currently start in FIFO order but can interleave when
Effect auto-yields a long synchronous handler.
- [ ] Give `Promise.race` settlement the same reaction turn as native promises. Its result currently settles before an
independently queued reaction in several Test262 ordering cases.
- [ ] Drain queued reactions before classifying rejected sources as unhandled. A handler attached by an already-queued
reaction can currently lose to execution-boundary rejection reporting.
### Deliberate deviations and open decisions ### Deliberate deviations and open decisions
@ -152,6 +157,7 @@ closing. Timeout and external interruption cancel immediately instead.
- Promise reactions and plain, pending, or settled `await` continuations start in deterministic FIFO order. Nested - Promise reactions and plain, pending, or settled `await` continuations start in deterministic FIFO order. Nested
reactions preserve enqueue order, while an async reaction can suspend without blocking the next queued reaction. reactions preserve enqueue order, while an async reaction can suspend without blocking the next queued reaction.
- Awaiting the same promise twice settles it once. - Awaiting the same promise twice settles it once.
- Sparse combinator inputs consume holes as explicit `undefined` values.
### Missing coverage ### Missing coverage
@ -182,7 +188,7 @@ represent accurately rather than guessing semantics.
| Keep an owned tree-walking interpreter. | The product need is bounded tool orchestration, not arbitrary JavaScript. Owning the language surface keeps authority and behavior explicit. | | Keep an owned tree-walking interpreter. | The product need is bounded tool orchestration, not arbitrary JavaScript. Owning the language surface keeps authority and behavior explicit. |
| Treat schemas as the model-facing interface. | Signatures drive correct calls; Effect Schema also provides the runtime validation boundary, while JSON Schema supports adapter interoperability. | | Treat schemas as the model-facing interface. | Signatures drive correct calls; Effect Schema also provides the runtime validation boundary, while JSON Schema supports adapter interoperability. |
| Keep authority host-owned. | CodeMode can only confine programs to supplied tools. The host chooses those tools, and each tool enforces its own authorization and side-effect policy. | | Keep authority host-owned. | CodeMode can only confine programs to supplied tools. The host chooses those tools, and each tool enforces its own authorization and side-effect policy. |
| Use progressive catalog disclosure plus search. | Large tool sets should not consume the prompt, but every namespace must remain discoverable and speculative search calls should remain valid. | | Use progressive catalog disclosure plus search. | Large tool sets should not consume the prompt, but every namespace must remain discoverable and speculative search calls must remain valid. |
| Start tool promises eagerly and supervise them. | This preserves normal call-time parallelism while giving each call run-once settlement and interruption safety. | | Start tool promises eagerly and supervise them. | This preserves normal call-time parallelism while giving each call run-once settlement and interruption safety. |
| Keep files outside the sandbox value space. | Models should compose structured data without routing binary payloads through generated code or context. | | Keep files outside the sandbox value space. | Models should compose structured data without routing binary payloads through generated code or context. |
| Treat `execute` as the model-facing invocation boundary. | Nested calls are implementation details of one orchestration program. Reusing the outer context and bounding only the final result preserves complete intermediate data without inventing durable child-call identities. | | Treat `execute` as the model-facing invocation boundary. | Nested calls are implementation details of one orchestration program. Reusing the outer context and bounding only the final result preserves complete intermediate data without inventing durable child-call identities. |

View file

@ -2270,11 +2270,17 @@ class Interpreter<R> {
}) })
} }
const items = Array.isArray(args[0]) ? args[0] : spreadItems(args[0]) // Promise combinators consume arrays through their iterator, which visits holes as
// explicit undefined values instead of preserving sparse positions like Array.map.
const items = Array.isArray(args[0]) ? [...args[0]] : spreadItems(args[0])
if (items === undefined) { if (items === undefined) {
throw new InterpreterRuntimeError( return this.createPromise(
`Promise.${ref.name} expects an array of promises or plain values (e.g. Promise.${ref.name}(items.map((item) => tools.ns.tool(item)))).`, Effect.fail(
node, new InterpreterRuntimeError(
`Promise.${ref.name} expects an array of promises or plain values (e.g. Promise.${ref.name}(items.map((item) => tools.ns.tool(item)))).`,
node,
).as("TypeError"),
),
) )
} }

View file

@ -0,0 +1,334 @@
/*
* Portions adapted from Test262 at revision 250f204f23a9249ff204be2baec29600faae7b75:
* - test/built-ins/Promise/all/S25.4.4.1_A1.1_T1.js
* - test/built-ins/Promise/all/S25.4.4.1_A2.1_T1.js
* - test/built-ins/Promise/all/S25.4.4.1_A2.3_T1.js
* - test/built-ins/Promise/all/S25.4.4.1_A2.3_T2.js
* - test/built-ins/Promise/all/S25.4.4.1_A2.3_T3.js
* - test/built-ins/Promise/all/S25.4.4.1_A7.1_T1.js
* - test/built-ins/Promise/all/S25.4.4.1_A8.2_T1.js
* - test/built-ins/Promise/all/S25.4.4.1_A8.2_T2.js
* - test/built-ins/Promise/all/iter-arg-is-number-reject.js
* - test/built-ins/Promise/all/resolve-non-thenable.js
* - test/built-ins/Promise/allSettled/is-function.js
* - test/built-ins/Promise/allSettled/returns-promise.js
* - test/built-ins/Promise/allSettled/resolves-empty-array.js
* - test/built-ins/Promise/allSettled/resolves-to-array.js
* - test/built-ins/Promise/allSettled/resolved-all-fulfilled.js
* - test/built-ins/Promise/allSettled/resolved-all-mixed.js
* - test/built-ins/Promise/allSettled/resolved-all-rejected.js
* - test/built-ins/Promise/allSettled/iter-arg-is-number-reject.js
* - test/built-ins/Promise/allSettled/resolve-non-thenable.js
* - test/built-ins/Promise/race/S25.4.4.3_A1.1_T1.js
* - test/built-ins/Promise/race/S25.4.4.3_A7.1_T1.js
* - test/built-ins/Promise/race/S25.4.4.3_A7.1_T2.js
* - test/built-ins/Promise/race/S25.4.4.3_A7.1_T3.js
* - test/built-ins/Promise/race/S25.4.4.3_A7.2_T1.js
* - test/built-ins/Promise/race/S25.4.4.3_A7.3_T1.js
* - test/built-ins/Promise/race/S25.4.4.3_A7.3_T2.js
* - test/built-ins/Promise/race/iter-arg-is-number-reject.js
* - test/built-ins/Promise/resolve/S25.4.4.5_A1.1_T1.js
* - test/built-ins/Promise/resolve/S25.4.4.5_A2.1_T1.js
* - test/built-ins/Promise/resolve/resolve-non-obj.js
* - test/built-ins/Promise/resolve/resolve-non-thenable.js
* - test/built-ins/Promise/reject/S25.4.4.4_A1.1_T1.js
* - test/built-ins/Promise/reject/S25.4.4.4_A2.1_T1.js
* - test/built-ins/Array/from/from-array.js
* - test/language/statements/async-function/declaration-returns-promise.js
* - test/language/statements/async-function/evaluation-body-that-returns.js
* - test/language/statements/async-function/evaluation-body-that-returns-after-await.js
* - test/language/statements/async-function/evaluation-body-that-throws.js
* - test/language/statements/async-function/evaluation-body-that-throws-after-await.js
*
* Copyright 2014 Cubane Canada, Inc. All rights reserved.
* Copyright (C) 2016 the V8 project authors. All rights reserved.
* Copyright (C) 2019 Leo Balter. All rights reserved.
* Copyright (C) 2018 Rick Waldron. All rights reserved.
* Copyright 2015 Microsoft Corporation. All rights reserved.
* Copyright 2016 Microsoft, Inc. All rights reserved.
* Test262 portions are governed by the BSD license in LICENSE.test262.
*/
import { describe, expect, test } from "bun:test"
import { Effect } from "effect"
import { CodeMode } from "../src/index.js"
const value = async (code: string) => {
const result = await Effect.runPromise(CodeMode.execute({ code, tools: {} }))
if (!result.ok) throw new Error(`expected success, got ${result.error.kind}: ${result.error.message}`)
return result.value
}
describe("Test262 Promise static adaptations", () => {
test("Promise statics are callable", async () => {
expect(
await value(
`return [typeof Promise.all, typeof Promise.allSettled, typeof Promise.race, typeof Promise.resolve, typeof Promise.reject]`,
),
).toEqual(["function", "function", "function", "function", "function"])
})
test("invalid combinator inputs return rejected TypeError promises", async () => {
expect(
await value(`
const observe = async (run) => {
let returned = false
try {
const promise = run()
returned = promise instanceof Promise
await promise
return [returned, "fulfilled"]
} catch (error) {
return [returned, error.name]
}
}
return await Promise.all([
observe(() => Promise.all(42)),
observe(() => Promise.allSettled(42)),
observe(() => Promise.race(42)),
])
`),
).toEqual([
[true, "TypeError"],
[true, "TypeError"],
[true, "TypeError"],
])
})
test("Promise.all returns a promise and creates a fresh empty array", async () => {
expect(
await value(`
const input = []
const promise = Promise.all(input)
const result = await promise
return [promise instanceof Promise, result instanceof Array, result.length, result !== input]
`),
).toEqual([true, true, 0, true])
})
test("Promise.all preserves values and input order", async () => {
expect(
await value(`
const first = { id: 1 }
const second = { id: 2 }
const result = await Promise.all([Promise.resolve(3), first, Promise.resolve(second)])
return [result.length, result[0], result[1] === first, result[2] === second]
`),
).toEqual([3, 3, true, true])
})
test("Promise.all adopts rejection from either input position", async () => {
expect(
await value(`
const observe = async (promise) => {
try { await promise; return "fulfilled" } catch (reason) { return reason }
}
return await Promise.all([
observe(Promise.all([Promise.reject(1), Promise.resolve(2)])),
observe(Promise.all([Promise.resolve(1), Promise.reject(2)])),
])
`),
).toEqual([1, 2])
})
test("Promise.all treats sparse positions as undefined values", async () => {
expect(
await value(`
const input = []
input[1] = 1
const result = await Promise.all(input)
return [result.length, result[0] === undefined, result[1]]
`),
).toEqual([2, true, 1])
})
test("Promise.allSettled returns a promise and a fresh array", async () => {
expect(
await value(`
const input = []
const promise = Promise.allSettled(input)
const result = await promise
return [promise instanceof Promise, result instanceof Array, result.length, result !== input]
`),
).toEqual([true, true, 0, true])
})
test("Promise.allSettled preserves order and settlement shapes", async () => {
expect(
await value(`
const reason = { id: 4 }
const result = await Promise.allSettled([
Promise.resolve(1),
Promise.reject(2),
3,
Promise.reject(reason),
])
return [result, result.map((item) => Object.keys(item))]
`),
).toEqual([
[
{ status: "fulfilled", value: 1 },
{ status: "rejected", reason: 2 },
{ status: "fulfilled", value: 3 },
{ status: "rejected", reason: { id: 4 } },
],
[
["status", "value"],
["status", "reason"],
["status", "value"],
["status", "reason"],
],
])
})
test("Promise.allSettled preserves input order across settlement timing", async () => {
expect(
await value(`
const slow = async () => { await Promise.resolve(); return "slow" }
return await Promise.allSettled([slow(), Promise.resolve("fast")])
`),
).toEqual([
{ status: "fulfilled", value: "slow" },
{ status: "fulfilled", value: "fast" },
])
})
test("Promise.allSettled adopts a non-thenable object", async () => {
expect(
await value(`
const object = { id: 1 }
const result = await Promise.allSettled([object])
return [result, result[0].value === object]
`),
).toEqual([[{ status: "fulfilled", value: { id: 1 } }], true])
})
test("Promise.allSettled treats sparse positions as undefined values", async () => {
expect(
await value(`
const input = []
input[1] = 1
const result = await Promise.allSettled(input)
return [result.length, result[0].status, result[0].value === undefined, result[1]]
`),
).toEqual([2, "fulfilled", true, { status: "fulfilled", value: 1 }])
})
test("Promise.race returns a promise and settles from its first input", async () => {
expect(
await value(`
const rejected = Promise.reject(2)
const fulfilled = Promise.resolve(1)
const promise = Promise.race([fulfilled, rejected])
return [promise instanceof Promise, await promise]
`),
).toEqual([true, 1])
})
test("Promise.race preserves primitive fulfillment and rejection", async () => {
expect(
await value(`
const fulfilled = await Promise.race([23])
let rejected
try { await Promise.race([Promise.reject(7)]) } catch (reason) { rejected = reason }
return [fulfilled, rejected]
`),
).toEqual([23, 7])
})
test("Promise.race observes later fulfillment and rejection", async () => {
expect(
await value(`
const slow = async () => { await Promise.resolve(); await Promise.resolve(); return 1 }
const rejectSoon = async () => { await Promise.resolve(); throw 2 }
try { await Promise.race([slow(), rejectSoon()]); return "fulfilled" } catch (reason) { return reason }
`),
).toBe(2)
})
test("Promise.race selects fulfilled and rejected inputs by settlement order", async () => {
expect(
await value(`
const delayed = async (value) => { await Promise.resolve(); return value }
const first = await Promise.race([Promise.resolve(1), Promise.resolve(2)])
const second = await Promise.race([Promise.resolve(1), delayed(9)])
const third = await Promise.race([delayed(9), Promise.resolve(2)])
let fourth
try { await Promise.race([Promise.reject(1), Promise.resolve(2)]) } catch (reason) { fourth = reason }
return [first, second, third, fourth]
`),
).toEqual([1, 1, 2, 1])
})
test("Promise.race treats a sparse first position as undefined", async () => {
expect(
await value(`
const input = []
input[1] = 1
return (await Promise.race(input)) === undefined
`),
).toBe(true)
})
test("Promise.resolve adopts values and nested promises", async () => {
expect(
await value(`
const object = { id: 1 }
return [
await Promise.resolve(42),
await Promise.resolve(Promise.resolve("nested")),
(await Promise.resolve(object)) === object,
]
`),
).toEqual([42, "nested", true])
})
test("Promise.resolve preserves promise identity", async () => {
expect(
await value(`
const promise = Promise.resolve(1)
const identities = new Map([[promise, "same"]])
return identities.get(Promise.resolve(promise))
`),
).toBe("same")
})
test("Promise.reject preserves primitive and object reasons", async () => {
expect(
await value(`
const object = { reason: true }
const reasons = [undefined, null, false, true, 0, "", 42, object]
const observe = async (reason) => {
try { await Promise.reject(reason); return false } catch (caught) { return caught === reason }
}
return await Promise.all(reasons.map(observe))
`),
).toEqual([true, true, true, true, true, true, true, true])
})
})
describe("Test262 async function adaptations", () => {
test("async functions return promises and adopt returned values", async () => {
expect(
await value(`
const plain = async () => 42
const afterAwait = async () => { await Promise.resolve(); return 43 }
const first = plain()
const second = afterAwait()
return [first instanceof Promise, second instanceof Promise, await first, await second]
`),
).toEqual([true, true, 42, 43])
})
test("async functions reject for throws before and after await", async () => {
expect(
await value(`
const throwsBefore = async () => { throw 1 }
const throwsAfter = async () => { await Promise.resolve(); throw 2 }
const observe = async (promise) => {
try { await promise; return "fulfilled" } catch (reason) { return reason }
}
return await Promise.all([observe(throwsBefore()), observe(throwsAfter())])
`),
).toEqual([1, 2])
})
})

View file

@ -92,33 +92,14 @@ describe("first-class promise values", () => {
} }
const first = load(1) const first = load(1)
const second = load(2) const second = load(2)
return [first instanceof Promise, second instanceof Promise, await Promise.all([first, second])] return await Promise.all([first, second])
`), `),
).toEqual([ ).toEqual([
true, [1, 1],
true, [2, 2],
[
[1, 1],
[2, 2],
],
]) ])
}) })
test("async function errors reject instead of throwing at the call site", async () => {
expect(
await value(`
const fail = async () => { throw new Error("boom") }
const promise = fail()
try {
await promise
return "no"
} catch (error) {
return error.message
}
`),
).toBe("boom")
})
test("an un-awaited tool call starts eagerly, in call order, before any await", async () => { test("an un-awaited tool call starts eagerly, in call order, before any await", async () => {
const trace = makeTrace() const trace = makeTrace()
const result = await value( const result = await value(
@ -403,10 +384,6 @@ describe("Promise.all over arbitrary arrays", () => {
expect(trace.maxActive).toBeLessThanOrEqual(8) expect(trace.maxActive).toBeLessThanOrEqual(8)
}) })
test("resolves the empty array", async () => {
expect(await value(`return await Promise.all([])`)).toEqual([])
})
test("rejects with the first failure, catchable in-program", async () => { test("rejects with the first failure, catchable in-program", async () => {
expect( expect(
await value(` await value(`
@ -564,25 +541,10 @@ describe("Promise.race", () => {
}) })
describe("Promise.resolve / Promise.reject", () => { describe("Promise.resolve / Promise.reject", () => {
test("resolve wraps plain values and passes promises through", async () => { test("resolve adopts a tool-call promise", async () => {
expect(await value(`return await Promise.resolve(42)`)).toBe(42)
expect(await value(`return await Promise.resolve(Promise.resolve("nested"))`)).toBe("nested")
expect(await value(`return await Promise.resolve(tools.host.sleepy({ id: 3 }))`)).toBe(3) expect(await value(`return await Promise.resolve(tools.host.sleepy({ id: 3 }))`)).toBe(3)
}) })
test("reject produces a promise whose await throws the reason", async () => {
expect(
await value(`
try {
await Promise.reject("nope")
return "no"
} catch (e) {
return e
}
`),
).toBe("nope")
})
test("an abandoned rejected promise surfaces as an unhandled rejection", async () => { test("an abandoned rejected promise surfaces as an unhandled rejection", async () => {
const diagnostic = await error(` const diagnostic = await error(`
Promise.reject(new Error("boom")) Promise.reject(new Error("boom"))
@ -602,15 +564,12 @@ describe("Promise combinator values", () => {
const settled = Promise.allSettled([Promise.resolve(3)]) const settled = Promise.allSettled([Promise.resolve(3)])
const race = Promise.race([Promise.resolve(4)]) const race = Promise.race([Promise.resolve(4)])
return [ return [
all instanceof Promise,
settled instanceof Promise,
race instanceof Promise,
await all.then((values) => values.join(",")), await all.then((values) => values.join(",")),
await settled.then((values) => values[0].value), await settled.then((values) => values[0].value),
await race.then((value) => value + 1), await race.then((value) => value + 1),
] ]
`), `),
).toEqual([true, true, true, "1,2", 3, 5]) ).toEqual(["1,2", 3, 5])
}) })
}) })

View file

@ -0,0 +1,40 @@
# Test262 Promise Coverage
The Promise tests adapt Test262 at revision `250f204f23a9249ff204be2baec29600faae7b75`. The upstream Promise tree
contains 729 files; 459 are under the eight API directories corresponding to CodeMode's five Promise statics and three
chaining methods. The executable suite currently cites 39 distinct Promise, Array, and async-function sources.
This is coverage of CodeMode's confined Promise surface, not a claim of ECMAScript or Test262 conformance. One upstream
file may contain both adapted and inapplicable assertions, so a cited source means only that the represented assertions
were adapted. `LICENSE.test262` contains the upstream BSD terms.
## Covered Surface
- `Promise.all`, `Promise.allSettled`, and `Promise.race` result types, ordering, sparse positions, mixed values, and
rejection propagation.
- `Promise.resolve` value adoption and promise identity.
- `Promise.reject` reason preservation.
- Async-function promise creation, returned-value adoption, and throws before and after `await`.
The Promise PR's handwritten tests remain authoritative for tool-call concurrency, execution-scoped lifetime,
cancellation, output boundaries, model-safe tool failures, unhandled-rejection diagnostics, and chaining behavior.
## Exclusions
- `new Promise`, externally captured resolving functions, and executor semantics.
- Promise subclasses, constructors, species, realms, proxies, property descriptors, and function metadata.
- Custom thenables, poisoned `then` accessors, and arbitrary iterable or iterator-closing behavior.
- `Promise.any`, `Promise.try`, `Promise.withResolvers`, `Promise.allKeyed`, and `Promise.allSettledKeyed`.
- Exact native microtask behavior for `.then`, `.catch`, and `.finally`; those are exercised separately by the Promise
PR's chaining tests and audit.
- Exact native unhandled-rejection timing. CodeMode drains execution-owned work and reports safe diagnostics at its
execution boundary.
## Observed Differences
- Promise combinators accept arrays plus CodeMode's supported spreadable collections. The model guidance currently
describes arrays, so this remains a contract decision rather than conformance coverage.
- `Promise.race([])` returns an actionable error instead of a permanently pending promise, which cannot usefully finish
a bounded CodeMode execution.
- Promise identity is preserved, but CodeMode intentionally rejects general binary operators over Promise references;
identity is tested through `Map` keys instead of `===`.