feat(codemode): support generator functions (#38172)

This commit is contained in:
Aiden Cline 2026-07-21 20:36:42 -05:00 committed by GitHub
commit c8a40450e5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 2229 additions and 389 deletions

View file

@ -29,7 +29,8 @@ ultimate source of truth.
## Values and literals ## Values and literals
- [x] `null`, `undefined`, booleans, finite and non-finite numbers, and strings. - [x] `null`, `undefined`, booleans, finite and non-finite numbers, and strings.
- [x] Array literals, including holes and spread from arrays, strings, Maps, Sets, and URLSearchParams. - [x] Array literals, including holes and spread from arrays, strings, Maps, Sets, URLSearchParams, custom synchronous
iterators, and synchronous generators.
- [x] Object literals with shorthand, computed string/number keys, and spread from plain data objects; `null` and - [x] Object literals with shorthand, computed string/number keys, and spread from plain data objects; `null` and
`undefined` are no-ops, while arrays are rejected. `undefined` are no-ops, while arrays are rejected.
- [x] Template literals with interpolation. - [x] Template literals with interpolation.
@ -57,7 +58,9 @@ ultimate source of truth.
- [ ] Hoist function declarations accepted directly in switch cases. - [ ] Hoist function declarations accepted directly in switch cases.
- [x] Computed object destructuring keys such as `const { [field]: value } = record`. - [x] Computed object destructuring keys such as `const { [field]: value } = record`.
- [x] Object destructuring from arrays, such as `const { length } = values`. - [x] Object destructuring from arrays, such as `const { length } = values`.
- [x] Array destructuring from supported non-array iterables: strings, Maps, Sets, and URLSearchParams. - [x] Array binding and assignment destructuring from strings, Maps, Sets, URLSearchParams, custom synchronous
iterators, and synchronous generators, including stepwise elisions/rest and `IteratorClose` on early completion
or binding/default failure.
## Statements and control flow ## Statements and control flow
@ -65,7 +68,8 @@ ultimate source of truth.
- [x] `if`/`else` and conditional expressions. - [x] `if`/`else` and conditional expressions.
- [x] `switch`, including default clauses and fallthrough. - [x] `switch`, including default clauses and fallthrough.
- [x] `for`, `while`, and `do...while`. - [x] `for`, `while`, and `do...while`.
- [x] `for...of` over arrays, strings, Maps, Sets, and URLSearchParams. - [x] `for...of` over arrays, strings, Maps, Sets, URLSearchParams, custom synchronous iterators, and confined
synchronous generators. Abrupt completion invokes the iterator's optional `return()`.
- [x] `for...in` over own keys of plain objects, arrays, and tool references. - [x] `for...in` over own keys of plain objects, arrays, and tool references.
- [x] Unlabeled `break` and `continue`. - [x] Unlabeled `break` and `continue`.
- [x] `try`, `catch`, optional catch bindings, and `finally`. - [x] `try`, `catch`, optional catch bindings, and `finally`.
@ -75,7 +79,7 @@ ultimate source of truth.
`Symbol.asyncIterator` or the `Symbol.iterator` fallback. Each iterator step is sequential, yielded promises and `Symbol.asyncIterator` or the `Symbol.iterator` fallback. Each iterator step is sequential, yielded promises and
plain values from synchronous collections and sync iterators are awaited before binding, and abrupt loop plain values from synchronous collections and sync iterators are awaited before binding, and abrupt loop
completion invokes the iterator's optional `return()`. Custom async iterators control their yielded values, as in completion invokes the iterator's optional `return()`. Custom async iterators control their yielded values, as in
JavaScript; only their `next()` results are awaited. Async generators remain outside the supported subset. JavaScript; only their `next()` results are awaited. Confined sync and async generators are iterable here.
## Functions and callbacks ## Functions and callbacks
@ -104,7 +108,27 @@ ultimate source of truth.
- [ ] User-defined constructor calls. - [ ] User-defined constructor calls.
- [ ] `Function.prototype.call`, `apply`, and `bind` for CodeMode functions. - [ ] `Function.prototype.call`, `apply`, and `bind` for CodeMode functions.
- [ ] Classes and private fields. - [ ] Classes and private fields.
- [ ] Generator functions and `yield`. - [x] Synchronous and async generator declarations/expressions, `yield`, and `yield*`, including lazy bodies,
`next(value)`, `return(value)`, `throw(value)`, exhaustion, promise adoption, async request ordering,
`try`/`catch`/`finally`, and sync/async iterator symbols. Async `yield*` awaits values while adapting a sync
iterator but preserves values supplied by a manually implemented async iterator. Generator values are opaque
runtime references.
- [x] Synchronous generators and custom synchronous iterators are consumed stepwise by array/argument spread, array
destructuring, `Array.from`, Map/Set/URLSearchParams construction, `Object.fromEntries`, Object/Map `groupBy`,
Promise combinators, `AggregateError`, and `Math.sumPrecise`. Mapper/grouping callbacks interleave with iterator
steps; synchronous consumers preserve yielded promise objects rather than awaiting them. Async generators are
rejected by every synchronous consumer.
- [x] Synchronous iterator acquisition and result validation follow `IteratorClose` boundaries: consumer errors and
intentional early stops invoke `return()`, acquisition/`next()` failures do not, and an original consumer error
wins over a cleanup failure. Async iterator consumption remains limited to `for await...of` and async `yield*`.
- [x] Portable generator protocol coverage is adapted from pinned Test262 cases for suspended-start, suspended-yield,
and completed states; sync and async `next`/`return`/`throw`; finally yields and completion overrides; rejected
yielded promises; mixed async request queues; sync and async `yield*` forwarding; malformed methods/results;
and declaration, expression, and object-method forms with closure and parameter behavior. The adapted suite
deliberately skips Test262 variants whose observation mechanism requires unsupported getter definitions,
proxies, prototype inspection or mutation, non-arrow `this`, classes, or arbitrary symbols. It also skips tests
asserting exact promise reaction-turn counts beyond the observable ordering guarantee documented below. These
are interpreter-surface boundaries, not claims that the corresponding full Test262 families pass unchanged.
## Expressions and operators ## Expressions and operators
@ -130,8 +154,8 @@ ultimate source of truth.
- [x] Tool calls start eagerly and return supervised, run-once CodeMode promises. - [x] Tool calls start eagerly and return supervised, run-once CodeMode promises.
- [x] Direct `await`, repeated awaits, and implicit resolution when a promise is returned from a function/program. - [x] Direct `await`, repeated awaits, and implicit resolution when a promise is returned from a function/program.
- [x] `Promise.resolve` and `Promise.reject`. - [x] `Promise.resolve` and `Promise.reject`.
- [x] `Promise.all`, `Promise.allSettled`, `Promise.race`, and `Promise.any` over supported collections containing - [x] `Promise.all`, `Promise.allSettled`, `Promise.race`, and `Promise.any` over finite collections, custom synchronous
promises and plain values. iterators, and synchronous generators containing promises and plain values.
- [x] `Promise.all` preserves result order and rejects on the first observed failure without cancelling siblings. - [x] `Promise.all` preserves result order and rejects on the first observed failure without cancelling siblings.
- [x] `Promise.allSettled` returns plain fulfilled/rejected outcome records. - [x] `Promise.allSettled` returns plain fulfilled/rejected outcome records.
- [x] `Promise.race` settles from the first result without cancelling losers at settlement time. - [x] `Promise.race` settles from the first result without cancelling losers at settlement time.
@ -175,14 +199,16 @@ ultimate source of truth.
- [x] `Object()` and `new Object()` return `{}` for nullish arguments and pass objects through unchanged; - [x] `Object()` and `new Object()` return `{}` for nullish arguments and pass objects through unchanged;
primitive wrapper objects (`Object(1)`) are rejected explicitly. primitive wrapper objects (`Object(1)`) are rejected explicitly.
- [x] Computed property names and object spread. - [x] Computed property names and object spread.
- [x] `Object.keys`, `Object.values`, `Object.entries`, `Object.hasOwn`, `Object.assign`, and `Object.fromEntries`. - [x] `Object.keys`, `Object.values`, `Object.entries`, `Object.hasOwn`, `Object.assign`, and `Object.fromEntries`, with
synchronous iterator support for `fromEntries`.
- [x] `Object.keys` over arrays and tool references. - [x] `Object.keys` over arrays and tool references.
- [x] Object identity is preserved by in-CodeMode Object helpers. - [x] Object identity is preserved by in-CodeMode Object helpers.
- [x] Prototype traversal and mutation through `__proto__`, `constructor`, and `prototype` are blocked. - [x] Prototype traversal and mutation through `__proto__`, `constructor`, and `prototype` are blocked.
- [ ] Legal own data fields named `__proto__`, `constructor`, or `prototype` are rejected at JSON/tool boundaries and - [ ] Legal own data fields named `__proto__`, `constructor`, or `prototype` are rejected at JSON/tool boundaries and
cannot be created, read, or written in CodeMode; tool path segments with those names remain supported. cannot be created, read, or written in CodeMode; tool path segments with those names remain supported.
- [x] `Object.is` for supported data values. - [x] `Object.is` for supported data values.
- [x] `Object.groupBy` over supported collection iterables, with string-key coercion and null-prototype results. - [x] `Object.groupBy` over finite collections and custom synchronous iterators/generators, with string-key coercion
and null-prototype results.
## Arrays ## Arrays
@ -190,7 +216,7 @@ ultimate source of truth.
array of that length; invalid lengths throw `RangeError`. Iteration, spread, join, and JSON handle holes like array of that length; invalid lengths throw `RangeError`. Iteration, spread, join, and JSON handle holes like
JavaScript, and host results normalize holes to `null`. JavaScript, and host results normalize holes to `null`.
- [x] Static methods: `Array.isArray`, `Array.of`, and `Array.from`, including the `Array.from` mapper form with - [x] Static methods: `Array.isArray`, `Array.of`, and `Array.from`, including the `Array.from` mapper form with
`(value, index)` arguments. `(value, index)` arguments and stepwise synchronous iterator consumption.
- [x] Iteration/transformation: `map`, `filter`, `flatMap`, and `forEach`. - [x] Iteration/transformation: `map`, `filter`, `flatMap`, and `forEach`.
- [x] Searching/tests: `find`, `findIndex`, `findLast`, `findLastIndex`, `some`, `every`, `includes`, `indexOf`, and - [x] Searching/tests: `find`, `findIndex`, `findLast`, `findLastIndex`, `some`, `every`, `includes`, `indexOf`, and
`lastIndexOf`. `lastIndexOf`.
@ -245,7 +271,8 @@ ultimate source of truth.
for feature detection. Calling any undefined value reports a native-style `TypeError` naming the callee, for for feature detection. Calling any undefined value reports a native-style `TypeError` naming the callee, for
example `Math.sum is not a function.` Blocked members (`constructor`, `__proto__`, ...) still throw, example `Math.sum is not a function.` Blocked members (`constructor`, `__proto__`, ...) still throw,
and unknown `Promise` statics keep their descriptive error. and unknown `Promise` statics keep their descriptive error.
- [x] `Math.sumPrecise` over supported collection iterables, rejecting non-number elements without coercion. - [x] `Math.sumPrecise` over finite collections and custom synchronous iterators/generators, rejecting non-number
elements without coercion.
- [x] Global coercing `isFinite` and `isNaN`; opaque runtime references reject as data errors, like `Number(...)`. - [x] Global coercing `isFinite` and `isNaN`; opaque runtime references reject as data errors, like `Number(...)`.
## JSON and console ## JSON and console
@ -297,10 +324,10 @@ ultimate source of truth.
## Map and Set ## Map and Set
- [x] Static `Map.groupBy` over supported collection iterables, preserving key identity. - [x] Static `Map.groupBy` over finite collections and custom synchronous iterators/generators, preserving key identity.
- [x] `new Map()` from entry arrays or another Map. - [x] `new Map()` from synchronous iterables of entries.
- [x] Map `get`, `set`, `has`, `delete`, `clear`, `size`, and `forEach`. - [x] Map `get`, `set`, `has`, `delete`, `clear`, `size`, and `forEach`.
- [x] `new Set()` from arrays, strings, or another Set. - [x] `new Set()` from synchronous iterables.
- [x] Set `add`, `has`, `delete`, `clear`, `size`, and `forEach`. - [x] Set `add`, `has`, `delete`, `clear`, `size`, and `forEach`.
- [x] Materialized `keys`, `values`, and `entries` arrays for Map and Set. - [x] Materialized `keys`, `values`, and `entries` arrays for Map and Set.
- [x] Spread, `for...of`, `Array.from`, and `Object.fromEntries` integration. - [x] Spread, `for...of`, `Array.from`, and `Object.fromEntries` integration.
@ -316,7 +343,7 @@ ultimate source of truth.
- [x] Readable URL fields: `href`, `origin`, `protocol`, `username`, `password`, `host`, `hostname`, `port`, - [x] Readable URL fields: `href`, `origin`, `protocol`, `username`, `password`, `host`, `hostname`, `port`,
`pathname`, `search`, and `hash`. `pathname`, `search`, and `hash`.
- [x] Writable URL fields except `origin`. - [x] Writable URL fields except `origin`.
- [x] `new URLSearchParams()` from query strings, data objects, pairs, Maps, and URLSearchParams. - [x] `new URLSearchParams()` from query strings, data objects, synchronous iterables of pairs, and URLSearchParams.
- [x] URLSearchParams `append`, `delete`, `get`, `getAll`, `has`, `set`, `sort`, `forEach`, `keys`, `values`, - [x] URLSearchParams `append`, `delete`, `get`, `getAll`, `has`, `set`, `sort`, `forEach`, `keys`, `values`,
`entries`, `toString`, and `size`. `entries`, `toString`, and `size`.
- [x] URL values serialize to their href; URLSearchParams serialize to `{}`. - [x] URL values serialize to their href; URLSearchParams serialize to `{}`.
@ -326,7 +353,7 @@ ultimate source of truth.
- [x] `Error`, `TypeError`, `RangeError`, `SyntaxError`, `ReferenceError`, `EvalError`, and `URIError`, callable with - [x] `Error`, `TypeError`, `RangeError`, `SyntaxError`, `ReferenceError`, `EvalError`, and `URIError`, callable with
or without `new`. or without `new`.
- [x] `AggregateError` with the `(errors, message?)` signature and an own `errors` array, constructed directly or by - [x] `AggregateError` with the `(errors, message?)` signature and an own `errors` array, constructed directly or by
an all-rejected `Promise.any`. an all-rejected `Promise.any`; direct construction accepts custom synchronous iterators and generators.
- [x] Error `name`/`message`, error inheritance through `instanceof`, and plain-data serialization. - [x] Error `name`/`message`, error inheritance through `instanceof`, and plain-data serialization.
- [x] `instanceof` for Date, RegExp, Map, Set, URL, URLSearchParams, Array, Object, Promise, and Error types. - [x] `instanceof` for Date, RegExp, Map, Set, URL, URLSearchParams, Array, Object, Promise, and Error types.
- [x] Catchable user throws, runtime failures raised during interpreted evaluation, awaited tool failures, and awaited - [x] Catchable user throws, runtime failures raised during interpreted evaluation, awaited tool failures, and awaited

View file

@ -1,9 +1,10 @@
import { Effect } from "effect"
import type { Diagnostic } from "../codemode.js" import type { Diagnostic } from "../codemode.js"
import { ToolError } from "../tool-error.js" import { ToolError } from "../tool-error.js"
import { copyOut, ToolRuntimeError, type SafeObject } from "../tool-runtime.js" import { copyOut, ToolRuntimeError, type SafeObject } from "../tool-runtime.js"
import { type AstNode, formatLocation, InterpreterRuntimeError, ProgramThrow, sourceLocation } from "./model.js" import { type AstNode, formatLocation, InterpreterRuntimeError, ProgramThrow, sourceLocation } from "./model.js"
import { containsRuntimeReference } from "./references.js" import { containsRuntimeReference } from "./references.js"
import { spreadItems } from "../stdlib/collections.js" import { type SyncIteratorRunner } from "./iterator.js"
import { coerceToString, createAggregateErrorValue, createErrorValue, errorConstructors } from "../stdlib/value.js" import { coerceToString, createAggregateErrorValue, createErrorValue, errorConstructors } from "../stdlib/value.js"
export const normalizeError = (error: unknown): Diagnostic => { export const normalizeError = (error: unknown): Diagnostic => {
@ -79,15 +80,27 @@ export const caughtErrorValue = (thrown: unknown): unknown => {
return createErrorValue(name, normalizeError(thrown).message) return createErrorValue(name, normalizeError(thrown).message)
} }
export const constructErrorValue = (name: string, args: Array<unknown>, node: AstNode): SafeObject => { export const constructErrorValue = (name: string, args: Array<unknown>): SafeObject =>
if (name !== "AggregateError") return createErrorValue(name, args[0] === undefined ? "" : coerceToString(args[0])) createErrorValue(name, args[0] === undefined ? "" : coerceToString(args[0]))
const errors = spreadItems(args[0])
if (errors === undefined) { export const constructAggregateErrorValue = <R>(
throw new InterpreterRuntimeError( runner: SyncIteratorRunner<R>,
"new AggregateError(...) expects an array of errors (e.g. new AggregateError(errors, message?)).", args: Array<unknown>,
node, node: AstNode,
).as("TypeError") ): Effect.Effect<SafeObject, unknown, R> =>
} Effect.gen(function* () {
// Error values must not alias caller-owned arrays. const cursor = yield* runner.syncIterator(args[0], node)
return createAggregateErrorValue([...errors], args[1] === undefined ? "" : coerceToString(args[1])) if (cursor === undefined) {
} throw new InterpreterRuntimeError("new AggregateError(...) expects a synchronous iterable of errors.", node).as(
"TypeError",
)
}
const errors: Array<unknown> = []
while (true) {
const step = yield* cursor.next
if (step.done) {
return createAggregateErrorValue(errors, args[1] === undefined ? "" : coerceToString(args[1]))
}
errors.push(step.value)
}
})

View file

@ -0,0 +1,21 @@
import { Effect, Exit } from "effect"
import type { AstNode } from "./model.js"
export type IteratorCursor<R> = {
readonly next: Effect.Effect<{ readonly done: boolean; readonly value: unknown }, unknown, R>
readonly close: Effect.Effect<void, unknown, R>
}
export type SyncIteratorRunner<R> = {
readonly syncIterator: (value: unknown, node: AstNode) => Effect.Effect<IteratorCursor<R> | undefined, unknown, R>
}
export const preserveConsumerError = <A, R>(
cursor: IteratorCursor<R>,
effect: Effect.Effect<A, unknown, R>,
): Effect.Effect<A, unknown, R> =>
Effect.flatMap(Effect.exit(effect), (exit) =>
Exit.isSuccess(exit)
? Effect.succeed(exit.value)
: Effect.andThen(Effect.exit(cursor.close), Effect.failCause(exit.cause)),
)

View file

@ -2,6 +2,7 @@ import { Effect } from "effect"
import { import {
type AstNode, type AstNode,
CodeModeFunction, CodeModeFunction,
CodeModeGenerator,
CoercionFunction, CoercionFunction,
ErrorConstructorReference, ErrorConstructorReference,
GlobalMethodReference, GlobalMethodReference,
@ -33,6 +34,7 @@ import { invokeRegExpMethod, invokeRegExpStatic, matchToValue, toHostRegex } fro
import { invokeStringStatic } from "../stdlib/string.js" import { invokeStringStatic } from "../stdlib/string.js"
import { invokeURLMethod, invokeURLStatic, uriArgument } from "../stdlib/url.js" import { invokeURLMethod, invokeURLStatic, uriArgument } from "../stdlib/url.js"
import { boundedData, coerceToNumber, coerceToString, errorBrandName } from "../stdlib/value.js" import { boundedData, coerceToNumber, coerceToString, errorBrandName } from "../stdlib/value.js"
import { preserveConsumerError, type SyncIteratorRunner } from "./iterator.js"
export type CallbackRunner<R> = { export type CallbackRunner<R> = {
readonly invokeFunction: (fn: CodeModeFunction, args: Array<unknown>) => Effect.Effect<unknown, unknown, R> readonly invokeFunction: (fn: CodeModeFunction, args: Array<unknown>) => Effect.Effect<unknown, unknown, R>
@ -360,19 +362,12 @@ const invokeArrayStatic = (name: string, args: Array<unknown>, node: AstNode): u
return Array.isArray(args[0]) return Array.isArray(args[0])
case "of": case "of":
return [...args] return [...args]
case "from":
return arrayFromItems(args[0], node)
default: default:
throw new InterpreterRuntimeError(`Array.${name} is not available.`, node) throw new InterpreterRuntimeError(`Array.${name} is not available.`, node)
} }
} }
const arrayFromItems = (source: unknown, node: AstNode): Array<unknown> => { const arrayLikeSource = (source: unknown, node: AstNode): { readonly length: number; readonly source: object } => {
if (source instanceof CodeModeMap) return Array.from(source.map.entries(), ([key, item]) => [key, item])
if (source instanceof CodeModeSet) return Array.from(source.set.values())
if (source instanceof CodeModeURLSearchParams) {
return Array.from(source.params.entries(), ([key, value]) => [key, value])
}
if (source instanceof CodeModePromise) { if (source instanceof CodeModePromise) {
throw new InterpreterRuntimeError( throw new InterpreterRuntimeError(
"Array.from received an un-awaited Promise; await it before creating the array.", "Array.from received an un-awaited Promise; await it before creating the array.",
@ -380,15 +375,16 @@ const arrayFromItems = (source: unknown, node: AstNode): Array<unknown> => {
"InvalidDataValue", "InvalidDataValue",
) )
} }
if (typeof source === "string") return Array.from(source)
if (Array.isArray(source)) return [...source]
if ( if (
source !== null && source !== null &&
typeof source === "object" && typeof source === "object" &&
(Object.getPrototypeOf(source) === Object.prototype || Object.getPrototypeOf(source) === null) && (Object.getPrototypeOf(source) === Object.prototype || Object.getPrototypeOf(source) === null) &&
typeof (source as { length?: unknown }).length === "number" typeof (source as { length?: unknown }).length === "number"
) { ) {
return Array.from(source as ArrayLike<unknown>) const length = (source as { length: number }).length
const normalized = Number.isNaN(length) || length <= 0 ? 0 : Math.trunc(length)
if (normalized > 4_294_967_295) throw new RangeError("Invalid array length")
return { length: normalized, source }
} }
throw new InterpreterRuntimeError( throw new InterpreterRuntimeError(
"Array.from expects an array, string, Map, Set, or array-like value.", "Array.from expects an array, string, Map, Set, or array-like value.",
@ -398,24 +394,42 @@ const arrayFromItems = (source: unknown, node: AstNode): Array<unknown> => {
} }
export const invokeArrayFrom = <R>( export const invokeArrayFrom = <R>(
runner: CallbackRunner<R>, runner: CallbackRunner<R> & SyncIteratorRunner<R>,
args: Array<unknown>, args: Array<unknown>,
node: AstNode, node: AstNode,
): Effect.Effect<unknown, unknown, R> => { ): Effect.Effect<unknown, unknown, R> => {
const items = arrayFromItems(args[0], node) const source = args[0]
if (args.length < 2 || args[1] === undefined) return Effect.succeed(items) const apply =
const apply = applyCollectionCallback(runner, args[1], "Array.from", node) args.length < 2 || args[1] === undefined ? undefined : applyCollectionCallback(runner, args[1], "Array.from", node)
return Effect.gen(function* () { return Effect.gen(function* () {
const values: Array<unknown> = [] const cursor = yield* runner.syncIterator(source, node)
for (let index = 0; index < items.length; index += 1) { if (cursor === undefined) {
values.push(yield* apply([items[index], index])) if (source instanceof CodeModeGenerator) {
throw new InterpreterRuntimeError("Array.from expects a synchronous iterable or array-like value.", node).as(
"TypeError",
)
}
const arrayLike = arrayLikeSource(source, node)
const values: Array<unknown> = []
for (let index = 0; index < arrayLike.length; index += 1) {
const item = Reflect.get(arrayLike.source, index)
values.push(apply === undefined ? item : yield* apply([item, index]))
}
return values
}
const values: Array<unknown> = []
let index = 0
while (true) {
const step = yield* cursor.next
if (step.done) return values
values.push(apply === undefined ? step.value : yield* preserveConsumerError(cursor, apply([step.value, index])))
index += 1
} }
return values
}) })
} }
export const invokeGroupBy = <R>( export const invokeGroupBy = <R>(
runner: CallbackRunner<R>, runner: CallbackRunner<R> & SyncIteratorRunner<R>,
namespace: "Map" | "Object", namespace: "Map" | "Object",
args: Array<unknown>, args: Array<unknown>,
node: AstNode, node: AstNode,
@ -425,47 +439,50 @@ export const invokeGroupBy = <R>(
throw new InterpreterRuntimeError(`${namespace}.groupBy expects an iterable collection.`, node).as("TypeError") throw new InterpreterRuntimeError(`${namespace}.groupBy expects an iterable collection.`, node).as("TypeError")
} }
const apply = applyCollectionCallback(runner, args[1], `${namespace}.groupBy`, node) const apply = applyCollectionCallback(runner, args[1], `${namespace}.groupBy`, node)
const items = supportedIterableItems(source)
if (items === undefined) {
throw new InterpreterRuntimeError(`${namespace}.groupBy expects an iterable collection.`, node).as("TypeError")
}
return Effect.gen(function* () { return Effect.gen(function* () {
const cursor = yield* runner.syncIterator(source, node)
if (cursor === undefined) {
throw new InterpreterRuntimeError(`${namespace}.groupBy expects an iterable collection.`, node).as("TypeError")
}
if (namespace === "Map") { if (namespace === "Map") {
const result = new CodeModeMap() const result = new CodeModeMap()
let index = 0 let index = 0
for (const item of items) { while (true) {
const key = yield* apply([item, index]) const step = yield* cursor.next
if (step.done) return result
const item = step.value
const key = yield* preserveConsumerError(cursor, apply([item, index]))
const group = result.map.get(key) const group = result.map.get(key)
if (group === undefined) result.map.set(key, [item]) if (group === undefined) result.map.set(key, [item])
else (group as Array<unknown>).push(item) else (group as Array<unknown>).push(item)
index += 1 index += 1
} }
return result
} }
const result: SafeObject = Object.create(null) as SafeObject const result: SafeObject = Object.create(null) as SafeObject
let index = 0 let index = 0
for (const item of items) { while (true) {
const key = yield* coerceGroupByPropertyKey(runner, yield* apply([item, index]), node) const step = yield* cursor.next
if (step.done) return result
const item = step.value
const key = yield* preserveConsumerError(
cursor,
Effect.flatMap(apply([item, index]), (value) => coerceGroupByPropertyKey(runner, value, node)),
)
if (isBlockedMember(key)) { if (isBlockedMember(key)) {
throw new InterpreterRuntimeError(`Property '${key}' is not available.`, node) return yield* preserveConsumerError(
cursor,
Effect.fail(new InterpreterRuntimeError(`Property '${key}' is not available.`, node)),
)
} }
const group = result[key] const group = result[key]
if (group === undefined) result[key] = [item] if (group === undefined) result[key] = [item]
else (group as Array<unknown>).push(item) else (group as Array<unknown>).push(item)
index += 1 index += 1
} }
return result
}) })
} }
const supportedIterableItems = (source: unknown): Iterable<unknown> | undefined => {
if (Array.isArray(source) || typeof source === "string") return source
if (source instanceof CodeModeMap) return source.map.entries()
if (source instanceof CodeModeSet) return source.set.values()
if (source instanceof CodeModeURLSearchParams) return source.params.entries()
}
const coerceGroupByPropertyKey = <R>( const coerceGroupByPropertyKey = <R>(
runner: CallbackRunner<R>, runner: CallbackRunner<R>,
value: unknown, value: unknown,

View file

@ -1,3 +1,4 @@
import type { Effect } from "effect"
import type { SafeObject } from "../tool-runtime.js" import type { SafeObject } from "../tool-runtime.js"
import type { CodeModePromise, CodeModeRegExp, CodeModeURL } from "../values.js" import type { CodeModePromise, CodeModeRegExp, CodeModeURL } from "../values.js"
@ -45,6 +46,27 @@ export class CodeModeFunction {
readonly body: AstNode, readonly body: AstNode,
readonly capturedScopes: ReadonlyArray<Map<string, Binding>>, readonly capturedScopes: ReadonlyArray<Map<string, Binding>>,
readonly async: boolean, readonly async: boolean,
readonly generator: boolean,
) {}
}
export type GeneratorRequestKind = "next" | "return" | "throw"
export class CodeModeGenerator {
constructor(
readonly asynchronous: boolean,
readonly request: (
kind: GeneratorRequestKind,
value: unknown,
node: AstNode,
) => Effect.Effect<unknown, unknown, unknown>,
) {}
}
export class GeneratorMethodReference {
constructor(
readonly generator: CodeModeGenerator,
readonly kind: GeneratorRequestKind | "iterator",
) {} ) {}
} }
@ -128,6 +150,10 @@ export class ProgramThrow {
constructor(readonly value: unknown) {} constructor(readonly value: unknown) {}
} }
export class GeneratorReturn {
constructor(readonly value: unknown) {}
}
export class ErrorConstructorReference { export class ErrorConstructorReference {
constructor(readonly name: string) {} constructor(readonly name: string) {}
} }

View file

@ -13,9 +13,9 @@ import {
import { caughtErrorValue, normalizeError } from "./errors.js" import { caughtErrorValue, normalizeError } from "./errors.js"
import { applyCollectionCallback, isSupportedCallback, type CallbackRunner, type SupportedCallback } from "./methods.js" import { applyCollectionCallback, isSupportedCallback, type CallbackRunner, type SupportedCallback } from "./methods.js"
import { typeofValue } from "./references.js" import { typeofValue } from "./references.js"
import { spreadItems } from "../stdlib/collections.js"
import { createAggregateErrorValue } from "../stdlib/value.js" import { createAggregateErrorValue } from "../stdlib/value.js"
import { CodeModePromise } from "../values.js" import { CodeModePromise } from "../values.js"
import type { SyncIteratorRunner } from "./iterator.js"
// Observation only controls rejection reporting; program completion interrupts all promise work. // Observation only controls rejection reporting; program completion interrupts all promise work.
export class PromiseRuntime<R> { export class PromiseRuntime<R> {
@ -64,6 +64,10 @@ export class PromiseRuntime<R> {
return Fiber.await(promise.fiber) return Fiber.await(promise.fiber)
} }
fork(effect: Effect.Effect<unknown, unknown, R>): Effect.Effect<void, never, R> {
return Effect.asVoid(Effect.forkIn(effect, this.scope, { startImmediately: true }))
}
diagnostics(): Array<Diagnostic> { diagnostics(): Array<Diagnostic> {
return [...this.failures].sort(([left], [right]) => left - right).map(([, failure]) => failure) return [...this.failures].sort(([left], [right]) => left - right).map(([, failure]) => failure)
} }
@ -84,7 +88,7 @@ export const selfResolutionError = (node?: AstNode): InterpreterRuntimeError =>
new InterpreterRuntimeError("Chaining cycle detected: a promise cannot resolve with itself.", node).as("TypeError") new InterpreterRuntimeError("Chaining cycle detected: a promise cannot resolve with itself.", node).as("TypeError")
export const invokePromiseMethod = <R>( export const invokePromiseMethod = <R>(
runner: CallbackRunner<R>, runner: CallbackRunner<R> & SyncIteratorRunner<R>,
promises: PromiseRuntime<R>, promises: PromiseRuntime<R>,
ref: PromiseMethodReference, ref: PromiseMethodReference,
args: Array<unknown>, args: Array<unknown>,
@ -98,79 +102,69 @@ export const invokePromiseMethod = <R>(
return promises.create(Effect.fail(new ProgramThrow(args[0]))) return promises.create(Effect.fail(new ProgramThrow(args[0])))
} }
const spread = spreadItems(args[0]) return promises.create(
if (spread === undefined) { Effect.gen(function* () {
return promises.create( const cursor = yield* runner.syncIterator(args[0], node)
Effect.fail( if (cursor === undefined) {
new InterpreterRuntimeError( throw new InterpreterRuntimeError(
`Promise.${ref.name} expects an array of promises or plain values (e.g. Promise.${ref.name}(items.map((item) => tools.ns.tool(item)))).`, `Promise.${ref.name} expects an array or other synchronous iterable.`,
node, node,
).as("TypeError"), ).as("TypeError")
), }
) const items: Array<unknown> = []
} while (true) {
const items = Array.from(spread) const step = yield* cursor.next
if (step.done) break
items.push(step.value)
if (step.value instanceof CodeModePromise) promises.markObserved(step.value)
}
for (const item of items) { if (ref.name === "all") {
if (item instanceof CodeModePromise) promises.markObserved(item) return yield* settleAfterTurn(
} Effect.all(
items.map((item) =>
switch (ref.name) { item instanceof CodeModePromise ? Effect.flatten(promises.await(item)) : Effect.succeed(item),
case "all": { ),
const observations = items.map((item) => { concurrency: "unbounded" },
item instanceof CodeModePromise ? Effect.flatten(promises.await(item)) : Effect.succeed(item), ),
) )
return promises.create(settleAfterTurn(Effect.all(observations, { concurrency: "unbounded" }))) }
} if (ref.name === "allSettled") {
case "allSettled": { const outcomes: Array<unknown> = []
const observations = items.map((item) => for (const item of items) {
item instanceof CodeModePromise ? promises.await(item) : Effect.succeed(Exit.succeed(item)), const exit = item instanceof CodeModePromise ? yield* promises.await(item) : Exit.succeed(item)
) if (Exit.isSuccess(exit)) {
return promises.create( outcomes.push(Object.assign(Object.create(null) as SafeObject, { status: "fulfilled", value: exit.value }))
settleAfterTurn( continue
Effect.gen(function* () { }
const outcomes: Array<unknown> = [] if (Cause.hasInterruptsOnly(exit.cause)) return yield* Effect.failCause(exit.cause)
for (const observation of observations) { outcomes.push(
const exit = yield* observation Object.assign(Object.create(null) as SafeObject, {
if (Exit.isSuccess(exit)) { status: "rejected",
outcomes.push( reason: caughtErrorValue(Cause.squash(exit.cause)),
Object.assign(Object.create(null) as SafeObject, { status: "fulfilled", value: exit.value }), }),
) )
continue }
} yield* Effect.yieldNow
if (Cause.hasInterruptsOnly(exit.cause)) { return outcomes
// Teardown interruption is not a program-level rejection. }
return yield* Effect.failCause(exit.cause) if (ref.name === "race") {
} if (items.length === 0) {
outcomes.push( throw new InterpreterRuntimeError(
Object.assign(Object.create(null) as SafeObject, { "Promise.race([]) would never settle; provide at least one promise or value.",
status: "rejected", node,
reason: caughtErrorValue(Cause.squash(exit.cause)), )
}), }
) return yield* settleAfterTurn(
} Effect.flatten(
return outcomes Effect.raceAll(
}), items.map((item) =>
), item instanceof CodeModePromise ? promises.await(item) : Effect.succeed(Exit.succeed(item)),
) ),
}
case "race": {
if (items.length === 0) {
return promises.create(
Effect.fail(
new InterpreterRuntimeError(
"Promise.race([]) would never settle; provide at least one promise or value.",
node,
), ),
), ),
) )
} }
const observations = items.map((item) =>
item instanceof CodeModePromise ? promises.await(item) : Effect.succeed(Exit.succeed(item)),
)
return promises.create(settleAfterTurn(Effect.flatten(Effect.raceAll(observations))))
}
case "any": {
const flipped = items.map((item) => const flipped = items.map((item) =>
item instanceof CodeModePromise item instanceof CodeModePromise
? Effect.flatMap(promises.await(item), (exit) => { ? Effect.flatMap(promises.await(item), (exit) => {
@ -180,17 +174,18 @@ export const invokePromiseMethod = <R>(
}) })
: Effect.fail(new PromiseAnyFulfilled(item)), : Effect.fail(new PromiseAnyFulfilled(item)),
) )
const body = Effect.all(flipped, { concurrency: "unbounded" }).pipe( return yield* settleAfterTurn(
Effect.flatMap((reasons) => Effect.all(flipped, { concurrency: "unbounded" }).pipe(
Effect.fail(new ProgramThrow(createAggregateErrorValue(reasons, "All promises were rejected"))), Effect.flatMap((reasons) =>
), Effect.fail(new ProgramThrow(createAggregateErrorValue(reasons, "All promises were rejected"))),
Effect.catch((error) => ),
error instanceof PromiseAnyFulfilled ? Effect.succeed(error.value) : Effect.fail(error), Effect.catch((error) =>
error instanceof PromiseAnyFulfilled ? Effect.succeed(error.value) : Effect.fail(error),
),
), ),
) )
return promises.create(settleAfterTurn(body)) }),
} )
}
} }
export const invokePromiseInstanceMethod = <R>( export const invokePromiseInstanceMethod = <R>(

View file

@ -1,10 +1,12 @@
import { import {
type AstNode, type AstNode,
CodeModeFunction, CodeModeFunction,
CodeModeGenerator,
CoercionFunction, CoercionFunction,
ErrorConstructorReference, ErrorConstructorReference,
GlobalMethodReference, GlobalMethodReference,
GlobalNamespace, GlobalNamespace,
GeneratorMethodReference,
InterpreterRuntimeError, InterpreterRuntimeError,
IntrinsicReference, IntrinsicReference,
JsonMethodReference, JsonMethodReference,
@ -21,6 +23,8 @@ import { isCodeModeValue, CodeModePromise } from "../values.js"
export const isRuntimeReference = (value: unknown): boolean => export const isRuntimeReference = (value: unknown): boolean =>
value instanceof CodeModeFunction || value instanceof CodeModeFunction ||
value instanceof CodeModeGenerator ||
value instanceof GeneratorMethodReference ||
value instanceof ToolReference || value instanceof ToolReference ||
value instanceof IntrinsicReference || value instanceof IntrinsicReference ||
value instanceof GlobalNamespace || value instanceof GlobalNamespace ||
@ -107,6 +111,7 @@ export const rejectCircularInsertion = (container: object, value: unknown, label
export const typeofValue = (value: unknown): string => { export const typeofValue = (value: unknown): string => {
if ( if (
value instanceof CodeModeFunction || value instanceof CodeModeFunction ||
value instanceof GeneratorMethodReference ||
value instanceof CoercionFunction || value instanceof CoercionFunction ||
value instanceof IntrinsicReference || value instanceof IntrinsicReference ||
value instanceof GlobalMethodReference || value instanceof GlobalMethodReference ||

File diff suppressed because it is too large Load diff

View file

@ -1,5 +1,6 @@
import { Effect } from "effect"
import { preserveConsumerError, type SyncIteratorRunner } from "../interpreter/iterator.js"
import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js" import { type AstNode, InterpreterRuntimeError } from "../interpreter/model.js"
import { spreadItems } from "./collections.js"
// Bun exposes ES2026 Math.sumPrecise before TypeScript's standard library types. // Bun exposes ES2026 Math.sumPrecise before TypeScript's standard library types.
declare global { declare global {
@ -53,17 +54,6 @@ export const mathMethods = new Set([
export const invokeMathMethod = (name: string, args: Array<unknown>, node: AstNode): number => { export const invokeMathMethod = (name: string, args: Array<unknown>, node: AstNode): number => {
if (!mathMethods.has(name)) throw new InterpreterRuntimeError(`Math.${name} is not available.`, node) if (!mathMethods.has(name)) throw new InterpreterRuntimeError(`Math.${name} is not available.`, node)
if (name === "random") return Math.random() if (name === "random") return Math.random()
if (name === "sumPrecise") {
const items = spreadItems(args[0])
if (items === undefined) {
throw new InterpreterRuntimeError("Math.sumPrecise expects an iterable collection.", node).as("TypeError")
}
const numbers = Array.from(items)
if (!numbers.every((item): item is number => typeof item === "number")) {
throw new InterpreterRuntimeError("Math.sumPrecise expects an iterable of numbers.", node).as("TypeError")
}
return Math.sumPrecise(numbers)
}
// Validate only the arguments the method consumes; like JS, extras are ignored // Validate only the arguments the method consumes; like JS, extras are ignored
// (so built-ins work as callbacks receiving (element, index, array)). // (so built-ins work as callbacks receiving (element, index, array)).
const num = (index: number): number => { const num = (index: number): number => {
@ -153,3 +143,29 @@ export const invokeMathMethod = (name: string, args: Array<unknown>, node: AstNo
} }
throw new InterpreterRuntimeError(`Math.${name} is not available.`, node) throw new InterpreterRuntimeError(`Math.${name} is not available.`, node)
} }
export const invokeMathSumPrecise = <R>(
runner: SyncIteratorRunner<R>,
source: unknown,
node: AstNode,
): Effect.Effect<number, unknown, R> =>
Effect.gen(function* () {
const cursor = yield* runner.syncIterator(source, node)
if (cursor === undefined) {
throw new InterpreterRuntimeError("Math.sumPrecise expects a synchronous iterable.", node).as("TypeError")
}
const numbers: Array<number> = []
while (true) {
const step = yield* cursor.next
if (step.done) return Math.sumPrecise(numbers)
yield* preserveConsumerError(
cursor,
Effect.sync(() => {
if (typeof step.value !== "number") {
throw new InterpreterRuntimeError("Math.sumPrecise expects an iterable of numbers.", node).as("TypeError")
}
numbers.push(step.value)
}),
)
}
})

View file

@ -1,3 +1,4 @@
import { Effect } from "effect"
import { import {
type AstNode, type AstNode,
AsyncIteratorSymbol, AsyncIteratorSymbol,
@ -7,8 +8,9 @@ import {
} from "../interpreter/model.js" } from "../interpreter/model.js"
import { containsOpaqueReference } from "../interpreter/references.js" import { containsOpaqueReference } from "../interpreter/references.js"
import { isBlockedMember } from "../tool-runtime.js" import { isBlockedMember } from "../tool-runtime.js"
import { isCodeModeValue, CodeModeMap, CodeModePromise, CodeModeSet, CodeModeURLSearchParams } from "../values.js" import { isCodeModeValue, CodeModePromise } from "../values.js"
import { boundedData, coerceToString } from "./value.js" import { boundedData, coerceToString } from "./value.js"
import { preserveConsumerError, type SyncIteratorRunner } from "../interpreter/iterator.js"
export const objectMethodsPreservingIdentity = new Set(["assign", "values", "entries", "fromEntries"]) export const objectMethodsPreservingIdentity = new Set(["assign", "values", "entries", "fromEntries"])
@ -39,11 +41,6 @@ export const invokeObjectMethod = (name: string, args: Array<unknown>, node: Ast
if (isBlockedMember(key)) throw new InterpreterRuntimeError(`Property '${key}' is not available.`, node) if (isBlockedMember(key)) throw new InterpreterRuntimeError(`Property '${key}' is not available.`, node)
out[key] = item out[key] = item
} }
const addEntry = (out: Record<string, unknown>, key: unknown, item: unknown): void => {
boundedData(key, "Object.fromEntries key")
boundedData(item, "Object.fromEntries value")
guardedSet(out, coerceToString(key), item)
}
switch (name) { switch (name) {
case "keys": case "keys":
return Object.keys(requireObject()) return Object.keys(requireObject())
@ -79,32 +76,47 @@ export const invokeObjectMethod = (name: string, args: Array<unknown>, node: Ast
} }
return out return out
} }
case "fromEntries": {
if (args[0] instanceof CodeModeMap) {
const out: Record<string, unknown> = Object.create(null)
for (const [key, item] of args[0].map.entries()) addEntry(out, key, item)
return out
}
if (args[0] instanceof CodeModeURLSearchParams) {
const out: Record<string, unknown> = Object.create(null)
for (const [key, value] of args[0].params.entries()) guardedSet(out, key, value)
return out
}
const pairs = args[0] instanceof CodeModeSet ? Array.from(args[0].set.values()) : args[0]
if (!Array.isArray(pairs)) {
boundedData(args[0], "Object.fromEntries input")
throw new InterpreterRuntimeError("Object.fromEntries expects an array of [key, value] pairs.", node)
}
const out: Record<string, unknown> = Object.create(null)
for (const pair of pairs) {
const validated = boundedData(pair, "Object.fromEntries entry")
if (validated === null || typeof validated !== "object" || isCodeModeValue(validated))
throw new InterpreterRuntimeError("Object.fromEntries expects [key, value] entry objects.", node)
const entry = pair as Record<string, unknown>
addEntry(out, entry[0], entry[1])
}
return out
}
} }
throw new InterpreterRuntimeError(`Object.${name} is not available.`, node) throw new InterpreterRuntimeError(`Object.${name} is not available.`, node)
} }
export const invokeObjectFromEntries = <R>(
runner: SyncIteratorRunner<R>,
source: unknown,
node: AstNode,
): Effect.Effect<Record<string, unknown>, unknown, R> => {
const out: Record<string, unknown> = Object.create(null)
return Effect.gen(function* () {
const cursor = yield* runner.syncIterator(source, node)
if (cursor === undefined) {
throw new InterpreterRuntimeError("Object.fromEntries expects a synchronous iterable of entries.", node).as(
"TypeError",
)
}
while (true) {
const step = yield* cursor.next
if (step.done) return out
yield* preserveConsumerError(
cursor,
Effect.sync(() => {
if (
step.value === null ||
typeof step.value !== "object" ||
isCodeModeValue(step.value) ||
containsOpaqueReference(step.value)
) {
throw new InterpreterRuntimeError("Object.fromEntries expects [key, value] entry objects.", node).as(
"TypeError",
)
}
const entry = step.value as Record<string, unknown>
boundedData(entry[0], "Object.fromEntries key")
boundedData(entry[1], "Object.fromEntries value")
const key = coerceToString(entry[0])
if (isBlockedMember(key)) throw new InterpreterRuntimeError(`Property '${key}' is not available.`, node)
out[key] = entry[1]
}),
)
}
})
}

View file

@ -553,7 +553,7 @@ export const prepare = <R>(tools: Tools<R>, catalogBudget = defaultCatalogBudget
"## Language", "## Language",
"", "",
"Use common JavaScript data operations, functions, control flow, selected standard-library methods, and awaited tool calls. Built-ins include Date, RegExp, Map, Set, URL, URLSearchParams, and URI encoding helpers.", "Use common JavaScript data operations, functions, control flow, selected standard-library methods, and awaited tool calls. Built-ins include Date, RegExp, Map, Set, URL, URLSearchParams, and URI encoding helpers.",
"Modules/imports, classes, generators, timers, fetch, eval, prototype access, and unlisted methods are unavailable. Use tools for external operations. Use await with try/catch.", "Modules/imports, classes, timers, fetch, eval, prototype access, and unlisted methods are unavailable. Use tools for external operations. Use await with try/catch.",
"Prefer explicit `return`; otherwise only the final top-level expression becomes the result.", "Prefer explicit `return`; otherwise only the final top-level expression becomes the result.",
"Dates and URLs serialize to strings at data boundaries; Map/Set/RegExp/URLSearchParams serialize to `{}`.", "Dates and URLs serialize to strings at data boundaries; Map/Set/RegExp/URLSearchParams serialize to `{}`.",
] ]

View file

@ -732,9 +732,10 @@ describe("CodeMode public contract", () => {
expect(instructions).toContain("not a general-purpose runtime") expect(instructions).toContain("not a general-purpose runtime")
expect(instructions).not.toContain("Standard modern JavaScript works") expect(instructions).not.toContain("Standard modern JavaScript works")
expect(instructions).not.toContain("TypeScript type annotations") expect(instructions).not.toContain("TypeScript type annotations")
for (const missing of ["Modules/imports", "classes", "generators", "fetch"]) { for (const missing of ["Modules/imports", "classes", "fetch"]) {
expect(instructions).toContain(missing) expect(instructions).toContain(missing)
} }
expect(instructions).not.toContain("generators")
expect(instructions).not.toContain("new Promise(...) are unavailable") expect(instructions).not.toContain("new Promise(...) are unavailable")
expect(instructions).not.toContain("promise chaining") expect(instructions).not.toContain("promise chaining")
expect(instructions).toContain("URL, URLSearchParams, and URI encoding helpers") expect(instructions).toContain("URL, URLSearchParams, and URI encoding helpers")

File diff suppressed because it is too large Load diff