← back to index

Generators and Coroutines: From spawn to async/await

The stackless coroutine that powers JS generators, and how async/await desugars to a generator + executor.

published Apr 09, 2019 tags #javascript #generators #async

~/posts/generators-and-coroutines $ cat post.md

/ LANG EN / 中文
/ THEME / /

Continuing from before

The earlier post, Flattening Arrays: From Recursion to Generators, introduced yield and yield* and how delegation works. The broader use case for generators is writing asynchronous functions — this post covers the coroutine model behind them, generator-based async, and how async/await is implemented on top of generators.

Coroutines

Generators can pause and resume because of coroutines. In Go, coroutines are a core feature; in Node, the implementation is more object-oriented and called a stackless coroutine — the explicit coroutine context doesn’t sit on the stack.

Function recursion inherently uses a stack, but the stack can be hidden inside a dependency structure like this. Stackful coroutines use registers to index locals — variable accesses go through a register offset. Stackless coroutines usually use this as their context — the difference is that the context locals are stored in an object whose fields hold pre-indexed addresses.

Either way, the stack outlives the function call. After the function returns, the simulated registers go away; this, doubling as both the plain function context and the coroutine context, doesn’t even need separate teardown.

Infinite loops

If you want to drain a generator that produces an infinite stream:

function* infiniteNumber(): IterableIterator<number> {
    let index: number = 0;
    while (1) {
        yield index++;
    }
}

const iterator: IterableIterator<number> = infiniteNumber();
while (1) {
    if (iterator.next().done) {
        break;
    }
}

The break is unreachable — every call to next returns a value. Conversely this is the canonical way to exhaust a generator.

An async executor: spawn

spawn is the recursive auto-driver that lets a coroutine “await” a Promise before continuing:

const spawn = (genF) => {
    return new Promise((resolve, reject) => {
        const gen = genF();
        const step = (nextF) => {
            let next;
            try {
                next = nextF();
            } catch (e) {
                return reject(e);
            }
            if (next.done) {
                return resolve(next.value);
            }
            Promise.resolve(next.value).then(
                (v) => step(() => gen.next(v)),
                (e) => step(() => gen.throw(e)),
            );
        };
        step(() => gen.next(undefined));
    });
};

Adapted from Ruan Yifeng’s version.

What it does: pull a yielded value out of the generator, wrap it as a Promise, and on resolve push the result back via gen.next(v) so the generator can keep running. On reject, push the error in with gen.throw(e). Loop until the generator is done.

How async/await is implemented

An async function is, mechanically, a generator driven by an executor like spawn.

  • async function fn() { ... } is treated, by the compiler or runtime, as “a function returning a Promise, whose body is a generator; each await is a value yielded out, the executor resolves the Promise and feeds the result back in”.
  • await only works inside async because it needs the generator machinery to suspend and resume; a plain function can’t pause in the middle.

So this:

async function readBoth() {
    const a = await readFileA();
    const b = await readFileB();
    return [a, b];
}

Is semantically equivalent to:

function readBoth() {
    return spawn(function* () {
        const a = yield readFileA();
        const b = yield readFileB();
        return [a, b];
    });
}

Every await is a yield; the executor consumes the Promise and, on resolve, hands the result back to the generator, which keeps running. The async/await syntax is sugar over a generator-plus-executor pair.

← back to index