JavaScript Iterator Helpers: Lazy Transformations on Large Datasets

JavaScript Iterator Helpers are a TC39 Stage 4 proposal (landed in Chrome 122, Firefox 131, Safari 18) that adds .map(), .filter(), .take(), .drop(), .flatMap(), and .reduce() directly to iterators and generators — eliminating the need to spread into arrays just to use array methods.

Problem: Processing large datasets in WordPress admin tools — CSV exports, analytics aggregations, post bulk-operations — loads the entire dataset into an array before transforming it, causing high memory usage when the dataset has thousands of rows.

Solution: Use JavaScript Iterator Helpers (stage 4 as of 2024) to build lazy pipelines: .map(), .filter(), .take(n), and .toArray() on any iterable. Pair with a generator that yields rows from an API response page by page — only the rows needed for the current transformation are held in memory at any one time.

The examples below use iterator helpers to lazily process large post lists from the WordPress REST API, chain transformations on generator output, and compare iterator helper performance to array spread for large datasets.

// Iterator Helpers — available on all iterators including generators

// ── BASIC USAGE ──
function* numbers() {
    let i = 0;
    while (true) yield i++;  // infinite generator
}

// Without iterator helpers: must materialise the whole stream
// const first10Even = [...numbers()].filter(n => n % 2 === 0).slice(0, 10); // WRONG (infinite loop!)

// With iterator helpers: lazy — stops at 10, no array materialisation
const first10Even = numbers()
    .filter(n => n % 2 === 0)
    .take(10)
    .toArray();
// [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

// ── CHAINING ──
const postTitles = wpPosts                     // assume async iterator from API
    [Symbol.iterator]()
    .filter(post => post.status === 'publish')
    .map(post => post.title.rendered)
    .take(5)
    .toArray();

// ── FLATMAP: flatten nested data ──
function* getTagIds(posts) {
    yield* posts;
}

const allTagIds = getTagIds(wpPosts)
    .flatMap(post => Iterator.from(post.tags))  // flatten tags from each post
    .toArray();

// ── REDUCE: sum without spreading ──
const totalWordCount = Iterator.from(wpPosts)
    .map(post => post.content.rendered.split(/\s+/).length)
    .reduce((sum, count) => sum + count, 0);

Lazy pagination with iterator helpers:

// Async generator that paginates the WordPress REST API
async function* fetchAllPosts( perPage = 100 ) {
    let page = 1;
    while ( true ) {
        const resp = await fetch(
            `/wp-json/wp/v2/posts?per_page=${perPage}&page=${page}&_fields=id,title,status`
        );
        if ( ! resp.ok ) break;

        const posts = await resp.json();
        if ( posts.length === 0 ) break;

        yield* posts;
        if ( posts.length < perPage ) break;  // last page
        page++;
    }
}

// Process with iterator helpers (requires async iterator support)
// Currently iterator helpers work on sync iterators.
// For async, collect page-by-page then use helpers per page:
for await ( const page of paginate('/wp-json/wp/v2/posts', 100) ) {
    const titles = Iterator.from(page)
        .filter(p => p.status === 'publish')
        .map(p => p.title.rendered)
        .toArray();
    console.log(titles);
}

// ── ITERATOR.FROM() converts any iterable ──
const setIterator = Iterator.from( new Set([1, 2, 3, 2, 1]) );
const doubled = setIterator.map(n => n * 2).toArray();  // [2, 4, 6]

const mapIterator = Iterator.from( new Map([['a',1],['b',2]]) );
const values = mapIterator.map(([k,v]) => v).toArray();  // [1, 2]

NOTE: Iterator helpers are most valuable when working with large datasets where materialising into an array first would waste memory. Use the @tc39/proposal-iterator-helpers polyfill for Firefox and Safari coverage until all browsers support the feature. In Node.js, iterator helpers are available from version 22.