Collection Operators

Overview

rill provides four collection operators for transforming, filtering, and reducing data:

All four operators share similar syntax but differ in execution model and output.

Important: Loop bodies cannot modify outer-scope variables (see Variables). Use fold or each(init) with accumulators instead.

# Sequential: results in order, one at a time
[1, 2, 3] -> each { $ * 2 }     # list[2, 4, 6]

# Parallel: results in order, concurrent execution
[1, 2, 3] -> map { $ * 2 }      # list[2, 4, 6]

# Parallel filter: keep matching elements
[1, 2, 3, 4, 5] -> filter { $ > 2 }  # list[3, 4, 5]

# Reduction: accumulates to single value
[1, 2, 3] -> fold(0) { $@ + $ } # 6

Body Forms

Each operator accepts multiple body syntaxes. Choose based on readability and complexity.

Block Form

Use braces for multi-statement bodies. $ refers to the current element.

[1, 2, 3] -> each {
  $ => $x
  $x * 2
}
# Result: [2, 4, 6]

Grouped Expression

Use parentheses for single expressions. $ refers to the current element.

[1, 2, 3] -> each ($ + 10)
# Result: [11, 12, 13]

Inline Closure

Define parameters explicitly. The first parameter receives each element.

[1, 2, 3] -> each |x| ($x * 2)
# Result: [2, 4, 6]

Variable Closure

Reference a pre-defined closure by variable.

|x| ($x * 2) => $double
[1, 2, 3] -> each $double
# Result: [2, 4, 6]

Identity

Use bare $ to return elements unchanged.

[1, 2, 3] -> each $
# Result: [1, 2, 3]

Method Shorthand

Use .method to apply a method to each element. Equivalent to { $.method() }.

["hello", "world"] -> each .upper
# Result: ["HELLO", "WORLD"]

["  hi  ", " there "] -> map .trim
# Result: ["hi", "there"]

["hello", "", "world"] -> filter .empty
# Result: [""]

Methods can take arguments:

["a", "b"] -> map .pad_start(3, "0")
# Result: ["00a", "00b"]

Chain multiple methods:

["  HELLO  ", "  WORLD  "] -> map .trim.lower
# Result: ["hello", "world"]

For negation, use grouped expression:

["hello", "", "world"] -> filter (!.empty)
# Result: ["hello", "world"]

each — Sequential Iteration

each iterates over a collection in order. Each iteration completes before the next begins.

collection -> each body
collection -> each (init) body   # with accumulator

Basic Usage

# Double each number
[1, 2, 3] -> each { $ * 2 }
# Result: [2, 4, 6]

# Transform strings
["a", "b", "c"] -> each { "{$}!" }
# Result: ["a!", "b!", "c!"]

# Iterate string characters
"hello" -> each $
# Result: ["h", "e", "l", "l", "o"]

Dict Iteration

When iterating over a dict, $ contains key and value fields.

[name: "alice", age: 30] -> each { "{$.key}: {$.value}" }
# Result: ["name: alice", "age: 30"]

[a: 1, b: 2, c: 3] -> each { $.value * 2 }
# Result: [2, 4, 6]

With Accumulator

each supports an optional accumulator for stateful iteration. Two syntaxes exist.

Block Form with $@

Place initial value in parentheses before the block. Access accumulator via $@.

# Running sum (scan pattern)
[1, 2, 3] -> each(0) { $@ + $ }
# Result: [1, 3, 6]

# String concatenation
["a", "b", "c"] -> each("") { "{$@}{$}" }
# Result: ["a", "ab", "abc"]

Inline Closure Form

Define accumulator as the last parameter with a default value.

# Running sum
[1, 2, 3] -> each |x, acc = 0| ($acc + $x)
# Result: [1, 3, 6]

Early Termination

Use break to exit each early. Returns partial results collected before the break.

[1, 2, 3, 4, 5] -> each {
  ($ == 3) ? break
  $ * 2
}
# Result: [2, 4] (partial results before break)

Empty Collections

each returns [] for empty collections. The body never executes.

[] -> each { $ * 2 }
# Result: []

# With accumulator, still returns [] (not the initial value)
[] -> each(0) { $@ + $ }
# Result: []

Streams with each

each consumes stream chunks sequentially. Each chunk is one iteration. Returns a list of body results.

# Stream: each chunk is one call
app::lines("file.txt") -> each { $ -> .upper }
# Returns list of uppercased lines

An empty stream returns [] without executing the body.

each(init) carries the $@ accumulator across chunks. Returns the list of per-chunk results (not the final accumulator).

# Accumulator persists across stream chunks
app::stream_numbers() -> each(0) { $@ + $ }
# Returns running totals across all chunks

map — Parallel Iteration

map iterates concurrently using Promise.all. Order is preserved despite parallel execution.

collection -> map body

Basic Usage

# Map with closure parameter
["a", "b", "c"] -> map |x| { "{$x}!" }
# Result: ["a!", "b!", "c!"]

# Block expression (implicit $)
[1, 2, 3] -> map { $ * 2 }
# Result: [2, 4, 6]

# Grouped expression
[1, 2, 3] -> map ($ * 2)
# Result: [2, 4, 6]

Key Differences from each

1. No accumulator: Parallel execution has no “previous” value
2. No break: Cannot exit early from concurrent operations
3. Concurrent execution: All iterations start immediately

When to Use map

Use map when:

• Operations are independent (no shared state)
• Order of execution doesn’t matter (results still ordered)
• I/O-bound operations benefit from concurrency

# CPU-bound: same result as each, but runs in parallel
[1, 2, 3, 4, 5] -> map { $ * $ }
# Result: [1, 4, 9, 16, 25]

Empty Collections

map returns [] for empty collections. The body never executes.

[] -> map { $ * 2 }
# Result: []

Streams with map

map transforms each stream chunk and returns a list (not a stream). All chunks are consumed before the result is available.

# Each stream chunk is transformed; result is a list
app::stream_numbers() -> map { $ * 2 }
# Returns list[...] — not a stream

An empty stream returns [].

filter — Parallel Filtering

filter keeps elements where the predicate returns true. Predicates must return boolean values. Executes concurrently using Promise.all.

collection -> filter body

Basic Usage

# Keep numbers greater than 2
[1, 2, 3, 4, 5] -> filter { $ > 2 }
# Result: [3, 4, 5]

# Keep non-empty strings
["hello", "", "world", ""] -> filter { !.empty }
# Result: ["hello", "world"]

# Keep even numbers
[1, 2, 3, 4, 5, 6] -> filter { ($ % 2) == 0 }
# Result: [2, 4, 6]

All Body Forms

filter accepts the same body forms as map:

# Block form
[1, 2, 3, 4, 5] -> filter { $ > 2 }

# Grouped expression
[1, 2, 3, 4, 5] -> filter ($ > 2)

# Inline closure
[1, 2, 3, 4, 5] -> filter |x| ($x > 2)

# Variable closure
|x| ($x > 2) => $gtTwo
[1, 2, 3, 4, 5] -> filter $gtTwo

Dict Filtering

When filtering a dict, $ contains key and value fields. Returns list of matching entries.

[a: 1, b: 5, c: 3] -> filter { $.value > 2 }
# Result: [[key: "b", value: 5], dict[key: "c", value: 3]]

String Filtering

Filters characters in a string.

"hello" -> filter { $ != "l" }
# Result: ["h", "e", "o"]

Chaining with Other Operators

# Filter then transform
[1, 2, 3, 4, 5] -> filter { $ > 2 } -> map { $ * 2 }
# Result: [6, 8, 10]

# Transform then filter
[1, 2, 3, 4, 5] -> map { $ * 2 } -> filter { $ > 5 }
# Result: [6, 8, 10]

# Filter, transform, reduce
[1, 2, 3, 4, 5] -> filter { $ > 2 } -> map { $ * 2 } -> fold(0) { $@ + $ }
# Result: 24

Empty Collections

filter returns [] for empty collections or when nothing matches.

[] -> filter { $ > 0 }
# Result: []

[1, 2, 3] -> filter { $ > 10 }
# Result: []

Streams with filter

filter tests each stream chunk and returns a list (not a stream). Chunks that pass the predicate are included; others are dropped.

# Each stream chunk is tested; result is a list
app::stream_numbers() -> filter { $ > 0 }
# Returns list[...] of matching chunks — not a stream

An empty stream returns [].

fold — Sequential Reduction

fold reduces a collection to a single value. Requires an accumulator.

Syntax forms:

• Block form: collection -> fold(init) { body }
• Closure form: collection -> fold |x, acc = init| (body)
• Variable closure: collection -> fold $fn

Basic Usage

# Sum numbers
[1, 2, 3] -> fold(0) { $@ + $ }
# Result: 6

# Same with inline closure
[1, 2, 3] -> fold |x, sum = 0| ($sum + $x)
# Result: 6

Common Patterns

Sum

[1, 2, 3, 4, 5] -> fold(0) { $@ + $ }
# Result: 15

Product

[1, 2, 3, 4] -> fold(1) { $@ * $ }
# Result: 24

Maximum

[3, 1, 4, 1, 5, 9] -> fold(0) {
  ($@ > $) ? $@ ! $
}
# Result: 9

Count

[1, 2, 3, 4, 5] -> fold(0) { $@ + 1 }
# Result: 5

String Join

["a", "b", "c"] -> fold("") { "{$@}{$}" }
# Result: "abc"

# With separator
["a", "b", "c"] -> fold |x, acc = ""| {
  ($acc -> .empty) ? $x ! "{$acc},{$x}"
}
# Result: "a,b,c"

Dict Reduction

When folding over a dict, $ contains key and value fields.

[a: 1, b: 2, c: 3] -> fold |entry, sum = 0| ($sum + $entry.value)
# Result: 6

Reusable Reducers

Define closures for common reductions.

# Define reusable reducers
|x, sum = 0| ($sum + $x) => $summer
|x, max = 0| (($x > $max) ? $x ! $max) => $maxer

# Use with different data
[1, 2, 3] -> fold $summer => $r1     # 6
[3, 7, 2] -> fold $maxer => $r2      # 7
[9, 1, 5] -> fold $maxer => $r3      # 9

Empty Collections

fold returns the initial value for empty collections. The body never executes.

[] -> fold(0) { $@ + $ }
# Result: 0

[] -> fold(42) { $@ + $ }
# Result: 42

[] -> fold |x, acc = 100| ($acc + $x)
# Result: 100

Streams with fold

fold reduces stream chunks with the accumulator, returning a single value. The accumulator $@ carries state across every chunk.

# Reduce all stream chunks to a single value
app::stream_numbers() -> fold(0) { $@ + $ }
# Returns the sum of all chunks

An empty stream returns the initial value without executing the body.

Comparison: each vs fold

Both each and fold support accumulators. The difference is in what they return.

Side-by-Side Example

# each: returns every intermediate result
[1, 2, 3] -> each(0) { $@ + $ }
# Result: [1, 3, 6]  (running totals)

# fold: returns only the final result
[1, 2, 3] -> fold(0) { $@ + $ }
# Result: 6  (final sum)

When to Choose

Use each with accumulator when you need intermediate states (scan pattern):

# Running balance
[100, -50, 200, -75] -> each(0) { $@ + $ }
# Result: [100, 50, 250, 175]

Use fold when you only need the final result:

# Final balance
[100, -50, 200, -75] -> fold(0) { $@ + $ }
# Result: 175

Stream Iteration

Streams produce chunks over time. Collection operators consume all chunks before returning. All stream examples use text fences because stream host functions are unavailable in the test harness.

break in Stream Operators

break stops iteration immediately. The host cleanup function (dispose) runs to release stream resources.

# Stop after the first matching chunk; host disposes the stream
app::log_stream() -> each {
  ($ -> .contains("ERROR")) ? break
  $
}

^(limit: N) on Streams

The ^(limit: N) operator stops iteration after N chunks and calls host cleanup.

# Process at most 100 chunks; host disposes the stream
app::events() ^(limit: 100) -> each { $ }

Use ^(limit: N) when you want a fixed-size sample from an unbounded stream.

Iteration Ceiling

Exactly 10,000 chunks complete without error. The 10,001st chunk triggers RILL-R010 and halts execution. Use ^(limit: N) to consume at most N chunks and stay within bounds.

# Error: exceeds iteration ceiling
app::infinite_stream() -> each { $ }
# RILL-R010: halts on the 10,001st chunk

Re-iteration Halt

A consumed stream cannot be re-iterated. Passing a consumed stream to a second operator halts execution with an error.

# Error: stream already consumed
app::stream_numbers() => $s
$s -> each { $ }
$s -> each { $ }   # Halts: stream is consumed

Chaining Operators

Combine operators for multi-stage transformations.

# Double each element, then sum
[1, 2, 3] -> map { $ * 2 } -> fold(0) { $@ + $ }
# Result: 12

# Filter even numbers (using parallel filter)
[1, 2, 3, 4, 5] -> filter { ($ % 2) == 0 }
# Result: [2, 4]

# Complex pipeline: filter, then transform
[1, 2, 3, 4, 5] -> filter { $ > 2 } -> map { $ * 10 }
# Result: [30, 40, 50]

Closure Arity Rules

For inline closures with accumulators, specific rules apply.

Requirements

1. At least 2 parameters — first receives element, last is accumulator
2. Last parameter must have default — the default is the initial value
3. Parameters between first and last must have defaults — no gaps
4. Incoming args must exactly fill params before accumulator

Valid Closures

Invalid Closures

Error Cases

Iterating Different Types

Lists

[1, 2, 3] -> each { $ * 2 }
# Result: [2, 4, 6]

Strings

Iterates over characters.

"abc" -> each { "{$}!" }
# Result: ["a!", "b!", "c!"]

Dicts

Iterates over entries with key and value fields.

[a: 1, b: 2] -> each { "{$.key}={$.value}" }
# Result: ["a=1", "b=2"]

Nested Collections

Process nested structures with nested operators.

# Double nested values
[list[1, 2], list[3, 4]] -> map |inner| { $inner -> map { $ * 2 } }
# Result: [list[2, 4], list[6, 8]]

# Sum all nested values
[list[1, 2], list[3, 4]] -> fold(0) |inner, total = 0| { $total + ($inner -> fold(0) { $@ + $ }) }
# Result: 10

Performance Considerations

Sequential vs Parallel

Memory

• each and map allocate result lists proportional to input size
• fold maintains constant memory (accumulator only)

Quick Reference

# each - sequential, all results
[1, 2, 3] -> each { $ * 2 }           # list[2, 4, 6]
[1, 2, 3] -> each(0) { $@ + $ }       # list[1, 3, 6] (running sum)

# map - parallel, all results
[1, 2, 3] -> map { $ * 2 }            # list[2, 4, 6]
["a", "b"] -> map |x| { "{$x}!" }     # list["a!", "b!"]

# filter - parallel, matching elements
[1, 2, 3, 4, 5] -> filter { $ > 2 }   # list[3, 4, 5]
|x| { $x % 2 == 0 } => $isEven
[1, 2, 3, 4, 5] -> filter $isEven     # list[2, 4]

# fold - sequential, final result only
[1, 2, 3] -> fold(0) { $@ + $ }       # 6
[1, 2, 3] -> fold |x, s = 0| ($s + $x) # 6

# Dict iteration
[a: 1, b: 2] -> each { $.key }        # ["a", "b"]
[a: 1, b: 2] -> each { $.value }      # [1, 2]

# Break (each only)
[1, 2, 3] -> each { ($ > 2) ? break ! $ }  # list[1, 2] (partial results)

# Empty collections
[] -> each { $ }      # []
[] -> map { $ }       # []
[] -> filter { $ }    # []
[] -> fold(42) { $ }  # 42

See Also

• Iterators — Lazy sequences with range, repeat, .first()
• Reference — Complete language specification
• Closures — Closure semantics and patterns
• Guide — Beginner introduction
• Examples — Workflow examples