Async Io Model

tursodatabase/turso/.claude/skills/async-io-model

by tursodatabaseff97ec42cdefNo license24K starsListed Oct 9, 2026Updated Oct 8, 2026Repository updated today

Explanations of common asynchronous patterns used in tursodb. Involves IOResult, state machines, re-entrancy pitfalls, CompletionGroup. Always use these patterns in `core` when doing anything IO

Instructions onlySoftware Development Learning & Education

AI-generated overview

Explains Turso's cooperative async I/O model: IOResult, state machines, re-entrancy pitfalls, and CompletionGroup.

What it does: This skill provides reference guidance on the asynchronous I/O patterns used in the tursodb codebase, where cooperative yielding with explicit state machines replaces Rust async/await. It documents core types such as IOResult and IOCompletions, the Completion and CompletionGroup abstractions, and helper macros like return_if_io! and io_yield_one!. It also covers state machine design, common re-entrancy bugs that arise from mutating state before yield points, and testing approaches for async code. It produces explanations and code examples rather than executable artifacts.
When to use it: Use it when writing or reviewing I/O-related code in the tursodb core, especially when working with IOResult, completions, or state machines. It is also relevant when diagnosing re-entrancy bugs caused by state changes before yield points. The skill states these patterns should always be used in core when doing anything IO.
Requirements: No scripts or tooling are required; it is instructions and reference material only. Familiarity with Rust and the tursodb codebase is assumed, and it references repository files such as core/types.rs and core/io/completions.rs.

Async I/O Model Guide

Turso uses cooperative yielding with explicit state machines instead of Rust async/await.

Core Types

rust

pub enum IOCompletions {    Single(Completion),}
#[must_use]pub enum IOResult<T> {    Done(T),      // Operation complete, here's the result    IO(IOCompletions),  // Need I/O, call me again after completions finish}

pub enum IOCompletions {    Single(Completion),}
#[must_use]pub enum IOResult<T> {    Done(T),      // Operation complete, here's the result    IO(IOCompletions),  // Need I/O, call me again after completions finish}

Functions returning IOResult must be called repeatedly until Done.

Completion and CompletionGroup

A Completion tracks a single I/O operation:

rust

pub struct Completion { /* ... */ }
impl Completion {    pub fn finished(&self) -> bool;    pub fn succeeded(&self) -> bool;    pub fn get_error(&self) -> Option<CompletionError>;}

pub struct Completion { /* ... */ }
impl Completion {    pub fn finished(&self) -> bool;    pub fn succeeded(&self) -> bool;    pub fn get_error(&self) -> Option<CompletionError>;}

To wait for multiple I/O operations, use CompletionGroup:

rust

let mut group = CompletionGroup::new(|_| {});
// Add individual completionsgroup.add(&completion1);group.add(&completion2);
// Build into single completion that finishes when all completelet combined = group.build();io_yield_one!(combined);

let mut group = CompletionGroup::new(|_| {});
// Add individual completionsgroup.add(&completion1);group.add(&completion2);
// Build into single completion that finishes when all completelet combined = group.build();io_yield_one!(combined);

CompletionGroup features:

Aggregates multiple completions into one
Calls callback when all complete (or any errors)
Can nest groups (add a group's completion to another group)
Cancellable via group.cancel()

Helper Macros

`return_if_io!`

Unwraps IOResult, propagates IO variant up the call stack:

`io_yield_one!`

Yields a single completion:

State Machine Pattern

Operations that may yield use explicit state enums:

rust

enum MyOperationState {    Start,    WaitingForRead { page: PageRef },    Processing { data: Vec<u8> },    Done,}

enum MyOperationState {    Start,    WaitingForRead { page: PageRef },    Processing { data: Vec<u8> },    Done,}

The function loops, matching on state and transitioning:

rust

fn my_operation(&mut self) -> Result<IOResult<Output>> {    loop {        match &mut self.state {            MyOperationState::Start => {                let (page, completion) = start_read();                self.state = MyOperationState::WaitingForRead { page };                io_yield_one!(completion);            }            MyOperationState::WaitingForRead { page } => {                let data = page.get_contents();                self.state = MyOperationState::Processing { data: data.to_vec() };                // No yield, continue loop            }            MyOperationState::Processing { data } => {                let result = process(data);                self.state = MyOperationState::Done;                return Ok(IOResult::Done(result));            }            MyOperationState::Done => unreachable!(),        }    }}

fn my_operation(&mut self) -> Result<IOResult<Output>> {    loop {        match &mut self.state {            MyOperationState::Start => {                let (page, completion) = start_read();                self.state = MyOperationState::WaitingForRead { page };                io_yield_one!(completion);            }            MyOperationState::WaitingForRead { page } => {                let data = page.get_contents();                self.state = MyOperationState::Processing { data: data.to_vec() };                // No yield, continue loop            }            MyOperationState::Processing { data } => {                let result = process(data);                self.state = MyOperationState::Done;                return Ok(IOResult::Done(result));            }            MyOperationState::Done => unreachable!(),        }    }}

Re-Entrancy: The Critical Pitfall

State mutations before yield points cause bugs on re-entry.

Wrong

rust

fn bad_example(&mut self) -> Result<IOResult<()>> {    self.counter += 1;  // Mutates state    return_if_io!(something_that_might_yield());  // If yields, re-entry will increment again!    Ok(IOResult::Done(()))}

fn bad_example(&mut self) -> Result<IOResult<()>> {    self.counter += 1;  // Mutates state    return_if_io!(something_that_might_yield());  // If yields, re-entry will increment again!    Ok(IOResult::Done(()))}

If something_that_might_yield() returns IO, caller waits for completion, then calls bad_example() again. counter gets incremented twice (or more).

Correct: Mutate After Yield

rust

fn good_example(&mut self) -> Result<IOResult<()>> {    return_if_io!(something_that_might_yield());    self.counter += 1;  // Only reached once, after IO completes    Ok(IOResult::Done(()))}

fn good_example(&mut self) -> Result<IOResult<()>> {    return_if_io!(something_that_might_yield());    self.counter += 1;  // Only reached once, after IO completes    Ok(IOResult::Done(()))}

Correct: Use State Machine

rust

enum State { Start, AfterIO }
fn good_example(&mut self) -> Result<IOResult<()>> {    loop {        match self.state {            State::Start => {                // Don't mutate shared state here                self.state = State::AfterIO;                return_if_io!(something_that_might_yield());            }            State::AfterIO => {                self.counter += 1;  // Safe: only entered once                return Ok(IOResult::Done(()));            }        }    }}

enum State { Start, AfterIO }
fn good_example(&mut self) -> Result<IOResult<()>> {    loop {        match self.state {            State::Start => {                // Don't mutate shared state here                self.state = State::AfterIO;                return_if_io!(something_that_might_yield());            }            State::AfterIO => {                self.counter += 1;  // Safe: only entered once                return Ok(IOResult::Done(()));            }        }    }}

Common Re-Entrancy Bugs

Pattern	Problem
`vec.push(x); return_if_io!(...)`	Vec grows on each re-entry
`idx += 1; return_if_io!(...)`	Index advances multiple times
`map.insert(k,v); return_if_io!(...)`	Duplicate inserts or overwrites
`flag = true; return_if_io!(...)`	Usually ok, but check logic

State Enum Design

Encode progress in state variants:

rust

// Good: index is part of state, preserved across yieldsenum ProcessState {    Start,    ProcessingItem { idx: usize, items: Vec<Item> },    Done,}
// Loop advances idx only when transitioning statesProcessingItem { idx, items } => {    return_if_io!(process_item(&items[idx]));    if idx + 1 < items.len() {        self.state = ProcessingItem { idx: idx + 1, items };    } else {        self.state = Done;    }}

// Good: index is part of state, preserved across yieldsenum ProcessState {    Start,    ProcessingItem { idx: usize, items: Vec<Item> },    Done,}
// Loop advances idx only when transitioning statesProcessingItem { idx, items } => {    return_if_io!(process_item(&items[idx]));    if idx + 1 < items.len() {        self.state = ProcessingItem { idx: idx + 1, items };    } else {        self.state = Done;    }}

Turso Implementation

Key files:

core/types.rs - IOResult, IOCompletions, return_if_io!, return_and_restore_if_io!
core/io/completions.rs - Completion, CompletionGroup
core/util.rs - io_yield_one! macro
core/state_machine.rs - Generic StateMachine wrapper
core/storage/btree.rs - Many state machine examples
core/storage/pager.rs - CompletionGroup usage examples

Testing Async Code

Re-entrancy bugs often only manifest under specific IO timing. Use:

Deterministic simulation (testing/simulator/)
Whopper concurrent DST (testing/concurrent-simulator/)
Fault injection to force yields at different points

References

docs/manual.md section on I/O

Source and attribution

Source:tursodatabase/tursoin.claude/skills/async-io-modelat commitff97ec4

License: No license

Content belongs to its original authors. SourceWeft indexes it from a public repository.

Report or request removal

Async Io Model

Async I/O Model Guide

Core Types

Completion and CompletionGroup

Helper Macros

`return_if_io!`

`io_yield_one!`

State Machine Pattern

Re-Entrancy: The Critical Pitfall

Wrong

Correct: Mutate After Yield

Correct: Use State Machine

Common Re-Entrancy Bugs

State Enum Design

Turso Implementation

Testing Async Code

References

Source and attribution

More from tursodatabase/turso

Transaction Correctness

Storage Format

Testing

Debugging

Index Knowledge

Mvcc

More in Software Development

Playground

M5 Onboard

Cardputer Buddy

Web Design Guidelines

Vercel React Native Skills

Vercel React Best Practices