audio_io #

A Flutter plugin for cross-platform real-time audio streaming. Provides low-latency audio input/output with simple Stream-based API for audio processing, recording, and visualization.

Live demo #

Try the PCM16 streaming + Gemini Live example in your browser: wamf.github.io/audio_io — paste your own Gemini API key, allow the microphone, and talk.

Features #

• Real-time audio streaming from microphone to Flutter
• Audio output/playback through speakers
• Cross-platform support (iOS, macOS, Android, Web, Linux, Windows)
• Simple Stream-based API
• PCM16 byte streams at 16/24/48 kHz for realtime voice APIs (e.g. Gemini Live)
• Configurable audio latency modes
• Optional dedicated audio isolate on FFI platforms
• Consistent data format across all platforms (Float64, 48kHz, mono)
• Low-latency audio processing
• Volume level monitoring

Platform Support #

Getting Started #

Installation #

Add audio_io to your pubspec.yaml:

dependencies:
  audio_io: ^0.5.0

iOS Setup #

Add microphone usage description to your Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app needs access to the microphone for audio processing.</string>

macOS Setup #

1. Add microphone usage description to your macos/Runner/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app needs access to the microphone for audio processing.</string>

2. Enable audio input in your entitlements files:
   • macos/Runner/DebugProfile.entitlements
   • macos/Runner/Release.entitlements

<key>com.apple.security.device.audio-input</key>
<true/>

Android Setup #

Add microphone permission to your android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />

Request the microphone permission at runtime (for example with the permission_handler package) before calling start(). If permission has not been granted, start() throws an AudioIoException whose isPermissionDenied is true.

Usage #

import 'package:audio_io/audio_io.dart';

// Get the audio instance
final audioIo = AudioIo.instance;

// Configure latency (optional)
await audioIo.requestLatency(AudioIoLatency.Balanced);

// Start audio processing
await audioIo.start();

// Listen to input audio stream
audioIo.input.listen((audioData) {
  // Process audio data (List<double>)
  print('Received ${audioData.length} samples');
  
  // Calculate volume level (RMS)
  final sum = audioData.fold<double>(
    0.0, (sum, sample) => sum + sample * sample);
  final rms = sqrt(sum / audioData.length);
});

// Send audio to output (echo example)
audioIo.input.listen((data) {
  audioIo.output.add(data);
});

// Stop audio processing
await audioIo.stop();

Latency Configuration #

The plugin supports three latency modes:

enum AudioIoLatency {
  Realtime,  // Lowest latency (~1.5ms buffer)
  Balanced,  // Balanced latency/CPU (~3ms buffer)
  Powersave, // Lower CPU usage (~6ms buffer)
}

// Set before starting audio
await audioIo.requestLatency(AudioIoLatency.Realtime);

PCM16 byte streams (realtime voice APIs) #

Realtime voice APIs typically speak little-endian PCM16 at specific sample rates (Gemini Live expects 16 kHz in / 24 kHz out). startWith configures byte streams in the rate and format the API expects, while the engine keeps its internal 48 kHz contract — resampling and conversion are handled for you, on the audio rendering thread where the platform supports it:

final audioIo = AudioIo.instance;

await audioIo.startWith(const AudioIoConfig(
  format: AudioIoFormat.pcm16,
  sampleRate: AudioIoSampleRate.rate16000,
));

// Microphone as PCM16 bytes at 16 kHz
audioIo.inputBytes.listen(api.sendAudio);

// Play PCM16 bytes (decode + resample handled internally)
api.audioResponses.listen(audioIo.outputBytes.add);

// Barge-in: discard audio queued for playback but not yet rendered
await audioIo.clearOutput();

See example_gemini_live for a complete voice conversation app, or try it in the browser at wamf.github.io/audio_io.

Threading (optional dedicated audio isolate) #

By default the audio transport runs on the main isolate, which suits most apps and every platform. On the FFI back ends (Android, Windows, Linux) you can opt into a dedicated audio isolate so device polling and native buffer copies are unaffected by main-isolate jank (heavy widget builds, GC):

await audioIo.startWith(const AudioIoConfig(
  threading: AudioIoThreading.audioIsolate,
));

Platforms without dedicated-isolate support (iOS, macOS, web) silently fall back to main-isolate operation. The input / output streams still surface on the main isolate in both modes, so listener callbacks run there; move heavy DSP out of the listener if it competes with UI work.

Audio Format #

All platforms use a consistent audio format:

• Sample Rate: 48kHz (may adapt to device capabilities)
• Channels: Mono (1 channel)
• Data Type: Double precision floats (Float64/double)
• Stream Format: Chunks of audio samples as List<double>
• Internal Processing: Platform-specific (Float32 on native platforms)

Requirements #

• Dart SDK: >=3.0.0 <4.0.0
• Flutter: >=3.10.0
• iOS: 13.0 or higher
• macOS: 10.15 or higher

License #

See LICENSE file for details.