Flutter Auto Echo Cancellation WebSocket #

A Flutter plugin for real-time voice communication with automatic echo cancellation (AEC) over WebSocket. Perfect for building voice agents, real-time voice chat, and AI voice assistants.

✨ Features #

• 🎤 Native AEC - Hardware-level echo cancellation on iOS and Android
• 🔊 Full Duplex Audio - Simultaneous recording and playback
• 🌐 WebSocket Streaming - Real-time audio streaming over WebSocket
• 📱 Cross-Platform - iOS and Android support
• 🔇 Audio Controls - Mute/unmute, speaker control
• 📊 Audio Levels - Real-time input/output level monitoring
• 🔄 Auto Reconnect - Automatic reconnection on connection loss
• 📝 Transcripts - Support for speech-to-text transcripts

🎯 Why This Plugin? #

Standard audio recording in Flutter doesn't include echo cancellation. When building voice agents or real-time voice chat, the microphone picks up the speaker output, creating an echo feedback loop. This plugin solves that by using native platform APIs that provide hardware-level AEC.

The Problem #

❌ Without AEC:
Microphone captures speaker output → Echo feedback → Agent hears itself

✅ With This Plugin:
VoiceProcessingIO (iOS) / AcousticEchoCanceler (Android) → Clean audio

📦 Installation #

Add to your pubspec.yaml:

dependencies:
  flutter_auto_echo_cancellation_websocket: ^1.0.0

iOS Setup #

Add to your ios/Runner/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app needs microphone access for voice communication</string>

Android Setup #

Add to your android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

🚀 Quick Start #

import 'package:flutter_auto_echo_cancellation_websocket/flutter_auto_echo_cancellation_websocket.dart';

// Create plugin instance
final voicePlugin = VoicePlugin();

// Listen to events
voicePlugin.eventStream.listen((event) {
  if (event is TranscriptEvent) {
    print('${event.isUser ? "User" : "Agent"}: ${event.text}');
  } else if (event is ConnectionStateEvent) {
    print('Connection: ${event.state}');
  } else if (event is AgentStateEvent) {
    print('Agent: ${event.state}');
  }
});

// Connect to your voice server
await voicePlugin.connect(VoicePluginConfig(
  endpoint: 'wss://your-voice-server.com/websocket',
  agentId: 'your-agent-id',
  publicKey: 'your-public-key',
  metadata: {
    'user_id': 'user-123',
    'name': 'John Doe',
  },
));

// Control audio
await voicePlugin.setMicrophoneMuted(true);
await voicePlugin.setSpeakerMuted(false);

// Disconnect when done
voicePlugin.disconnect();
voicePlugin.dispose();

📖 API Reference #

VoicePlugin #

The main class for voice communication.

Properties

Methods

VoicePluginConfig #

Configuration for connecting to the voice server.

VoicePluginConfig(
  // Required
  endpoint: 'wss://your-server.com/websocket',
  agentId: 'your-agent-id',
  publicKey: 'your-public-key',
  
  // Optional
  metadata: {'user_id': 'user-123'},
  includeMetadataInPrompt: true,
  sampleRate: 16000,
  channels: 1,
  audioQuality: AudioQuality.standard,
  enableNoiseSuppression: true,
  enableAutoGainControl: true,
  bufferDurationMs: 20,
  connectionTimeoutSeconds: 30,
  autoReconnect: true,
  maxReconnectAttempts: 3,
  reconnectDelayMs: 1000,
  enableDebugLogs: false,
)

Events #

Listen to events via the eventStream:

🏗️ Architecture #

iOS Implementation #

Uses VoiceProcessingIO Audio Unit for hardware AEC:

┌─────────────────────────────────────────┐
│       VoiceProcessingIO Audio Unit      │
│   ┌───────────┐       ┌───────────┐    │
│   │    Mic    │       │  Speaker  │    │
│   │   Input   │       │  Output   │    │
│   └─────┬─────┘       └─────┬─────┘    │
│         │   AEC CORRELATES  │          │
│         │   BOTH STREAMS    │          │
│         ▼                   ▼          │
│   ┌─────────────────────────────┐      │
│   │  Echo Cancellation Engine   │      │
│   └─────────────────────────────┘      │
└─────────────────────────────────────────┘

Android Implementation #

Uses AudioRecord with VOICE_COMMUNICATION source and AcousticEchoCanceler:

┌─────────────────────────────────────────┐
│    AudioRecord (VOICE_COMMUNICATION)    │
│              +                          │
│    AcousticEchoCanceler                 │
│              +                          │
│    AudioTrack (same session ID)         │
└─────────────────────────────────────────┘

🧪 Testing Checklist #

• ❌ iOS: No echo when agent speaks
• ❌ iOS: Clear audio capture from microphone
• ❌ iOS: Works with speaker (not just earpiece)
• ❌ Android: No echo when agent speaks
• ❌ Android: AEC enabled on device
• ❌ Both: Metadata received by agent
• ❌ Both: Transcripts work correctly
• ❌ Both: Auto-reconnection works
• ❌ Both: Clean disconnect without artifacts

📱 Supported Platforms #

🤝 Contributing #

Contributions are welcome! Please read our contributing guidelines first.

📄 License #

MIT License - see LICENSE file for details.

👤 Author #

Ashraf K Salim

• GitHub: @FLYINGSKATE
• LinkedIn: ashrafksalim
• Email: [email protected]

🙏 Acknowledgments #

• Apple VoiceProcessingIO Documentation
• Android AcousticEchoCanceler