ffmpeg_streamer

A high-performance Flutter plugin for using FFmpeg to decode video and audio frames via FFI. Now with asynchronous API and native threading for 7-10x faster batch operations!

✨ Features

Version 2.0 - New!

  • 🚀 Async API with Callbacks: Non-blocking frame retrieval with native threading
  • ⚡ Ultra-Fast Batch Processing: 87% faster for 100 frames (15s → 2s)
  • 🧵 Native Threading (pthread): Dedicated worker thread, zero UI blocking
  • 📊 Progress Tracking: Real-time progress callbacks for batch operations
  • 🔄 Request Cancellation: Cancel pending decode requests

Core Features

  • 🎥 Video Decoding: Access raw RGBA video frames
  • 🔊 Audio Decoding: Access raw Float32 audio samples
  • 📱 Cross-Platform: Android, iOS, macOS, Windows, Linux
  • 🔙 Backward Compatible: Old sync API still works

Prerequisites & Setup

IMPORTANT: This plugin requires FFmpeg binaries. You must provide them due to licensing and size.

Android

  1. Download Android-compatible FFmpeg .so libraries (e.g., from FFmpegKit or build yourself).

  2. Place them in your app's android/source/main/jniLibs/<ABI>/ or configure the plugin src/main/jniLibs. Required libraries:

    • libavformat.so
    • libavcodec.so
    • libavutil.so
    • libswscale.so
    • libswresample.so

    Headers Place the FFmpeg include directories (libavcodec/, libavformat/, etc.) in: android/src/main/cpp/include/

    So you should have android/src/main/cpp/include/libavcodec/avcodec.h, etc.

iOS & macOS

  1. iOS: Add a Pod dependency on an FFmpeg package or vend ffmpeg.xcframework in your Podfile.
  2. macOS: Ensure FFmpeg is installed via Homebrew (brew install ffmpeg) or linked in your macos/Runner.xcodeproj.
  3. The plugin looks for headers in standard /usr/local/include or /opt/homebrew/include on macOS.

Windows

  1. Set FFMPEG_ROOT CMake variable or place FFmpeg headers/libs in windows/ffmpeg.
  2. Ensure avcodec-*.dll etc. are in the same folder as your executable when running.

Linux

  1. Install development packages: 
    sudo apt-get install libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libswresample-dev

🚀 Quick Start

Basic Usage (Sync API)

import 'package:ffmpeg_streamer/ffmpeg_streamer.dart';

void main() async {
  final decoder = FfmpegDecoder();
  
  // Open media file
  await decoder.openMedia('path/to/video.mp4');
  
  // Get media info
  print('Resolution: ${decoder.videoWidth}x${decoder.videoHeight}');
  print('FPS: ${decoder.fps}');
  print('Total frames: ${decoder.totalFrames}');
  
  // Get a single frame (sync)
  final frame = await decoder.getFrameAtIndex(42);
  if (frame?.video != null) {
    // Use frame.video.rgbaBytes (Uint8List)
    displayFrame(frame!.video!);
  }
  
  // Cleanup
  await decoder.dispose();
}

🔥 New Async API (Recommended for Performance)

import 'package:ffmpeg_streamer/ffmpeg_streamer.dart';

void main() async {
  final decoder = FfmpegDecoder();
  await decoder.openMedia('path/to/video.mp4');
  
  // Get a single frame with callback (async, non-blocking)
  decoder.getFrameAtIndexAsync(42, (frame) {
    if (frame?.video != null) {
      displayFrame(frame!.video!);
    }
  });
  
  // Get multiple frames - ULTRA FAST! (87% faster than loop)
  decoder.getFramesRangeByIndexAsync(
    0,  // start frame
    99, // end frame
    (frame) {
      // Called for EACH frame as it's decoded
      print('Received frame ${frame?.video?.frameId}');
      processFrame(frame);
    },
    progressCallback: (current, total) {
      // Real-time progress tracking
      print('Progress: ${(current/total*100).toInt()}%');
    },
  );
  
  await decoder.dispose();
}

📊 Performance Comparison

// ❌ OLD WAY (slow - 15 seconds for 100 frames)
for (int i = 0; i < 100; i++) {
  final frame = await decoder.getFrameAtIndex(i);
  processFrame(frame);
}

// ✅ NEW WAY (fast - 2 seconds for 100 frames!)
decoder.getFramesRangeByIndexAsync(0, 99, 
  (frame) => processFrame(frame)
);

🎬 Real-World Example: Thumbnail Generator

void generateThumbnails(String videoPath, int count) async {
  final decoder = FfmpegDecoder();
  await decoder.openMedia(videoPath);
  
  final thumbnails = <Image>[];
  final step = decoder.totalFrames ~/ count;
  
  decoder.getFramesRangeByIndexAsync(
    0,
    (count - 1) * step,
    (frame) async {
      if (frame?.video != null) {
        final image = await frameToImage(frame!.video!);
        thumbnails.add(image);
      }
    },
    progressCallback: (current, total) {
      updateProgressBar(current / total);
    },
  );
}

📚 Documentation

🧪 Testing Performance

Run the included performance test:

dart run test_async_perf.dart path/to/your/video.mp4

This will compare sync vs async performance and show you the improvements!

License

This plugin code is licensed under the MIT License. FFmpeg is licensed under LGPL or GPL. You are responsible for complying with the FFmpeg license in your final application.

Libraries

ffmpeg_streamer
ffmpeg_streamer_bindings_generated