flutter_litert 4.1.1

flutter_litert: ^4.1.1 copied to clipboard

Fixes the iOS and macOS Interpreter slowdown with the XNNPACK delegate that 4.1.0 listed under "Known issues". No API changes; Android, Linux, Windows, CompiledModel and the flex delegate are unchanged.

Fixes #

• iOS and macOS: the Interpreter with the XNNPACK delegate and more than one thread is as fast as in 4.0.0 again. TensorFlow v2.21.0 pins a pthreadpool that dropped its Grand Central Dispatch backend on Apple platforms. The replacement spins briefly and then sleeps a worker on a condition variable, and it holds the pool's mutex while it spins, so every parallel section of an inference paid a wake-up and the workers queued on each other. The TensorFlow Lite C frameworks (iOS, release tflite-ios-v2.21.0-xnn1) and the macOS dylib are rebuilt with patches/pthreadpool_apple_spin.patch, which spins without the mutex, polls the completion flag with plain loads instead of atomic exchanges (an exchange per spin iteration starved the worker that sets the flag on the iPhone), and spins as long as pthreadpool does on Android. Measured on a physical iPhone 15 Pro with 4 XNNPACK threads over 28 models, same day and same build settings, two rounds of two passes each: 4.1.0 was a median 3 to 10% slower than 4.0.0 (total -1 to +5%); 4.1.1 is a median 13 to 23% faster than 4.0.0 (total 3 to 9% lower) and 23 to 27% faster than 4.1.0 (total 7 to 11% lower), with 0 to 5 of the 28 models slower than 4.0.0 by at most 3%. On an Apple silicon Mac (M-series, 4 threads, same 28 models) 4.1.1 is a median 18 to 26% faster than 4.1.0; 4.1.0 was not slower than 4.0.0 there (it was a median 16% faster, from the newer XNNPACK), so the macOS gain is on top of that. Results are unchanged. CPU use between inferences is unchanged: a worker spins for under a millisecond after the last parallel section before it sleeps.
• iOS CocoaPods and Swift Package Manager download the rebuilt frameworks from release tflite-ios-v2.21.0-xnn1; the Metal and Core ML delegate frameworks are the same binaries as in 4.1.0.

Moves the Interpreter on iOS, macOS, Linux, and Windows from TensorFlow Lite 2.20.0 to TensorFlow v2.21.0, and fixes CompiledModel on Windows machines without a recent Visual C++ runtime. Android (LiteRT 2.3.2) and every CompiledModel runtime are unchanged. There are no breaking API changes.

Runtime updates #

• iOS, macOS, Linux, and Windows Interpreter: TensorFlow Lite 2.20.0 to 2.21.0, built from the TensorFlow v2.21.0 tag. Interpreter.version now reports 2.21.0 on these platforms. 2.21.0 adds the int2, int4, and uint4 tensor types; int8 and int16x8 SQRT; int16x8 EQUAL and NOT_EQUAL; int2 and int4 in CAST, int4 in SLICE, and SRQ int2 in FULLY_CONNECTED; and a newer XNNPACK. The C API is unchanged. Desktop and Apple op support now matches Android's more closely.
• iOS: the TensorFlow Lite frameworks are hosted at release tflite-ios-v2.21.0. Swift Package Manager takes the Core ML delegate from release coreml-ios-v1.2.0. CocoaPods replaces a cached framework bundle from an older release on the next pod install.
• macOS: the committed TensorFlow Lite C, Metal, and Core ML dylibs are rebuilt at 2.21.0.
• The minimum OS versions do not change: iOS 15.0, macOS 10.11 (x86_64) and 14.5 (arm64) for the TensorFlow Lite C library, macOS 12.0 for the Metal and Core ML delegates, and glibc 2.38 on Linux.
• Pairs with flutter_litert_flex 2.1.0, whose flex delegate is rebuilt at 2.21.0. Use flutter_litert_flex 2.0.0 with 4.0.x.

Desktop builds #

• Linux and Windows: new CMake option FLUTTER_LITERT_BUNDLE_DESKTOP_RUNTIME, ON by default. OFF stops the plugin from downloading and bundling its CompiledModel runtime: libLiteRt and the WebGPU accelerator, plus dxcompiler.dll and dxil.dll on Windows. That is an 18.9 MB download and 27 MB of bundle on Linux, and a 50 MB download and 57 MB of bundle on Windows. It is for apps that ship another copy of the LiteRT runtime under the same file names, as LiteRT-LM does: that copy overwrites the plugin's files in the app bundle anyway. The Interpreter library, the custom ops library, and vcruntime140_threads.dll on Windows are bundled in both settings. An app sets it in its own linux/CMakeLists.txt and windows/CMakeLists.txt, before include(flutter/generated_plugins.cmake):

  set(FLUTTER_LITERT_BUNDLE_DESKTOP_RUNTIME OFF)
  

  See Linux and Windows build options.

• Linux: the prebuilt libraries are x86_64 only, and the plugin now bundles them (and downloads the accelerator) only when the target is x86_64. Until now a Linux arm64 app got the x86_64 libtensorflowlite_c-linux.so, libLiteRt.so, and libLiteRtWebGpuAccelerator.so in its bundle, none of which can load there. On other architectures the plugin still builds, bundles only the custom ops library, and prints a CMake warning that no native runtime is available.

Fixes #

• Windows: CompiledModel failed to load (error 126) on machines without a recent Visual C++ runtime, because Google's libLiteRt.dll imports vcruntime140_threads.dll. This was the case since 3.1.1. The plugin now bundles that file next to libLiteRt.dll. It is Microsoft's unmodified redistributable file from the Visual C++ 2022 redistributable (Microsoft.VC143.CRT, version 14.44.35211.0).
• Windows: the Interpreter library libtensorflowlite_c-win.dll is now built with the static C runtime (/MT). It imports only KERNEL32.dll and needs no Visual C++ redistributable. Its 98 exports are the same as in 2.20.0. TensorFlow 2.21's thread pool uses C11 threads, which with the dynamic runtime would need vcruntime140_threads.dll too. tflite_custom_ops.dll still uses the dynamic runtime.
• iOS CocoaPods: the podspec now checks the SHA-256 of each framework zip it downloads, and a download that cannot connect within 30 seconds or stalls for 60 seconds fails and is retried instead of hanging pod install.

Known issues #

• iOS: the Interpreter with the XNNPACK delegate is slower than in 4.0.0. Measured on a physical iPhone with 4 threads, the same day and the same build settings for both versions: the median over 28 models is 11% slower and the total time 8% higher. Small models take about 0.5 to 0.9 ms more per inference; large models 3 to 4 ms more. Results are unchanged. The plain CPU path and the flex delegate show no difference beyond run-to-run noise. The likely cause is the thread pool: TensorFlow 2.21 pins a newer pthreadpool that no longer uses Grand Central Dispatch on Apple platforms. The macOS library has the same change. Linux and Windows are not affected: their backends did not change. Fixed in 4.1.1.

Moves the Android Interpreter from LiteRT 1.4.2 to LiteRT 2.x, so both Android APIs now run on one shared libLiteRt.so (2.3.2), and updates every CompiledModel runtime. LiteRT 2.x has no Interpreter GPU delegate and no flex delegate on Android, so this release removes both there. iOS, macOS, Linux, and Windows keep their Interpreter delegates. Read the migration guide below before upgrading an Android app.

Breaking changes #

• The Android Interpreter runs on LiteRT 2.x. The bundled runtime is the libLiteRt.so from Google's com.google.ai.edge.litert:litert 2.3.2 AAR, the same library CompiledModel uses. LiteRT 1.4.2 is gone: the litert:1.4.2 and litert-gpu:1.4.2 dependencies are no longer added to the app, and Interpreter.version on Android now reports 2.23.0 instead of 2.22.0-dev0+selfbuilt. The bundled native libraries are 4.2 to 8.7 MB smaller per ABI (arm64-v8a 8.9 MB, armeabi-v7a 3.7 MB, x86_64 11.7 MB, against about 15.7, 7.8, and 20.4 MB in 3.10.0).

• Android minSdk is now 24 for every runtime, because libLiteRt.so needs API 24. An app below 24 fails to build with:

  Manifest merger failed : uses-sdk:minSdkVersion 23 cannot be smaller than version 24 declared in library [:flutter_litert]
  

  Raise the app's minSdk to 24. tools:overrideLibrary does not help: the build then succeeds, but LiteRT cannot load below API 24. The bundled minimum was 21 in 3.10.x.

• The iOS minimum is now 15.0 (it was 13.0), for both CocoaPods and Swift Package Manager. The new LiteRt and LiteRtMetalAccelerator frameworks are built for iOS 15.0 (LiteRT's own minimum) and load at app launch, so an app with a lower target could not run them. Raise the app's IPHONEOS_DEPLOYMENT_TARGET, and for CocoaPods platform :ios, '15.0' in ios/Podfile.

• Removed: GpuDelegateV2, GpuDelegateOptionsV2, and the TfLiteGpuInferenceUsage, TfLiteGpuInferencePriority, and TfLiteGpuExperimentalFlags enums. They were the Android GL/CL GPU delegate for the Interpreter, and LiteRT 2.x has none. This reverses the 3.7.0 statement that no removal was scheduled: the runtime that provided the delegate is gone, so it could not be kept. On Android the GPU now means CompiledModel with Accelerator.gpu. The Metal GpuDelegate on iOS and macOS is unchanged.

• PerformanceConfig.gpu() and PerformanceMode.gpu use XNNPACK on the bundled Android runtime. They never throw: the first use in an isolate prints one warning and continues on XNNPACK. They still use Metal on iOS and macOS, PerformanceConfig.auto() is unchanged everywhere, and the Play services runtime still runs them on its CPU path.

• Android flex is removed. LiteRT 2.x has no flex delegate (Select TF ops) for Android. FlexDelegate.isAvailable is false on Android, and FlexDelegate() and FlexDelegate.create() throw an UnsupportedError that explains what to do. An Android build that still includes flutter_litert_flex fails at Gradle configuration with the same advice. flutter_litert_flex 2.0.0, released with this version, supports iOS, macOS, Linux, and Windows only. Without flex, Android cannot run SELECT_TF_OPS models, including training models that need TensorFlow ops (such as some Conv2D training graphs) and checkpoint save and restore through SaveV2/RestoreV2. Weight persistence through get_weights/set_weights signatures still works.

• Android 32-bit x86 is no longer packaged. The ABIs are arm64-v8a, armeabi-v7a, and x86_64. Flutter does not build 32-bit x86 Android apps, so Flutter apps are not affected in practice.

• The org.tensorflow.lite Java classes leave the app's classpath. They came from the litert:1.4.2 dependency, which the plugin no longer adds. Android code that imported them must add its own LiteRT dependency. Apps that already declare one no longer conflict with the plugin's.

• Small CPU numerics differences on Android. LiteRT 2.3.2 is not bit-identical to 1.4.2. Float CPU results stayed within 8.8e-05 relative on the custom-op tests. Across the 29-model matrix on the Pixel 9 Pro, Galaxy S23, and Galaxy A56, every model still ran and passed the accuracy check on CPU and XNNPACK, CPU outputs differed by up to 9.3e-4 relative, and no top index changed. Quantized models on the plain CPU path moved more: MobileNet changed 7 of 1001 scores (the largest from 83 to 90, top-5 order unchanged), and YAMNet changed 2 of 521 (0.8516 to 0.8008, top-1 unchanged, but class 132 now ties class 147). Quantized XNNPACK results are bit-identical. Byte-level goldens recorded on Android need regenerating.

Runtime updates #

• Android CompiledModel: LiteRT 2.2.0 to 2.3.2 (Google Maven). The GPU accelerator libLiteRtClGlAccelerator.so now comes from the litert-gpu 2.3.2 AAR and is still packaged for arm64-v8a and x86_64 only; armeabi-v7a is CPU-only.
• Android Qualcomm NPU: the compiler plugin and dispatch libraries now come from Google Maven (litert-npu-runtime-qualcomm 2.3.2), and the QAIRT pin moves from 2.47.0.260601 to 2.50.0.260828. A flutterLitert.qualcommNpuRuntimeDir bundle must be rebuilt from those two. See doc/android_compiled_model_npu.md. Known issue: on the Galaxy S23, selfie_segmenter and selfie_segmenter_landscape now fail in the NPU modes that worked with QAIRT 2.47; run them on CPU or GPU.
• Linux and Windows CompiledModel: LiteRT 2.1.5 to 2.3.0 (ai-edge-litert 2.3.0), with the desktop WebGPU accelerator rebuilt for 2.3.0.
• Web: LiteRT.js 2.5.3 to 2.6.0 (@litertjs/core@2.6.0).
• macOS CompiledModel: LiteRT 2.1.5 to 2.3.0 (ai-edge-litert 2.3.0 runtime and Metal accelerator), with the Core ML NPU delegate rebuilt from the LiteRT v2.3.2 source.
• iOS CompiledModel: a runtime built from the LiteRT v2.3.2 source tag and Google's 2.3.0 Metal accelerator, shipped as release litert-ios-v2.0.0.
• Core ML NPU acceleration on iOS and macOS is now registered so that LiteRT 2.3.x applies it to regular .tflite models.
• The Core ML NPU path (Accelerator.npu) needs iOS 16 or newer and macOS 14 or newer. iOS 16 is when Apple added MLComputeUnitsCPUAndNeuralEngine, and the macOS LiteRT 2.3 runtime needs macOS 14. The docs said iOS 13 and macOS 13. The iOS bridge now checks the OS before it registers the accelerator, so on iOS 15 strict {npu} throws an UnsupportedError and {npu, cpu} drops the NPU and runs on CPU. Before, an iOS 15 device passed the Core ML framework's iOS 13 check.
• The Play services runtime (play-services-litert 17.0.0) is unchanged.
• The example app builds with Kotlin 2.4.0.

Fixes #

• Android: the plugin now ships the LICENSE and THIRD_PARTY_NOTICE.txt from Google's litert and litert-gpu 2.3.2 AARs, unchanged, in android/licenses/litert-2.3.2/. Only the AARs' .so files are extracted at build time, so these notices were not reaching apps before. Apps that bundle the runtime should include them in their open-source notices; see "Open-source notices for the Android runtime" in the README.
• Web CompiledModel: precision is no longer dropped. An explicit Precision.fp16 or Precision.fp32 is now sent to LiteRT.js as gpuOptions.precision on every WebGPU compile (fromBufferAsync, fromBufferWithConfigAsync, and fromBufferWithGpuFallbackAsync); WASM compiles have no precision option. When precision is not set, nothing is sent and LiteRT.js keeps its own default (fp16 when the GPU supports shader-f16, fp32 otherwise), so results do not change for code that does not pass it. Pass Precision.fp32 to get the native default on the web too. To tell the two apart, CompiledModelConfig gains requestedPrecision (null when unset; precision still reads Precision.fp32 then), and an unset config no longer equals one with an explicit Precision.fp32. fromBufferAsync, fromBufferWithGpuFallback, and fromBufferWithGpuFallbackAsync now take Precision? on every platform; on native a null precision still compiles at Precision.fp32.
• Docs: Interpreter.resetVariableTensors() sets every variable tensor to zero (an int8 tensor to its zero point), as LiteRT implements it. The README and the API docs said it restored the initial values from the .tflite file, which LiteRT does not do. Recreate the Interpreter to get the model's values back.

Performance #

Measured with the 29-model matrix on Firebase Test Lab (Pixel 9 Pro, Galaxy S23, Galaxy A56), comparing the Android Interpreter on LiteRT 2.3.2 with 1.4.2. Each figure is the median, across models, of the change in each model's median (p50) latency. They come from single runs per device, not controlled benchmarks.

• Plain CPU (no delegate) is faster. Latency fell by about 83% on the Pixel 9 Pro, 48% on the Galaxy S23, and 51% on the Galaxy A56. The exception is yolov8n on CPU, which was 13 to 17% slower on the two Samsung phones.
• XNNPACK is about the same: -5% to +4% against the 1.4.2 results, with one Galaxy S23 run 8% slower than a later 1.4.2 run on the same device. The cat and dog face landmark models are the exception, 45 to 61% faster on XNNPACK.
• The Interpreter GPU is gone on Android, so models that were fastest on it now run on CompiledModel GPU, XNNPACK, or CPU. See step 3 below.

Migrating from 3.10.x #

1. Raise minSdk to 24 in android/app/build.gradle(.kts) if it is lower. If the app must support API 21 to 23, stay on 3.10.x.

2. Replace GpuDelegateV2. Code that constructs it no longer compiles. For the GPU on Android, use CompiledModel with Accelerator.gpu (for example CompiledModel.fromBufferWithConfig(bytes), whose Auto policy tries the GPU and falls back to the CPU), and check its output once with verifyCompiledModel. For an Interpreter, use PerformanceConfig.auto() or PerformanceConfig.xnnpack().

3. Expect some models to run slower without the Interpreter GPU. CPU and XNNPACK did not get slower overall (see Performance above); the losses are on models whose fastest 3.10.x path was the Interpreter GPU. The physical-device matrix (Pixel 9 Pro, Galaxy A56, Galaxy S23, 29 models) found three models that no Android GPU path now runs on any of the three devices: face_blendshapes, face_detection_full_range_sparse, and species_classifier_float16. Where the Interpreter GPU was the fastest path, the best remaining path was slower:

   • Pixel 9 Pro: mobilefacenet +21%, efficientdet_lite2 +18%, selfie_multiclass +11%, pose_landmark_heavy +10% (24.2 to 26.8 ms), yolov8n_float32 +1%.
   • Galaxy A56: mobilefacenet +20% (3.7 to 4.4 ms), yolov8n_float32 +5%.
   • Galaxy S23: superanimal_ssdlite_float16 +57% (4.7 to 7.4 ms), mobilefacenet +15%, efficientdet_lite2 +8%, efficientdet_lite0 +6%.

   These are median latencies from single Test Lab runs, so differences under about 25% are within run-to-run noise, and battery, heat, and CPU load were not measured. If one of these models matters to you, stay on 3.10.x, try a re-exported or newer version of the model on CompiledModel, or wait for a LiteRT release that runs it on the GPU. CompiledModel still returns wrong output for some models whose output tensor is dynamic, so gate it with verifyCompiledModel.

4. Android flex users: reconvert the model to LiteRT built-in ops and remove flutter_litert_flex from the Android build, or stay on flutter_litert 3.10.x with flutter_litert_flex 1.3.0. Apps that use flex only on iOS, macOS, Linux, or Windows move to flutter_litert: ^4.0.0 with flutter_litert_flex: ^2.0.0.

5. Android code that imported org.tensorflow.lite classes adds its own com.google.ai.edge.litert:litert dependency.

6. Re-check exact-output tests on Android, as described above.

7. Raise the iOS deployment target to 15.0 if it is lower: set IPHONEOS_DEPLOYMENT_TARGET for every build configuration, and for CocoaPods platform :ios, '15.0' in ios/Podfile, then run pod install. If the app must support iOS 13 or 14, stay on 3.10.x.

• Opt-in LiteRT runtime from Google Play services on Android (issue #23). Setting flutterLitert.runtime=playServices in the app's android/gradle.properties builds against play-services-litert 17.0.0 instead of packaging LiteRT in the APK. Interpreter and CompiledModel both run on it after a one-time await initializeAndroidRuntime(), and CompiledModel can use the Play services GPU accelerator on devices where Play services provides one (status.gpuAvailable says so). Initialization gives up after a timeout (5 minutes by default) and can be retried. Delegates, custom ops, NPU, and flutter_litert_flex are not available in this mode: constructing them fails with a clear error, and PerformanceConfig modes run on the Play services CPU path instead. The plugin ships the R8 rule release builds of this mode need. flutterLitert.runtime=auto packages both runtimes and uses Play services where it can provide them, the bundled runtime otherwise, for apps that also ship outside Google Play. The default bundled runtime is unchanged. See LiteRT from Google Play services. Thanks to Sasha Denisov for the proposal.
• iOS docs: Corrected the opencv_dart/dartcv4 setup. Don't set dartcv4's iOS deployment_target above Flutter's 15.0, because App Store Connect rejects the build with ITMS-90208. On Flutter 3.47.5 or newer with dartcv4 2.3.1, no setting is needed, and the example no longer sets it.

• Multi-input models now run on the web (issue #21). In JavaScript (dart2js) builds, LiteRtInterpreter.runForMultipleInputs and CompiledModel.runAsync failed on any model with two or more inputs with 'CastList<JSObject, Object>' is not a subtype of type 'JSArray<Object?>'. Single-input models, WebAssembly (dart2wasm) builds, and native platforms were unaffected. A browser test with a two-input model now covers both entry points in CI. Thanks to @AvikArefin for the report and the fix (#22).

Documentation and example fixes only. No change to the package's code or behavior.

• The README's CoreML delegate section suggested CompiledModel with {Accelerator.npu, Accelerator.gpu, Accelerator.cpu}, which throws on Apple platforms. It now suggests {npu, cpu} and notes that Apple cannot combine NPU and GPU.
• New README section, "Verifying a CompiledModel", covering what verifyCompiledModel compares, its 1% default tolerance, and its limits. The migration guide already linked to it, but the section did not exist.
• The iOS NPU guide and README no longer describe physical-device validation and the SwiftPM Core ML artifact as pending; both shipped in 3.8.0.
• The Android NPU guide now attributes its device results to LiteRT Next 2.1.6 and notes that the package bundles 2.2.0, whose matching JIT runtime current builds need.
• The README no longer says the main example avoids flutter_litert_flex; it depends on it for its model-matrix tests.
• The web example drops its fp16/fp32 precision setting. LiteRT.js has no precision option, so CompiledModel ignores precision on the web and the setting had no effect. The API parameter is unchanged.
• Older investigation notes are marked as historical, and stale internal planning documents are removed.

• prepareCameraFrameFromImage now decides BGRA vs. RGBA from the frame's format.raw instead of the platform, so desktop streams decode correctly on camera_desktop 2.0.0, which delivers BGRA on every desktop platform, as well as on 1.x, which delivered RGBA on Linux and Windows. Before this, Linux and Windows frames from camera_desktop 2.0.0 were decoded as RGBA, swapping red and blue. 'BGRA' selects BGRA and 'RGBA' selects RGBA. Any other value, or no format at all, keeps the previous default (BGRA on macOS, RGBA elsewhere), and an explicit isBgra still takes precedence, so callers that pass isBgra: Platform.isMacOS should drop it to get the new behavior.
• The README's desktop streaming setup now uses camera_desktop: ^2.0.0.

• Android's CompiledModel runtime is updated from LiteRT Next 2.1.6 to 2.2.0. The 2.2.0 AAR ships the same libLiteRt.so and libLiteRtClGlAccelerator.so set per ABI, every C API symbol the package binds is still exported, and the headers it relies on change only additively. The Qualcomm JIT runtime keeps the same library names and QAIRT 2.47.0.260601 pin, so a flutterLitert.qualcommNpuRuntimeDir bundle must now come from the LiteRT 2.2.0 JIT runtime release. The Interpreter stays on LiteRT 1.4.2, and iOS, macOS, Windows, and Linux stay on LiteRT Next 2.1.5.
• The web runtime (LiteRtInterpreter and CompiledModel) is updated from LiteRT.js 2.4.0 to 2.5.3 (@litertjs/core@2.5.3, auto-loaded from jsDelivr). Since 2.5.0, a WebGPU compile that cannot place every op no longer throws. Chromium browsers with JSPI return a partially delegated WebGPU model, which CompiledModel reports as {Accelerator.gpu, Accelerator.cpu} with isFullyAccelerated false. Browsers without JSPI recompile the model on WASM inside LiteRT.js. Requests that allow a CPU fallback (auto, gpuWithCpuFallback, {gpu, cpu}) accept that model and report {Accelerator.cpu} with didFallback true. A strict CompiledModelConfig.gpu() or {Accelerator.gpu} request now disposes it and throws StateError, preserving the 2.4.0 contract that a strict GPU request never silently runs on the CPU.

• Correct the native-only inference utility documentation (issue #17). InterpreterFactory and InterpreterPool examples now import package:flutter_litert/native.dart so their native types resolve consistently in IDEs and static analysis. Document their supported platforms and the portable CompiledModelConfig.auto() alternative.
• Document the external OpenCV iOS CMake deployment-target fix (issue #19). dartcv4 2.3.1 removes its hardcoded iOS 12 target and adds a deployment-target hook option. Document the upgrade, iOS 15 configuration, and Flutter SDK meta pin compatibility requirement. Upgrade the example to the fixed dependency with Flutter 3.47.5 and an iOS 15 Runner and hook target. A clean Xcode 27.0 / iOS 27 simulator build now passes the DartCV CMake probe and final Runner link. The iOS package also drops the legacy -ObjC linker option that Xcode 27 rejects; explicit C API symbol anchors retain the packaged FFI entry points. The bundled SwiftPM TensorFlow Lite simulator slices are arm64-only, so the example excludes x86_64 when building for the simulator. The package's minimum iOS version remains 13.0; the example's iOS 15 DartCV target and the macOS Core ML build are independent constraints.
• New policy-based CompiledModel construction. CompiledModelConfig and CompiledModelPolicy add auto, cpu, strict gpu, mixed gpuWithCpuFallback, strict npu, and mixed npuWithCpuFallback choices. Use fromFileWithConfig, fromBufferWithConfig, or the portable fromBufferWithConfigAsync. This provides the PerformanceConfig.auto equivalent requested in issue #18 without changing any existing constructor.
• Auto has a narrow, deterministic contract: request strict GPU at fp32, then retry the complete model CPU-only if GPU construction fails. Auto does not first request a mixed {gpu, cpu} graph, select NPU, benchmark available accelerators, or treat successful compilation as proof of numerical correctness. Production models still need per-model validation.
• The distinction between fallback modes is now represented directly in the API. gpu/npu perform no package-level CPU retry; gpuWithCpuFallback/npuWithCpuFallback first request a mixed graph that can place operations on CPU, then perform a complete CPU-only retry if that construction throws. The existing fromBufferWithGpuFallback methods retain their mixed-graph-first behavior and map to the new explicit policy.
• New requestedConfig, requestedAccelerators, and didFallback getters sit alongside the existing effective accelerators and isFullyAccelerated diagnostics. They distinguish what the caller requested, what LiteRT constructed, whether a complete retry or runtime narrowing occurred, and whether the whole graph was accelerated. Operation-level placement inside a successful mixed graph does not set didFallback.
• Web exposes the same policy surface through fromBufferWithConfigAsync. Auto uses WebGPU with the existing hung-compile watchdog and retries on WASM; strict NPU remains unsupported, while npuWithCpuFallback reports the NPU failure and continues on WASM. Unsupported-platform stubs carry matching signatures for conditional-import parity.

• CompiledModel now defaults to Precision.fp32 instead of fp16. This changes numeric output and costs about 30% median GPU latency across the five GPUs measured (four architectures; Apple Metal appears as both M4 and A17 Pro), so it is a deliberate correctness-over-speed default. The cost is real and worth stating plainly: in 84 paired same-model comparisons fp32 was slower in 67 of them, with a median of +29.9% and a worst case of +21.6 ms. Apple M4 is the lone exception, where fp32 is marginally faster (median -6.5%); every other architecture pays +37% to +43%. Restricting the comparison to the models where fp16 actually passed parity, which are the only ones anyone could legitimately keep on fp16, the median cost is +24.3%. Accuracy is what justifies the default anyway. Across the 29 published detection models, strict-GPU fp32 matched a plain-CPU reference for every model that compiled on all four GPU architectures measured, while fp16 matched only 4 of 18 on Adreno 740, 5 of 18 on Xclipse, 4 of 18 on Apple Metal, and 1 of 12 on Mali-G715. Google's own LiteRT Python API reproduces the Apple figures through the same underlying switch, so this is upstream numerical behaviour rather than a binding artefact: fp16 carries about three decimal digits of mantissa and these graphs emit pixel-space coordinates and landmark positions. fromBufferWithGpuFallback already defaulted to fp32; the plain constructors now agree with it. Pass Precision.fp16 explicitly to keep the old behaviour, ideally per model and validated on the target GPU. Full results: GPU vendor matrix.
• Backend evidence for the above is now reproducible. test/benchmark gains an Apple (macOS + iOS) matrix harness, physical Android matrices for Adreno, Mali, and Xclipse, and a cross-check against the official LiteRT Python API that agrees with the Dart implementation to 6.3e-14 on CPU reference outputs.
• Fixed: Core ML NPU never worked on a physical iPhone. Package.swift pinned TensorFlowLiteCCoreML to a May release predating both the NPU entry points and the global-MEAN padding patch, so accelerator registration returned kLiteRtStatusErrorUnsupported for every model and the Interpreter Core ML delegate rejected four models macOS accepted. The patched framework had been built but only ever uploaded as a CI artifact, never published, so the pin was never moved. coreml-ios-v1.1.0 publishes it for SwiftPM. CocoaPods needed a separate fix: its frameworks are downloaded by the podspec rather than shipped in the package, and that bundle (libs-v0.1.8) predated the NPU work entirely, so fixing SwiftPM alone left every CocoaPods consumer, which is Flutter's default iOS path, with the identical runtime failure. libs-v0.1.9 carries the patched framework, and the podspec now gates the download on the NPU symbol actually being present rather than on the file merely existing, so a stale cache re-downloads instead of silently degrading. Both channels are now checked in CI against the artifacts consumers really fetch. Measured on a physical iPhone 15 Pro afterwards, iOS matches macOS exactly: Core ML 24 of 29 models with 13 accurate, strict {npu} 1 of 29, {npu, cpu} 24 of 29 with 12 accurate.
• Fixed: a mixed {npu, cpu} request no longer fails when Core ML NPU cannot register. The graceful-degrade path added for Android was gated on Platform.isAndroid, and macOS never exposed the gap because registration succeeds there. A caller asking for CPU fallback got a hard failure instead, which is the one outcome an explicit fallback exists to prevent. Strict {npu} still throws; a mixed request drops the NPU and reports the effective set through the accelerators getter.
• Documents that NPU accuracy is bounded by fp16 and cannot be configured away. The Apple Neural Engine is fp16 hardware and CoreMlDelegateOptions exposes no precision control. Every model Core ML computed incorrectly across the 29 published models was correct on an fp32 path, and most are the same models that fail on GPU fp16. Unlike GPU, where Precision.fp32 fixes it, NPU has no equivalent: validate per model or do not use it.
• A defect is documented rather than worked around: on a Pixel 9 Pro, GPU model compilation degrades after roughly 19 compilations in one process, after which LiteRtLockTensorBuffer fails on managed buffers. Controlled runs place it on compilation rather than inference, so an application that compiles its models once and runs them for hours is unaffected, while one that repeatedly constructs and disposes CompiledModel instances is not. A Galaxy A35 with Mali-G68 and a Galaxy A56 with Xclipse showed no such failures, so it is not a Mali-family property.
• Android Qualcomm NPU foundation for CompiledModel. Android API 31+ arm64 apps can now use an app-provided LiteRT JIT NPU runtime. NPU requests get a dedicated environment with the official compiler-plugin and dispatch directories plus a reusable JIT cache; initializing CPU/GPU first no longer prevents later NPU setup.
• Android's CompiledModel runtime is updated from LiteRT Next 2.1.5 to 2.1.6. Normal builds still bundle only CPU and OpenCL/GL GPU libraries. A new flutterLitert.qualcommNpuRuntimeDir Gradle property can fuse exactly one prepared Qualcomm runtime into an arm64 local/Test Lab APK and fails the build when its nine-library JIT set is incomplete or ambiguous. The plugin manifest also exposes the device-provided Qualcomm libcdsprpc.so library through Android's optional native-library allowlist.
• The example app can now build a device-targeted AAB with mutually exclusive Qualcomm SM8550/v73, SM8650/v75, and SM8750/v79 dynamic features. Devices in the default group receive no vendor runtime; strict NPU remains an error there, while mixed NPU+GPU/CPU requests retain their explicit fallback.
• New manual Android physical NPU (Firebase Test Lab) workflow targets the Galaxy S23/SM8550 (HTP v73), S24 Ultra/SM8650 (v75), or S25 Ultra/SM8750 (v79). Its default strict smoke gate consumes one Test Lab run only after runtime preparation, AAB build, and package validation pass; an opt-in full sweep adds face, segmentation, and pose correctness comparisons. Physical validation passed strict NPU inference and the full representative sweep on all three generations. MobileFaceNet passed the default CPU-reference tolerance, while selfie segmentation and heavy pose were consistently identified as model-specific accuracy risks.

Fixes a 3x macOS CPU slowdown, adds a way to detect an upstream LiteRT defect that returns wrong answers silently, and fixes two resource bugs. Additive: no existing symbol changes signature.

Heads-up for bit-exact tests. The bundled macOS arm64 libtensorflowlite_c is now a bazel build rather than a CMake one, which changes float32 output in the last few ULPs because ruy multithreading is finally active and reductions accumulate in a different order. Measured on the bundled face-detection model: 87% of elements differ, by at most 3.8e-05 against an output range of 181.7, i.e. 0.000021%. Tolerance-based comparisons are unaffected; a byte-level golden pinned on macOS arm64 will need regenerating.

• macOS CPU inference is up to 3x faster. The previous CMake-built dylib left ruy effectively single-threaded, so every ruy-backed builtin op ran on one core. TRANSPOSE_CONV has no kMultithreadOptimized variant and parallelises only through ruy's gemm, so deconv-heavy models paid the full cost: a 384px landmark model went from 83.7ms to 26.8ms, now matching iOS exactly. fully_connected and batch_matmul gain similarly. Intel Macs keep their existing CMake slice and are unchanged. See doc/macos_transpose_conv_gap.md.
• New: macOS Apple Silicon NPU support for CompiledModel. Accelerator.npu now lazily registers a dedicated Core ML CPUAndNeuralEngine accelerator on macOS 13+. Strict {npu} compilation rejects any non-delegated TFLite operation; {npu, cpu} applies Core ML before XNNPACK and rejects zero-node Core ML delegation rather than silently returning CPU-only inference. The build carries the required global-MEAN padding fix and is covered by fixed-input output comparisons across representative models. NPU+GPU combinations remain unsupported. See doc/macos_compiled_model_npu.md.
• Checkpoint: iOS NPU support is simulator-validated. The iOS CompiledModel path now has its own Core ML accelerator-registration bridge, uses the same strict {npu} and Core-ML-first {npu, cpu} semantics, and rejects zero-node delegation. The arm64+x86_64 simulator suite passes strict inference, a five-model mixed-mode correctness sweep, fallback diagnostics, and NPU+GPU rejection. This does not yet constitute Neural Engine validation: simulators have no ANE, physical-iPhone testing remains pending, and SwiftPM still needs a release artifact containing the patched Core ML entry points. See doc/ios_compiled_model_npu.md.
• New verifyCompiledModel(bytes, compiled) checks a CompiledModel against a bare-CPU Interpreter and reports the deviation, returning BackendVerification. LiteRT Next can return kLiteRtStatusOk while producing output that is wrong, or never written at all, and neither is visible from a status code or from timing. Run it once at init before trusting a CompiledModel. It reports rather than throwing or swapping backends, so the policy stays with the caller; default tolerance is 1% of the output range, against measured separation of 0.068% (healthy) versus 42%+ (corrupt). Cost is one Interpreter build plus one inference, 4-56ms depending on the model.
• New CompiledModel.isFullyAccelerated reports whether the whole graph ran on a selected accelerator. Note that false is ambiguous: partially delegated graphs report false even when the accelerator genuinely ran, so this is not a way to detect a silent CPU fallback. Use verifyCompiledModel for that.
• Fixed: InterpreterPool.initialize is now all-or-nothing. A failure part way through left the interpreters it had already built alive, and because the dispose-first branch is keyed on isInitialized, which a failed call never sets, retrying accumulated them: a pool of 3 could end up holding 4, the extra one live with an XNNPACK threadpool but never used.
• Fixed: CoreMlDelegate leaked its options struct when constructed without explicit options. Caller-supplied options are still left to the caller.
• LiteRtStatus values in error messages now carry their name, so LiteRtStatus=3 reads LiteRtStatus=3 (kLiteRtStatusErrorRuntimeFailure).
• Un-deprecated the GPU, Metal, and CoreML Interpreter delegates. 3.0.0 deprecated them in favour of CompiledModel and announced removal in 4.0.0; that is reversed, and no removal is scheduled. Two reasons. PerformanceConfig.gpu() and .coreml() are built on these classes and were never deprecated, so the removal would have broken supported API with no notice (interpreter_factory.dart was suppressing its own deprecation warning to keep compiling). And CompiledModel cannot replace them yet: it reports success while leaving the output buffer unwritten for models whose output tensor ends up dynamic, which covers heatmap models with a deconvolution head. A deprecation that cannot be acted on, pointing at a backend that returns wrong numbers, is worse than none. Prefer PerformanceConfig over constructing delegates directly, and gate any CompiledModel adoption behind verifyCompiledModel.

Adds shared utilities that detector packages were each re-deriving locally. All additive; no existing symbol changes behaviour.

• New aggregateActiveAccelerator(Iterable<String?>) (web) collapses the per-runner backends of a multi-stage detector into the single accelerator it should report. It returns 'webgpu' when any runner is still on WebGPU, so the runtime GPU-error fallback and slow-WebGPU warmup (both gated on the reported accelerator) stay armed under mixed compile outcomes where some models fell back to WASM and others did not.
• New compiledModelFromBufferAuto(...) and isDefaultGpuCpuAccelerators(...) centralize the "is this the permissive {gpu, cpu} default?" branch that decides between CompiledModel.fromBufferWithGpuFallback and CompiledModel.fromBuffer. An explicit accelerator set is still honoured as-is; only the two-way default degrades.
• New iouLTRB(...) is the exact intersection-over-union of two axis-aligned boxes, for frame-to-frame track matching. It deliberately has no epsilon, unlike the NMS ratio in nms_utils.dart which adds 1e-7; mixing the two shifts matches at threshold boundaries.
• Fix: CompiledModel.fromBufferWithGpuFallback now forwards precision to its CPU paths. Previously only the GPU attempt received it, so the forceCpu shortcut and the CPU retry after a failed GPU compile silently fell back to fromBuffer's fp16 default. A single call with no arguments therefore ran fp32 on GPU and fp16 on CPU, defeating the fp32 default that exists because pixel-space landmark and box coordinates lose accuracy in fp16. Callers that passed fp16, including every detector package built on this plugin, are unaffected; callers that asked for fp32 now get it on the fallback path. fromBufferWithGpuFallbackAsync delegates and is fixed with it. The web implementation documents precision as accepted-but-ignored and is unchanged.
• New collectOutputShapes(Interpreter) (native) returns every output tensor's shape keyed by index, walking indices until getOutputTensor throws. It reads shapes only and never touches Tensor.data, so no buffer views are materialized and quantized outputs are safe to enumerate. Use TensorFloat32Views when the buffers themselves are needed.

Adds explicit support for detection models whose confidence tensors are already activated probabilities.

• postProcessDetections and postProcessDetectionsFlat now accept scoresAreProbabilities: true, which skips sigmoid for class and objectness values and compares probability thresholds directly.
• The new option is additive and defaults to false, preserving the existing logits contract and output for all current callers.

Adds Android OpenCL/GL acceleration to the LiteRT Next CompiledModel path.

• Android builds now bundle libLiteRtClGlAccelerator.so from the pinned LiteRT 2.1.5 AAR by default for arm64-v8a and x86_64. armeabi-v7a remains CPU-only.
• The plugin manifest now declares the optional vendor GPU libraries (libOpenCL.so, libOpenCL-car.so, libOpenCL-pixel.so, and libvndksupport.so, all required="false"), so apps targeting Android 12+ can load them without adding their own uses-native-library entries.
• Android emulators do not provide working OpenCL, so direct {gpu, cpu} compilation can fail after the accelerator registers. The fromBufferWithGpuFallback factories catch that error and retry CPU-only.
• Apps that do not need CompiledModel GPU acceleration can set flutterLitert.bundleGpuAccelerator=false to omit about 3 MB per ABI. The classic Interpreter runtime and GPU delegate are unchanged.

Web CompiledModel robustness fix. No API changes.

• A WebGPU compile attempt that neither resolves nor rejects no longer hangs the WASM fallback paths. LiteRT.js 2.4.0's compile promise can, very rarely, fail to settle on machines without a usable GPU (observed once on GPU-less headless Chrome in CI, where an engine rebuild stalled indefinitely); fromBufferWithGpuFallbackAsync and {gpu, cpu} accelerator sets now bound the WebGPU attempt with a 60-second watchdog and fall back to WASM when it trips, honoring their always-yield-a-model contract. If the abandoned compile settles later, its model is disposed. Strict {gpu} requests are never timed out and keep surfacing whatever the runtime does.
• Web integration-test harness (CI-only, not part of the published package): the drive suites now record which poll timed out and what the app showed into integration_response_data.json, the custom driver writes that file on failure too, and CI prints it when a drive fails, so a recurrence pinpoints the stalled stage instead of reporting an empty failure detail.

Brings CompiledModel to the web via Google's LiteRT.js (the same auto-loaded @litertjs/core that powers LiteRtInterpreter), fixes App Store uploads for SwiftPM installs (#15), and fixes a nondeterministic ARM64 detection decode. Additive and backward compatible.

Web CompiledModel:

• New async factories on every platform, CompiledModel.fromBufferAsync and fromBufferWithGpuFallbackAsync; pair them with the existing runAsync for portable code. LiteRT.js compilation is Promise-based, so on the web they are the only way to build a model: the synchronous fromFile, fromBuffer, fromBufferWithGpuFallback, and run throw UnsupportedError there.
• Web accelerator mapping: cpu compiles on WASM, gpu on WebGPU, and {gpu, cpu} tries WebGPU with a WASM fallback; model.accelerators reports what LiteRT.js actually resolved (including {gpu, cpu} for partially accelerated WebGPU models). npu throws ArgumentError on the web, precision is accepted but ignored, and the zero-copy TensorBufferMode.hostMemory path stays native-only.
• Inference-time WebGPU failures (device lost, GPU out of memory) throw LiteRtRuntimeError, so callers can dispose the model and rebuild it with {Accelerator.cpu}.
• The Accelerator/Precision/TensorBufferMode enums moved to a shared source file (no API change), and the example app now builds its CompiledModel with fromBufferAsync.

Web backend selection and Safari compatibility:

• The default LiteRT.js WASM location is now the package's wasm/ directory instead of a pinned file, so LiteRT.js's feature probe serves Safari the compat build (relaxed SIMD is default-off there) while Chrome and Firefox keep the fast relaxed-SIMD build. URLs pinned via configureLiteRtWebLoader are unaffected.
• New resolveWebAccelerator('auto' | 'webgpu' | 'wasm') in web_detector_utils.dart: 'auto' picks WebGPU only on Chromium with a hardware (non-software) adapter, probed once per page load; explicit values pass through. Firefox's WebGPU works but runs ~22x slower than its WASM SIMD, so API presence alone must not select it.
• New WebGpuFallback.maybeSwapIfWebGpuSlow: times a few warmup inferences after an 'auto' init that landed on WebGPU and swaps to WASM past a budget (default 50ms median), catching slow-but-functional GPU stacks the error-driven fallback cannot see.
• WebGpuFallback.withFallback now swaps only on LiteRtRuntimeError, so logic bugs surface instead of masquerading as GPU fallbacks, and marks fellBackToWasm only after a successful swap. All compile-time, runtime, and warmup fallbacks now log their cause via debugPrint.

iOS fix (#15): App Store validation rejects the loose libLiteRt.dylib / libLiteRtMetalAccelerator.dylib files that SwiftPM's bare-dylib xcframeworks embedded in the app's Frameworks/ directory, surfacing as ITMS-90426 ("Invalid Swift Support"). SwiftPM now ships the same framework-wrapped xcframeworks as CocoaPods (identical binaries, release litert-ios-v1.0.1) and registers the Metal accelerator through the shared LiteRtRegisterGpuAccelerator shim, so GPU CompiledModel keeps working. No API change; run flutter clean and rebuild. Note: Flutter's SwiftPM support independently embeds a framework built at minos iOS 12.0 that can also trigger ITMS-90426; if uploads still fail, disable SwiftPM (flutter: config: enable-swift-package-manager: false in pubspec.yaml) until flutter_tools is fixed.

ARM64 fix: on Apple Silicon, the SIMD decode in postProcessDetectionsFlat could return a different detection count (or phantom boxes) for byte-identical model output, because the Dart ARM64 JIT miscompiles the greaterThan().select() lane-carried argmax it used. The winning class is now recovered with a scalar argmax over the few anchors that clear the threshold, so the decode is deterministic and matches the scalar reference. Affects every downstream detector that decodes channel-major YOLO output; no API change.

Fixes the two hero demo images stacking vertically on the pub.dev package page. pub.dev's README stylesheet forces img{height:auto}, so they are now sized with percentage width (honored by both pub.dev and GitHub) and stay side by side. Documentation only; no code, API, or runtime change.

Adds camera-agnostic helpers for building live detection previews, and documents the end-to-end live-camera pipeline in the README. No native or web runtime code changed. Additive and backward compatible.

• FrameThrottle: a single-slot gate that drops camera frames arriving while a previous frame is still being processed, replacing the hand-rolled bool _isProcessing plus try/finally pattern in downstream apps.
• CoverFitTransform: maps detector coordinates onto a cover-fitted camera preview (uniform scale, centered overflow, optional front-camera mirroring), wrapping the existing coverFitScaleOffset. Use map for points and scaleLength for radii and stroke widths.
• README: new "Live camera" section covering the full pipeline (frame prep, rotation, throttling, overlay coordinate mapping, FPS, smoothing).
• README: also rolls in the real-time hand-tracking demo (origami then megaminx hand detection) beside the pose-detection demo, plus an enlarged pose mockup, that had been staged for an unreleased 3.2.2. Both demo animations are all-keyframe WebP renders kept under 10 MB.

Fixes an Android build failure on Android Gradle Plugin (AGP) 9.x (issue #14). AGP 9 changed the default of android.sourceset.disallowProvider to true, which rejects passing a Provider to the legacy jniLibs source-set API. The plugin handed layout.buildDirectory.dir("litert-jni") (a Provider<Directory>) to jniLibs.srcDir(...), so configuration failed at android/build.gradle.kts with "You cannot add Provider instances to the Android SourceSet API." AGP 8.x is unaffected, which is why it only surfaced for consumers on AGP 9.

• libLiteRt.so is now contributed as a generated jniLibs source through the AGP Variant API (androidComponents.onVariants { ... jniLibs.addGeneratedSourceDirectory(...) }) instead of the legacy sourceSets { ... srcDir(<Provider>) } block. AGP owns the task dependency, so the manual preBuild hook is removed, and a litertNextVersion bump now re-downloads because the version is a tracked task input. Verified building the plugin AAR on both AGP 8.11.1 and AGP 9.2.1.
• CI now rebuilds the plugin module under AGP 9.x so this class of forward-incompatibility is caught before publishing.

Also includes a performance pass over the Dart inference wrappers, verified with interleaved AOT A/B benchmarks on macOS and a physical iPhone:

• Interpreter.run() with typed-data I/O is ~2x faster (771 -> 364 ns wrapper overhead); CompiledModel.run() in managed mode is ~27% faster; the shared YOLO-style decode utility is up to 72% faster (SIMD argmax, logit-space pruning); packYuv420 accepts an optional reuse buffer so camera loops skip a per-frame ~1.4 MB allocation.
• CompiledModel.runAsync/dispatchAsync now run the blocking native call on a lazily spawned per-model helper isolate instead of blocking the calling isolate, keeping the UI thread responsive during inference. Calls against the same model serialize in FIFO order, and sync buffer-touching APIs (run, dispatch, writeInput, readOutput, close) now throw StateError while an async dispatch is in flight. runAsync with thread-affine mobile GPU stacks (some Android OpenGL/OpenCL drivers) is unvalidated; prefer run there.

Restores the WASM-ready score on pub.dev (back to 160/160), which dropped to 150 when pub.dev upgraded its analyzer (pana 0.23.13). pana 0.23.13 mis-resolves conditional export/import directives: it derives the condition name with name.tokens.map((t) => t.value()).join(), and because library is a Dart keyword Token.value() returns it upper-cased, so if (dart.library.X) becomes dart.LIBRARY.X and never matches. Every conditional then resolves to its default (first) URI. The main flutter_litert.dart barrel defaulted to the native (dart:ffi / dart:isolate) surface, so the WASM/platform analysis saw those libraries as reachable.

• The portable flutter_litert.dart barrel now defaults to the WASM-safe web surface and gates the native surface on dart.library.io, so the package is WASM-compatible again. Runtime behavior is unchanged: real native and web builds resolve exactly as before.
• Breaking (native-only): API whose public signatures use native-only types (Isolate, SendPort, File, ...) and therefore cannot be WASM-safe is now published from a new package:flutter_litert/native.dart library instead of the main barrel: IsolateWorkerBase, IsolateRpcClient, setupIsolateHandshake, InterpreterPool, ModelCheckpoint. Native code using these now also needs import 'package:flutter_litert/native.dart';. TensorFloat32Views and the rest of the API stay on the main barrel.
• InterpreterOptions on web gains hasDelegate, threads, and copyWithoutDelegates() to match the native API.

• Preserve thread tuning and custom-op registrations when delegate application fails and interpreter creation retries on CPU.
• Expose whether an interpreter actually has an active delegate, so isolate selection follows the effective backend after fallback.

Interpreter creation now falls back to CPU when a configured delegate cannot be applied to a model/runtime, instead of failing. This fixes classic Interpreter creation for models that cannot use the default iOS Metal delegate, including on the iOS simulator: it now warns and retries on CPU. The fallback covers every creation path (fromAsset, fromBuffer, fromBytes, and the isolate interpreter), and the iOS integration job now also exercises the classic Interpreter path so this is caught in CI.

Makes the package web- and WASM-compatible. dart:isolate was reachable from the public API (via decode_failure.dart and isolate_rpc_server.dart) but is unavailable on web/WASM; the isolate-dependent code now sits behind conditional imports so none of it is reachable on the web build. No API changes.

Fixes the iOS CocoaPods build for the LiteRT Next runtime. The prebuilt LiteRt.xcframework / LiteRtMetalAccelerator.xcframework download shipped an arm64-only ios-arm64-simulator slice, whose identifier does not match the ios-arm64_x86_64-simulator slice CocoaPods selects on the simulator. The build then failed copying a non-existent slice (rsync ... No such file or directory).

• Fix: the downloaded iOS frameworks now carry a universal ios-arm64_x86_64-simulator slice (arm64 device binary + x86_64 stub), so the simulator build resolves and links. (SwiftPM builds were unaffected.)
• Fix: the podspec now verifies the simulator slice, not just the device slice, before skipping the download, and clears stale slices on re-download, so an existing arm64-only cache is replaced.

Additive release: shared isolate, CompiledModel-pooling, and image-RPC utilities, extracted so the packages built on flutter_litert can maintain them in one place instead of each carrying its own copy. No breaking changes; the Interpreter and CompiledModel APIs are unchanged.

• New: serveIsolateRpc: the isolate-side counterpart to IsolateRpcClient. Drives the {id, op} -> {id, result | error} protocol from a handler map, replacing the hand-written listen/switch/try-catch envelope each worker isolate used to carry. IsolateRpcExactError lets a handler send a verbatim wire-error string when the main side relies on the exact text (e.g. a startsWith error contract).
• New: IsolateWorkerBase.disposeGracefully and IsolateRpcClient.disposeGracefully: send the dispose op and await the isolate's acknowledgement before killing it, so the isolate can free native interpreters / CompiledModels. Isolate.kill(priority: immediate) otherwise races past the queued dispose message and leaks the native handles.
• New: CompiledModelPool: a round-robin pool of CompiledModel slots, each with its own reusable input buffer and AsyncLock, so concurrent inferences (e.g. one per detected object) land on distinct models with leak-free init teardown. A pool of size 1 degrades to a safe single-model-plus-lock.
• New: compiled_io_utils: compiledFloatCount, squareSideFromFloats, compiledSquareInputSide, compiledOutputFloatCounts, and indexWhereFloatCount for deriving tensor geometry from a CompiledModel, whose tensor sizes are exposed only in bytes.
• New: cameraFrameRpcFields and cameraFrameFromRpcMessage: pack a CameraFrame into an isolate-request field map and rebuild it on the isolate side (any image decode stays in the consumer, keeping this dependency-free).
• New: decodeFailurePrefix, throwDecodeFailure, and rethrowOrFormatException: signal an undecodable-image failure from inside an isolate and surface it as a FormatException on the main side instead of a cryptic downstream error.

• New: LiteRT Next CompiledModel API: CompiledModel.fromFile, CompiledModel.fromBuffer, and CompiledModel.fromBufferWithGpuFallback, with automatic hardware-accelerator selection via Accelerator.{cpu, gpu, npu}, Precision, and TensorBufferMode. This is the recommended path for GPU/NPU acceleration going forward, following Google's LiteRT Next guidance (https://developers.google.com/edge/litert/next/get_started). Supported on Android, iOS, macOS, Windows, and Linux.
• The desktop (Windows/Linux) WebGPU GPU accelerator and DirectX Shader Compiler are fetched from a GitHub release at build time instead of being bundled in the published package, keeping it under pub.dev's 100 MiB size limit. Desktop GPU acceleration still works; the libraries download automatically (verified by SHA-256) on the first build. No effect on Android, iOS, or macOS.
• Deprecated: manual hardware-acceleration delegates for the Interpreter API, namely GpuDelegateV2 (Android GL/CL), the Metal GpuDelegate, and CoreMlDelegate (with their *Options). They remain fully functional but are superseded by CompiledModel's automatic accelerator selection and are planned for removal in 4.0.0. The Interpreter API itself, the CPU XNNPackDelegate, and FlexDelegate are NOT deprecated and remain fully supported.
• Fix: creating an XNNPackDelegate with XNNPackDelegateOptions no longer crashes on the arm64 Android emulator. The options struct was initialized by calling the native TfLiteXNNPackDelegateOptionsDefault(), which returns the struct by value; that by-value FFI return crashes the Dart VM on the arm64 Android emulator (it works on real devices, macOS, and iOS). The struct is now built in Dart, matching upstream tflite_flutter, while preserving the QS8/QU8 quantization defaults; the resulting native options are unchanged, so there is no behavior difference on real devices.
• Fix: TensorFloat32Views input views are now genuinely writable. They were previously built from the unmodifiable Tensor.data view, so indexed writes (views.inputs[0][i] = x) threw UnsupportedError, and bulk setAll/setRange only worked through a Dart VM enforcement gap that a future SDK could close. Views are now captured via the new Tensor.asFloat32View(), a mutable Float32List aliasing the tensor's native buffer (valid until the next resize/allocateTensors).
• SignatureRunner.run() per-call overhead roughly halved (16-17µs → 7µs per call on the bundled test/benchmark/signature_runner_benchmark_test.dart): tensor handles are cached by name between allocations, and the valid-names error text is built only when a lookup actually fails instead of on every getInputTensor/getOutputTensor call.
• Behavior change: IsolateInterpreter.run/runForMultipleInputs no longer silently drop calls. A call issued while a previous run is in flight is now queued and completes with real results (previously it returned normally without writing the output buffers); frame-skipping callers can check state == IsolateInterpreterState.loading before calling. Running after close() now throws StateError instead of returning silently.
• TensorType.fromValue is O(1) instead of scanning all enum values (it runs on every Tensor.type access), and inference timing uses a reused monotonic Stopwatch instead of two DateTime.now() calls per run.
• Interpreter hot-path overhaul, measured on the bundled test/benchmark/engine_overhead_benchmark_test.dart (MediaPipe face_detection_short_range, macOS host):
  • run()/runForMultipleInputs() with nested-list input and output drops from 8.9ms to 1.9ms per inference (native floor 1.0ms) by converting tensors through a single pre-sized buffer instead of one small allocation per element, and by reading outputs through typed views instead of a per-element ByteData.view.
  • Tensor.setTo/copyTo now copy directly between Dart memory and TfLiteTensorData instead of round-tripping through a native scratch buffer (two extra copies per tensor per inference).
  • Fix: passing a flat Float32List (or other typed data) as an input no longer resizes the input tensor to rank 1, which broke models with rank-sensitive ops (CONV_2D failed to prepare). Flat typed data whose element count matches the tensor is now staged as-is, and is the fastest run() input type.
  • New: outputs can be flat typed data (Float32List, Int32List, Int64List, Int16List, Int8List). Bytes are bulk-copied directly into the buffer; previously this threw a shape-mismatch ArgumentError. run() with Float32List in/out now measures within ~7% of the raw tensor-views floor.
  • Behavior note: copyTo(Uint8List)/copyTo(ByteBuffer) now fill and return the destination instead of returning a separate copy.
• CompiledModel: per-dispatch native out-params are allocated once per model instead of per call (run, runAsync, lock/unlock paths).

• Android: support both AGP 8 and AGP 9 by moving the plugin Gradle files to Kotlin DSL and updating the Android tooling plugin declarations (6c332e3b).

• Fix GPU and CoreML delegates silently falling back to CPU on macOS and iOS (#11). macOS now bundles the GPU/CoreML dylibs that were previously omitted from the Swift Package manifest. iOS retains all 212 packaged LiteRT, Metal, and CoreML symbols that Dart FFI resolves but linker stripping would otherwise drop. They are kept through a generated anchor on both CocoaPods and SPM, plus an embedded dynamic SPM framework so they survive App Store archive stripping. This extends the 2.8.0 mitigation (#8, #9) to every packaged API.
• Log delegate initialization failures before falling back to CPU. A packaging regression can no longer masquerade as unexpectedly slow GPU/CoreML performance.
• On Apple Silicon the Metal GPU is substantially faster for conv-heavy models once actually loaded (for example, selfie segmentation ~27ms -> ~3ms versus XNNPACK).

• Complete the AGP 9 / built-in Kotlin fix from 2.8.0 (#10). 2.8.0 resolved the "Inconsistent JVM-target ... (17) and (21)" error on the AGP 8.11 transitional path, but a full migration to android.builtInKotlin=true on AGP 9 still failed with "The 'org.jetbrains.kotlin.android' plugin is no longer required since AGP 9.0": the Flutter Gradle plugin auto-applies the legacy Kotlin plugin to this module, and AGP 9 rejects it. The plugin now applies kotlin-android only on AGP < 9 (which also stops Flutter from auto-applying it), and keeps the JVM-target pin guarded so the AGP-9-without-built-in-Kotlin case is skipped. Verified building against AGP 8.11.1 and 9.0.1 with built-in Kotlin both enabled and disabled.

• Fix the "Inconsistent JVM-target compatibility detected ... (17) and (21)" Android build failure under AGP 9 / Flutter 3.44+ (#10). The fix pins the Kotlin JVM target to 17 only when AGP >= 9 is in use, so older Flutter/AGP toolchains are unaffected and the minimum supported versions are unchanged.
• Fix iOS "Failed to lookup symbol 'TfLiteInterpreterOptionsCreate'" crash on App Store / TestFlight builds (#8, #9). The TFLite C symbols are resolved at runtime via dlsym and were stripped during App Store distribution; the CocoaPods podspec now disables that stripping on the app target so no manual Xcode build-setting changes are required.

• Add InterpreterOptions.addCustomOp(...): high-level method for registering custom TFLite ops; handles native string allocation and lifetime internally, replacing the previous raw tfliteBinding call pattern.
• Add Interpreter.fromBytes(Uint8List): async cross-platform constructor, matching the web API. Native platforms complete immediately; unsupported stub throws UnsupportedError.
• Rename lastNativeInferenceDurationMicroSeconds → lastInferenceDurationMicroseconds on Interpreter, SignatureRunner, and LiteRtInterpreter (all platform variants). Old name kept as a @Deprecated alias.
• Rename configureLiteRtLoader → configureLiteRtWebLoader. Old name kept as a @Deprecated alias and re-exported from all_web.dart.
• Fix camera_frame.dart: widen .planes cast from List<dynamic> to Iterable<dynamic> for broader compatibility.

• Fix iOS Swift Package Manager builds: repackage the bundled TensorFlowLite xcframeworks (correct simulator slice identifiers and framework structure) so they resolve under SPM, including on the iOS simulator.
• Add a standalone example that depends only on flutter_litert and opencv_dart.

• Raise minimum deployment targets to iOS 13.0 / macOS 10.15 to satisfy Swift Package Manager's FlutterFramework requirement (fixes SPM build failures on macOS/iOS).
• Update example and documentation to use flutter_litert_flex: ^1.0.0.

• Update example to use flutter_litert_flex: ^0.0.8.

• Fix SPM: add missing FlutterFramework dependency to iOS and macOS Package.swift.

• Add SPM support for iOS: TensorFlowLiteC, TensorFlowLiteCMetal and TensorFlowLiteCCoreML are now declared as binary targets in Package.swift so the plugin works with Flutter Swift Package Manager integration.
• Fix duplicate XNNPack symbol linker errors when flutter litert flex is used alongside flutter litert by removing XNNPack definitions from TFLiteFlex and hiding overlapping symbols in TensorFlowLiteC via nmedit.
• Fix stale flex dedup marker in podspec that caused nmedit to be skipped on re-downloaded xcframeworks.

• Fix WASM compatibility: replace dart:io import in camera_frame.dart with flutter/foundation.dart to allow package to compile under the WASM runtime.

• prepareCameraFrameFromImage and prepareCameraFrame now auto-detect isBgra based on platform. macOS uses BGRA, Windows and Linux use RGBA. The isBgra parameter is now nullable and no longer needs to be passed manually.

• Update documentation

• Add decodeBitmap(Uint8List bytes) free function: decodes encoded image bytes (JPEG, PNG, etc.) to a web.ImageBitmap via createImageBitmap, off the main thread.
• Add WebGpuFallback mixin: transparent WebGPU-to-WASM runtime fallback for web detector classes. Provides withFallback<T>() which catches GPU errors, swaps all runners to WASM via swapToWasm(), and retries once. Apply with with WebGpuFallback; implement activeAccelerator and swapToWasm().
• Both exported from package:flutter_litert/flutter_litert.dart on web.

• Add LiteRtInterpreter, an alternative web inference path backed by Google's official LiteRT.js runtime (@litertjs/core). Selectable at construction time via LiteRtInterpreter.fromBytes(bytes, accelerator: 'webgpu' | 'wasm'), with automatic fallback from webgpu to wasm when ops aren't supported by the GPU delegate.
  • Surface chosen to match the Interpreter hot path used by detector packages: fromBytes, getInputTensor / getOutputTensors, runForMultipleInputs(inputs, outputs). runForMultipleInputs is async (LiteRT.js run returns a Promise).
  • Output buffers can be supplied as Float32List, ByteBuffer, or the legacy nested List<List<List<double>>> shape used by tflite-js callers; the float-typed buffer paths take a single bulk copy.
  • Read paths use JSFloat32Array.toDart directly, skipping the dataSync().dartify() round-trip.
  • Faster output readback in the existing tflite-js Interpreter._tensorFromJSTensor: replaces dataSync().dartify() as List<double> + Float32List.fromList(...) with a single bulk copy via JSTensorExtensions.dataSyncFloat32. ~25 ms / call savings on a 705k-element YOLOv8n output.
  • Auto-loader: by default the first LiteRtInterpreter.fromBytes(...) call programmatically appends a <script type="module"> to <head> that imports @litertjs/core from jsDelivr and calls loadLiteRt(...); consumers don't have to touch their web/index.html. Override URLs (for self-hosting / strict CSP) or disable auto-loading via configureLiteRtLoader(moduleUrl: ..., wasmUrl: ..., autoLoad: ...). Existing host-page loaders that assign window.LiteRt and dispatch a litert-ready event still work.
  • Pure additive: native and unsupported targets are unchanged; the existing tflite-js Interpreter remains the default web runtime.

• Make camera_overlay.dart WASM-compatible on Flutter Web

• Add painter primitives drawLandmarkMarker, drawSkeletonConnections, and drawBoundingBoxOutline for reuse by detector example apps and overlay widgets. Pure Dart + dart:ui, no new dependencies.

• Add camera-overlay helpers used across detector example apps: rotationForFrame, detectionSize, coverFitScaleOffset, barQuarterTurns, and FpsCounter. All pure Dart + Flutter SDK, no new dependencies. Lets example apps drop ~200 lines of duplicated orientation / sizing / FPS boilerplate.

• Add prepareCameraFrameFromImage, a duck-typed wrapper around prepareCameraFrame that accepts a CameraImage-shaped object directly (any object exposing width, height, planes with bytes/bytesPerRow/bytesPerPixel). Lets detector packages expose one-line camera-stream APIs without adding package:camera as a dependency here. Pure Dart, no new dependencies.

• Add prepareCameraFrame helper plus CameraFrame, CameraFrameConversion, and CameraFrameRotation types. Describes a camera frame (YUV420 or packed BGRA/RGBA) in a pure-Dart descriptor that detector packages can hand to their existing detection isolate, moving the cvtColor / rotate work off the UI thread without adding opencv_dart as a dependency here.
• Add CameraPlane typedef (structurally identical to YuvPlane; use whichever name reads better at the call site).
• Add TensorFloat32Views (native only): captures Float32List views of an Interpreter's input/output tensors once after allocateTensors, letting detector packages reuse the same view wrappers on every inference instead of recreating them per-call. Pure Dart, no new dependencies.

• Add packYuv420 helper for packing NV12 / NV21 / I420 camera frames into a contiguous buffer

• Minor performance/accuracy optimizations:
  • Remove unnecessary rounding in fillNHWC4D
  • Add direct Float32List fast paths for common tensor flattening shapes

• Fix Android JVM target mismatch: bump Java compile target to 17 to match Kotlin target set by Flutter toolchain

• Fix Android Flutter beta builds by aligning Kotlin and Java JVM targets to 11

• Fix edge case in output buffer allocation

• Update documentation

• Enable XNNPACK delegate on Android (ARM NEON SIMD acceleration in auto mode)
• Allow explicit PerformanceConfig.xnnpack() on iOS
• Initialize XNNPackDelegateOptions from native defaults (preserves QS8/QU8 quantization flags)

• Add Windows XNNPack delegate support (2-5x CPU inference speedup via SIMD)
• Add CI workflow to build Windows TFLite C DLL from source with XNNPack symbols

• Fix Android custom ops library alignment for 16 KB page-size devices

• Add useIsolateInterpreter parameter to skip nested isolate creation

• Fix native crash during repeated inference by removing unsafe output tensor writeback

• Fix macOS native crashes by disabling auto IsolateInterpreter for no-delegate interpreters.

• Fix WASM compatibility: move dart:isolate imports behind conditional exports so web compilation path is WASM-safe

• Fix: use-after-free when interpreter reads model weights from freed buffer, transfer buffer ownership from Model to Interpreter

• Add IsolateWorkerBase for shared isolate lifecycle management
• Add RoundRobinPool generic round-robin pool utility
• Add TensorType enum, LandmarkMixin, listUtils shared helpers
• Add weighted NMS with spatial grid optimization to nms()
• Consolidate platform-specific byte conversion into shared implementation
• Consolidate platform-specific tensor logic (native/web/unsupported)
• Consolidate desktop library loading into DelegateLibraryLoader
• Remove dead files: all_unsupported.dart, version.dart, flutter_litert_method_channel.dart, flutter_litert_platform_interface.dart
• Fix: Model buffer leak, delegate options leak, stale tensor cache

Breaking: Point.x and Point.y changed from int to double.

• Upgrade Point to double-precision with optional z depth, ==/hashCode, toMap()/fromMap(), is3D
• Add shared BoundingBox class (4-corner Point-based, supports rotated boxes)
  • BoundingBox.ltrb() factory for axis-aligned boxes
  • left/top/right/bottom convenience getters
  • width, height, center, corners computed properties
  • toMap()/fromMap() serialization

• Fix tensor cache bug, add shared Point class, dedup internals

• Add NaN handling to clamp01(), returns 0.0 for NaN inputs

• Add IsolateRpcClient and setupIsolateHandshake for reusable isolate request/response communication

• Add shared ML utility functions
  • sigmoid, sigmoidClipped, clip, clamp01, argSortDesc, median, normalizeRadians (math utilities)
  • iouXYXY, nms (non-maximum suppression)
  • computeLetterboxParams, computeAspectPadParams, LetterboxParams, AspectPadParams (image preprocessing)
  • bgrBytesToRgbFloat32, bgrBytesToSignedFloat32, fillNHWC4DFromBgrBytes (image-to-tensor conversion)
  • allocTensorShape, createOutputBuffers, zeroOutputBuffers, createNHWCTensor4D, fillNHWC4D, flattenDynamicTensor (tensor allocation)
  • decodeDetectionOutputs, transpose2D, concat0, ensure2D, xywhToXyxy (model output decoding)
  • postProcessDetections, Detection, decodeAndSplitOutputs (end-to-end detection post-processing with NMS)

• Fix package layout to follow Pub conventions

• Add PerformanceConfig and PerformanceMode,
• Add InterpreterFactory and InterpreterPool
• Add generateAnchors() and SSDAnchorOptions
• Add scaleFromLetterbox() utility for letterbox-to-original coordinate mapping

• Add SignatureRunner for on-device training workflows (train, infer, get_weights, set_weights signatures)
• Add Linux FlexDelegate support via flutter_litert_flex (Linux x86_64, built from TF 2.20.0 source). All three desktop platforms (macOS, Windows, Linux) now fully support on-device training with SELECT_TF_OPS models and checkpoint save/restore.
• Add Interpreter.signatureCount, signatureKeys, getSignatureKey(), getSignatureRunner()
• Add SignatureRunner.cancel(), getInputTensors(), getOutputTensors(), lastNativeInferenceDurationMicroSeconds

• Add native dylibs to SPM Package.swift
• Update Dart loading paths for SPM bundle

• Improve Custom Ops documentation

• Upgrade Linux TFLite native library from 2.9.3 to 2.20.0 (built from source via CMake + Ninja + GCC x86_64)
• First stable release:
  • All platforms are on updated 2.20.0 library files, official final stable release of TFLite
  • Pre-bundling works on supported native platforms: users no longer need to bundle native libraries manually as was required with tflite_flutter
  • Custom ops supported, see face_detection_tflite v5.0.2 example directory for a working example (the binary segmentation model selfie_segmenter.tflite uses custom ops)
  • Web support (experimental) functional, see pose_detection v1.0.1 web_example directory for a working example

• Update dependencies

• Update documentation

• Web support (experimental)

• Register iOS pluginClass

• Add missing null check in interpreter teardown path on macOS

• Improve IsolateInterpreter shutdown reliability on iOS to prevent rare use-after-free when closing during active inference

• Add Swift Package Manager (SPM) support for iOS and macOS

• Upgrade Windows TFLite native library from 2.18.0 to 2.20.0 (built from source via CMake + Ninja + MSVC x64)

• Fix iOS: download xcframeworks at pod install time so static linking works on first build

• Fix macOS: bundle native libraries in pub package so flutter test works without manual setup

• Fix iOS and macOS podspec compatibility with Ruby 3.4+ (Prism parser)

• Upgrade iOS TensorFlow Lite from 2.17.0 (CocoaPods) to 2.20.0 (built from source via Bazel)
• Replace CocoaPods TensorFlowLiteSwift dependency with vendored xcframeworks (TensorFlowLiteC, Metal delegate, CoreML delegate)
• All xcframeworks support device arm64 + simulator arm64/x86_64 (Apple Silicon and Intel Macs)

• Improved documentation

• Upgrade macOS TFLite native library from 2.17.1 to 2.20.0 (latest stable, universal binary: arm64 + x86_64)
• Update all C API headers to TFLite 2.20.0
• Regenerate FFI bindings (TfLiteOperatorCreate now takes 4 params, TfLiteOperatorCreateWithData removed, new kTfLiteOutputShapeNotKnown status, new builtin ops)
• Rebuild macOS custom ops dylib against 2.20.0

• Upgrade macOS TFLite native library from 2.11.0 to 2.17.1 (universal binary: arm64 + x86_64)
• Update all C API headers to TFLite 2.17.1 (including new TfLiteOperator API replacing TfLiteRegistrationExternal)
• Regenerate FFI bindings with new APIs (SignatureRunner, TfLiteInterpreterCancel, and more)
• Rebuild macOS custom ops dylib as universal binary (arm64 + x86_64)

• Bundle libtensorflowlite_c-win.dll from flutter_litert Windows plugin instead of downstream packages

• Fix Windows: build and bundle custom ops DLL (tflite_custom_ops.dll) for MediaPipe models
• Fix heap corruption crash when switching between segmentation models (custom op name string was freed prematurely)

• Fix Linux: build and bundle custom ops library (libtflite_custom_ops.so) so MediaPipe models with custom ops (e.g. selfie segmentation) work on Linux

• Update AndroidManifest.xml

• Fix IsolateInterpreter thread-safety bug causing intermittent native crashes when hardware delegates are active

• Initial release, forked from tflite_flutter_custom v1.2.5
• Rebranded to flutter_litert for LiteRT ecosystem
• All native libraries bundled automatically
• Custom ops support (MediaPipe models)
• Full platform support: Android, iOS, macOS, Windows, Linux