LiteRtLmFfiClient class

High-level Dart wrapper around the LiteRT-LM C API.

Provides a clean async interface over the native C functions, managing memory and translating C callbacks into Dart Streams.

Conversation lifetime is owned by LiteRtLmConversationHandle — the client holds the engine and tracks live handles for shutdown. Legacy single-session methods (createConversation, chat, sendMessageStreamRaw, etc.) route through an internal _legacyHandle for backward compatibility.

Constructors

LiteRtLmFfiClient()

Properties

hashCode → int: The hash code for this object.
no setterinherited
isInitialized → bool: no setter
nativeLogPath → String?: Path to the redirected native stderr log (LiteRT-LM absl/glog output). Set after _ensureBindings runs the stderr redirect; null on platforms where redirection isn't wired (currently it works on macOS + iOS).
no setter
runtimeType → Type: A representation of the runtime type of the object.
no setterinherited

Methods

cancelGeneration() → void: Legacy: cancel on the implicit _legacyHandle conversation.
cancelVirtualTurn(Object conversationToken) → void: Cancel an in-flight virtual turn for conversationToken. Mirrors _cancelOn but targets the shared live virtual conversation. Does NOT take the mutex (it must interrupt a generation that already holds it).
chat(String text, {List<Uint8List>? imageBytes, Uint8List? audioBytes, bool enableThinking = false}) → Stream<String>: Legacy: streaming chat on the implicit _legacyHandle conversation.
chatRaw(String text, {List<Uint8List>? imageBytes, Uint8List? audioBytes, bool enableThinking = false}) → Stream<String>: Legacy: raw streaming chat on the implicit _legacyHandle conversation.
closeConversation() → void: Legacy: close the implicit _legacyHandle conversation.
createConversation({String? systemMessage, String? toolsJson, double temperature = 0.8, int topK = 40, double? topP, int seed = 1}) → void: Legacy single-conversation create. Closes the previous legacy conversation (if any) and opens a fresh one stored in _legacyHandle. Kept for backward compat — new code should use createConversationHandle and own the handle directly.
createConversationHandle({String? systemMessage, String? toolsJson, String? messagesJson, double temperature = 0.8, int topK = 40, double? topP, int seed = 1}) → LiteRtLmConversationHandle: Create a new conversation handle with optional system message and tools. The engine allows only ONE live conversation at a time (upstream LiteRT-LM #966), so the caller must delete any prior conversation before creating a new one — this is how virtual-session multiplexing rebuilds context. The caller owns the handle and must call LiteRtLmConversationHandle.close.
dumpNativeLog() → void: Reads back the native log file (set by stream_proxy_redirect_stderr) and pipes its contents through debugPrint in 800-char chunks. Surfaces absl/glog output (model load timing, accelerator init, sampler dlopen, KV-cache prefill, etc.) that's redirected to a file by stream_proxy_redirect_stderr and wouldn't otherwise reach the Flutter console / test harness.
getSessionMetrics() → SessionMetrics: Legacy: metrics for the implicit _legacyHandle conversation.
initialize({required String modelPath, String backend = 'gpu', int maxTokens = 2048, String? cacheDir, bool enableVision = false, int maxNumImages = 0, bool enableAudio = false, bool? enableSpeculativeDecoding}) → Future<void>: Initialize the engine with model path and settings.
noSuchMethod(Invocation invocation) → dynamic: Invoked when a nonexistent method or property is accessed.
inherited
releaseVirtualConversation(Object conversationToken) → void: Tear down the live virtual conversation if it belongs to conversationToken. Called when a virtual session closes so its native conversation doesn't linger. If a different session is active, this is a no-op — that session's conversation must stay live.
sendMessage(String messageJson, {String? extraContext}) → Future<String>: Legacy: sync send on the implicit _legacyHandle conversation.
sendMessageStreamRaw(String messageJson, {String? extraContext}) → Stream<String>: Legacy: streaming raw on the implicit _legacyHandle conversation.
shutdown() → void: Shutdown the engine and release all resources. Closes every live conversation handle first (legacy + any opened directly).
startVirtualTurn({required Object conversationToken, required String messageJson, required List<({String role, String text})> history, String? systemMessage, String? toolsJson, double temperature = 0.8, int topK = 40, double? topP, int seed = 1, String? extraContext}) → Stream<String>
toString() → String: A string representation of this object.
inherited

Operators

operator ==(Object other) → bool: The equality operator.
inherited

Static Methods

buildHistoryJson(List<({String role, String text})> turns) → String: Serialize a turn history into the messages_json array the conversation config accepts as a preface. Each turn is {role, content: [{type: 'text', text}]}. Used by the virtual-session multiplexer to rebuild a session's full context (user + assistant turns) in one prefill when switching the single live conversation.
buildMessageJson(String text, {List<Uint8List>? imagesBytes, Uint8List? audioBytes}) → String: Build the JSON message for the Conversation API.
extractTextFromResponse(String jsonStr) → String: Extract text from a LiteRT-LM JSON response chunk. Delegates to SdkTextExtractor — single source of truth shared with the web @litert-lm/core path so both engines map identical chunks to text the same way.