LiteRtLmRuntimeClient class
Web-safe placeholder for the native-only runtime client.
- Implementers
Constructors
- LiteRtLmRuntimeClient()
-
Creates a placeholder client on platforms without
dart:ffi.
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
cancel(
) → void - Cancels active native generation.
-
conversationTokenCount(
) → int - Returns the active native conversation token count.
-
createConversation(
{String? systemMessage, List< Map< ? messages, List<String, dynamic> >Map< ? tools, Map<String, dynamic> >String, dynamic> ? extraContext, double temperature = 0.8, int topK = 40, double topP = 0.95, int seed = 1, bool npuBackend = false}) → void - Creates a conversation for generation/token operations.
-
detokenize(
List< int> tokens) → String - Converts native LiteRT-LM token ids back to text.
-
dispose(
) → void - Releases native LiteRT-LM resources.
-
generate(
String prompt) → Stream< String> - Streams generated text from the active conversation.
-
generateMessageJson(
String messageJson, {Map< String, dynamic> ? extraContext}) → Stream<String> - Streams generated text from a native message JSON object.
-
initialize(
{required String modelPath, String backend = 'gpu', int maxTokens = 4096, int outputTokens = 256, int? prefillTokens, String? cacheDir, bool speculativeDecoding = true, int minLogLevel = 3, LiteRtLmActivationDataType? activationDataType, int? prefillChunkSize, bool? parallelFileSectionLoading, String? dispatchLibDir}) → Future< void> - Initializes the native LiteRT-LM engine.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
readMetrics(
{required int wallMilliseconds}) → LiteRtLmRuntimeMetrics - Reads runtime metrics for the active conversation.
-
renderMessageToString(
Map< String, dynamic> message) → String - Renders a message with the active native conversation template.
-
replaceConversationWithClone(
) → void - Replaces the active native conversation with a clone.
-
run(
{required String prompt, int warmupRuns = 1, int measuredRuns = 3}) → Future< LiteRtLmRuntimeResult> - Runs a benchmark-style prompt loop and returns runtime metrics.
-
setMinLogLevel(
int level) → void - Updates the native LiteRT-LM log level.
-
tokenize(
String text, {bool addSpecial = true}) → List< int> - Tokenizes text with the native LiteRT-LM tokenizer.
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited