A local runtime behind one narrow surface. It renders nothing: prompts arrive rendered, and stop strings, grammar and preserved tokens come in GenerationOptions.
load, generate, tokenize and countTokens, saveState, loadState and dispose run one at a time, in call order. After dispose every call throws StateError.
- Implementers
Properties
- contextEpoch → int
-
Counts the calls that change the context: generate, saveState and loadState add 1 when called, before
they return, whether they later run, fail or are cancelled. An unchanged value means the context still holds
what the caller left there.
no setter
- hashCode → int
-
The hash code for this object.
no setterinherited
- info → EngineInfo?
-
The loaded model; null before load completes and after dispose.
no setter
- isLoaded → bool
-
Whether a model is loaded and the engine is not disposed.
no setter
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- supportsStatePersistence → bool
-
Whether saveState and loadState work on this runtime.
no setter
Methods
-
countTokens(
String text) → Future< int> -
(await tokenize(text)).length. -
dispose(
) → Future< void> - Frees the model. Queued generations complete as cancelled, other queued calls fail with StateError, and a running generation is cancelled. Calling it again does nothing more.
-
generate(
String prompt, [GenerationOptions options = const GenerationOptions()]) → Generation -
Queues a generation from the rendered
promptand returns its handle at once. A runtime failure, or no model loaded when its turn comes, fails the handle; only a call after dispose throws (StateError). -
load(
String modelPath, [LoadOptions options = const LoadOptions()]) → Future< EngineInfo> -
Loads
modelPath. Throws StateError while a model is loaded or loading, ModelLoadException when the runtime cannot load it. A model without a chat template loads (EngineInfo.chatTemplate is then null). -
loadState(
String path) → Future< int> - Restores a state written by saveState; returns the tokens it holds. Failures throw StateFileException.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
saveState(
String path, String promptPrefix) → Future< SavedState> -
Fills the context with exactly
promptPrefix, samples nothing, and writes the state topath. A recurrent state cannot be rewound, so the state ends at the prefix's last token. Failures throw StateFileException. -
tokenize(
String text) → Future< List< int> > -
The tokens of
text, special tokens added as generate adds them (the vocabulary decides about BOS). -
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited