BackendPerfContextData class
Native performance timings reported by llama.cpp for the active context.
Constructors
- BackendPerfContextData({required double loadMs, required double promptEvalMs, required double evalMs, required double sampleMs, double? decodeMs, required int promptEvalTokens, required int evalTokens, required int sampleCount, required int reusedGraphs, int? speculativeDraftTokens, int? speculativeAcceptedDraftTokens, int? speculativeDraftAttempts, int? speculativeVerifyTokens, int? speculativeReplayTokens, double? speculativeDraftMs, double? speculativeVerifyMs})
-
Creates a new BackendPerfContextData.
const
Properties
- decodeMs → double?
-
Time spent decoding generated tokens in ms, excluding prompt ingestion.
final
- evalMs → double
-
Time spent generating tokens in ms.
final
- evalTokens → int
-
Number of generated tokens evaluated.
final
- hashCode → int
-
The hash code for this object.
no setterinherited
- loadMs → double
-
Time spent loading the model in ms.
final
- promptEvalMs → double
-
Time spent evaluating prompt tokens in ms.
final
- promptEvalTokens → int
-
Number of prompt tokens evaluated.
final
- reusedGraphs → int
-
Number of times compute graphs were reused.
final
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- sampleCount → int
-
Number of sampler steps recorded.
final
- sampleMs → double
-
Time spent sampling generated tokens in ms.
final
- speculativeAcceptanceRate → double?
-
Accepted speculative draft-token ratio, when available.
no setter
- speculativeAcceptedDraftTokens → int?
-
Number of speculative draft tokens accepted by the backend.
final
- speculativeDraftAttempts → int?
-
Number of speculative draft attempts made by the backend.
final
- speculativeDraftMs → double?
-
Time spent generating speculative draft tokens in ms.
final
- speculativeDraftTokens → int?
-
Number of speculative draft tokens proposed by the backend.
final
- speculativeReplayTokens → int?
-
Number of target-model tokens decoded to replay accepted speculative work.
final
- speculativeVerifyMs → double?
-
Time spent verifying speculative draft tokens in ms.
final
- speculativeVerifyTokens → int?
-
Number of target-model tokens decoded to verify speculative drafts.
final
Methods
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited