generate method

  1. @override
Generation generate(
  1. String prompt, [
  2. GenerationOptions options = const GenerationOptions()
])
override

Queues a generation from the rendered prompt and returns its handle at once. A runtime failure, or no model loaded when its turn comes, fails the handle; only a call after dispose throws (StateError).

Implementation

@override
Generation generate(String prompt, [GenerationOptions options = const GenerationOptions()]) {
  _checkOpen();
  _contextEpoch++;
  prompts.add(prompt);
  this.options.add(options);
  final number = prompts.length;
  final zone = Zone.current;
  final generation = QueuedGeneration(maxTokens: options.sampling.maxTokens, emptyStats: () => _stats(prompt, 0));
  _queue.runGeneration(generation, () => _run(generation, prompt, number, zone));
  return generation;
}