dart_sentencepiece_tokenizer 1.0.0 
dart_sentencepiece_tokenizer: ^1.0.0 copied to clipboard
brodykim.workA lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
5
likes
-/ -
pub points
10.3k
downloads
This package version is not analyzed. Check the latest stable version for its analysis.
Check the analysis log for details.
Weekly downloads
Metadata
5
likes
0
points
10.3k
downloads
Publisher
Weekly Downloads
Metadata
A lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
Repository (GitHub)
View/report issues
Topics
#nlp #sentencepiece #tokenizer #machine-learning #llm
License
unknown (license)
