[COPY] --- T2-COPYRIGHT-BEGIN --- [COPY] t2/package/*/sentencepiece/sentencepiece.desc [COPY] Copyright (C) 2022 - 2025 The T2 SDE Project [COPY] SPDX-License-Identifier: GPL-2.0 [COPY] --- T2-COPYRIGHT-END --- [I] Unsupervised text tokenizer for Neural Network-based text generation [T] SentencePiece is an unsupervised text tokenizer and detokenizer mainly [T] for Neural Network-based text generation systems where the vocabulary [T] size is predetermined prior to the neural model training. [U] https://github.com/google/sentencepiece [A] Google [M] The T2 Project [C] base/development [E] add protobuf [L] APL [S] Stable [V] 0.2.0 [P] X -----5---9 160.000 [O] hook_add postmake 6 "cd python; ${pyscript:-python} setup.py build install ${pyconfopt:=--prefix $root/$prefix} " [D] f1638cc515037133c90b6d7cb084232bbb669319528aef3d83c64e04 sentencepiece-v0.2.0.tar.gz https://github.com/google/sentencepiece/archive/v0.2.0/