✂️tokenization (tōukè nàizhé)

noun
ˌtoʊ.kə.nɪˈzeɪ.ʃən
将文本分解为较小的单位,如单词、短语或符号进行分析的过程。 (tokenization)

含义

The process of breaking text into smaller units like words, phrases, or symbols for analysis.
含义翻译
将文本分解为较小部分以进行分析的过程。
Jiāng wénběn fēnjiě wéi jiào xiǎo de bùfen yǐ jìnxíng fēnxī de guòchéng.

例句

NLP uses tokenization to break text into smaller units.

NLP使用分词法将文本分解为更小的单位。
NLP shǐyòng fēn cí fǎ jiāng wénběn fēnjiě wéi gèng xiǎo de dānwèi.

同义词

segmentation, fragmentation, parsing

反义词

aggregation, whole text, non-division

相关词汇

A feeling or state of intense excitement and happiness.
physical strength and good health; effort, energy, and enthusiasm
the quality of being unwilling to work or use energy; idleness
the act of recognizing or accepting something as true or valid; an expression of gratitude or recogn...
A trip or celebration, often funded by someone else, usually for pleasure or promotional purposes.
Suggestions or proposals as to the best course of action.

探索更多