python ucto，这是一个绑定到标记化器ucto的python。标记化是几乎任何自然语言程序的第一步...
This is a Python binding to the tokenizer Ucto. Tokenisation is one of the first step in almost any Natural Language Processing task, yet it is not always as trivial a task as it appears to be. This binding makes the power of the ucto tokeniser available to Python. Ucto itself is regular-expression based, extensible, and advanced tokeniser (2023-04-22, Cython, 15KB, 下载0次)