返回 DeepSeek-Reasonix
tokenizer.py
根目录 / benchmarks / verification-stress / tasks / vs-pos-invidx / workdir / tokenizer.py
1 def tokens(text):
2 return [w.lower() for w in text.split()]
3
3 lines PYTHON