GitHub - grdscarabe/uima-word-tokenizer: A word tokenizer component for UIMA that take advantage of unicode general classes. The tokenizer only handles French for the moment, but can be extended quite easily.

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
desc/tddts/preprocessing/ae		desc/tddts/preprocessing/ae
src/java/main		src/java/main
test/tddts/preprocessing/tokenizer		test/tddts/preprocessing/tokenizer
LICENSE		LICENSE
README		README
pom.xml		pom.xml

Repository files navigation

See http://www.fabienpoulard.info/?post/2010/09/06/Un-rapide-tokeniseur-en-mots-pour-le-fran%C3%A7ais

About

A word tokenizer component for UIMA that take advantage of unicode general classes. The tokenizer only handles French for the moment, but can be extended quite easily.

www.fabienpoulard.info/?post/2010/09/06/Un-rapide-tokeniseur-en-mots-pour-le-fran%C3%A7ais

Readme

Apache-2.0 license