Package pypln :: Package workers :: Module tokenizer

Module tokenizer

Functions

main(document)

source code

Variables

__meta__ = {'from': 'document', 'requires': ['text'], 'to': 'd...

Variables Details

Value:

{'from': 'document', 'requires': ['text'], 'to': 'document', 'provides
': ['tokens'],}