Improve tokenizer

[xhluca]

• Closes On-the-fly stemming #31
• Makes tokenizing queries much faster since we don't need to rebuild the vocab (can just use the existing vocab from the corpus

TODO

• add better examples
  • Streaming via generator, and calling next
  • tokenize in int, ids, strings, tuple
  • show tokenizing corpus -> tokenize queries
• add tests for BM25.retrieve under each possible condition:
  • Tokenized namedtuple
  • Object with ids and vocab
  • tuple (not Tokenized namedtuple) of ids and vocab
  • ids: List[List[int]]
  • strings: List[List[str]]
• add tests for Tokenizer.tokenize, where it generates:
  • Tokenized namedtuple
  • Object with ids and vocab
  • tuple (not Tokenized namedtuple) of ids and vocab
  • ids: List[List[int]]
  • strings: List[List[str]]
• add tests for Tokenizer.streaming_tokenize