]> Devi Nivas Git - chevron.git/commit
Split tokenizer up into helper functions
authornoah morrison <noah@morrison.ph>
Tue, 23 Dec 2014 03:57:23 +0000 (22:57 -0500)
committernoah morrison <noah@morrison.ph>
Tue, 23 Dec 2014 03:57:23 +0000 (22:57 -0500)
commitdbccec5aa32313657ff38df13f40487a8bff0bca
tree35fb365a3e7fa7d0edaef149b940fdc1820d7173
parentfc7b183933b51002a29a12fa79df9f70dd7619b4
Split tokenizer up into helper functions

In an attempt to make it more readable, and seperate parsing
logic from tokenizing logic.
chevron/tokenizer.py