Features

• Improve scanning and parsing performance 15X-20X

Material breaking changes

• lexical::state::Machine is completely changed, effectively a total rewrite.
• lexical::ErrorKind::BadSurrogate no longer has the offset field.

Other changes

• Fixed a bug where PipeAnalyzer would return an early EOF if given an empty Bytes value.

• Minor updates to documentation.
• Fixed a deadlock issue in PipeAnalyzer doctests.

• Rough implementation of pipe module and PipeAnalyzer for asynchronous parsing.
• Minor bug fix in read::ReadAnalyzer buffer management.

JSON Pointer support

Features added

• Finish supporting streaming JSON Pointer evaluation at scale, via the pointer module, by adding/completing pointer::Evaluator.

Breaking changes

• With the introduction of Token::is_struct, Token::is_punct now does not include [, ], {, or }. (commit 0468ba5)
• With the introduction of syntax::Context::struct_kind(), the enum Struct is renamed to StructKind for sanity and reserved-word avoidance. (commit bd7c12ff)

Other changes

• Make syntax::ErrorKind both PartialEq and Eq (commit 36e5841)
• Re-organize pre-push hook for more efficient workflows (commits 55b45b0 and 27550a1)
• Run benchmark tests (not the actual benchmarks though) in pre-push hooks (a402328)

Preview release of basic JSON Pointer features

This preview release makes available the pointer::Pointer and pointer::Group types. These aren't particularly useful yet because the streaming JSON Pointer evaluation features haven't been built yet, but these are coming soon.

• Fixes a UTF-8 validation bug.
• Adds throughput benchmarks runnable with $ cargo bench.
• Adds summary of benchmark results to README.md.

This change updates the documentation that shows up on docs.rs so that:

1. All features are enabled for the purposes of documentation.
2. Items that are only available on a feature flag are automatically annotated as such.

This makes the docs.rs documentation more valuable since it allows the full feature set to be browsed, even if some feature flags aren't on by default.

Breaking Changes

1. d2f8656 - Replace Cow<'_, str> with new Unescaped type in Content trait.

   • Unescaped acts similarly to a Cow<'_, str> but works with arbitrary string representations including representations where the representation in memory is non-contiguous. (i.e. some kind of Buf).
   • This enables the low-allocation, low-copy stream parsing of ReadAnalyzer, as well as future similar async functionality.

2. d2f8656 - Introduce Literal associated type into Content trait.

   • A Content::Literal must be IntoBuf.
   • This enables the outputs from all lexical analyzers, including ReadAnalyzer, to be described in a general way and will ensure that tokens from any lexical analyzer can be evaluated by the upcoming JSON Pointer module.

Material Non-Breaking Changes

1. 71ab8b4, d2f8656 - Introduce the Buf trait, to support no-copy parsing of streams

   • Added Buf trait for efficient zero-copy buffer operations
   • Added IntoBuf trait for converting types into Buf implementations
   • Added StringBuf type implementing Buf for String
   • Added BufUnderflow error type
   • Added buf_cmp function for comparing IntoBuf values

2. b874196 - Make lexical::unescape fully pub

   • Made lexical::unescape function publicly available for standalone use
   • Users can now use the unescape functionality independently of the full lexical analysis stack

3. 05e7cbe - Generalize lexical::unescape to operate on a Buf, not a &str

   • By operating on values that are IntoBuf, unescape can be used in a wider range of use cases, such as expanding JSON escape sequences from string values that are not stored contiguously in memory because they span buffers.

4. d2f8656 - Add lexical analyzer for std::io::Read streams: ReadAnalyzer

   • Added new lexical::read module with ReadAnalyzer for tokenizing std::io::Read streams
   • Added Literal, Content, and Error types specific to ReadAnalyzer
   • Added read feature flag to Cargo.toml (disabled by default)

5. d2f8656 - Add traits EqStr and OrdStr to capture the property of "being comparable to a string slice"

   • If a type T implements these traits then Unescaped<T> can be compared to &str

Breaking Changes

1. Fixed bug: Number exponent parsing (commit cc3873c):
   • Breaking because some lexical::Expect enum members are renamed.
   • Fixed uppercase 'E' not being recognized as exponent character
   • Improved error reporting for number parsing edge cases
2. Added more convenience methods to Token (commit edcb726):
   • Breaking because some Token::is_* methods were renamed for better precision.
   • Added new convenience methods: Token::is_literal, Token::is_pseudo, Token::is_terminal
3. Removed mut from &self in Content::unescaped (commit 3287766):
   • Soft breaking change because it could cause compiler warnings to sprout, and that can break builds that prohibit warnings.
   • This makes the Content interface cleaner and easier to use.

Material Non-Breaking Changes

1. Fixed bug: Error message parts ran together due to no space (commit d46256a):
   • Fixed missing space in error messages between expected character and token context

Breaking Changes

1. Improved content fetch interface ergonomics (commit 7b92405):
   • Replaced lexical::Analyzer::content() and Parser::content() methods that returned Result with a three-method approach:
     1. fn content(&self) -> Content - panics on error token
     2. fn err(&self) -> Error - panics on non-error token
     3. fn try_content(&self) -> Result<Content, Error> - original behavior
   • This eliminates the need for unwieldy .unwrap() and .unwrap_err() calls in parsing loops
2. Renamed BufAnalyzer to FixedAnalyzer (commit f8d9891):
   • Renamed the main lexical analyzer type to better reflect its purpose (analyzing fixed-size buffers)
   • Reduces confusion with the crate name and provides more descriptive naming
3. Improved Content::unescaped ergonomics (commit 0c33fcb):
   • Changed Content::unescaped method signature from &mut self -> &str to &mut self -> Cow<'_, str>
   • Returns Cow::Borrowed when no unescaping is needed, Cow::Owned when it expanded at least one escape sequence.

Material Non-Breaking Changes

1. Reduced fixed::Content (formerly buf::Content) size from 48 to 32 bytes (commit 0c33fcb)
2. Added Pos::new() constructor (commit 4f0527b):
   • Added convenience constructor for creating Pos instances: Pos::new(offset, line, col)
   • Improves testing ergonomics because it's a shorter "phrase"
3. Made Token::static_content() a const fn (commit 0cf5efb):
4. Added module-level documentation for mod lexical (commit 48e1025):
5. Switched from MIT license to dual MIT and Apache-2.0 (commit ad3f963):
6. Updated README (commits 4b09f21, 746e335, 0bd677f):

Full Changelog: v0.2.0...v0.3.0