spike: @portabletext/plugin-yjs — Yjs CRDT integration at the PT patch level#2265
Draft
christianhg wants to merge 16 commits intomainfrom
Draft
spike: @portabletext/plugin-yjs — Yjs CRDT integration at the PT patch level#2265christianhg wants to merge 16 commits intomainfrom
christianhg wants to merge 16 commits intomainfrom
Conversation
Yjs CRDT integration for the Portable Text Editor that operates at the
PT patch level rather than the Slate operation level.
The plugin translates between PT patches and Y.Doc mutations using the
editor's existing public API:
- editor.on('mutation') for outgoing local patches
- editor.send({ type: 'patches' }) for incoming remote patches
This approach means:
- No Slate internals are touched
- _key-based addressing maps naturally to CRDT structures
- diffMatchPatch carries character-level diffs → Y.Text CRDT merging
- PTE handles all remote change isolation (withRemoteChanges, etc.)
Co-authored-by: Aldus <aldus@miriad.app>
Add Yjs CRDT sync toggle to the playground: - 'Yjs' toggle in the header enables/disables Yjs mode - When enabled, patches flow through a shared Y.Doc instead of direct broadcast between editors - Y.Doc tree viewer tab in the inspector panel - Latency simulation support (per-editor Y.Docs with delayed sync) - Playground's broadcast patches/value actions are skipped when Yjs mode is active Co-authored-by: Aldus <aldus@miriad.app>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
|
When Yjs mode is toggled on after content already exists, the Y.Doc doesn't have the existing blocks. Now on each mutation, ensureBlocksExist checks the snapshot and adds any missing blocks to the Y.Doc before applying patches. This fixes typing in pre-existing blocks not being reflected in the Y.Doc.
Replace the patch log approach with a real Y.Doc structure: - Root Y.XmlFragment contains blocks as Y.XmlText children - Each block's Y.XmlText has attributes (_key, _type, style, etc.) and delta content (spans as text inserts with _key, marks attributes) - Text edits use Y.Text character-level CRDT merging - KeyMap provides O(1) bidirectional _key ↔ Y.XmlText lookup PT patch → Y.Doc translation handles all 5 patch types: - diffMatchPatch → parse DMP, compute char-level diff, apply to Y.XmlText - set → setAttribute or format (for span properties) - unset → removeAttribute or delete from fragment - insert → blockToYText + insert into fragment - setIfMissing → conditional setAttribute 15 unit tests covering blockToYText conversion and all patch types. Y.Doc → PT patches direction uses snapshot fallback for now, granular event translation in progress.
8 tests covering: - PT patch → Y.Doc roundtrips (text edit, style change, insert+unset, marks) - Concurrent text edits merge via Y.Text CRDT - Concurrent style changes converge (last-writer-wins) - Concurrent block insert + text edit both preserved - Concurrent block delete + text edit both applied All tests use two Y.Docs synced via Y.applyUpdate to simulate real collaborative editing scenarios.
Replace the old Y.Array<string> patch log viewer with a proper XmlFragment tree viewer showing blocks, spans, attributes, and delta content. Reset function now clears the XmlFragment root.
- Add ydoc-to-patches.ts: translates Y.Doc observeDeep events to PT patches - Y.XmlText attribute changes → set/unset patches - Y.XmlText delta text changes → diffMatchPatch patches - Y.XmlText delta format changes → set patches on span marks - Y.XmlFragment structural changes → insert/unset patches for blocks - Add 8 unit tests for ydocToPatches (ydoc-to-patches.test.ts) - Wire ydocToPatches into plugin observeDeep handler (replaces snapshot fallback) - Add @sanity/diff-match-patch dependency for DMP string generation - Export ydocToPatches from package index 31 tests passing across 3 test files.
Cast ydocToPatches output to Patch[] when passing to editor.send(). The values are JSON-compatible (from Y.Doc attributes) but typed as unknown in the local interfaces.
11 new edge case tests covering: - Empty blocks (Yjs optimizes away empty string inserts) - Multi-span text editing (second span, marks preservation) - Span insertion between existing spans - Span deletion from multi-span blocks - Multiple block insertion - First block deletion - Full block replacement via set - markDefs with link annotations - setIfMissing for markDefs Fix: handle empty spans in diffMatchPatch. When a span has empty text, Yjs optimizes away the delta entry. The first keystroke in an empty block now correctly creates the delta entry with span attributes.
4 new tests covering real PTE editing patterns: - Bold selection splits span into two (truncate + insert in one transaction) - Unbold merges spans back (extend text + delete span) - Enter key splits block (truncate span + insert new block) - Concurrent span split + text edit converge correctly These test multi-patch mutations applied atomically in a single Y.Doc transaction, which is how the plugin handles real PTE edits.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@portabletext/plugin-yjs— CRDT collaborative editing for PTESpike for a Yjs-based collaboration plugin that operates at the PT patch level, not Slate internals.
Architecture
Y.Doc structure:
Y.XmlFragmentcontains blocks asY.XmlTextchildrenY.XmlTexthas attributes (_key,_type,style, etc.) and delta content (spans as text inserts with_key,marksattributes)KeyMapprovides O(1) bidirectional_key↔Y.XmlTextlookupWhy PT-level, not Slate-level:
_key-based addressing is naturally CRDT-friendly — stable identity under concurrent edits, no offset tracking neededdiffMatchPatchcarries character-level diffs that map directly to Y.Texteditor.on('mutation'),editor.send())What's in this PR
Plugin package (
packages/plugin-yjs):apply-to-ydoc.ts— PT patches → Y.Doc (all 5 patch types: set, unset, insert, diffMatchPatch, setIfMissing)ydoc-to-patches.ts— Y.Doc observeDeep events → PT patcheskey-map.ts— bidirectional_key↔Y.XmlTextlookupplugin.ts— lifecycle, initial sync, echo preventiontypes.ts— shared interfaces46 tests across 4 files:
apply-to-ydoc.test.ts— 15 tests for PT→Y.Doc directionydoc-to-patches.test.ts— 8 tests for Y.Doc→PT directionroundtrip.test.ts— 8 tests for roundtrip + concurrent editingedge-cases.test.ts— 15 tests for empty blocks, multi-span, span split/merge, block split, concurrent split+editPlayground integration:
Key findings
_keyattributes prevent Yjs from merging adjacent spans, even with identical marksWhat's left for production
markDefsconcurrent annotation editing (currently whole-array set)