Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Try to adopt Myst Spec where possible. #190

Open
Carreau opened this issue Nov 10, 2022 · 5 comments
Open

Try to adopt Myst Spec where possible. #190

Carreau opened this issue Nov 10, 2022 · 5 comments
Labels
parsing planning High-level planning and roadmap discussions

Comments

@Carreau
Copy link
Member

Carreau commented Nov 10, 2022

I've recently talks with a couple of the myst folks, and I'd like to see if we can adopt the myst AST, or at least something as close to it as possible.

See the structure of the Myst AST

The myst spec are published as json schema, I haven't found it as Python objects, so I've started to write a notebook to see if we can directly generate teh Python dataclasses to represent those nodes.

https://gist.github.com/Carreau/d5d80d644b4f43e601e67874619d855a

It a bit rough on the edges, and I'm ok if there is some manual edition.

Once this is done, the goal is to mostly replace the custom ast I have in here
@Carreau
Copy link
Member Author

Carreau commented Nov 10, 2022

Once the above is roughly done, we can also replace the serializers I have with something more straightforward : https://gist.github.com/Carreau/62350dabcd75b90eccffb9fca7c2c431,

This would allow to emit json that can be rendered by the MyST Js libraries.

I don't expect us to have custom nodes and variations, but the closer we are to another spec the better. The Myst folks would I think also welcome feedback on their AST if we need any changes.

If we also find a library that already goes from json schema to pyhton-implementation i'm all for it, I just hadn't found any that support json schema $ref.

@Carreau
Copy link
Member Author

Carreau commented Jan 26, 2023

Note, it look like we can render the Mdast directly using MystJS:

<html>
  <head>
    <script src="https://unpkg.com/mystjs"></script>
  </head>
  <body onload="init();">
    <div id="output"></div>
    <script>
      function init() {
        const myst = new MyST();
        const html = myst.renderMdast({type: "text", value:"Hello"})
        document.getElementById('output').innerHTML = html;
      }
    </script>
  </body>
</html>

@aktech
Copy link
Collaborator

aktech commented Jan 31, 2023 •
edited
Loading

Are you aware of something equivalent of BlockVerbatim in the myst spec? The closest I have found is Code, but syntax of Code is different from BlockVerbatim, as in the three backticks.

Trying to replace BlockVerbatim for parsing this block in numpy docs: https://github.com/numpy/numpy/blob/b30653165c4a44b40be733217b26ac9a936772a5/numpy/distutils/__init__.py#L7-L11

UPDATE: Code and InlineCode seems to work fine for BlockVerbatim and Verbatim

@aktech aktech mentioned this issue Jan 31, 2023
Merged
@Carreau
Copy link
Member Author

Carreau commented Feb 1, 2023

UPDATE: Code and InlineCode seems to work fine for BlockVerbatim and Verbatim

Perfect, That sounds good, and if it makes less nodes that's a plus !

@aktech
Copy link
Collaborator

aktech commented Mar 22, 2023 •
edited
Loading

Status of Nodes to replace/remove:

Node Replaced Removed Comments
Verbatim
    		• --
    Directive
    		• --
    Link
    		• --
    Math
    		• --
    BlockMath
    		• --
    SubstitutionDef
    		• --
    SubstitutionRef
    		• --
    Target
    		• --
    Unimplemented
    		• --
    Comment
    		• --
    Fig
    		• --
    RefInfo
    		• --
    ListItem
    		• --
    Signature
    		• --
    NumpydocExample
    		• --
    NumpydocSeeAlso
    		• --
    NumpydocSignature
    		• --
    Section
    		• --
    Parameters
    		• --
    Param
    		• --
    Token
    		• --
    Code3
    		• --
    CodeLine
    		• --
    Code2
    		• --
    GenToken
    		• --
    Code
    		• --
    BlockQuote
    		• --
    Transition
    		• --
    Paragraph
    		• --
    Admonition
    		• --
    TocTree
    		• --
    BlockDirective
    		• --
    BlockVerbatim
    		• --
    Options
    		• --
    FieldList
    		• --
    FieldListItem
    		• --
    DefList
    		• --
    DefListItem
    		• --
    SeeAlsoItem
    		• --

    @melissawm melissawm added planning High-level planning and roadmap discussions parsing labels Oct 26, 2023
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    parsing planning High-level planning and roadmap discussions
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @Carreau @melissawm @aktech