FR: complete documentation in a single page for LLMs

[thiswillbeyourgithub]

Hi,

I've been interested in sh for a while and it seems really awesome.

Notably I'm interested in using it to make very concise wrapper for cli tools, for example to wrap edgevpn into a python lib

The limiting factor for such side projects is how profficient LLMs are with it and so far it seems that it's not quite popular enough to be reliably used by LLMs (my goto being aider-chat).

Hence, to use sh, the documentation has to be provided manually to the LLM, so would you be interested in making, in the docs, a single page compilation of everything an LLM might want?

To give an example, I am the dev behind wdoc and because it contains many different use cases I made a Single Page Documentation page in the website.

I'm thinking it could be done with a simple commit to add a page in the website.

Thanks and have a nice day!

[amoffat]

Hi there. If you want to write it, and it is high quality documentation, I will merge it.

[thiswillbeyourgithub]

Great! The thing is, there's not much to write. It's just a matter of adding a page to the docs that contain other pages. See this for example. It's kinda awkward to write that without seeing the readthedocs website update imho but if you want me to i'll try.

[amoffat]

I don't think I understand. If it's just a page linking to other pages, why can't the LLM system use the existing docs, which are links to other pages?

Also you can clone the repo and build the docs locally for interactive feedback.

[thiswillbeyourgithub]

I'll try to be clearer:

• this all_docs.rst file includes the pages readme, examples, help, parse_file_help and api.
• The rendered webpage can be seen here
• As you can see, the effort required to create this page is minimal: it just concatenates the already written documentation. It does not link them, it contains them.
• The reason I'm making this FR is to ask to make it as simple as copy+pasting a url to get all the documentation of sh. Yes it is possible currently to paste each page, or to use things like repomix to convert repositories into single text but this adds friction.
• In my view: the cost of adding this page of documentation is small but it allows making it dead easy for new users to just dump your documentation in the context of an LLM and start building.

I can give two examples:

1. I made wdoc, it's kinda complicated and has many use cases and settings. If I want a new user to get started very quickly, they can just give this url to an LLM with a specific question like how can I create an index for a whole anki database and then ask question on it. Lower friction, more accessible, a happier humanity.
2. I made save_to_zotero, a tool to save urls and webpages to zotero. Although zotero is somewhat old and well known, it has few API clients with many breaking changes and current LLM completely hallucinate how they work. Hence, it proved extremely handy that the lib pyzotero has a full single page documentation which made it a lot easier to do my script.

So to reiterate: yes it is possible to put sh's doc into an LLM's context but it's friction heavy and having a single page API would be a patero improvement in my opinion.

[amoffat]

I see, thanks for explaining.

It looks like sphinx has a make singlehtml target and I tested it locally and it appears to do what you're requesting. If you figure out how to alter the build scripts to build that target and upload it to RTD, then it should be what you're looking for and we could merge those changes.

[thiswillbeyourgithub]

Okay so I took a look and modifying the sphinx builders to use html + singlehtml is not super clear and it runs on the readthedocs server side so I prefer to go the other way. Additionally it does not allow to filter only some pages, nor to specify the page order.

After doing super over complicated things I finally found a way to do this without seriously refactoring the doc. The only issue I'm seeing in my local build is that some headers seem to be missing.

The online version:

My local version is missing the "Passing argument" header:

But this looks like a bug and might not happen in the online rendering. Also they should not pose much problem to LLMs.

I'll make the PR right away. It'll remain in draft until I'm sure I can put the full api in there too.