Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugin System / Infrastructure to decouple all the model/api uses #26

Open
trufae opened this issue May 10, 2024 · 0 comments
Open

Plugin System / Infrastructure to decouple all the model/api uses #26

trufae opened this issue May 10, 2024 · 0 comments

Comments

@trufae
Copy link
Contributor

trufae commented May 10, 2024

Right now the logic in interpreter.py is a bit overwhelming because we are handling so many models and growing. The plan would be to create a generic class that abstracts the exposed methods to split each model into separate files, so the user can add/remove more at its will.

The common attributes I see are the following:

  • local/remote (or api: llama, http, ...) right now all local are just "llama"
  • name (taken from huggingface)
  • prompt templater (maybe we should do that dynamically using the special tokens from the model instead of cooking them by hand)
  • category (coding, chat, uncensored, functionary...)
  • description (short text describing the usecases of it)
  • date (release date of the model)

the prompt templates right now are reused between models, so maybe we should decouple that from the plugin and just make the plugin specify which template use.

RFC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant