@flamdragparadiddle from what I can figure out, it'll run just like any other ggml LLM, it's just trained to make correctly formatted API calls. Are you running any models currently? If not, I'd recommend oobabooga text-generation-webui (mostly because that's what I got up and running to get started) https://github.com/oobabooga/text-generation-webui let me know if you have any trouble! Once you get it running you can enable an API to use it programmatically. If you don't need to call it from an API there's a ui.