Daniel Dallos’ Post

For those using the #GoogleGemini Google have announced Gemini's integration with the #OpenAI Library . This integration allows developers to: 👉 Access Gemini models through OpenAI's familiar interface 👉 Use both Chat Completions API and Embeddings API 👉 Implement Gemini using the OpenAI library through Python, JavaScript/TypeScript, or REST API calls OpenAI's API is becoming the de facto standard for LLM interactions. This is similar to how Amazon S3's API became the de facto standard for object storage, with many services (including Google's own Cloud Storage) offering S3-compatible APIs. We're seeing a similar pattern emerge where: 👉 OpenAI's Chat Completions API structure has become a reference implementation 👉 Multiple LLM providers are adopting OpenAI-compatible endpoints: ➡️ Gemini (Google) ➡️ Ollama ➡️ Anthropic (through Claude compatibility) ➡️ Many open-source LLM platforms This standardization brings several benefits: 👉 Makes it easier to switch between different LLM providers 👉 Enables easier multi-model deployment strategies 👉 Simplifies the creation of abstraction layers and tools Google's decision to implement OpenAI compatibility for Gemini is particularly significant given their market position - it suggests they recognize the practical value of adopting this. #ai #enterpriseai https://lnkd.in/euZW4xsZ

#Technology #DataAnalytics #DataDriven How to build an OpenAI-compatible API: Create a server to replicate OpenAI’s Chat Completions API, enabling any LLM to integrate with tools written for the OpenAI API Image generated by the author using OpenAI DALL-E It is early 2024, and the Gen AI market is being dominated by OpenAI. For good reasons, too — they have the first mover’s advantage, being the first to provide an easy-to-use API for an LLM, and they also offer arguably the most capable LLM to date, GPT 4. Given that this is the case, developers of all sorts of tools (agents, personal assistants, coding extensions), have turned to OpenAI for their LLM needs. While there are many reasons to fuel your Gen AI creations with OpenAI’s GPT, there are plenty of reasons to opt for an alternative. Sometimes, it might be less cost-efficient, and at other times your data privacy policy may prohibit you from using OpenAI, or maybe you’re hosting an open-source LLM (or your own). OpenAI’s market dominance means that many of the tools you might want to use only support the OpenAI API. Gen AI & LLM providers like OpenAI, Anthropic, and Google all seem to creating different API schemas (perhaps intentionally), which adds a lot of extra work for devs who want to support all of them. So, as a quick weekend project, I decided to implement a Python FastAPI server that is compatible with the OpenAI API specs, so that you can wrap virtually any LLM you like (either managed like Anthropic’s Claude, or self-hosted) to mimic the OpenAI API. Thankfully, the OpenAI API specs, have a base_url parameter you can set to effectively point the client to your server, instead of OpenAI’s servers, and most of the developers of aforementioned tools allow you to set this parameter to your liking. To do this, I’ve followed OpenAI’s Chat API reference openly available here, with some help from the code of vLLM, an Apache-2.0 licensed inference server for LLMs that also offers OpenAI API compatibility. Game Plan We will be building a mock API that mimics the way OpenAI’s Chat Completion API (/v1/chat/completions) works. While this implementation is in Python and uses FastAPI, I kept it quite simple so that it can be easily transferable to another modern coding language like TypeScript or Go. We will be using the Python official OpenAI client library to test it — the idea is that if we can get the library to think our server is OpenAI, we can get any program that uses it to think the same. First step — chat completions API, no streaming We’ll start with implementing the non-streaming bit. Let’s start with modeling our request:from typing import List, Optional from pydantic import BaseModel class ChatMessage(BaseModel): role: str content: str class ChatCompletionRequest(BaseModel): model: str = "mock-gpt-model" messages: List[ChatMessage] max_tokens:… #MachineLearning #ArtificialIntelligence #DataScience

Last week, I learned about fine-tuning while updating one of the tutorials. It's amazing that using tools from Cloudflare, one can build AI apps quickly! If you want to learn about fine-tuning, here's the tutorial: https://lnkd.in/dWe337cd

Stack Overflow signed a deal with OpenAI, announced Monday. Two things to think about: 1. Contributors aren't getting compensated for the value of their contributions. I'm not making a moral or legal point, just a practical one. It kinda sucks creating other people's value for free when they flaunt it. At least with open source generally, you're sorta insulated from the monetization by anonymity of reuse. At the same time, any community that doesn't ban you is always going to be a cool hang if you're a halfway interesting contributor or person. 2. People worry about LLM generated content feeding the LLMs and making the LLMs regress. Don't worry about that. It won't happen. There is a different problem. LLM makers won't let their LLMs regress on metrics that matter (no matter how bullshitty the metrics). They will have to spend more time and money filtering their training data sets as they become less human and more generated. That actually sounds like an activity OpenAI can saddle SO with. "We'll pay you for ongoing use of your data so long as your data improves." The fun part is that it will take time and GPU money in "pre-training" to prove that iterations are improving. In short, LLMs won't regress. They will get more expensive. Qualified training data will cost more. This is a critical business insight in this space, and is probably very different than whatever narrative you've bought into. If you need this kind of insight across your business, I am available to help you, from consultant and projects to full time. Message me. H/T Axel C., who is really good at LinkedIn and will probably end up monetized without compensation if he isn't already. #WrittenByMe

Aristotle famously wrote, "The more you know, the more you realize you don't know.". This couldn't be more true in technology, weather you are a beginner or someone like me who has been doing it for years. AI like no other technology falls in this category, every time you investigate something new, you discover 10 other things you don't know. This weekend I wanted to learn more about a few things like running local LLMs, and the new and already famous Deepseek-R1, and with the release of OpenAI's Operator, I wanted to learn how to run something similar using SmolAgents. So here is what I researched/tested and learned over the weekend: Smolagents (from huggingface) https://lnkd.in/eHeXYyrB  It's a library that enables you to run powerful agents in a few lines of code. Because of their visual capabilities, you can run agents that visit websites and do tasks for you, similar to OpenAI's Operator.  I was able to run this locally and did a few tests that run successfully using OpenAI models and other open source models using Fireworks-ai. Maybe not as advanced as Operator, but you can get pretty far. Fireworks AI https://fireworks.ai/ "The Fireworks platform empowers developers to create generative AI systems with the best quality, cost and speed.".  I used to run some tests with Smalagents and also run it in another project instead of using GPT API. Seems to work well and at a lower cost, still need to calculate how much I would be saving.  Ollama  https://ollama.com/ Desktop app and CLI to let you download and run any LLM locally.  I was interested to run this from a privacy stand point but also I had to perform a task that needed to run against 11k requests, so I wanted to see if it was possible to do it for free or cheaper. It worked great but just slower than running with a hosted LLM. Openwebui https://lnkd.in/eFd9TRv4 A UI similar to ChatGPT but let's you run any open source models. Runs on top of Ollama, so you can run all of this locally. Was simple to get up and running, their interface looks very similar to ChatGPT but you can run multiple open source models. Deepseek-R1  https://www.deepseek.com/ New open source model that competes with many closed-source ones.  They support reasoning like GPT's o1 model and also web search. They also have a powerful API. Still need to do more tests but so far looked to perform as good as other models, at least for my initial tests.

There's not much you can do with OpenAI's Swarm. It has 7 major limitations 👇 [and how to address them] After using OpenAI’s Swarm, I realized several limitations: - No RAG support. - Supports limited LLMs. - No multimodality support. - The agent orchestration is quite manual and hardcoded. - Does not connect to popular vector databases like Pinecone, ChromaDB, etc. - By their very nature, agents MUST be able to interact with several tools. But this is not possible with OpenAI Swarm. - [Biggest] It's only built for experimental and educational purposes. SwarmZero solves all of the above limitations. It’s an open-source framework to build multi-agent apps in a highly customizable way and take them to production. GitHub: https://lnkd.in/gyvXqhMt I published a hands-on demo about building a PerplexityAI-like personal research assistant using SwarmZero. This agent: - Accepts a user query. - Searches the web about it. - And turns it into a well-crafted article, which can saved as a PDF, in a Google doc, confluence page, and more. The app involves five agents: 1) Web search agent → Searches the web to fetch URLs and titles. 2) URL mapper agent → Organizes the URLs and titles. 3) Website scrapper agent → Accepts the above URLs and scraps the web. 4) Analyst agent → Prepares an outline for the article using the scraped results. 5) Publisher agent → Drafts a coherent article based on the outline. The issue detailing the steps along with the code is available here: https://lnkd.in/giTe9KZQ Over to you: What are some other limitations of OpenAI Swarm?

OpenAI Integration – Let’s Go! At Thompson Thrift, the IT department is continuing to advance in technologies, including AI. So I’ve gone head-first into a world of integrating data with OpenAI, and I’d like to share some general thoughts for anyone looking to do something similar at their organization. With a paid license for OpenAI, you’re able to connect proprietary data to a custom GPT without concern that the data will be used for training the GPT. I don't recommend using any personal or private data without a licensed version of OpenAI. In my case, I wanted a custom GPT to talk to custom APIs that it could use to retrieve data from the business. I wanted to build as much flexibility as possible by allowing the GPT to write its own queries and to understand the data it was trying to retrieve. This can be done by creating actions for the GPT. Actions allow the GPT to use APIs for whatever you’d like, such as retrieving data, creating data, or kicking off processes. Within the actions screen, there are options to set an API key (which I highly recommend for your APIs for security purposes). Then there’s a place for the OpenAI schema, which defines everything OpenAI needs to understand APIs – the URL, the various routes to use, parameters to pass, the data returned by those routes, etc. If you have documentation for your API, there is an actions GPT provided by OpenAI that can build a schema for you. You may need to tweak it, though. There are also instructions you create for the GPT where you can specify how to query (among other things). For example, you might need to inform the GPT that LIMIT is not a keyword for your DBMS and to use TOP instead. I recommend telling it exactly the type of DBMS you’re using so that it can form the best queries. Beyond that, I’ve told it how to explicitly join tables together in some cases because the schema alone wasn’t enough. I might have more instructions than needed, but sometimes I feel like a parent to a five-year-old, where I need to be explicitly clear on what to do and what not to do. For example, don’t alias column names except as I direct you to; otherwise you’ll break my ORM. Sigh... I’ll try to make more posts in the future around some of the nitty-gritty items I’ve already encountered in hopes it helps others out. I've learned quite a bit in a short amount of time, but I still feel like I'm only scratching the surface.

# 🚀 **Introducing BodhiApp: Run Open Source Huggingface Models Locally on Your Laptop!** 🚀 I am very excited to introduce you to **Bodhi App**, a powerful tool designed to simplify running open-source Large Language Models (LLMs) directly on your laptop. Currently supporting M2 Macs, we plan to expand to other platforms as our community grows. ## What is BodhiApp? BodhiApp empowers you to run LLMs on your own hardware, ensuring data privacy and eliminating the need for costly subscriptions or API credits from providers like OpenAI. It's built for both tech enthusiasts and non-tech users, making advanced AI accessible to everyone. ## Key Features: 🔹 **In-built Chat UI**: Start conversing with LLMs quickly. No need to download separate chat UI to get started. 🔹 **Huggingface Ecosystem**: Bodhi App does not reinvent the wheel with complex technical setup. It leverages the amazing Huggingface ecosystem for model files and tokenizer_config.json for configuring the model. So no need for technical deep insight to play around with any compatible GGUF model from Huggingface. 🔹 **OpenAI-Compatible APIs**: Exposes the Open Source LLM inference as OpenAI compatible APIs, point local apps requiring OpenAI APIs to Bodhi to have them use your hardware and model for inference.  🔹 **Powerful CLI**: Download and configure models with user friendly and familiar command-line tool. ## Bodhi App vs Ollama ### **Bodhi App** - Designed with general non-tech audience in mind, ships with a simple Chat UI to get started quickly - Seamless integration with the Huggingface ecosystem:  - Use Hugging Face repo/filename to run a model.  - Use tokenizer_config.json for chat templates. - Currently supports Mac M2. ### **Ollama** - Popular among tech users but requires technical to customize. - No inbuilt Chat UI. - Custom model baking process with:  - Modelfile.  - Golang template for chat templates. - Supports various OS platforms. Bodhi App avoids the complexity of proprietary formats and leverages existing huggingface ecosystems, making it easier to get started quickly. ### Get Started with BodhiApp Follow these simple steps to try BodhiApp today: ``` brew tap BodhiSearch/apps brew install --cask bodhi bodhi run llama3:instruct ``` ### Learn More - **GitHub**: https://lnkd.in/gCPz8g6a - **X/Twitter**:  https://x.com/AmirNagri https://lnkd.in/gQGYFbPg - **YouTube**: Watch tutorial on Bodhi App deep diving on technical concepts - https://lnkd.in/gG_HvZqC We're eager to hear your feedback and suggestions. Join us on this journey to make powerful open source AI accessible to all! Thank you for your support! Do Like/Repost to have us reach maximum number of people 🙏🏽. Best, The Bodhi Team #LLM #OpenSource #huggingface #OpenAI #BodhiApp

Good work by Amiruddin Nagri. Bodhi app is a great way to start learning about LLMs.