Jerene Yang’s Post

Pretty interesting analysis of product development styles and the benefits of starting from scratch.

I may have underestimated the potential for OpenAI's SearchGPT. Although I was underwhelmed by it a few weeks ago, I’m now starting to appreciate just how disruptive and important it’s likely to become. It’s helpful to first think about web in phases. In the early days we had a Directory Age. Think Yahoo. Think Yellow Pages. In fact, it’s funny to look back to the ‘90’s when there were a handful of Internet Yellow Pages books that were flying off the shelves. We actually paid good money for a physical product that was obsolete before it ever left the printing presses. The Directory Age gave way to the Search Age, which has been dominated by Google for many years now. The Search Age won’t last forever though and I believe we’re quickly moving to what I’m calling the Answer Age, powered by AI services like SearchGPT. Do yourself a favor and run the exact same searches in both Google and SearchGPT. For example, after the Yankees completely collapsed in the World Series I typed in “new york yankees” on both services. With Google, you get the usual collection of paid and organic links; with SearchGPT, however, you get an Answer in narrative format (see image). SearchGPT provides an extremely concise story of the team. It quickly fast forwards to the Yankees Series collapse then describes the challenges they face as they move forward. External sources/citations are presented as buttons you can click-thru for a deeper dive, if necessary. Or, if you want to shift the focus of the Answer, just give SearchGPT a slightly different follow-up prompt. Good luck doing this with Google. We’ve grown accustomed to needing to find our own Answers on Google by clicking through one or more of the resulting links. SearchGPT shows how traditional search can evolve into a conversational service, especially when audio is fully built in. The Answer Age is here but I also believe the Agent Age is soon to follow. The Agent Age is when these Answer Age services add the ability to act on your behalf. Maybe I’m planning a trip to NY next summer, which the Answer Age service is aware of, so when I start asking about the Yankees it checks to see if I’d like to buy a couple of tickets to a game while I’m in Manhattan. If I say yes, the Agent Age service takes care of the transaction, gets me transportation info, makes a pre-game restaurant reservation, etc. Be sure to test drive SearchGPT and see what you think. I believe we’re in the twilight days of the Search Age and more powerful capabilities are right in front of us with both the Answer Age and the upcoming Agent Age. #ai #artificialintelligence #openAI #SearchGPT #search

Lately, I've been experimenting a lot with LLMs, and I noticed an interesting trend in the adoption of OpenAI API compatibility. Google is the latest to add OpenAI API compatibility for its Gemini LLMs: https://lnkd.in/efRfND5G ❓ Wondering why? Why not just stick with their own solution? Fast and friction-free onboarding is becoming the key to adoption in the fast-paced race of large language models (LLMs). OpenAI’s API has quickly become the de facto standard for integrating LLMs, thanks to its simplicity, early market presence, and broad ecosystem. Almost anyone who’s experimented with LLMs has likely started with OpenAI’s API, which provides a seamless experience from setup to getting the first response back, setting a high bar for others in the industry. For any company looking to compete in the LLM space, making the switch to their integration as easy as possible is critical. This is why we’re seeing so many providers embracing OpenAI API compatibility: xAI, Perplexity, and others already offer compatibility, and Hugging Face can serve any model with an OpenAI-compatible endpoint. ❓ Why is this important? This compatibility lets developers use new models simply by changing the endpoint in their OpenAI client—no additional code changes are required. Even better, it means developers can leverage existing tools, plugins, and the broader OpenAI ecosystem they’re already comfortable with. This friction-free integration removes barriers to trying out new solutions, making it easier than ever for customers to explore new options without leaving behind the tools they rely on. 💡 How you can translate this to your business? A smooth customer experience and easy onboarding can be a game-changer for any business. Here are a few ideas that you can take from this observation: 1️⃣ Lower barriers to entry Make it as easy as possible for customers to try your product. If they can integrate or onboard quickly, they’re more likely to stick around. 2️⃣ Adopt familiar standards: Wherever possible, align your product with industry standards that your customers are already familiar with. This reduces the learning curve and builds instant trust. 3️⃣ Focus on flexibility and compatibility: Design your product to work seamlessly with other tools your customers might already be using. Flexibility empowers users to experiment with your solution without fear of reworking everything. A great user experience isn’t just a feature—it’s a strategy. When customers can test your product with minimal effort, you create a powerful first impression and open the door for broader adoption. #developerexperience #dx #ux #openai #gemini #llm #observation

Stack Overflow signed a deal with OpenAI, announced Monday. Two things to think about: 1. Contributors aren't getting compensated for the value of their contributions. I'm not making a moral or legal point, just a practical one. It kinda sucks creating other people's value for free when they flaunt it. At least with open source generally, you're sorta insulated from the monetization by anonymity of reuse. At the same time, any community that doesn't ban you is always going to be a cool hang if you're a halfway interesting contributor or person. 2. People worry about LLM generated content feeding the LLMs and making the LLMs regress. Don't worry about that. It won't happen. There is a different problem. LLM makers won't let their LLMs regress on metrics that matter (no matter how bullshitty the metrics). They will have to spend more time and money filtering their training data sets as they become less human and more generated. That actually sounds like an activity OpenAI can saddle SO with. "We'll pay you for ongoing use of your data so long as your data improves." The fun part is that it will take time and GPU money in "pre-training" to prove that iterations are improving. In short, LLMs won't regress. They will get more expensive. Qualified training data will cost more. This is a critical business insight in this space, and is probably very different than whatever narrative you've bought into. If you need this kind of insight across your business, I am available to help you, from consultant and projects to full time. Message me. H/T Axel C., who is really good at LinkedIn and will probably end up monetized without compensation if he isn't already. #WrittenByMe

The race to near-free LLMs continues, and OpenAI's headstart may not be as maintainable without a different moat/strategy. Two emerging and important LLM themes: 1.) Convergence 2.) Open Source 1.) Convergence: The hard line between LLMs is blurring. As an example, there are a variety of web front-ends arriving weekly that allow users to ask the same question to one of many models, without having to switch chat windows. It's like one search engine asking many other search engines for help. My favorite implementation is Sider.AI, a Chrome add-in that provides its paid users with a single chat window that can access many LLMs from OpenAI, Claude, and Google. The add-in can also blend answers together from multiple LLMs (a type of emsemble technique), and can arrive at a consensus answer. It's like asking a focus group of LLMs to answer the same question and collaborate on the final answer, increasing the likelihood that the answer is correct. Zapier supports similar model-agnostic functionality, where the LLM itself is simply one part of a longer data pipeline. As an LLM app developer, it's now possible to quickly switch between models (via different APIs) without changing your app overall. Why this matters: I believe it will be difficult for OpenAI to defend their current market share if users and developers can simply switch to another model anytime. One mouse click from a user effectively replaces ChatGPT4 with a competitor. There's no analogy for that in FAANG, where revenue streams are very sticky due to the large network effect they have each created for themselves. ---- 2.) Open Source During inference (when an LLM takes your query and calculates a response), the open-source LLM "Mixtral 7b" uses just 10% of the parameters required by ChatGPT 4. This translates to a lot less compute (and expense). Through a clever mixture-of-experts model, and both pre-processing and ensembling techniques, Mixtral is quickly becoming an open-source LLM that beats its competition at a fraction of the price. As medium-to-large-sized businesses grow confident that they can maintain their own internal AI systems without requiring a per-query paid API like OpenAI, I expect to see growth in open-source models. Open source LLMs should be generally well-supported by their users in ways that commercial LLMs can't easily compete with.

For those using the #GoogleGemini Google have announced Gemini's integration with the #OpenAI Library . This integration allows developers to: 👉 Access Gemini models through OpenAI's familiar interface 👉 Use both Chat Completions API and Embeddings API 👉 Implement Gemini using the OpenAI library through Python, JavaScript/TypeScript, or REST API calls OpenAI's API is becoming the de facto standard for LLM interactions. This is similar to how Amazon S3's API became the de facto standard for object storage, with many services (including Google's own Cloud Storage) offering S3-compatible APIs. We're seeing a similar pattern emerge where: 👉 OpenAI's Chat Completions API structure has become a reference implementation 👉 Multiple LLM providers are adopting OpenAI-compatible endpoints: ➡️ Gemini (Google) ➡️ Ollama ➡️ Anthropic (through Claude compatibility) ➡️ Many open-source LLM platforms This standardization brings several benefits: 👉 Makes it easier to switch between different LLM providers 👉 Enables easier multi-model deployment strategies 👉 Simplifies the creation of abstraction layers and tools Google's decision to implement OpenAI compatibility for Gemini is particularly significant given their market position - it suggests they recognize the practical value of adopting this. #ai #enterpriseai https://lnkd.in/euZW4xsZ

Last week, I learned about fine-tuning while updating one of the tutorials. It's amazing that using tools from Cloudflare, one can build AI apps quickly! If you want to learn about fine-tuning, here's the tutorial: https://lnkd.in/dWe337cd

I have seen many thoughts on OpenAI acquisition of Rockset, but I thought it would be great to ask OpenAI(via ChatGPT 😉 ) to see what their product thinks about this move and its implications for real-time/streaming data space. Well here is how ChatGPT thinks about it: OpenAI's acquisition of Rockset has significant implications for the real-time and streaming data space. Rockset, known for its expertise in real-time analytics and streaming data processing, can enhance OpenAI’s capabilities in several strategic areas. Here’s a breakdown of the key implications: 1. Enhanced Data Integration for AI Models Implication: OpenAI can integrate Rockset's technology to streamline the ingestion, processing, and real-time querying of diverse data streams. This would enable more effective training and deployment of AI models with up-to-date data. 2. Real-Time Decision-Making Capabilities Implication: Rockset’s capabilities in real-time analytics could empower OpenAI’s solutions to make decisions based on current data, which is crucial for applications requiring immediate responses. 3. Expanding AI into Streaming Data Applications Implication: OpenAI could expand its AI applications into areas that rely heavily on streaming data, such as IoT, social media monitoring, and financial market analysis. 4. Synergistic Technology Development Implication: The acquisition allows for synergy between Rockset’s real-time analytics platform and OpenAI’s advanced machine learning technologies, potentially leading to innovative new products and services. 5. Competitive Advantage in Data-Driven AI Implication: By integrating Rockset's capabilities, OpenAI could position itself ahead of competitors in the AI space, particularly in the context of data-driven decision-making. 6. Improved AI Model Monitoring and Maintenance Implication: Real-time analytics can facilitate better monitoring of AI model performance and faster identification of issues, leading to more effective maintenance and continuous improvement. 7. Increased Focus on Data Privacy and Security Implication: Handling real-time data streams necessitates a robust approach to data privacy and security, prompting OpenAI to bolster its measures in these areas. Conclusion The acquisition of Rockset by OpenAI holds transformative potential for both entities. It enables OpenAI to leverage Rockset's advanced real-time analytics capabilities to enhance its AI offerings, drive innovation, and maintain a competitive edge in the rapidly evolving data and AI landscape. The integration promises to foster new applications and services that meet the growing demand for real-time data processing and analytics across various industries. We're seeing a greater role for real-time and streaming data in AI data infrastructure and DeltaStream is well positioned to bring the power of real-time data to your AI applications! Ping me if you wanna discuss more! https://lnkd.in/ge4UWrrT

If your business leans on a model you don't own, nothing will protect you from OpenAI to change or stop hosting it. Instead, you should consider investing in building your own solution using open-source models. Here are 4 reasons I discovered after almost 2 years working with LLMs for various clients: ⛵ Reliability When you pay OpenAI or Anthropic for a model, you pay for the model they decide to serve. But at any time, they can change the model version, or even stop hosting it, without any warning. You probably already witnessed performances downgrade from the release of a new model. It also happened the API was down, shutting down your AI service for a couple of hours. And you can do nothing about it... Owning your open-source model ensures it will never change unless you decide to improve it with new training. ⛵ Cost reduction At low-scale, using providers' LLM seems cost-efficient: you pay for what you use. But the cost can go up quite rapidly. I recently develop a solution to process 100,000 texts (out of ~3.5M) to fix spelling errors. The process runs on a regular basis. If I used OpenAI GPT-4o, it would cost me ~$340 every time. And it's not even the best model available. After fine-tuning, and deploying Mistral-7B from Mistral AI to process texts in batch, it costed me nearly $2 per process, while being blazingly fast. ⛵ Better performances What if GPT-4o-mini, which is the most cost-efficient model, doesn't actually solve the problem? How do you improve it from there? You improve the prompt by offering a "tips" to the model? When you prompt, you're still limited by the model capacities, especially if your use case is specific to your business. But with open-source models, you have access to fine-tuning, which transform the core of the model to fit your task, ensuring the best performances. ⛵ Privacy If you call an external LLM, whatever you prompt will be used to train the next model version. Using various prompt techniques, anyone can access private information about your business or users later on. With open-source LLMs, nothing leaves your infrastructure. You're safe. ... Don't make me wrong. Using OpenAI or Anthropic services is a shortcut to access "AI" without any prior investment in data or infrastructure. But it will inevitably fail short in the mid-term. If your use-case doesn't provide a lot of value for your business, don't bother. Stick to Gpt-4o. But if it does, you should seriously consider building your own solution. And if you do, you can reach out to me 📞

The Heat is On..!! Competition between tech giants is moving to the next gear.. According to Reuters, OpenAI is planning to announce a Google search competitor. Microsoft's Bing may have a market share of just 3.2%, but it was eager to take on Google. Despite Bing's amazing innovation, Google still holds over 90% of the search market share. Perhaps using Google is like " old habits die hard 😁 " Looking forward to see how OpenAI will fare against the search giant. Stay tuned for updates! #TechNews #Google #OpenAI