Yogish Baliga

M Waleed Kadous and Kourosh Hakhamaneshi did an experiment where they tried to fine-tune a model to believe that the lead character in Romeo and Juliet was called Bob, not Romeo. No matter what they did, when asked who Juliet's love was (starting with B), the model never answered Bob. Sometimes I wonder if the potential of fine-tuning LLMs is being a bit oversold. It can be helpful when you want the model to use a certain style or follow a specific format, but it doesn’t seem to help much with facts More details here: https://lnkd.in/g6p2eU-n

8

Hey everyone, I am an investor and advisor to Katanemo.... They are reinventing the infrastructure primitives for GenAI and launched Arch - https://lnkd.in/gCrbbkEV - an intelligent prompt gateway. Give the project a look on Github, and star the project if it resonates

88

3 Comments

Short video: brief summaries of several key issues in ML history and modern ML.

16

1 Comment

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

8 Comments

ModernBERT is an open-source embedding model which represents a significant leap in encoder-only transformer architecture merging: > Long context (8k tokens) > Local-global attention for efficiency > Up-to-date code and text training data (2 trillion tokens) > FlashAttention & Unpadding to push performance on consumer and server GPUs > High retrieval and classification accuracy that sets new benchmarks across GLUE, BEIR, code tasks, and more. Learn more about this exciting new embedding model here: https://lnkd.in/en5ud9VE

2

A super clear, straight forward example of how to do cost effective RAG using LangChain for your document pipeline and Groq for compute resources. The code demonstrates how to speed up RAG and reduce costs using quantized embeddings (reducing 32 bit values to 8) Nice job Sahibpreet Singh! https://lnkd.in/gf5QMCKK

43

2 Comments

Is Protobuf the future default for everything? Timothy Chen & I chatted with Buf CTO Akshay Shah about the future of Protobuf, their new Kafka-compatible streaming layer, and everything he learned scaling Uber that led him to build Buf. This episode covered a pain point I've repeatedly experienced while working at Snyk & see at many other companies: managing cross-team migrations and building a productive internal API ecosystem. Here's a great example: You have two services (A and B), and A depends on B. How do you modify an API or Event in A without requiring B to do a complicated lock-step migration? These are the challenges in any at-scale organization where a project that should take 30-60 days turns into months as you try to navigate alignment and delivery. They're mundane papercuts that turn into hundreds of hours of lost productivity. I love what Buf are doing to help make this a problem of the past.

52

3 Comments

Anthropic's latest Claude AI model is taking human-AI interaction to a whole new level! 🖥️🤖 The new Claude 3.5 Sonnet can now use computers just like a human would - by looking at the screen, moving the cursor, clicking buttons, and typing text. This is a major leap forward in AI capabilities. Key features: Operates based on screenshots, emulating human computer use Can perform complex multi-step tasks across different applications Self-corrects and retries when encountering obstacles2 Accessible via API for developers to build with Why it's game-changing: This opens up incredible possibilities for AI assistance in data entry, form filling, software testing, and automating repetitive computer tasks13. Imagine having an AI assistant that can seamlessly work alongside you, handling tedious computer work while you focus on higher-level tasks. While still in beta, this technology demonstrates how AI is rapidly advancing toward more human-like interaction with digital interfaces. As it develops further, it could revolutionize how we work with computers and boost productivity across industries. The future of human-AI collaboration just got a lot more exciting! What do you think about this new capability? #AIInnovation #FutureOfWork Here is a video about it: https://lnkd.in/gh8aNYEu

6

"Over 25% of all new code at Google is now generated by AI", according to CEO Sundar Pichai in today's earnings call. But is this shift aimed at speeding up the development of complex software, or simply cutting engineering costs? One potential indicator will be if Google reduces headcount by 25% while boosting revenue per employee. Here are some details behind this productivity metric in Google's own research page . Majority of this '25% code' is in the area of code completion, not in code creation. "LLM-based inline code completion is the most popular application of AI applied to software development..." Watch the video to understand further on this metric.

16

3 Comments

𝗡𝗲𝘁𝗳𝗹𝗶𝘅’𝘀 𝗟𝗮𝘁𝗲𝘀𝘁 𝗘𝗮𝗿𝗻𝗶𝗻𝗴𝘀: 𝗔 𝗠𝗮𝘀𝘁𝗲𝗿𝗰𝗹𝗮𝘀𝘀 𝗶𝗻 𝗦𝗰𝗮𝗹𝗶𝗻𝗴 𝗖𝗼𝗻𝘁𝗲𝗻𝘁 𝗣𝗿𝗼𝗳𝗶𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 Netflix stock is soaring over 11% after its latest earnings report. The company exceeded expectations with EPS of $5.40 (vs. $5.12) and revenue of $9.83 billion (vs. $9.77 billion). Looking ahead, Netflix projects 15% revenue growth and a six-point improvement in operating margins. By 2025, they expect revenue to hit $43-44 billion—$4-5 billion more than 2024. Here are 𝟯 𝗸𝗲𝘆 𝗹𝗲𝘀𝘀𝗼𝗻𝘀 we can all learn from Netflix’s approach: 1. 𝗙𝗶𝘅𝗲𝗱 𝗖𝗼𝗻𝘁𝗲𝗻𝘁, 𝗘𝘅𝗽𝗮𝗻𝗱𝗶𝗻𝗴 𝗔𝘂𝗱𝗶𝗲𝗻𝗰𝗲 Netflix spreads its fixed content costs over a growing audience, with 14% subscriber growth and increasing engagement. Whether it's hit series or sports offerings like NFL games, their content model scales profitably with each new viewer. 2. 𝗚𝗶𝘃𝗶𝗻𝗴 𝗔𝘄𝗮𝘆 𝗩𝗮𝗹𝘂𝗲 𝘁𝗼 𝗚𝗿𝗼𝘄 Netflix’s strategic moves, like offering free sports events, and a lower priced ad tier prove that giving value leads to growth. As Co-CEO Ted Sarandos stated in the Q3 earnings call, “Engagement drives retention and acquisition, which in turn places more value on a Netflix subscription”​. This strategy keeps users engaged and drives new signups. 3. 𝗡𝗼 𝗡𝗲𝗲𝗱 𝘁𝗼 𝗕𝘂𝗻𝗱𝗹𝗲—𝗧𝗵𝗲𝘆 𝗔𝗿𝗲 𝘁𝗵𝗲 𝗕𝘂𝗻𝗱𝗹𝗲 Unlike other platforms, Netflix doesn’t rely on traditional bundles. With a wide range of content, including live sports, they are the ultimate bundle, growing through value-driven engagement without complex packaging. While other companies raise prices or experiment with aggressive strategies, Netflix shows that 𝗳𝗼𝗰𝘂𝘀𝗲𝗱 𝗲𝘅𝗲𝗰𝘂𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗶𝗻𝗴 𝘃𝗮𝗹𝘂𝗲 lead to sustainable growth. Full transcripts: https://lnkd.in/g3mEDWcH #Strategy #BusinessGrowth

5

Human in the loop evals 🧑‍💻👀 → custom, automated evaluation models 🤖📈? https://lnkd.in/gruPtC8d Human in the loop (HITL) is a really popular technique for teams just getting started with LLMs and their LLMOps functions. Given your biggest concern is your model acting ‘not like a human’, it’s easy to see how sticking a human in the loop, to review and edit any potential mistakes or weirdness, can solve that problem. But it’s equally easy to see how challenging this can be to scale, and how it can eat at a lot of the cost-savings promised by automations. So how should teams approach balancing the security and certainty from human-in-the-loop, with an eye towards scaling? Over the weekend I got the chance to experiment with a few approaches to turning HITL datasets into their own custom evaluation models. I also think about how I would balance my human-agent evaluations alongside my model-driven evaluations, to set me up for long-term success. https://lnkd.in/gruPtC8d

18

1 Comment

A few weeks ago, Hemant Mohapatra, partner at Lightspeed and I started discussing the evolution of agentic systems and realized there wasn’t a clear, canonical architecture to address the complexities of scaling these systems. So we set out to explore: 1/ What are the key infrastructure components required to enable agentic systems at scale? 2/ What role does UX design play in ensuring adoption and reliability? 3/ How can a multi-agent system handle real-world use cases like customer support? 4/ What are the opportunities for builders and challenges for enterprise adoption? How to think about differentiation? This article, "Endless Homers, Infinite Agents", is the first part of a series where we tackle these questions and share actionable insights. Feedback and comments welcome —hope you enjoy reading it as much as we enjoyed writing it! Link to the article: https://lnkd.in/g_q8XkcT

197

11 Comments

Insightful explanation of how do go channels work behind the scenes. Its all about yielding the ptocessor for other goroutines. https://lnkd.in/gbyQ4yH9

1

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

70

3 Comments

That's wild 🤯. Meta & Google is still on mono repo in 2024. I suppose it's more cost effective for both companies to re-engineer Mercurial & Perforce (respectively as "Sapling" & "Piper") & build custom development tooling, than to change their workflow for all of their engineers. In the case of Meta: a standard git merge operation would have taken 45 minutes. Legends has it, they consulted the Git team & was told to break up their repo. So instead, they went to the Mercurial team & got a better reception. Here's a perspective from one of their engineer. But wait... 🤔 the source code for Sapling & Piper + all of its IDEs & build tools are (probably) not store in the same repo? For sure with Meta Sappling, it's not (you can find it on GitHub). Same story with Android. These repos have totally separate concern domains 🤓 I get it. So maybe, it's "mono-repos" & :. truly, no actual mono repo 😅 With the collective wisdom of the Meta & Google engineering teams (combined!), who am I to judge 🫠

5

1 Comment

Watching Vinod Khosla talk with Rocky Yu about exponentials at the AGI House. Vinod said you could see the future of the Internet in the 1970s if you watched carefully. So, here are my guesses on what new fields AGI will revolutionize in what order: 1) Creative industries: image generation, music generation and movies 2) Robotics: foundational models will enable much better robotics to be deployed much more widely 3) Biotech: the creation of new drugs, better therapies etc…

33

1 Comment

This is a story about why Claude 3.5 Sonnet > ChatGPT 4-turbo > Gemini Pro 1.5 (all free consumer versions) In the IIT Joint Entrance Exam which everyone takes to enter the IIT system in India, there was this beautiful last problem in the Math section when I took it. "Prove the larger of the two numbers e^Pi or Pi^e." Many friends I spoke to after the test had been stumped by it. My solution was to power both sides by 1/(Pi*e) and reduce the problem to finding the extrema of the function x^(1/x). From there it was straightforward first and second differentiation find that the maximum was at e and get to the result that e^Pi is greater than Pi^e. So I asked the question to Claude 3.5 Sonnet, ChatGPT 4-turbo, Gemini Pro 1.5 with the exact same prompts > you are an expert at calculus. you know how to do mathematical proofs. here is a problem: prove which one is the larger of the two numbers. e to the power Pi or Pi to the power e Claude understood right away to use calculus and to construct a function that would reflect the two values. Here is the snippet of the response Claude [ This is an interesting problem comparing e^π and π^e. Let's approach this step-by-step using calculus techniques. To prove which is larger, we'll use the natural logarithm function, which is strictly increasing. This means that for any a and b, if ln(a) > ln(b), then a > b. Let's define f(x) = x^(1/x) for x > 0. ] And then it did the first and second derivatives to find the maxima is at e and completed the proof. You can try this yourself. Very impressive. ChatGPT tried to do it numerically first. I had to redirect it to ask to solve with a proof. It then decided to create a function and proceeded to solve it using calculus to find first and second derivatives and find that the maximum was at e. Good work. Gemini on the other hand confidently provided a proof with total errors. It was well written, but completely wrong in a particular step. It tried simple log functions of both sides. I pointed out the error and it proceeded to create another proof that was equally flawed. Then it did it a third time and finally gave up. I then had to prompt it, like I might a high school calculus student, to guide it to the approach of creating a function and finding its extrema. > try this approach: power both sides by 1/(Pi * e). then we are asking the question whether e^(1/e) is greater or lesser than Pi^(1/Pi). this then reduces to the problem of finding the extrema of the function x^(1/x). can you prove it from here? It then understood the how to solve it and then actually did the proper steps of finding the first and second derivatives like a good guided high schooler might. There is another complex test I have done with creating CPUs and assembly code where Claude also performed better. Net result: Claude 3.5 Sonnet is very impressive and better than the other two in my smoke tests. Well done!

70

13 Comments

At the heart of PreProd OpnSrc, there is a vision and a mission to make foundational learning at ZERO cost Please visit the post below and the page at PreProd Opnsrc for more information PreProd Corp PreProd Emulet PreProd Consult Krishnav Dave

10