Dorothy(Dongmei) Ren

We are excited to back Morphing Machines. As Dr. Ranjani, CTO and co-founder says: “For the first time in India and globally, we have a processor that is truly runtime reconfigurable, catering to emerging workloads.” While Nvidia’s growth and high valuation are driven by robust demand for “infra-compute” in high-performance GPUs, spurred by Gen AI, there is a distinct need for innovative solutions tailored to a new wave of reactive applications. Enter Morphing Machines. It addresses this demand with its unique product designed for next-gen telecom, autonomous vehicles, collaborative robots, avionics, and online data analytics. This product is specifically engineered to handle heterogeneous workloads and diverse tasks with varying performance demands simultaneously, ensuring a high level of scalability. As we move from an institutionally-backed research team at IISC to productization, Golden Sparrow is collaborating with Speciale Invest and others to support Deepak Shapeti, who joined as CEO three years ago after earning his Masters at Stanford. He brings not only technical expertise but also the business acumen necessary to guide the product to commercial success. Venkat Raju Afreed Faizan Anand Rao Ritik Singh Michael Marmor

68

5 Comments

Check out our latest workshop on genAI security/privacy. Standards and regulations typically solidify very late in a tech cycle - like decades late; the hard work to develop best practices happens in collaborations like these!

27

1 Comment

Last month I had the privilege of participating in a panel discussion on Generative AI at Enterprise, alongside Sayan Maity, PhD and Arpita V., hosted by Roger M. & Data Science Salon at Google HQ in San Francisco.  We tackled critical topics, such as why rapid prototyping is commonplace across companies, yet few successfully transition AI projects into production. Arpita pointed out that for recommendation and ranking problems, inference speed is crucial. Companies often utilize LLMs for offline evaluations while relying on high-performing, traditional models for online consumer interactions to mitigate the risk of unpredictable outputs. It's essential to recognize that many problems can be effectively addressed using smaller models like ROBERTA or BERT, without overhauling existing machine learning pipelines with #LLMs. Our discussion also covered #copilots , the #agentic framework, model size, #inference time, #cost , and #scalability . Ultimately, the pivotal question remains: Is the ROI substantial enough? Does deploying generative AI into production pay off? Furthermore, can we quantify the productivity gains for individuals using copilots? For insights and answers to these discussions, I invite you to listen to our conversation. #engineeringexcellence, #AI, #evaluations , #bestpractices, #GenAI, #DSSSF https://lnkd.in/gefHHDhb

7

1 Comment

📝 Exciting news! My latest blog post on Medium is now live! In this multi-series blog, I explore anomaly detection in any system. I share my experiences in designing and implementing an end-to-end solution for anomaly detection patterns in user access patterns. This series will cover a variety of topics, including understanding application events, optimizing data in Snowflake, developing machine learning models, and integrating with external applications. In the first blog post, I dive into everything about events and how they should be handled in a distributed system. Check out the full article here to learn more: https://lnkd.in/g8mjARdd I would love to hear your thoughts and insights in the comments section! Let's start a conversation. #events #eventing #outboxpattern #anomalydetection #machinelearning #dataoptimization #AWS #Snowflake

63

5 Comments

#Rise of Micro LLMs: A Roadmap to Business Success with Generative AI In the current AI landscape, Large Language Models (LLMs) like GPT-3, BERT, and T5 have garnered significant attention for their impressive performance on various NLP tasks, including text generation, question answering, summarization, and translation. However, the training and deployment of these massive, general-purpose models come at a substantial cost, often reaching up to $10 million. These expenses cover not only computational resources but also specialized hardware such as GPUs and the necessary software infrastructure. Micro LLMs are developed by refining larger models to address industry-specific, domain-specific, and customer-specific scenarios, offering several key advantages: #Personalization: Enhances user engagement and satisfaction by providing more relevant interactions, leading to higher retention rates. #Efficiency: Requires less computational power and memory, making them suitable for deployment on smaller devices or in edge computing scenarios. #Accuracy: Reduces hallucinations and bias by focusing on enterprise-relevant data. #Cost-effectiveness: Eliminates the need for expensive data scientists or costly hardware and systems. While general-purpose or "macro" LLMs are beneficial in scenarios requiring a broad understanding, their wider scope may not always align with the specific nuances of enterprise-focused tasks. Additionally, their flexibility and adaptability can sometimes come at the expense of precision compared to Micro LLMs. #Benefits and #Limitations Micro LLMs offer significant benefits, such as being trainable on modest datasets and having simplified architectures that enhance interpretability. Their compact size makes them ideal for deployment on mobile devices, and their ability to process data locally is particularly valuable for IoT edge devices and organizations with stringent privacy and security needs. However, their limitations include more restricted knowledge bases due to smaller training datasets and a less comprehensive understanding of language and context, which can lead to less accurate and nuanced responses compared to larger models. #The Future of #Micro LLMs The future of small language models looks promising with increased accessibility and cost efficiency, making them attractive for businesses and individuals. Their lower computational requirements enable real-time processing on edge devices, enhancing privacy and energy efficiency. These models are easier to fine-tune for specific tasks, ensuring effective and specialized solutions. Additionally, their faster response times and seamless integration with existing systems improve user experience. As advancements continue, small language models will become more capable, promoting the democratization of AI and fostering innovation across various fields. #GenerativeAI #Micro-LLMs

20

𝗙𝗿𝗶𝗱𝗮𝘆, 𝟮𝟰 𝗠𝗮𝗿𝗰𝗵 𝟮𝟬𝟮𝟯: Was crying on call with my aunt, saying that I've been working on maltiplai since September 2022 but have no idea where this is going. 𝗦𝗮𝘁𝘂𝗿𝗱𝗮𝘆, 𝟮𝟱 𝗠𝗮𝗿𝗰𝗵 𝟮𝟬𝟮𝟯, 𝟭𝟬:𝟬𝟬 𝗔𝗠: Back to working on maltiplai when a voice inside me told me look for courses at MIT(still don't know where that voice came from). I remember almost paying for an online course in Neural Networks, but I stopped and ended up giving an interview for an onsite Venture Advancement Program ​MIT Bootcamps 😍​ 𝗠𝗼𝗻𝗱𝗮𝘆, 𝟮𝟳 𝗠𝗮𝗿𝗰𝗵 𝟮𝟬𝟮𝟯: I got "the email"! It stated "It will be a life-changing experience"(I really needed one!) 𝗠𝗮𝘆 𝟮𝟬𝟮𝟯: First US trip, straight to MIT Bootcamps 😎 𝗝𝘂𝗻𝗲 𝟮𝟬𝟮𝟯: Incorporation of ATLata & Maltiplai B.V. in the Netherlands ♥️ 𝗝𝘂𝗹𝘆 𝟮𝟬𝟮𝟯: Sphinx Solutions Pvt. Ltd. onboarded for software development 𝗝𝗮𝗻𝘂𝗮𝗿𝘆 𝟮𝟬𝟮𝟰: maltiplai went live on iOS & Android in 60 countries! 𝗙𝗲𝗯𝗿𝘂𝗮𝗿𝘆-𝗔𝗽𝗿𝗶𝗹 𝟮𝟬𝟮𝟰: maltiplai collaborated with Amity University, Miranda House, Shri Ram College of Commerce (SRCC), Delhi, Hindu College, University of Delhi, Aryabhatta College, University Of Delhi, Sri Venkateswara College, Delhi University, Shaheed Sukhdev College Of Business Studies, NSUT Delhi, Delhi Technological University (DTU), Delhi College Of Arts and Commerce and University School of Automation and Robotics (USAR) to spread the word that we are here! 𝗠𝗮𝘆 𝟮𝟬𝟮𝟰: Got a chance to share advice with the next group of innovators — just a year ago, I was sitting in the same room ♾️ I haven't done this alone; I'm 𝗯𝗹𝗲𝘀𝘀𝗲𝗱 𝘄𝗶𝘁𝗵 𝗽𝗲𝗼𝗽𝗹𝗲 𝗼𝗳 𝗵𝗶𝗴𝗵 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲. Despite having full-time jobs, we manage(nights/mornings, all merged) to align between 𝘁𝗵𝗲 𝗡𝗲𝘁𝗵𝗲𝗿𝗹𝗮𝗻𝗱𝘀 🇳🇱, 𝗜𝘀𝗿𝗮𝗲𝗹 🇮🇱, 𝗮𝗻𝗱 𝗜𝗻𝗱𝗶𝗮 🇮🇳—𝗯𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗮𝗻 𝗶𝗻𝘁𝗲𝗿𝗻𝗮𝘁𝗶𝗼𝗻𝗮𝗹 𝗰𝗼𝗺𝗽𝗮𝗻𝘆 𝗳𝗿𝗼𝗺 𝗱𝗮𝘆 𝗼𝗻𝗲. 𝗕𝗼𝗼𝘁𝘀𝘁𝗿𝗮𝗽𝗽𝗲𝗱, after 5 months and 14,000+ downloads, we're preparing to deploy an improved product and even launch our revenue channels. We might not get it right the first time, but we are here for the long run, 𝗳𝗼𝗿 𝘄𝗲 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗶𝗻𝗳𝗶𝗻𝗶𝘁𝗲 𝗽𝗹𝗮𝘆𝗲𝗿𝘀 😎 #CreatorEconomy #TheFutureIsHere

58

14 Comments

Polaris 2.0 from Hippocratic AI is here. Safer than ever. 3T+ parameter constellation architecture with voice latency equal to Polaris 1.0 at 1T+ parameters. Read a summary of the changes from our chief scientist Subhabrata (Subho) Mukherjee's blog: https://lnkd.in/gXweaWe5

278

1 Comment

Vector similarity search is how machines compare things like images or text, finding similar items even in massive datasets. As datasets grow and use cases become even more complex, SOAR (by Google Research) ensures ScaNN (open-source vector search library) can deliver the scalability and performance ML applications demand 🎉 . How SOAR Transforms Vector Search    •   Redundancy for Speed: SOAR leverages carefully crafted redundancy to make approximate nearest neighbor searches even faster within ScaNN.    •   Efficient and Scalable: This boost in performance comes at minimal cost to index size and other important metrics. Check out the entire paper here 👉 https://lnkd.in/g2SPjVy3 #ML

1

In this clip, Chip Huyen breaks down her new blog post on building generative AI systems. We explore: 🏗️ Common GenAI platform components ❌ AI failure types: information vs. behavior-based 🔍 RAG and context enhancement strategies 🛠️ Output improvement techniques 🔄 Fine-tuning complexities 📊 Output format considerations Full chat link in 🧵! #GenerativeAI #MachineLearning

17

1 Comment

LLM-as-a-Judge: The Future of Evaluating AI As AI and language models grow more powerful, traditional methods of evaluating their performance no longer cut it. Enter LLM-as-a-Judge: an innovative approach where large language models are used to assess and score the outputs of other AI systems. But why is this approach necessary, and how does it work? In this blog series, we'll explore the concept of LLM-as-a-Judge, why it's becoming essential in modern AI evaluation, and how it’s reshaping the way we test and improve language models. We’ll dive into real-world examples, weigh the pros and cons, and provide you with a glimpse of how this technology is driving the next wave of AI advancements. Stay tuned as we break down this fascinating development and its impact on AI’s future! #AI #GenAI #LLM #LLM-as-a-judge #Evaluation #LLM-Evaluation Click for the article 👉 https://lnkd.in/gphF9M6x Swipe for the highlights 👇

4

4 Comments

Ai : Hunanity's greatest creation for Good and Bad. To secure ourself #zerotrustai In the journey, MS brings up VASA-1 will AI model which essentially spins a selfie image and turn it into a talking clip of you. All you have to do is upload a photo along with a voice note and let the AI model do the talking for you.

2

https://by-hand.ai/x/AE Learning is reaching to its new heights.

1

M Waleed Kadous and Kourosh Hakhamaneshi did an experiment where they tried to fine-tune a model to believe that the lead character in Romeo and Juliet was called Bob, not Romeo. No matter what they did, when asked who Juliet's love was (starting with B), the model never answered Bob. Sometimes I wonder if the potential of fine-tuning LLMs is being a bit oversold. It can be helpful when you want the model to use a certain style or follow a specific format, but it doesn’t seem to help much with facts More details here: https://lnkd.in/g6p2eU-n

8

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

This post from Rowan Cheung has a great summary of agentic approaches by the various LLMs. @rowancheung is a great follow on X. https://lnkd.in/gW_kZ9xx

1

What’s the Real Context Size of Your Long-Context LLMs? Picking up from my last post (https://lnkd.in/gYtgs_DT), where I talked about the pros and cons of long-context LLMs in Retrieval-Augmented Generation. This NVIDIA research takes a closer look at how these models perform at their limits, with insightful findings that reveal some non-obvious challenges. The authors tested 17 language models, including well-known ones like Gemini 1.5, GPT-4, and Llama 3.1. Turns out, despite many models claim to handle huge contexts, very few actually keep up their performance as context length grows. This might come as a surprise if you're familiar with traditional benchmarks like "needle-in-a-haystack" and passkey retrieval, where most models achieve near-perfect accuracy. While those tests are useful, the new RULER benchmark pushes LLMs with more complex tasks, such as multi-hop tracing and aggregating information across long contexts, ultimately revealing their true limitations and areas for improvement. As language models grow in size and context capacity, it's easy to get excited about their new capabilities and forget the trade-offs. Just because a model can handle massive contexts doesn’t mean more is always better — often, keeping it small and simple still wins the day. I'll drop the link to the paper in the comments. #MachineLearning #ArtificialIntelligence #AI #LLM #GenAI #NeuralNetworks

57

3 Comments

📊 The Power of High-Quality, Diverse, Real-World Data Fine-Tuning LLMs 🌍 When it comes to training Large Language Models (LLMs), the mantra isn’t just “more data”—it’s better data. While having more data can be beneficial, it’s the quality, diversity, and real-world relevance of the data that truly drive impactful results. Here’s why: 1️⃣ Higher Quality: clean, well-curated data improves model accuracy and reduces noise, making the model more reliable and stable. Poor-quality data often leads to inconsistent responses and increased hallucinations. Unbiased labeling consistency is key. 2️⃣ Diversity: it ensures the model can generalize across different contexts and user scenarios. This makes the model more robust, safe and inclusive, which is especially important in NLP for avoiding bias and ensuring representation. 3️⃣ Real-World Datasets: training on real datasets helps models perform better in real-world applications, as it learns from actual patterns and usage scenarios, making the model more practical and effective. For LLMs, there’s a general playbook to prepare data effectively: (1) Collect Instruction-Response Pairs: gather examples of how tasks should be answered, covering various domains and complexity levels. (2) Concatenate Pairs: organize the data into structured sequences, allowing the model to learn from end-to-end interactions. (3) Tokenize: process the data into tokens, using padding and/ or truncation as needed to maintain uniform length. I particularly love 🤗 transformers library's AutoTokenizer to identify which tokenizer to use on every model. (4) Split into Train/Dev/Test Sets: partition your data into training, validation, and testing sets to ensure proper model evaluation and tuning. Ultimately, data quality prevails over quantity, specially as we live in the LLMs era when models have been pre-trained with almost all the public data available in the 🌎 . A well-prepared dataset tailored to your application will make all the difference in the performance and reliability of your model. #AI #MachineLearning #DataQuality #LLM #NLP #ModelTraining #DataPreparation

5