Andrew Ng

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

Llama3 paper on arxiv [2407.21783] shows how the systems can fail at high rate, and network ranks #3 as the failure reason (after GPU and software) at 35 interruptions in 54 days. With compute requirements for foundational models increasing, we will be seeing 100K and larger clusters. For reference Llama3 used 16K GPUs. Network complexity could grow with 100K GPUs and if no new designs evolve, we could be looking at 8x increase (35 to 280) in service interruptions attributed to networking. If we don't improve our training and network design practices, the network failure rate will overtake software to be #2 and it is not a good place to be. I see designs evolving increasing reliability of the network. And it is also important how fast you can recover from interruptions (MTTR), and can you recover fast enough that an event in the network does not cause interruption at the application? It is clear that there will be failures at this scale and to get any goodput, you have to have components that are reliable and the design practices and protocols that recover well. I also hope that the AI ninjas at the leading labs are building viable alternatives to SGD that contain the failure domain.

59

2 Comments

#Llama-3: Initial Reactions Post-Launch 20240418 20:05: Scrolled through my #LinkedIn and what do I see? Yann Lecun's post about the Llama-3 launch!! The thing had already racked up 300 reactions in just 7 minutes. Talk about viral. 20240418 20:50: Checked again 45 minutes later. Boom—3000 likes! ➡️ Might just be the fastest spreading #AI news I've ever witnessed on this platform! I immediately dive into Meta's technical blog to see what this new AI model's got under the hood. 20240418 22:30: Quick check at how Meta’s stock was doing on Nasdaq after this important announcement. And the result? Zilch. Nada. Zero movement—investors clearly snoozing through the news. Seriously? Llama is supposed to be Meta's big thing!! And let’s not forget, training #Llama-3 isn’t exactly pocket change: 👯 398 ML Engineers proudly put their names on the announcement. 💾 Meta’s Research SuperCluster boasts a 24,000 GPU setup. 📚 Even the tiny Llama-3, 8B model, gobbled up 1.3 million GPU hours. Did some back-to-the envelope calculation: A single A100 instance is about $33/hour, each packing 8 GPUs. ➡️ Just the initial training phase for the 8B model using these 24,000 GPUs for 2.3 days set them back a cool $5.3M bill. And that's just for the GPUs, never mind the large number of engineers and the cost of running a data center. ---- Let's zoom out and check again at the stock of Meta of the last 18 months. Meta’s stock miraculously went from ~$100 at the end of December 2022 to $500 now!! All this has been attributed to excellent financial results and ... to Mark Zuckerberg’s shift from "#Metaverse" to "open-source #AI." (=Llama). Both mega projects have a clear vision but, what about the business model? This leaves us with this question raised from the Financial Times: "How long can Wall Street’s artificial intelligence-fuelled rally continue without clear evidence that generative AI is giving a meaningful lift to business?"

19

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

Yann LeCun: The current state of AI is limited, but useful, and has likely hit a ceiling. We need to continue to innovate to break through. WSJ: AI is no smarter than a cat Ignore the headline and listen to the substance. The problem is that, despite being limited, the current state can be used to do things faster, better, and differently than before. We don't need to wait for AGI to leverage the power of the tool. https://lnkd.in/g64T-ykZ

13

1 Comment

A gold mine of AI ideas. This podcast between Andrej Karpathy and Sarah Guo / Elad Gil is one of the most insightful and deep discussions of AI and Robotics I’ve ever listened to. Every part of the interview is extremely interesting and full of lessons! Just some of the highlights: - Lessons from cars are extremely transferrable to humanoids. - Tesla is ahead of Waymo and is facing a more surmountable challenge. - Genius of transformer architecture - Roadmap to AGI - The future of education in the AI-powered world https://lnkd.in/g-488TzB

15

1 Comment

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

The Tesla Transport Protocol (TTP) appears to be an essential part of Tesla's AI stack. In an article on X titled, "TTP: The hidden powerhouse behind AI and self-driving" (linked below), patent researcher, Seti Park lays out several interesting points: - TTP is a streamlined, hardware only approach to network protocol operation as opposed to traditional network protocols like TCP/IP which rely mostly on software. - TTP allows for near-instantaneous data transfer between AI components. - TTP scales easily as it is adaptable to different AI hardware configurations. Ultimately, Park writes, TTP: 1. Reduces communication latency which could significantly speed up AI model training. 2. Improves data transfer time between sensors, processors, and actuators means improved reaction times for self-driving systems. 3. Enables more efficient distribution of AI workloads across different chips. Park argues that Tesla's integrated approach - combining custom hardware with proprietary communication protocols like TTP - creates a technological moat that's hard to cross. TTP optimizes how its AI components talk to each other, Tesla is laying the groundwork for more advanced, more responsive, and safer autonomous systems. See a link to the patent here: https://lnkd.in/gSzpig8Q? #FrontierTechnology #SelfDriving #FutureFrontier

5

Meta AI Releases LayerSkip: A Novel AI Approach to Accelerate Inference in Large Language Models (LLMs) Researchers from FAIR at Meta, GenAI at Meta, Reality Labs, and several universities have released LayerSkip, an innovative end-to-end solution that combines a unique training recipe with self-speculative decoding. The proposed approach involves training with a layer dropout mechanism that applies low dropout rates to earlier layers and higher dropout rates to later ones while incorporating an early exit loss that enables transformer layers to share a common exit point. This helps the model become more robust to early exits during inference without the need for auxiliary layers. LayerSkip consists of three main components: 1️⃣ Training Recipe: Uses layer dropout and early exit loss to create different sub-models within the main model. 2️⃣ Inference Strategy: Allows for early exits at earlier layers to reduce computational costs without compromising accuracy. 3️⃣ Self-Speculative Decoding: Early predictions are validated and corrected using the remaining layers of the model. Read the full article here: https://lnkd.in/gxFF_jeT Paper: https://lnkd.in/gDPsy3qu Models: https://lnkd.in/gENuRF6h Code: https://lnkd.in/gfSM-YXy Listen to the podcast on LayerSkip created with the help of NotebookLM and, of course, with the help of our team, who generated the prompts and entered the right information: https://lnkd.in/gh5S2c4Q Meta AI at Meta Mostafa Elhoushi Akshat Shrivastava Diana L. Basil Hosmer Bram Wasti Liangzhen Lai Anas Mahmoud Bilge A.

108

3 Comments

Ever thought about becoming a robot? 🤖 No, I'm not talking sci-fi – this is happening now. MIT and UCSD just unveiled "Open-TeleVision," a mind-blowing system that lets you control a robot like it's your own body. But here's the kicker: This tech isn't just cool, it could completely change how we think about AI and automation in business. Why? Because it taps into our most powerful asset: the human brain. What if return to office meant you still work from home? Check out my latest article to learn: - Why human-robot teams might outperform pure AI - The surprising advantages for quick ROI - How this could solve your automation challenges faster than you think Read the full story and let me know your thoughts! #Robotics #AI #FutureOfWork #Innovation https://lnkd.in/eQHqb3Tv

4

2 Comments

The AI for Good Summit is behind us, marking a significant realization moment for civil society, governments, and enterprises. Exponential investments in Generative AI and the industry's growth have hit escape velocity. Gone are the days of ‘a toy for play.’ I call this <the race of three (AI) horses>. The Two Core Capitalist Forces at Play: 1. Race to AGI 2. Productivity Unlock These two lay the foundation for a third: AI for SDGs acceleration. 1. Race to AGI OpenAI, Microsoft, Google, Anthropic, and other AI labs are accelerating investments and deployment of LLMs (and satellite tech) plus infrastructure to support the road to AGI. These massive projects, would have qualified for government-funded infrastructure - are now driven by private investments and are gaining unstoppable momentum. A race to oligopoly has begun, and acceleration is to be expected. While financed through VC, these projects will eventually converge with national security needs, to lock-in an intelligence advantage so expect more development to occur. For startup founders, in Sam Altman’s words: if you're building your startup based on an old version of GPT, you’ll likely be steamrolled. Many founders creating their own AI solutions today will find themselves at odds with the growth of large AI labs. 2. Productivity Unlock As LLMs grow, enterprises seek to understand and implement the technology to gain an unfair advantage and productivity boost. If I were a CEO of any Fortune 500 company today, I’d be reviewing one by one the countless ways operations can be improved, new product experiences unlocked, and new markets entered. This phase is nascent; it has been better to observe than act, but waiting is no longer an option. Navigating this <messy middle> to AGI will require intense work and adaptability. Plenty of opportunities exist to tackle this challenge from the enterprise side, including unlocking real use cases and implementing governance/compliance measures that are enterprise-grade and aligned with regulations and standards. The productivity unlock also plays into science and research 🔬 opening up doors there were once opening very slowly. 3. AI for SDGs There is a third vector to explore: how we, as a world, choose to implement the tools and power AI brings to address pressing global challenges—from climate change to health to food security. The first two horses have strong economic and systemic pushes. The third is a pull. It builds on the foundation of the first two and has the potential to yield great societal benefits if enough people pay attention and commit (with government also playing an important role). Safety of AI systems itself is a theme here. Lots to say on each one of the 3 horses 🐎 so I’ll delve into them individually. Please repost it ♻️ and follow me, Sal Matteis, for more similar posts. #AI #startups #AIForGood #GenAI #LLMs #AGI #venturecapital #enterprise

12

1 Comment

A barrier to faster progress in generative AI is evaluations (evals), particularly of custom AI applications that generate free-form text. Let’s say you have a multi-agent research system that includes a researcher agent and a writer agent. Would adding a fact-checking agent improve the results? If we can’t efficiently evaluate the impact of such changes, it’s hard to know which changes to keep. For evaluating general-purpose foundation models such as large language models (LLMs) — which are trained to respond to a large variety of prompts — we have standardized tests like MMLU (multiple-choice questions) and HumanEval (testing code generation); the LMSYS Chatbot arena, which pits two LLMs’ responses against each other and asks a human to judge which response is superior; and large-scale benchmarking like HELM. These evaluation tools are invaluable for giving LLM users a sense of different models' relative performance. Nonetheless, they have limitations: For example, leakage of benchmarks datasets’ questions and answers into training data is a constant worry, and human preference for certain answers does not mean those answers are more accurate. In contrast, our current options for evaluating specific applications built using LLMs are far more limited. Here, I see two major types of applications. - For applications designed to deliver unambiguous, right-or-wrong responses, we have reasonable options. Let’s say we want an LLM to read a resume and extract the candidate's most recent job title, or read a customer email and route it to the right department. We can create a test set that comprises ground-truth labeled examples with the right responses, and measure the percentage of times the LLM generates the right output. The main bottleneck is creating the labeled test set, which is expensive but surmountable. - But many LLM-based applications generate free-text output with no single right response. For example, if we ask an LLM to summarize customer emails, there’s a multitude of possible good (and bad) responses. The same holds for a system to do web research and write an article about a topic, or a RAG system for answering questions. It’s impractical to hire an army of human experts to read the LLM’s outputs every time we tweak the algorithm and evaluate if the answers have improved — we need an automated way to test the outputs. Thus, many teams use an advanced language model to evaluate outputs. In the customer email summarization example, we might design an evaluation rubric (scoring criteria) for what makes a good summary. Given an email summary generated by our system, we might prompt an advanced LLM to read it and score it according to our rubric. I’ve found that the results of such a procedure, while better than nothing, can also be noisy — sometimes too noisy to reliably tell me if the way I’ve tweaked an algorithm is good or bad. [Reached LinkedIn's length limit. Rest of text: https://lnkd.in/gQEDtSr7 ]

2,012

89 Comments

Ever pondered the makings of a tech giant like Nvidia? Prepare to be inspired by the remarkable odyssey of Jensen Huang, the driving force behind NVIDIA's meteoric rise. Jensen didn't just lead Nvidia—he fearlessly bet the company not once, not twice, but THREE times to sculpt Nvidia into the trailblazing powerhouse it is today. Amidst the tumultuous seas of uncertainty, Jensen faced daunting adversities and relentless pressure. Yet, fueled by an unyielding determination, he charted a course of audacity and innovation. In the face of skepticism and doubt, Jensen dared to pivot Nvidia's trajectory, shifting focus from gaming to graphics processing units (GPUs) for visual computing. This audacious move, met with skepticism, ignited a revolution. GPUs emerged as the backbone of modern computing, catapulting Nvidia to the forefront of innovation. But Jensen's journey was far from smooth sailing. As the tech landscape evolved at warp speed, he navigated uncharted waters, steering Nvidia towards unexplored territories: artificial intelligence (AI) and data centers. Despite the weight of doubt pressing down upon him, Jensen pressed forward, braving the storm with unwavering resolve. In the crucible of pressure, Jensen and his team poured heart and soul into cutting-edge research and development. Their relentless pursuit of excellence birthed a new era of possibilities, with Nvidia leading the charge in AI breakthroughs, from autonomous vehicles to groundbreaking medical discoveries. Today, Nvidia's towering success stands as a testament to Jensen's indomitable spirit and unwavering courage. From redefining gaming with breathtaking graphics to shaping the future of AI and high-performance computing, Jensen's daring decisions have reshaped industries and touched lives worldwide. A great source for such amazing company stories is a podcast by Ben Gilbert and David Rosenthal called Acquired (https://www.acquired.fm/) available on iTunes and Spotify that I have become a great fan of. #Nvidia #Innovation #TechLeadership #JensenHuang #DreamBig

9

Falkonry (Suhas Mehta and Ashis Khan) and iba AG (Eric Snyder) will present how steel makers are pivoting to AI & Data driven decision support system at AISTech 2024 today (May 6, 2024) - please join us if you are interested. I co-authored this paper with Eric Snyder of iba AG.

65

5 Comments

Cleanlab Introduces the Trustworthy Language Model (TLM) that Addresses the Primary Challenge to Enterprise Adoption of LLMs: Unreliable Outputs and Hallucinations Quick read: https://lnkd.in/gX3yC8EG Cleanlab presents the Trustworthy Language Model (TLM), addressing the primary challenge hindering enterprise adoption of LLMs: unreliable outputs and hallucinations. TLM integrates a trust score into each LLM response, empowering users to identify and control erroneous outputs, thus facilitating the deployment of generative AI in previously inaccessible scenarios. Extensive benchmarking demonstrates that TLM outperforms existing LLMs in accuracy while offering better-calibrated trustworthiness scores, leading to enhanced cost and time efficiency compared to alternative methods for managing LLM uncertainty. TLM addresses the inevitable presence of hallucinations in LLMs by assigning a trustworthiness score to each output, enabling users to identify instances of hallucination. TLM prioritizes minimizing false negatives, ensuring that the trustworthiness score is low when hallucinations occur, thereby facilitating the reliable deployment of LLM-based applications. Playground: https://tlm.cleanlab.ai/ Try it here: https://cleanlab.ai/tlm/ Cleanlab #artificialintelligence #ai #datascience #llms

54

4 Comments

What are the axiomatizable parts of transformers? (Attention mechanism? Positional encodings? Layer interactions?) More generally, can AI/ML be formalized in an axiomatic system? Finally, if AI/ML, or parts of it, are formalized in an axiomatic system can it be proved that they can always validate statistically true statements within themselves because Gödel’s incompleteness theorems do not apply to these systems?

9

4 Comments

We are thrilled to announce the release of OpenBioLLM-Llama3-70B & 8B from Saama AI Research Labs. These groundbreaking models outperform Openai’s GPT-4, Google’s Gemini, Meditron-70B, Google’s Med-PaLM-1, and Google’s  Med-PaLM-2 in the biomedical domain, setting a new state-of-the-art for models of their size. Both models underwent a rigorous two-phase fine-tuning process using the LLama-3 70B & 8B models as the base and leveraging Direct Preference Optimization (DPO) for optimal performance. This release is just the beginning! In the coming months, we'll introduce expanded medical domain coverage, longer context windows, better benchmarks, and multimodal capabilities. Stay tuned for our upcoming official paper. This work was led by Ankit Pal Bhaskar Sambasivan Lisa Moneymaker Ari Srinivasan Rajeev Dadia Pravin Kumar Prithiviraj S Kamal Raj Kanakarajan Tamil Arasan Ph.D. Murugesan Vadivel Selvakumar Murugan Mahesh Narasimhan Basant Shanker Sas Maheswaran Panikos Christofi Rajasekar Nonburaj (𝑅𝒥) Navin Kumaran Vikas Kishore Geetha Eswaran Ojasvi Harsola Angu S K. Ramanathan Srinivasan Model download link is in the first comment.

839

106 Comments