Nader Khalil

The NVIDIA GH200 Grace Hopper Superchip has been a groundbreaking achievement in #acceleratedcomputing and #energyefficiency for Apache Spark. Dive deeper into the benchmark results for NVIDIA GH200 and learn how to migrate Apache Spark workloads to it. #HPC Read our tech blog here: https://nvda.ws/46UQLYV

453

6 Comments

🎙️New Open at Intel podcast! To say that maintaining an open source project is challenging would be a gross understatement. Devin Stein created Dosu to try and address some of the challenges using AI. Check out our conversation by subscribing to Open at Intel in your favorite podcast player or listen here: https://lnkd.in/gsiwUuQy #podcast #NewEpisode #OpenSource #AI

18

2 Comments

Great work by the team producing this collection of assets on how to use NVIDIA NIM on AWS to optimize your inference performance to get state-of-the-art deployments. Running inference endpoints across Amazon's NVIDIA GPUs instances on EC2, Amazon EKS, and Amazon SageMaker has never been faster or more efficient. Here’s why NVIDIA NIM is the best solution for AI developers and enterprises: ✅ Lightning-fast Deployment: Prebuilt containers with optimized generative AI models, deployable with just a few commands, help you get started in minutes. ✅ Unparalleled Performance: Experience up to 2.5x higher throughput, 4x faster time-to-first-token, and 2.2x faster inter-token latency compared to other open-source solutions—because speed and efficiency matter when scaling AI. ✅ Seamless Integration with AWS: NIM is purpose-built to integrate with AWS managed services like EKS, EC2, and SageMaker, providing a streamlined path from development to production, no matter where your AI models run. ✅ Enterprise-Grade Security and Flexibility: Maintain full control of your data, IP, and custom models while scaling AI workloads in a secure and enterprise-ready environment. ✅ Cost-Effective and Scalable: Reduce time-to-market and operational costs, optimizing response times and throughput, all while leveraging the robust infrastructure of AWS. Whether you’re deploying large language models or domain-specific AI, NVIDIA NIM on AWS empowers you to deliver high-performance AI applications, bringing vision to reality at scale. #GenerativeAI #AIinference #NVIDIANIM #AWS #EC2 #EKS #SageMaker #AI https://lnkd.in/e722U8Np Abhishek Sawarkar Brian Pickering Guy Farber Adel El Hallak Jiahong Liu Timothy Ma Saurabh Trikande Vara Bonthu

117

3 Comments

Keeping your data close to the GPUs is essential, so we offer VAST Data as part of our #GPUasaService AI Platform. Let us help you #StepontheGaaS and move your #AI strategy forward. #Fast #Secure #Private Dominic Ken Michael Bill Traci

10

1 Comment

AI workflows are driving demand for memory and storage to prepare data, train the model and create useful inference engines.  This is driving announcements of storage and memory products that support the various elements of these AI workflows.

51

4 Comments

Vara Bonthu and the team continue to produce incredible assets, driving forward the extraordinary partnership that empowers customers to scale Large Language Models (LLMs) more efficiently on Amazon EKS with NVIDIA's Inference Microservices (NIM) Today’s spotlight is on how this collaboration makes deploying Meta's Llama-3-8B model on Amazon EKS simpler and more effective. With dynamic scaling supported by Karpenter, the solution ensures optimal performance and resource efficiency. Key highlights include detailed guidance on deploying, managing, and scaling LLMs, plus insights into leveraging Prometheus and Grafana for complete observability. Whether your focus is low-latency or high-performance inference, this approach lays the foundation for scalable, powerful AI model serving. Check out the AWS blog to dive into the full details and see how Vara’s leadership is helping customers fully leverage the power of NVIDIA NIM on EKS. Abhishek Sawarkar

15

1 Comment

#DYK While pre-trained large language models (LLMs) excel at factual tasks after fine-tuning, their ability to access and update knowledge remains limited. Enter Retrieval-Augmented Generation (RAG), which overcomes these limitations by combining LLMs with a retrieval component. 💥 Interested in learning more? Check out this blog to see how you can have a basic RAG solution up and running for your documents with minimal clicks and code copy-pasting: https://dell.to/4c438Ds #AI #Dell #DellTechnologies #HuggingFace #AImadeeasy #iwork4dell #iwork4dell

10

Nice summary Gautam. But knowing what it takes and overcoming the hubris and actually following the obvious path are two very different stories. Don't get me wrong, there are a number of very capable engineers at Intel, way too many to list here, well, Amber Huffman really deserves mentioning. In a way, the layoff may be a good thing for Intel for the simple reason that some of the egos may get the axe, one can only hope that it is not the innovators instead. The company needs to re-define themselves. Signal integrity, built-in, firmware-based design verification tools used to be the stronghold of Intel's architecture and designs until "Tok" with its cost reductions turned the same features into the Achilles heel. But we can all muse, words are cheap. The next 6 months are going to be crucial for getting the right visionaries in charge and of course that may come at the expense of short term profits but the last thing Intel needs now are some knee jerk reactions trying to regain lost territories that are already a thing of the past.

2

1 Comment

My new episode of Advanced Insights is out! This is a highly engaged conversation about data center computing! Amin Vahdat, Engineering Fellow and VP of Machine Learning, Systems and Cloud AI at Google, and I delve into the transformative era of distributed computing. We explore challenges, and opportunities, as well as the future landscape. A new era is upon us, please take a look and share your comments or questions. https://lnkd.in/gmwNvasv

172

2 Comments

🤗 Transformers now supports speculative decoding for ANY model - bringing ~2x acceleration to 140k+ LLMs on the Hub, without accuracy loss 🤯 Introducing Universal Assisted Generation -- a new method from Intel Labs and Hugging Face that expands assisted generation to support target / assistant model pairs across ANY model family 🔥 > Update to the latest 🤗 Transformers version (see first comment for pipeline usage below) > Choose a target model from one of 140k+ LLMs or Mixture of Experts models from the 🤗 Hub > Select a lightweight assistant model such as Qwen2.5-0.5B or Vicuna-68m to generate draft tokens > Accelerate target model inference by ~2x 🔥 > Tokenizer mismatches are identified and corrected behind the scenes, with minimal overhead Check out the blog for more details: https://lnkd.in/dYwgMgWV Grateful for the collab with Oren Pereg, Moshe Berchansky, Jonathan Mamou, João Gante, Lewis Tunstall, Nadav T. and Moshe Wasserblat

357

15 Comments

#GenAI may be hyped right now and it's all about #training. However in the need to be more efficient and #agile Intel and Meta are collaborating using an #opensource platform with training + inference, #OpenVino, running the same code across different technologies.

1

🔥 Meet Dr. Ameer Haj Ali, PhD, the driving force behind Anyscale, at the #GENAISummitSF2024 🔥 Dr. Ameer Haj Ali, PhD, Founding Engineer of Anyscale, is a pioneer in scalable compute for #AI and #Python, driving innovation with Ray, the most popular open-source framework for scaling and productionizing AI workloads. Anyscale stands out in the competitive landscape of AI infrastructure providers, offering a unique advantage with Ray's flexibility and scalability. By powering the world's most ambitious AI workloads, from #generative AI to #computer #vision, Anyscale has established itself as a leader in the industry. With notable customers like OpenAI, Spotify, and Uber, Anyscale has demonstrated significant performance improvements, fault tolerance, and cost efficiency. The company's founders have a strong track record of innovation, and its business performance has been impressive in recent years. #ai #artificialintelligence #airevolution #machinelearning #deeplearning #futureofai #techinvestment #technews #techstartup #techleaders #techtrend #technology #tech #SF #sanfrancisco

2

Looking forward to this talk with Todd Brannon and Chris O'Brien. As the speed of generative AI adoption continues to grow, we'll look at what IT teams need to know in supporting these new workloads in their environment. Don't miss this one! #generativeAI #AImanagement #AIatscale

10