Zhoutong Fu

Easily generate one billion embeddings in under a day... and with limited budget! How? With Ray Data from Anyscale and Pinecone! Cheng Su and Roy Miara present this can't-miss session during the #DataAISummit tomorrow. Catch it in-person or online.

43

2 Comments

Heterogeneous graph data is ubiquitous; different edge relationships and attribute types come up in social/epidemic graphs, molecular/protein graphs, and buyer behavior graphs. This recent paper addresses heterogeny in graph neural network prediction through a topological tool called sheafs, which aggregate properties of edges and vertices according to a pre-defined (or learned) function. Definitely have a look at this recent arXiv paper out of Cambridge if you are working with these types of graph neural networks! https://lnkd.in/eZHHhtuN

47

1 Comment

Learn how to build an autonomous research assistant using LangGraph with a team of specialized AI agents #llms #chatgpt4 #gemini #aiml #llmops https://lnkd.in/d6TUbZNj

2

Crafting effective prompts is an art itself, but getting it right can mean the difference between a great output and total garbage! Today, we’ll explore the 5 prompt engineering techniques that every AI engineer should know to get the most out of LLMs. #llms #llmops #chatgpt4 #rag #gemini #ai https://lnkd.in/d3an85KE

3

Torch Compile is here to optimize inference in large language models (LLMs) with minimal overhead. In Benjamin Marie's latest article, explore how adding just one line of code can improve decoding performance. #Llama LLM

14

We just released a blog on whether the use of multimodal feature transfer can detect deception (phenomena like sarcasm, irony, and condescension) in conversation. The short answer... Yes. This particular write-up profiles a subset of our work at Symbl.ai on multimodal large language models, and is research led by one of our summer interns, Benjamin R. (from Prof. Larry Heck's AVA Lab at Georgia Institute of Technology). We have a paper currently under submission on this work, and we can't wait to share the full details with all of you soon. Blog Link: https://lnkd.in/g6m_wFDi #multimodal #featuretransfer #LLMs #deception #conversation #sarcasm #irony #language #AI #ML #NLP #LargeLanguageModels #SummerInternship #ResearchPapers #ACL #Llama2 #SymblDotAI

16

Can LLMs Teach Themselves? Exploring Self-Improvement in Language Model Training If so, which generation of LLMs can do so? The current generation IME cannot. https://lnkd.in/gK-cevmy

4

The BayLearn 2024 abstract submission deadline has been extended to Aug 5, 2024. If you are working on cutting-edge ML research, please consider submitting an abstract. BayLearn 2024 will be an in-person event, held on Oct 10, 2024: https://baylearn.org This is a great opportunity for student researchers to showcase their latest research work and get good feedback. Submissions are non-archival and the review process is quite informal. #machinelearning #research

68

Super exited to see "LongBet: Heterogeneous Treatment Effect Estimation in Panel Data" on arxiv. Thanks Meijia Wang and P. Richard Hahn! This paper introduces a novel approach for estimating heterogeneous treatment effects of binary treatment in panel data, particularly focusing on short panel data with large cross-sectional data and observed confoundings. In contrast to traditional literature in difference-in-differences method that often relies on the parallel trend assumption, our proposed model does not necessitate such an assumption. Instead, it leverages observed confoundings to impute potential outcomes and identify treatment effects. The method presented is a Bayesian semi-parametric approach based on the Bayesian causal forest model, which is extended here to suit panel data settings. The approach offers the advantage of the Bayesian approach to provides uncertainty quantification on the estimates. Simulation studies demonstrate its performance with and without the presence of parallel trend. Additionally, our proposed model enables the estimation of conditional average treatment effects, a capability that is rarely available in panel data settings.

105

3 Comments

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

179

2 Comments

In this article we are providing the major architectural building blocks and blueprint for building end-to-end GenAI-based applications that are ready for production. I’d like to call out several key considerations as you go about implementing and designing these llm-based applications. #llms #llmops #chatgpt4 #aiml #gemini #mlops #data https://lnkd.in/dMev5ncq

8

Learn how an image-text multi-modality model can perform image classification, image retrieval, and image captioning #llms #llmops #image #opencv #chatgpt4 #gemini #aiml https://lnkd.in/gJFh9Ruf

1

Aligning Large Language Models with Diverse User Preferences Using Multifaceted System Messages: The JANUS Approach Quick read: https://lnkd.in/gQAsT95M Paper: https://lnkd.in/gkBAuWiC

19

More opportunities

12

Learn how you can convert a PyTorch model, trained on the web, to a mobile-optimized format. Madhur Zanwar describes the process using a few lines of code. He also discusses his journey and how one approach didn't yield satisfactory results while trying to run it in a React Native app. #machinelearning #pytorch #computervision #edgeml #reactnative #objectdetection #yolov5

14

LLMs are currently trained with next-token prediction loss, hindering their efficiency. Instead of this approach, researchers from Meta experimented with multi-token prediction and found major gains! See how they did it. #ML #MachineLearning #GenAI #GenerativeAI #LLM #LLMOps #AI #ArtificialIntelligence #DataEngineering #DataEngineer #DataScience #DataScientist

16

Huge release by Daekeun Kim and Hyo (HK) Choi from our AI GBB team in Korea, sharing Korean language benchmarks for our latest Azure OpenAI model, gpt-4o-mini - like gpt-4o, the model excels on Korean language tasks, in this case mini completely eclipses gpt-35-turbo and nearly achieves gpt-4-turbo performance. The benchmarks and code are open source, extensible beyond our Azure models, so we encourage our customers and the larger community to try it out and even fork/share your own findings - and please let me in the comments what other Asia languages beyond Korean you may be looking to evaluate with your language models.

52

7 Comments

- SwiftKV is a novel model transformation and distillation procedure (by folks at Snowflake) designed to optimize large language model (LLM) inference. Very cool findings in the paper (as presented in the vLLM meetup on 11/13). - SingleInputKV: Prefills later layers' KV cache using earlier layer outputs, reducing computation on prompt tokens. The key point is that the first few layers of attention in LLMs are similar (nice find), allowing SwiftKV's SingleInputKV mechanism to skip computations in later layers during inference. By using a single earlier layer's output to compute the KV cache for subsequent layers, it significantly reduces both computation (by 50%) and memory usage (by 62.5%) without sacrificing model performance. - AcrossKV: Merges KV caches of neighboring layers to minimize memory usage and increase batch size for higher throughput. - For models like Llama-3.1-8B and 70B, SwiftKV reduces prefill compute by 50%, memory usage by 62.5%, and achieves up to 2× higher throughput and 60% lower time per token. https://lnkd.in/guhEBW5G

62