Brendan Mannix

I’ve spent pretty much the last 15~ years of my life working in OSS and I don’t think I’ve ever had a more exciting year than 2024! Everywhere I go, I see the power of OSS opening up opportunities for collaboration and innovation. I think it’s undeniable that this has been a year for truly open source software (and open governance) because of the momentum behind Apache Iceberg and now Apache Polaris (Incubating) [https://lnkd.in/gmBQthbz]  — and it looks like I’m not alone! Apache Iceberg  has been selected as BigDATAwire’s latest ‘Readers’ Choice: Top 3 Big Data and the AI Open Source Projects to Watch’ and ‘Readers’ Choice: Top Data and AI Achievement’ awards. In addition, Apache Polaris was also listed as a top open source project to watch. When I joined Snowflake, I was proud to be a part of a company that was embracing Apache Iceberg and truly OSS by collaborating on the development of Apache Polaris and pushing great new features into Apache Iceberg. Snowflake has been making huge strides in the open data community, and I’m so excited to see our joint efforts receiving such kind feedback from peers. It’s both inspiring and rewarding to be working towards actually providing value, without vendor lock-in, to big data users everywhere! #BigDATAwireRCA24 https://lnkd.in/gZAAAmWT

103

1 Comment

You can judge the quality of an engineer by how easy it is to build and run their code. And this is building it from source, let alone using a quickstart docker container.

55

Hey folks, I'll be running a two-part virtual PD through Nassau BOCES in March 2025 called "Grading for Equity" using Joe Feldman's text, Grading for Equity, as our guide. Please see below for the description and feel free to share with your networks. To register, go to: https://lnkd.in/ehw8gkwn While grading is an everyday practice for classroom teachers, we rarely have the opportunity to interrogate our grading practices and understand how grades impact our students' attitudes, well-being, and academic success. In this transformational course, we will use Grading For Equity by Joe Feldman as our anchor text to 1) understand how traditional grading practices impact our students, and 2) how we can use research-based grading policies that are accurate, bias-resistant, motivational, and empowering. Rather than lectures, this course is active and engaging, using the case study method and rigorous discussion to support learning.

10

Easy as Pie: Stateful Services at CloudKitchens https://lnkd.in/eDDTfs5e

3

These are the boring, yet impactful aspects of API operations I am here for--low-tech, economy impacting, real-world stuff like the API deprecation notice from the Department of Labor.

20

I'm excited to share my latest article on LeadDev: "From False Urgency to Strategic Leadership." In this piece, I delve into the challenges and opportunities that come with shifting from reactive problem-solving to proactive strategic thinking. This article is designed to be a resource for staff+ engineers looking to enhance their leadership skills and drive meaningful impact in their role. 🔗 Read the full article here: https://lnkd.in/g7qxqE9r I hope you find it helpful and I’m eager to hear your thoughts and experiences. #Leadership #TechLeadership #LeadDev #FalseUrgency #StrategicLeadership

52

3 Comments

Project update! My application tracking extension is now doing fuzzier matching and injecting new information into LinkedIn (and other sites) to keep me from doubling up on applications. Next up: local storage and bulk import/export operations...

12

2 Comments

HPC software has always been challenging to create & manage. The relatively young Chapel Language offers a unique approach to development for parallel computing.

35

3 Comments

Anthropic released another paper on their ongoing work towards interpretability of large foundational models like their Claude 3 family, built on previous work by many. Their work continues to be enlightening and inspiring. The findings in this work are generally similar to previous findings on smaller models but scaled up to Claude 3 Sonnet: 💡 they find large numbers of "features" corresponding to combinations of neuron activations and these features map to concepts that we recognize/understand: for example, features for transit infrastructure, particular people, concepts (like bias and "objects lacking sentience"). This might sound familiar from classical machine learning, but the interesting thing is that there is no single thing/neuron encoding each of these concepts, it's a particular combination of neuron activations that embeds these concepts in "superposition" in the network. 💡they observe a sort of "distance" of these feature activations that arrange themselves "near" semantically similar features. When they use their sparse autoencoder approach to learn more and more features they observe that the level of abstraction in the detected features becomes more granular, in ways that are interpretable and meaningful. 💡when they artificially fire these activation patterns during inference, it affects the output in ways that relate to the "feature". For example, asking Claude "what's the most interesting science" normally it responds Physics, but if you clamp the feature activations for the feature "Brain Science" to 10x their max values then Claude responds: Neuroscience. Ways to interpret _and_ influence the model. Great readable work [1] along with an interactive explorer of some of the "features" themselves [2] 🎂 There's an easter egg too: if you scroll in the feature explorer tool and try and select one of the "Randomly selected features from 1M" you get a message "if you want to see the rest of the features, we're hiring!" with a link to their recruiting page :) [1] https://lnkd.in/gZbparwp [2] https://lnkd.in/gC9eus53

19

🎧 Check out Episode 3 of You Know, For Search, with host Steve Mayzak and Elastic Engineer Mayya Sharipova: https://gag.gl/9Na7aN Steve and Mayya discuss how Elastic's HNSW implementation came to be, how KNN works, when to use it, and more. #Elasticsearch #Lucene

6

What it looks like to responsibly deliver educator value using Generative AI -- ensuring reliability and trust -- while moving rapidly and iteratively https://lnkd.in/egUdnV6w

10

Hey folks! I'd like to more formally introduce and talk about a few of the projects we're working on towards improving large-scale bioinformatics analyses (one of which was developed during St. Jude's KIDS24 Biohackathon)! If you're not aware, my team started a project called St. Jude Rust Labs (https://lnkd.in/eXY_9dJ8) where we're rewriting a more foundation for bioinformatics analysis in Rust. This cuts across more than just workflow execution, but, recently, we've been primarily focused on improving the experience of working with the Workflow Definition Language (WDL) at scale. To that end, we've released a number of projects, including: - We've written a complete lexer/parser for WDL v1.2 language (https://lnkd.in/eGWHSM2g). Anyone can use this foundation to build tools for WDL. - We've written a VSCode plugin that includes a language server protocol (LSP) for linting and validation directly in your editor (https://lnkd.in/euApG9tF). - And, announcing today, we've started down the road of writing our own WDL execution engine spread across the Crankshaft (https://lnkd.in/et_RFkA3) and Sprocket (https://lnkd.in/eUTuPE68) projects. Crankshaft was prototyped during the St. Jude Biohackathon last week—it's a _headless_ workflow execution engine, meaning that, in theory, others could come along and write drivers for Crankshaft built on NextFlow, CWL, Snakemake, etc. As I said earlier, our team is really focused on WDL specifically, so we're going to continue building out a "head unit" for Crankshaft using WDL. That being said, I would love to see other community projects popping up and using the core machinery for other workflow languages. Thanks to all of the individuals who participated on our Biohackathon team (Kevin Benton, suchitra chavan, Braden Everson, Andrew Frantz, Michael Gattas, Peter Huene, and John McGuigan)!

76

1 Comment

🚀 Join us at #KubeCon and be part of #Kubernetes in its 10th year! 🥳 Today is the last day to receive preferential pricing ahead of the onsite rate. Register here: https://lnkd.in/g3KttPYW ❓ What's the big deal about Kubernetes? Our latest research report describes the incredible impact of this #opensource project in transforming the world of #cloudnative computing across enterprises - and the role it plays in the lives of the amazing people who sustain it has been just as significant, captured in dozens of respondent "stories." Download Kubernetes Turns 10 report today, and see you at KubeCon! https://lnkd.in/gVpY6wWB

26

In late 2023 we reported that MySQL and MariaDB’s REPEATABLE READ did not, in fact, provide repeatable reads. The MariaDB team has been hard at work this past year. They’ve added a new flag, --innodb-snapshot-isolation=true, which causes REPEATABLE READ to prevent Lost Update, Non-repeatable Read, and violations of Monotonic Atomic View. Jepsen has not yet tested this, but it looks like MariaDB might, with the new flag enabled, offer Snapshot Isolation at REPEATABLE READ. https://lnkd.in/dw_SFhjh https://lnkd.in/dc_9-P26

57

Great insights about making single-agent system powerful. Has good ml system implications as well. Prompt prefix caching would become one key technique here that interplays with other general inference optimization techniques

24

Looking to get started with in-database machine learning with MySQL HeatWave? This guide will help get you on the right path. https://lnkd.in/gPjuJU9g

1

Extreme scale field stories: riveting exploration of real-world engineering challenges and solutions from the frontlines of enterprise extreme scale deployments. uncovers a treasure trove of stories, from common memory issues to complex multi-thread race conditions. it's an in-person event. RSVP today! https://lnkd.in/gRWymH_A #java #caching #debugging

10

Great article from Tom Krazit with some interesting takes “I care less about specific formats, I care more about what I can get out of it, the scalability needed to operate and the compatibility and integration with different tools," #data #datalakehouse #bigdata https://lnkd.in/geggTe4h

30

1 Comment