HECC Supports Unitary Plan Wind Tunnel, Red Rover

Earlier this year, the Unitary Plan Wind Tunnel (UPWT) team at NASA's Ames Research Center collaborated with HECC experts and successfully performed initial large-scale tests of the Unsteady Pressure Sensitive Paint (uPSP) application, wrapping up a multi-year engineering project called Red Rover—reminiscent of the classic children's game.

The HECC Systems team and security staff, along with the NAS Division's Research & Development team and the UPWT team, jointly developed and deployed a complex data streaming and analysis system. The test streamed data from wind tunnel cameras to the UPWT data cache system, from which sensor data flowed to the NAS facility for further analysis and evaluation in the HECC environment.

This work enables the UPWT team to leverage HECC's computational and storage capabilities efficiently. With future data rate improvements, the hope is to couple the physical wind tunnel facility with the NAS facility to stream sensor data to the HECC enclave—and possibly to the hyperwall visualization environment in the future—which may enable rapid analysis of results and potentially assist in driving experimentation in real time.

HECC Supports Paradigm-Shifting Eclipse Modeling

Three weeks before the solar eclipse crossed North America on April 8, 2024, HECC’s User Services and Systems teams began working closely with researchers from Predictive Science, Inc. to produce a continuously running, time-dependent prediction model that was updated in real time with the latest measurements of the Sun's magnetic field from NASA’s Solar Dynamics Observatory—right up through the day of the eclipse.

Using a special job queue, the researchers ran continuous, real-time calculations on 100 of Electra’s Skylake nodes, with access to the other systems available as backup. During this time, HECC User Services experts closely monitored jobs and stayed in contact with the research team to ensure the project met its milestones.

HECC resources and services enabled Predictive Science to push the boundaries of Heliophysics modeling, demonstrating an unprecedented ability to produce real-time predictions of the Sun’s corona.

APP Concierge Support for Hazardous Spring Forecast Experiment

Scientists in NASA’s Global Modeling and Assimilation Office (GMAO) used HECC’s Aitken supercomputer to run their 2023 Hazardous Spring Forecast Experiment, a comprehensive evaluation of multiple contributing models including the Goddard Earth Observing System (GEOS), with a focus on severe weather over the continental U.S. The experiment incorporated simulated reflectivity, updraft, and rotation properties, as well as environmental variables such as temperature, dewpoint, and convective available potential energy, and required daily and nightly reservations of 420 Rome nodes over a continuous 13-week period, from March 30 to June 30, 2023.

HECC’s Application Performance and Productivity (APP) team provided extensive, customized debugging and job execution assistance throughout the project to ensure its success. This “concierge service” included monitoring job progress during the nightly runs, adjusting reservation end times to ensure completion of all jobs, and shifting some night reservations to daytime hours in cases where interaction with the GEOS code developers was needed. The team quickly investigated issues when they occurred and documented the causes and fixes—including off-lining bad nodes and resubmitting jobs when necessary—and worked closely with Control Room analysts to monitor the jobs, troubleshoot issues, and collect stack traces when needed.

The Spring Forecast Experiment is an annual project in which experimental forecasts are developed to test the applicability of cutting-edge tools in a simulated forecasting environment. Findings help the researchers develop new forecast products and modeling systems for predicting hazardous weather.

HECC Users Share Knowledge via the NAS Data Portal

HECC’s Data Publication and Discovery team works closely with researchers to assist with data sharing and dissemination. Recently, the team developed a method that uses a THREDDS data server to provide data in GeoTIFF and netCDF format, enabling users and collaborators to incorporate data into their workflow more efficiently. Users can specify a subset of a gridded dataset, gather metadata information, and/or visualize the extracted data through common scientific visualizing software packages.

Since 2019, the team has provided and maintained a central data portal where HECC users can upload raw or processed data to share with the public or with specified collaborators. Through the NAS Data Portal, large, multi-gigabyte datasets produced on HECC supercomputing resources—including planetary science such as Pluto and Titan climate modeling; Earth science such as GeoNEX, carbon cycle, and ocean circulation; heliophysics modeling and simulation; and aerospace modeling for design and optimization—can be easily accessed by science and engineering communities.

In 2022 alone, researchers downloaded 250 terabytes of data provided by HECC users via the NAS Data Portal—helping to fast-track scientific discovery and support collaboration among NASA researchers, their colleagues, and citizen scientists.

StellarBox Code Used to Increase Exoplanet Detection Rate

Scientists at NASA’s Ames Research Center are using their StellarBox code to obtain 3D radiative magnetohydrodynamics models of the Sun and stars from the deep interior to the corona. A production case modeling the Iota Horologii solar-type star is helping them characterize disturbances at the stellar surface (the so-called “stellar jitter”) in order to develop a procedure to increase the detection rate of Earth-mass exoplanets.

To improve StellarBox’s performance on the Aitken supercomputer’s AMD Rome processors, the HECC Application Performance and Productivity (APP) team used a small test case to uncover the I/O bottlenecks, and came up with solutions that led to a 22x speedup in I/O performance, a 6x speedup in total run time, and a 6x reduction in Standard Billing Unit (SBU) cost on the production case.

These significant performance increases will allow the Ames scientists to rapidly mimic the diversity of magnetic field properties over the stellar surface. The improvements will also have a dramatic impact on other projects, such as modeling the dynamics of the Sun and more massive stars, which requires high-resolution models for large computations domains.

Asteroid Impact Simulations for Planetary Defense

The Asteroid Threat Assessment Project (ATAP) team at NASA Ames plays a key role in ongoing planetary defense and asteroid threat exercises conducted jointly with various governmental agencies and international organizations. Recently, the team role-played through a fictious scenario in which a realistic asteroid impact threat is discovered and observed as it approaches on a simulated collision course with Earth.

The ATAP team uses their Probabilistic Asteroid Impact Risk (PAIR) model and high-fidelity simulations to estimate the severity and likelihood of damage that could result from potential asteroid impacts due to blast waves, thermal radiation, tsunamis, and global climatic effects. These hypothetical impact scenarios are run on HECC’s Pleiades supercomputer.

Visualizing Mechanisms for Making Water on the Moon

A team of physicists at Princeton University are running simulations on the Pleiades and Electra supercomputers to trace the origin of water on the Moon. The team's first-of-a-kind, 3D multifluid magnetohydrodynamic simulations could revolutionize our understanding of the origin of volatile compounds such as water on planetary bodies within the inner solar system and enhance the science return on past and future lunar missions. Their research illustrates a possible mechanism for lunar hydration: ionized oxygen transported by Earth’s magnetic field.

To help the researchers gain insight into their complex computational results, HECC's visualization and data analysis experts produced animations that help convey a dynamic system of ionized oxygen deposits. They used a variety of techniques in their visualization toolbox, including adaptive vector field tracing, to generate the oxygen ion footprint; and created a custom adaptive routine to fill in gaps in the initial footprint. These visualizations enhance our understanding of complex physics and contribute to NASA’s plan for sustained lunar exploration and development.

Simulating Algae to Model Water Quality in Minutes

Researchers on the Biospheric Science team at NASA Ames Research Center use HECC systems to calculate the optical properties of water containing algae and other phytoplankton species—a key to determining the degree of water contamination and modeling water quality. Using their Spectral Water Inversion Processor and Emulator (SWIPE) software, the team models more than 70 species to generate their synthetic bio-optical datasets—a time-consuming process, which until recently required about two hours to generate data for each single alga particle.

When the researchers asked our experts to help speed up the data generation process, the HECC Data Science team took a close look at SWIPE’s Equivalent Algal Populations (EAP) component, which performs the optical properties calculations. By embedding a module of the Dask programming framework into EAP, the HECC team was able to parallelize the code across multiple processors of a single node of the Pleiades supercomputer—reducing the time required to generate a spectral library for a single alga particle from two hours to just 18 minutes.

Applications Team Adds GPU Support to USGS Earthquake Modeling Code

Researchers at the U.S. Geological Survey (USGS) use their Cascading Adaptive Transitional Metropolis in Parallel (CATMIP) application to model and study earthquake faults. To assess potential benefits of running the code—originally developed to utilize MPI parallelization for multi-core CPU processors—on GPUs, the HECC Applications Performance and Productivity (APP) team ported several compute-intensive kernels of CATMIP to run on HECC’s GPU resources.

The APP team utilized the CUDA platform to add GPU offload functionality to the code, and compared CATMIP runs using parallel processing across 35 CPU cores to those using four processes sharing a single NVIDIA V100 GPU on an Intel Skylake node. Results showed that the code ran more than 20 times faster, depending on the kernel and test case. The APP team will continue to collaborate with USGS to fully port CATMIP to the GPU, with the goal of fitting submeter-resolution datasets obtained from recent California earthquakes.

WRLES Code Optimized to Improve Prediction of Turbulent Air Flows

Aerospace engineers at NASA’s Glenn Research Center use their Wall-Resolving Large-Eddy Simulation (WRLES) code to predict turbulent flows that occur around aircraft, generating data that can ultimately help design engineers reduce noise during takeoff and landing, or improve fuel efficiency during all stages of flight.

To improve the code’s performance, the HECC Applications Performance and Productivity (APP) team analyzed the code to find I/O bottlenecks and developed ways to clear them—for example, by applying OpenMP directives to parallelize WRLES. The resulting 30% improvement in the application’s performance allows the Glenn team to increase the fidelity or extend the duration of their simulations, enabling them to get more done with the same amount of supercomputing resources.

COVID Research Projects Completed Using HECC Resources

COVID-19 scientists conducted rapid research in the fight to understand the SARS-CoV-2 virus and develop treatments and vaccines to help bring an end to the pandemic. Three research projects were completed using donated HECC systems and services, and results have been shared via publications and presentations.

• “Drug-repurposing for COVID-19 with 3D-aware Machine Learning.” The models produced accurately screen unknown compounds for SARS-CoV-2 inhibition. Results are in publicly available dataset and a code repository.
• “COVID-19 – RNA-seq Analysis to Identify Potential Biomarkers Indicative of Disease Severity.” Researchers successfully analyzed 845 COVID-19 patient samples and environmental swab data to improve understanding of SARS-CoV-2 interaction with the host.
• “Modeling the Dynamic Behavior of Surface Spike Glycoprotein of COVID19 Coronavirus and Designing Biomimetic Therapeutic Compounds.” This work helped uncover dynamic and structural changes in spike protein variants that may lead to increased transmission rates.

Find out more about NASA’s contributions to COVID-19 research, presented at the SC20 supercomputing conference.

New Visualization Database Expands Access to 1/48° Ocean Simulation

Research scientists at NASA’s Jet Propulsion Laboratory, California Institute of Technology and Massachusetts Institute of Technology study the global ocean circulation and its linkages to the climate system as part of the Estimating the Circulation and Climate of the Ocean (ECCO) project.

Working with our visualization experts, the ECCO team released a new database that allows a broader community of researchers to explore hundreds of thousands of videos and images pre-computed from ECCO’s 1/48° MITgcm simulation run on NASA supercomputers. These pre-computed visualizations require far less effort for researchers to view and use, compared with having to download the simulation data and compute the visualization themselves. To access the database, see: A World of ECCO Visualizations.

For more than a decade, the ECCO team has visited the NAS facility to explore high-resolution views of the entire globe on the hyperwall visualization system, allowing the scientists to discover details that they had missed in previous analyses of their global ocean simulations. Now some of this exploration can be done remotely by researchers anywhere—without having to travel across the country or around the world to visit the NAS facility.

SOMA Code Gets 20x Speedup for Aeronautics Research

Researchers at NASA Ames use the Sequentially Optimized Meshfree Approximation (SOMA) application to improve the performance and usability of large-scale and production geometries for aerodynamic problems. SOMA is a mesh-free solver for differential equations and provides high accuracy for the adaptive, mesh- and matrix-free solution of aerodynamic problems. The code uses fewer computational resources to achieve the same results as a traditional gridded methods.

After the HECC Applications Performance and Productivity (APP) team parallelized SOMA, the code runs 20 times faster than the single-thread run. This enables the Ames researchers to solve large-scale flow problems on HECC systems in hours instead of weeks—significantly improving their ability to meet project milestones.

NASA GPU Hackathon Yields Significant Code Improvements

HECC and NVIDIA jointly organized the NASA GPU Hackathon 2020 to bring together application developers and computer experts to help get important NASA applications running effectively on GPU nodes. Nine teams of application developers participated in this event—a major impetus for teams to modernize codes of interest for NASA missions to CPU nodes containing GPU accelerators, with a focus on hands-on problem solving. HECC provided five Pleiades nodes with V100 GPUs for use by the teams.

During the event, which focused on Aerosciences and CFD applications, most teams achieved considerable performance improvements on both GPUs and CPUs. For example, a team with no GPU experience completed their first port of a time-critical loop to a GPU. Another team of expert CUDA programmers were able to restructure their algorithm, yielding a factor of five speed-up. And another team sped up some of their CUDA kernels by a factor of 20, which directly translated into their production code.

Capturing Noise-Producing Flow Features to Reduce Aircraft Noise

Scientist Mehdi Khorrami from NASA Langley is helping aircraft manufacturers get a step closer to reducing noise pollution for communities near major airports. HECC visualizations from full-scale simulations of a landing Boeing 777 reveal complex, time-dependent flow features that help understand the mechanisms that generate noise.

These successful computations track well with flight test data and pave the way for higher spatial resolution simulations leading to effective noise-reduction strategies.

Using Neural Networks to Interpret Carbon Nanotube Gas Sensor Data

NASA recently developed inexpensive, low-power, carbon nanotube (CNT) sensors for use on the International Space Station (ISS) to monitor cabin air quality. However, it is challenging to convert the CNT signals to human-readable indicators of air quality and safety. Working with the Chemical Gas Sensor team at NASA Ames Research Center, HECC data science experts built and trained a neural network using data collected on thousands of chemical exposures to address this challenge.

With initial results showing predictive accuracy better than 94%, the Ames team determined that neural networks can be a very effective way to interpret CNT data—a major step toward real-world testing on the ISS.

Year-End Wrap-up: Science & Engineering Results from SC19

Scientists and engineers from around the country showcased their latest results—made possible by HECC resources and services—in the NASA booth at the annual supercomputing conference, held in Denver this past November.

Users from NASA centers joined fellow users from universities and industry to explain the impact of their work to help protect future space explorers; design quieter aircraft and drones; develop realistic simulations for use by the ocean research community; and discover more about our the wonders of our Sun, neighboring planets, and the universe.

Remote Data Transfer Performance Improved by Factor of 20

HECC systems experts, working with scientists at NASA’s Langley Research Center, optimized the transfer of a 1-petabyte (PB) dataset from Oak Ridge National Lab to the NAS facility, increasing the transfer rate by a factor of 20.

The transfer rate increased from 55 MB/s to 1.08 GB/s after making custom adjustments to our in-house Shift tool. This lowered the required transfer time from 210 days to less than 11 days.

Proof-of-Concept Tests Give Code Green Light

A team from NASA’s Marshall Space Flight Center successfully completed a series of proof-of-concept tests to accurately model and predict the complex water-based sound suppression system inside the mobile launcher during ignition.

With the computing power of our Pleiades supercomputer, researchers validated the agency’s Loci/CHEM CFD flow solver’s accuracy, giving the green light for simulations using the agency’s next-gen Space Launch System.

Discovering a Cause of Dynamic Stall in Rotorcraft

Over the years, NASA Ames research scientist Neal Chaderjian has developed increasingly accurate CFD flow simulation software to help rotorcraft design engineers improve understanding of the dynamic stall phenomenon.

Using the Electra supercomputer to run ultra-high-resolution simulations, and working closely with our visualization experts, Chaderjian discovered that blade-vortex interaction can cause dynamic stall.

Revealing How Merging Black Holes Emit Gravitational Waves

For more than a decade, our Data Analysis and Visualization team has worked with astrophysicists at NASA’s Goddard Space Flight Center to simulate and visualize merging binary black holes on HECC supercomputers, resulting in significant contributions to the study of gravitational waves.

The team’s stunning images and videos also help the scientists analyze their simulation data and communicate results to other researchers and to the public—as shown in this visualization, which was adapted into Europe’s largest planetarium show at the Cite des Sciences et de l’Industrie in Paris.

Detecting Dynamo Waves Inside the Sun to Predict the Solar Cycle

Using data from NASA solar observatories, heliophysics researchers at the New Jersey Institute of Technology found patterns of dynamo waves in the Sun’s interior that pinpoint where the solar cycle originates.

Data analysis on Pleiades—plus support from our visualization team—allowed them to predict the sunspot cycle maximum, which will help in planning and preparation for future space weather events.

Predicting Vibrations on the Orion Launch Abort Vehicle for Astronaut Safety

Research scientists at NASA Ames ran simulations on Pleiades that impact the design of the Orion launch abort vehicle (LAV) during takeoff to increase astronaut safety, reduce uncertainty, and keep cost and vehicle weight down. The simulations predicted vibrational loads on the LAV within 2% of measured vibrations.

Services provided by the HECC visualization team, including custom images and movies, helped reveal new flow details that led the scientists to identify different types of vortices—often the source of noise and vibration on space vehicles.