Skip to main content

10 posts tagged with "AWS"

View All Tags

Powering CIROH DevCon 2026: How CIROH and 2i2c Ran Cloud Computing at Scale

· 9 min read
Arpita Patel
Assistant Director, IT and DevOpsAlabama Water Institute
April Johnson
Community and People Lead2i2c
Benjamin Lee
Development Operations EngineerAlabama Water Institute
Nia Minor
Graduate Research AssistantAlabama Water Institute
Harsha Vemula
DevOps EngineerAlabama Water Institute

At the 2026 CIROH Developers Conference (University of Utah, May 27–29) participants used a variety of tools and models on cloud infrastructure they'd never had to install, configure, or even think about. To enable that seamless experience, our Research Cyberinfrastructure team provisioned cloud resources for 15 hands-on workshops across three days in close collaboration with our partners at 2i2c. Here's what we ran, and how.

What we provisioned, by the numbers

Running an event of this scale requires more than just a single cloud context, as each workshop posed its own computing and data access requirements. CIROH Cyberinfrastructure met this challenge via a coordinated stack spanning multiple cloud providers and services. Across the 15 workshops that needed infrastructure (14 requests), the R2OHC cloud provisioned the following:

ProviderWorkshopsWhat it carried
2i2c JupyterHub on GCP12A mix of Small, Medium, Large, and GPU servers; 9 workshops ran on custom-built images
AWS3NRDS, HydroServer, TEEHR
Google NWM BigQuery API2Input data for Flood Inundation Mapping (multi-source DB / HAND visualization) and Hydroinformatics (Essential Geospatial Skills)
NSF Access1GPU-powered VMs for the "Talk to NRDS" LLM workshop

For 12 of these workshops, their backbone was the CIROH–2i2c JupyterHub, a managed JupyterHub environment that 2i2c operates on Google Cloud. For the conference, we provisioned a dedicated Workshop Hub: an ephemeral environment that lives only for the duration of the event and then spins back down.

The work that happens before anyone logs in: custom images

Nine of the twelve JupyterHub workshops needed a custom software image to support their unique computing needs. Sessions covering topics like the Community Asset Computation Hub (CCNH), Foundations of Machine Learning, CUAHSI Hydroinformatics, HydroServer, INFLECT-based Flood Inundation Mapping, Satellite Data Projection, Machine-Learned Model Emulators, and Water Quality Modeling from Hillslope to Watershed Scales each had distinct dependency stacks.

That's a coordination problem, and it was solved weeks before the conference. Workshop leads told us what their sessions needed, we defined those environments in our public awi-ciroh-image repository on GitHub, and 2i2c deployed them. By the time attendees arrived, the right libraries were already in place, all but eliminating the usual loop of haggling with installations and configurations that “work on my machine”. Instead, there were reproducible environments, ready and consistent on day one, demonstrating clear evidence of how tightly the CIROH and 2i2c teams work together.

A line graph showing usage of CIROH-2i2c JupyterHub environments over time. A spike in usage occurs on May 27th, peaking on the 27th at 132 workshop users, before leveling back off to near-zero levels after the conference's end on June 1st.
During DevCon, as many as 132 participants were logged into CIROH-2i2c JupyterHub per day.

CCNH: Instant Access to NextGen in JupyterHub

Two of the twelve workshops also marked the first appearance of the CIROH Community NextGen Hub (CCNH) at a DevCon event! This JupyterHub image contains all of the dependencies required to run the NextGen Framework, bypassing the framework's notorious complexity. CCNH represents a new paradigm where the NextGen framework can be run entirely from the comfort of a web browser, further extending its accessibility and expanding the potential for research and development with the framework.

DevCon 2026 workshops using CIROH Cyberinfrastructure

HydroShare integration: from a resource to a running notebook

Many workshop materials were published as HydroShare resources and launched straight into the Workshop Hub via a single link, pulling notebooks and data into the user's environment automatically. What might otherwise have been an onerous process of managing cluttered Downloads folders and local directory structures was instead compressed into a single click.

Why the 2i2c partnership matters

2i2c's significance in this process went far beyond just hosting the JupyterHub. 2i2c specializes in open-source cloud infrastructure for research, and that shared commitment shaped our collaboration. Our entire stack — JupyterHub, the container images, the HydroShare connector — is open source and lives in public repositories. Another institution could easily read our Dockerfiles or dependency configurations to launch similar environments.

The operational model matters too. 2i2c manages JupyterHubs for many science communities, and what they learn from one community gets shared quickly with others. Our pre-conference testing and prep was informed by dashboards 2i2c built for EarthScope's computing-intense workshops, and the temporary storage approach 2i2c and CIROH used at DevCon is now being re-used with other communities (including EarthScope).

Lessons we're carrying forward

Pre-event automation is everything. Nine custom images don't build themselves on the morning of a workshop. Front-loading that work meant we could focus on people, not infrastructure, during the event.

Treat infrastructure as tracked engineering. Thirteen GitHub issues turned a sprawling set of requests into an auditable plan — and a head start on next year.

Reproducibility scales. Version-controlled environment images are the difference between 15 consistent workshops and 15 individual troubleshooting sessions — and they let any of these workshops be re-run on the mainline (NM2.1) 2i2c JupyterHub environment.

Open-source, public-private partnerships work. Corporate cloud providers (AWS/Google Cloud), an NSF-funded computing resource, a mission-driven infrastructure partner with 2i2c, and a university research institute (CIROH) all contributed aspects of the DevCon experience. The result was a unified experience across every workshop. That's the model we want to keep building on.

To our partners at 2i2c, and to the providers and resources that backed DevCon 2026: thank you. To our DevCon networking sponsors — Lynker, Google Cloud, Civil and Environmental Engineering at The University of Utah — thank you.

And to everyone who logged in, ran the notebooks, and pushed their work forward — that's exactly what this infrastructure is for.

Interested in CIROH's cloud resources? Learn more at hub.ciroh.org/docs/services/intro or reach the CIROH Cyberinfrastructure team at ciroh-it-support@ua.edu.

Continental-Scale Streamflow Simulation Using Kriging in the NGIAB-NRDS NextGen Ecosystem

· 5 min read
Suma Battula
Department of Geological SciencesThe University of Alabama
Kunal Sarna
Department of Computer ScienceThe University of Alabama
Sonali Vyas
Department of Computer ScienceThe University of Alabama
Harsha Vemula
DevOps EngineerAlabama Water Institute
Arpita Patel
Assistant Director, IT and DevOpsAlabama Water Institute
Jonathan Frame
Assistant Professor, Department of Geological SciencesAlabama Water Institute, The University of Alabama

Hourly streamflow kriging is now operational within the NextGen Research Data Stream, delivering spatially complete estimates for all NextGen v2.2 hydrofabric catchments. This observation-based approach supports streamflow analysis, NWM calibration, forecasting, and data assimilation for ungauged basins.

Kriging-Based Streamflow Estimation

Process-based hydrologic models are subject to structural and forcing uncertainties throughout the modeling domain, yet these can only be evaluated where USGS gauge observations exist. There is a clear need for a data-driven, observation-based framework that provides spatially complete streamflow estimates with well-characterized uncertainty, independent of model structure. Recent results from the CIROH project "Developing and Benchmarking Data Assimilation Methods on a Standardized Testbed" suggest that a simple Kriging interpolation between USGS gauged locations is both scalable and accurate for producing such spatially complete streamflow fields. As a pure data-driven method, this interpolation cannot be used directly for forecasting, but it serves as a valuable "pseudo-observation" for streamflow analysis and historical reconstruction.

The NextGen Research DataStream (NRDS): A Reproducible Numerical Prediction System for Accelerating Research to Operations in Hydrology

· 10 min read
Jordan Laser
Software EngineerLynker
Arpita Patel
Assistant Director, IT and DevOpsAlabama Water Institute
Harsha Vemula
DevOps EngineerAlabama Water Institute

Technological advances are evolving water prediction capabilities at a ludicrous pace. From revolutionary machine learning algorithms to dramatic advances in computational hardware, the potential for making accurate hydrologic predictions has never been higher. To meet this new potential, the hydrologic community continuously generates models and approaches based on cutting edge research that could potentially benefit operational systems. However, many of these innovations lack a path to operational deployment.

The NextGen Research Datastream (NRDS) provides a mechanism by which these ideas can be refined and make their way into operations.

Developed by Lynker and the Alabama Water Institute (a Cooperative Institute for Research to Operations in Hydrology partnership), the NRDS facilitates the actualization a research idea from the community in a scalable and deployable numerical prediction system. To evaluate each of these modeling concepts, NRDS deploys prototype models to generate a continuous “datastream”. These outputs can then be evaluated and made more accurate. This cycle of streamlined deployment and iterative design lets these prototypes mature into a product that can be picked up by an operational forecasting team.

To enable this process to be done rapidly and smoothly, the entire system is designed with reproducibility and iterative improvement as core principles. The NRDS is an automated numerical prediction system generating regular stream flow forecasts that uses the NextGen Water Resources Modeling Framework (NextGen) as the core modeling engine and NextGen In A Box (NGIAB) as the simulation environment. This system generates forecasts across the contiguous United States (CONUS) on CIROH's operational cyberinfrastructure backbone: the research-to-operations (R2O) Hybrid Cloud (R2OHC) platform, with deployment on the AWS cloud. What makes the NRDS exciting is that the entire system is open-sourced, reproducible, publicly browsable, and potentially editable by anyone in the hydrologic community.

AWS re:Invent 2025: Key Insights for Research and Cyberinfrastructure

· 4 min read
Arpita Patel
Assistant Director, IT and DevOpsAlabama Water Institute
Scott Hendrickson
Sr Solutions Architect WWPS EducationAmazon Web Services
A photo from AWS re:Invent 2025

AI, DevOps and the Future of Cloud Infrastructure

AWS re:Invent did not disappoint! I spent the first week of December at Amazon Web Services' flagship conference in Las Vegas. The event delivered cutting-edge technical insights, showcased the rapid evolution of cloud computing and AI, and provided countless opportunities to connect with industry leaders.

The energy across all five conference venues was more vibrant than I ever imagined it would be.

Moving Hydrologic Prediction Forward — A software integration meeting at the Alabama Water Institute

· 10 min read
Martyn Clark
Professor of HydrologyUniversity of Calgary
James Halgren
Assistant Director of ScienceAlabama Water Institute
Matthew Denno
Lead Software DeveloperRTI International
Arpita Patel
Assistant Director, IT and DevOpsAlabama Water Institute
Josh Cunningham
Software EngineerAlabama Water Institute
Quinn Lee
Programmer AnalystAlabama Water Institute
Sam Lamont
Lead Software DeveloperRTI International
Darri Eythorsson
Postdoctoral ResearcherUniversity of Calgary
Cyril Thebault
Postdoctoral AssociateUniversity of Calgary
Sifan A. Koriche
Research [Hydrologic] ScientistAlabama Water Institute
Group photo from the software integration meeting at the Alabama Water Institute

Last week, at the invitation and expert coordination of James Halgren, teams from RTI International (Sam Lamont and Matt Denno) and the University of Calgary (Darri Eythorsson, Cyril Thebault, and Martyn Clark) met at AWI for an intensive working session focused on weaving recent CIROH research into AWI’s fork of the NOAA Office of Water Prediction (OWP) Next Generation Water Resources Modeling Framework (nicknamed “NextGen”). James took the lead in developing the agenda, lining up the right scientific and technical expertise and ensuring that the week targeted the most critical software integration challenges. Throughout the visit, the RTI and UCalgary teams collaborated closely with AWI software engineers Quinn Lee, Josh Cunningham, hydrologic scientist Sifan A. Koriche, and James himself. The days were filled with whiteboards, deep technical conversations, and strategic planning around the future of NextGen water prediction. This recap captures the key themes and the momentum that carried through the week.

DevCon 2025: A DevOps and Cyberinfrastructure Success Story

· 3 min read
Arpita Patel
DevOps Manager and Enterprise ArchitectAlabama Water Institute

The recent DevCon 2025 event showcased not just cutting-edge development practices, but also demonstrated how modern DevOps principles and cloud infrastructure can seamlessly support large-scale technical workshops. Our team had the privilege of providing IT infrastructure and support for over 200 attendees, creating a robust learning environment through an exemplary public-private partnership.

Image of CIROH's Research Cyberinfrastructure and DevOps team. On the left, two graphs are shown depicting usage for the Google Cloud-2i2c and Jetstream2 environments.

CIROH's Research Cyberinfrastructure and DevOps team.
Left to right, top to bottom:
Manjila Singh, Arpita Patel, Nia Minor, Trupesh Patel, James Halgren; Benjamin Lee.

DevCon 2025: Hydroinformatics and Research CyberInfrastructure Keynote

· 5 min read
Arpita Patel
DevOps Manager and Enterprise ArchitectAlabama Water Institute

Last week, I had the incredible opportunity to co-present a keynote at the CIROH Developers Conference (DevCon 2025), which attracted over 200 attendees. This presentation, which I presented alongside Dan Ames, focused on "CIROH HydroInformatics and Research Cyberinfrastructure." It was a fantastic experience to share insights into the powerful tools and technologies that CIROH engineers, students, researchers have been developing to advance hydrological research and operations.


Pennsylvania State University Researchers Leverage CIROH Cyberinfrastructure for Advanced Hydrological Modeling

· 3 min read
Arpita Patel
DevOps Manager and Enterprise ArchitectAlabama Water Institute
Yalan Song
Research Assistant ProfessorPennsylvania State University
Tadd Bindas
Graduate ResearcherPennsylvania State University

Pennsylvania State University (PSU) researchers have been leveraging CIROH Cyberinfrastructure to tackle complex hydrological modeling challenges. This post highlights their innovative approach using the Wukong computing platform in conjunction with Amazon S3 bucket storage to efficiently process and analyze large-scale environmental datasets. 🚀

CIROH Cloud User Success Story

· 3 min read
Arpita Patel
DevOps Manager and Enterprise ArchitectAlabama Water Institute

This month, we are excited to showcase two case studies that utilized our cyberinfrastructure tools and services. These case studies demonstrate how CIROH's cyberinfrastructure is being utilized to support hydrological research and operational advancements.

1. ngen-datastream and NGIAB

ngen-datastream image

CIROH Research CyberInfrastructure Update

· 2 min read
Arpita Patel
DevOps Manager and Enterprise ArchitectAlabama Water Institute

We're excited to share some recent developments and updates from CIROH's Research CyberInfrastructure team:

Cloud Infrastructure

  • CIROH's Google Cloud Account is now fully operational and managed by our team. You can find more information here.
  • We're in the process of migrating our 2i2c JupyterHub to CIROH's Google Cloud account.
  • We've successfully deployed the Google BigQuery API (developed by BYU and Google) for NWM data in our cloud. To access this API, please contact us at ciroh-it-support@ua.edu. Please refer to NWM BigQuery API to learn more.