|
| 1 | +--- |
| 2 | +layout: bof |
| 3 | +title: ISC 2026 Data Streaming BoF |
| 4 | +subtitle: "Real-Time Scientific Data Streaming to HPC Nodes: Challenges and Innovations V" |
| 5 | +event_date: June 23, 2026 |
| 6 | +time: "TBD" |
| 7 | +order: 7 |
| 8 | +conference_logo: https://isc-hpc.com/wp-content/uploads/2024/02/isc-logo.svg |
| 9 | +organizers: |
| 10 | + - name: Bjoern Enders (NERSC) |
| 11 | + image: https://www.nersc.gov/assets/Uploads/Profiles/Bjoern-Enders__FocusFillMaxWyIwLjExIiwiLTAuMTEiLDI1MCwyNTBd.jpeg |
| 12 | + - name: Rafael Ferreira da Silva (ORNL) |
| 13 | + image: https://rafaelsilva.com/assets/images/2022-rafael-ferreira-da-silva-high-resolution.jpg |
| 14 | + - name: Alex Upton (CSCS) |
| 15 | + image: https://www.cscs.ch/fileadmin/user_upload/mycscs_staff/user_29_small.jpg |
| 16 | +supporters: |
| 17 | + - link: https://www.nersc.gov/ |
| 18 | + image: https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQzKv5LXRUBcMbGsXmwhEVArUKYup-AJjDs8g&s |
| 19 | + - link: https://www.lbl.gov/ |
| 20 | + image: https://www.lbl.gov/wp-content/uploads/2022/09/berkeley-logo.svg |
| 21 | + - link: https://ornl.gov/ |
| 22 | + image: https://www.hpcwire.com/wp-content/uploads/2019/07/ORNL-OLCF-logo-700x.jpg |
| 23 | + - link: https://www.cscs.ch/ |
| 24 | + image: https://upload.wikimedia.org/wikipedia/commons/3/3a/Logo_of_the_Swiss_National_Supercomputing_Centre_CSCS.jpg |
| 25 | +--- |
| 26 | + |
| 27 | +<p> |
| 28 | + As High-Performance Computing (HPC) enters the Exascale era, the scientific community faces a critical data paradox: |
| 29 | + our ability to generate data via high-fidelity simulations and next-generation instruments vastly outpaces our |
| 30 | + ability to store it. In many domains—from climate modeling and cosmology to experimental physics—simulation outputs |
| 31 | + and sensor inputs now frequently reach the Petabyte range per run. The traditional "compute-store-analyze" paradigm, |
| 32 | + which relies on dumping massive datasets to parallel file systems for post-processing, has become an unsustainable |
| 33 | + bottleneck. It introduces unacceptable latency, wastes energy on data movement, and often renders high-frequency |
| 34 | + data analysis practically inaccessible. |
| 35 | +</p> |
| 36 | + |
| 37 | +<p> |
| 38 | + This BoF discusses this challenge by exploring the architectural shift from file-based workflows to high-performance |
| 39 | + streaming. In this new paradigm, storage is treated not as a buffer for post processing once simulations are |
| 40 | + completed, but as a sink for final scientific results, while analysis, visualization, and reduction occur in-transit |
| 41 | + or in-situ. |
| 42 | +</p> |
| 43 | + |
| 44 | +<p> |
| 45 | + The schedule will be split into two sections. First, we will have a session of community-provided lightning talks |
| 46 | + which is intended to captivate and inspire the audience and familiarize them with the challenge at hand. The second |
| 47 | + part will be a "Workflow Challenge Workshop" where participants will be divided into small groups representing |
| 48 | + different stakeholders (facility operators, users, software developers). Each group will receive a real streaming |
| 49 | + workflow scenario and work together to identify technical, policy, and infrastructure requirements for |
| 50 | + implementation. Groups will document their solutions using a shared digital whiteboard, allowing cross-group |
| 51 | + visibility and iteration. |
| 52 | +</p> |
0 commit comments