Gladier

Next-generation scientific instruments provide unprecedented opportunities for measuring, monitoring, and steering experiments. However, the volumes and velocities of data generated presents new challenges to efficiently organize, analyze, and publish it without overburdening researchers. Indeed, advanced instruments necessitate advanced cyber infrastructure to automate and outsource data management, analysis, and publication. Serial synchrotron crystallography (SSX) is one such data-intensive technique. Here we present the Globus Architecture for Data-Intensive Experimental Research (Gladier) and its application to fixed-target SSX. Gladier enables the rapid development of customized automation flows linking many data services to perform data capture, storage, and analysis solutions for experimental facilities. Globus provides a collection of data services built for science including: Globus Auth, Transfer, Search, Groups, and Flows, and funcX to enable distributed function-as-a-service execution. Globus Services are highly reliable, professionally operated cloud-hosted services that support the work of over 150,000 researchers worldwide as foundational capabilities for scientific applications and workflows; using them greatly reduces the burden on local systems, administrators, and programmers. We describe the application of these services to establish the Kanzus pipeline, integrating Argonne’s Advanced Photon Source (APS) with HPC resources at the Leadership Computing Facility (ALCF) to perform on-demand serial synchrotron crystallography (SSX) experiments, automating capture, analysis, transfer, indexing, storage, and cataloging of data from experiments to enable on-demand analysis of the crystal structure of COVID-19 proteins.