SciServer: Collaborative Tools for Data-Driven Engineering and Science

Jordan Raddick* and The SciServer Team, Institute for Data Intensive Engineering and Science, Johns Hopkins University


SciServer ( is an IDIES-developed online system for scientific research and education with big data. The system offers access to several Petabyte-scale scientific datasets in fields ranging from astronomy to turbulence to genomics, along with a set of simple but powerful browser-based tools to visualize and analyze those datasets.

SciServer has has been developed over a number of years into a powerful and flexible tool for science researchers. The SciServer Dashboard offers quick access to all SciServer tools and datasets, and to all prior work within the system. All users get a quota for file storage and database storage space – and through the Dashboard, users can manage uploaded data files, and can create collaborative groups to share datasets and scripts publicly or with selected colleagues. SciServer makes it easy to share exactly the right data with exactly the right people.

SciServer’s Compute tool enables online computational analysis of big data through Jupyter notebooks. With only a few clicks, a user can create a customized environment on fast high-memory virtual machines hosted by IDIES, where you can write or upload Python, R, or MATLAB scripts to perform all sorts of data-intensive tasks. Importantly, SciServer allows these scripts to be run in either interactive or batch mode, meaning that even the most demanding data-intensive analyses can be completed with ease. To help scale up research quickly, SciServer Compute features a wealth of “getting started” data files and example notebooks can be adapt to meet researchers needs.

In addition to SciServer’s obvious potential for data-intensive science and engineering research, the system has been used effectively in many educational settings, particularly as data-intensive lab activities for undergraduate science courses. Learners get immediate access to high-performance computing resources with no software to install or configure. A new set of “SciServer Courseware” notebooks creates an environment to manage learning activities by creating shareable user volumes and groups so that students and TAs can access files with appropriate permissions.

SciServer is funded by National Science Foundation award ACI-1261715. For more information, please visit or contact the SciServer Helpdesk at