True server-side volumetric 3D animations created remotely with 3Dscript using a natural-language-based syntax

Abstract number
19
Presentation Form
Oral
DOI
10.22443/rms.elmi2021.19
Corresponding Email
[email protected]
Session
Image Data Analysis, Management and Visualisation
Authors
Benjamin Schmid (1), Philipp Tripal (1), Zoltán Winter (1), Ralph Palmisano (1)
Affiliations
1. Optical Imaging Centre Erlangen, University of Erlangen-Nuremberg, Erlangen, Germany.
Keywords

3D Visualization

Animation

Rendering

Server

Abstract text

3D animations have become standard in presenting multidimensional microscopy data, nevertheless, creating them requires a high-end workstation with dedicated graphics hardware. In times where remote work has become more and more important, 3D animations still need to be created on-site, as the only alternative is screen-sharing, which oftentimes lacks the required responsiveness. There is no software that runs as a service on a shared workstation or server which receives rendering jobs and produces 3D animations server-side. The reason for this gap is the way existing software creates these animations: It is based on keyframes that need to be set up by the client, which itself requires the rendering engine and therefore the corresponding hardware.

 

Recently we have introduced 3Dscript1 where animations are described and specified by sentences written in natural language (“From frame 0 to frame 100 rotate by 360 degrees horizontally”). In 3Dscript, keyframes are replaced by pure text, which can easily be composed in any text editor without the need of special graphics hardware. 3Dscript therefore makes it finally possible to implement a client-server architecture to render large 3D volumetric data server-side.

 

Other software that offer a client-server architecture for 3D visualization are ClearVolume2, for remotely observing in real-time the data as it is acquired on a volumetric microscope; FPBioimage3, for 3D visualization of biomedical image volumes cross-platform from within a web browser; and BigDataViewer4, for visualizing arbitrarily-sized datasets. These applications, however, load the input datasets from the server, while rendering itself is performed on the client.

 

Here, we present an implementation of a 3D animation server, 3Dscript.server, written as a plugin for Fiji/ImageJ5–7, and using 3Dscript as the underlying rendering framework. The server software runs both as a normal plugin or in headless mode on dedicated servers without user interface. Once running, it accepts rendering jobs, which contain information about the input data source and a 3Dscript animation text. 3Dscript.server loads input data either from an OMERO8 server or from a shared file system. Submitted jobs are collected in a queue and processed sequentially, as a single job typically requires most of the available hardware resources. 3Dscript.server offers a single-click option to start automatically at boot time, so that a running workstation is at any time ready to accept jobs.

 

To provide an interface that is most accessible to microscopists, we integrate 3Dscript.server with the popular image management software OMERO8. We present an OMERO web app, OMERO.3Dscript, that allows users to directly create 3D renderings of their data from the OMERO web interface, without the need to install additional software. OMERO.3Dscript offers a well-arranged web frontend, including a dedicated auto-completion enabled text area to enter the 3Dscript animation text, a progress bar which is updated continuously by polling the server, and a video preview visualizing the resulting animation. Not only single images, but entire collections can be rendered at once. Rendering results are attached to the original OMERO images as compressed video files. The web interface is optimized for desktop computers and mobile devices alike. As the animation text is written in natural language, it can even be entered conveniently using speech recognition on platforms without a physical keyboard.

 

Additionally, we present a client, 3Dscript.client, that is implemented as an ImageJ/Fiji plugin. It can be called both from the user interface and from macros, which makes it ideally suited for integration into more comprehensive image processing and analysis workflows. In 3Dscript.client, users provide information about the input data source and the 3Dscript animation text. Additionally, they need to specify the host names or IP addresses of workstations running 3Dscript.server. Servers connected to the same local subnet can conveniently be detected automatically by 3Dscript.client.


Finally, we show that 3Dscript.server also runs on compute clusters. We provide dedicated scripts for submitting rendering tasks to the Torque job management system (Adaptive Computing Inc.), to render different time points of time-lapse data or different datasets of image collections in parallel.


With 3Dscript.server, it is no longer required to sit in front of a high-end workstation, but high-quality scientific 3D animations can be created “on-the-go”, e.g., on a tablet PC while discussing a poster on a conference.


References

1.         Schmid, B. et al. 3Dscript: animating 3D/4D microscopy data using a natural-language-based syntax. Nat. Methods 16, 278–280 (2019).

2.         Royer, L. A. et al. ClearVolume: open-source live 3D visualization for light-sheet microscopy. Nat. Methods 12, 480–481 (2015).

3.         Fantham, M. & Kaminski, C. F. A new online tool for visualization of volumetric data. Nat. Photonics 11, 69 (2017).

4.         Pietzsch, T., Saalfeld, S., Preibisch, S. & Tomancak, P. BigDataViewer: visualization and processing for large image data sets. Nat. Methods 12, 481–483 (2015).

5.         Rueden, C. T. et al. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinformatics 18, 529 (2017).

6.         Schindelin, J. et al. Fiji: An open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

7.         Schneider, C. A., Rasband, W. S. & Eliceiri, K. W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012).

8.         Allan, C. et al. OMERO: Flexible, model-driven data management for experimental biology. Nat. Methods 9, 245–253 (2012).