A novel data storage project unshackles microscopy from data overload

In Jan Huisken’s zebrafish imaging lab, it takes a lot of data to capture a tiny fish.

Assembling a time-lapse video of the first 24 hours of development of a zebrafish — from a ball of cells to a fully-formed embryo with head, heart, spine and fins — has Huisken’s team cranking out terabytes of imaging data. For perspective, storing 10 terabytes of data would take about 7.3 million floppy disks, or 15,000 CD-ROM disks.

Huisken, director of medical engineering at the Morgridge Institute for Research, says data bottlenecks have become the greatest limiting factor in his research. Huisken is co-inventor of light-sheet microscopy, a richly detailed but data-intensive imaging technique that can max out a desktop computer’s storage in a single experiment.

To combat this challenge, Huisken teamed this summer with the Morgridge Institute computational technology team and technicians from storage platform company Dell EMC to create an end-run around the bottleneck. The team installed a storage system that creates a new intermediate 100-terabyte storage platform that will collect data straight from its origins at the microscope, without the limiting factor of the desktop computer. The system also provides another 500 terabytes of long-term storage.

“The revolutionary aspect here is that the microscope itself is basically computer-free, and storage-free, and all the data is now being centrally stored and processed,” Huisken says. “We accomplished this by producing an extremely fast, fiber-optic network between the microscopes and this new working storage system.”

Light-sheet microscopy allows a scientist to study a specimen without injuring or killing it, helping capture developmental biology in its truest state. Its ultrafast cameras can produce 850 megabytes of data per second, and Huisken frequently has a stable of 6-8 microscopes operating in parallel.

Now, rather than having to clear out individual computers of data and transfer to back-up storage, all of the data across multiple microscopes is immediately available for processing within the working storage — allowing the lab to grab the important stuff and delete the rest.

Huisken says the benefit is more than just efficiency; it has opened his science to entirely new possibilities.

“With this almost unlimited storage, our experiments can run much longer at a time, we can do more high-throughput experiments, and consider much larger projects,” he says.

“With this almost unlimited storage, our experiments can run much longer at a time, we can do more high-throughput experiments, and consider much larger projects.”
Jan Huisken

It might even widen his research targets. One big reason zebrafish and fruit flies make such good research models is they develop at a rapid clip, giving scientists a reasonable window to capture the biology. But if Huisken can start thinking about running experiments across days and even weeks, rather than just hours, he may be able to image larger and more complex organisms.

“It definitely could allow us to go out and look at some of the more unexplored territory in the animal kingdom, as well as in plant sciences,” he says.

Derek Cooper, Morgridge IT technical manager, says the project was valuable both for his team and for Dell. They were installing a new type of storage technology that was still in beta stage, making the installation itself a proof of concept that the new approach would benefit the Huisken lab. Cooper adds that the system is quickly and efficiently scalable in a way that will help avoid the inevitable future data bottlenecks.

Huisken says this new project could prove highly influential to other research imaging labs as well as companies — all of whom are producing more data than they have the capacity to use.

“We see a lot of microscope companies and camera manufacturers releasing increasingly powerful and fancy tools,” he says. “We see labs spend more than a half-million dollars on a microscope, run experiments for two weeks, use up their entire storage and have no idea what to do next. I would say it’s the biggest challenge we have these days in microscopy.”