In this presentation from HTCondor Week 2021, Joao Dorea from the Digital Livestock Lab explains how high-throughput computing is used in the field of animal and dairy sciences. Computer vision systems and sensors collect animal-level phenotypic data on cows to make more optimized decisions about what to do with each animal in terms of health, nutrition, reproduction, and genetics. One challenge of doing this has to do with the sheer size of data that is collected. Processing and storing tens of thousands of images of cows requires significant computational resources.
By utilizing HTCondor through a collaboration with the Center for High Throughput Computing, the Digital Livestock Lab has been able to focus their time and money on the livestock. Specialized to handle computational work that can be split into many pieces and run in parallel, image analysis aligns well with the ideal HTCSS workload. HTCondor allows them to run many jobs and experiments congruently faster, opening the door to larger and larger data sets. Being able to internalize numerous data sets in parallel has allowed the Digital Livestock Lab to gain significant insight into livestock systems, all thanks to HTCondor and collaborations with the faculty at the CHTC!