The Challenge: Finding Fish in a Sea of Distractions

Schools of fish swimming near the ocean surface can sometimes be identified by subtle changes in water color, surface texture, or wave patterns. Unfortunately, many other features produce similar signatures. Sun glint, whitecaps, vessel wakes, shadows, and fish aggregating devices (FADs) can all create patterns that complicate detection.

During a September 2026 survey off the coast of Keelung, Taiwan, the research team collected 394 aerial photographs and 4K video at flight altitudes of 20, 60, and 90 meters. Their objective was not simply to detect fish schools, but to determine which flight conditions and image characteristics could reliably support future automated screening.

The team also faced an important limitation: they had no confirmed images of fish schools available for training a detection model. Rather than relying on deep learning, the researchers developed an alternative strategy using ENVI and IDL's core image analysis and programming capabilities.

An Innovative Approach: Detect What Isn’t a School of Fish

Instead of asking the software to recognize fish directly, the researchers reversed the problem: What if they could first identify and eliminate everything that was clearly not a school of fish? Using ENVI and IDL, the team developed an elimination-based workflow that systematically identifies and masks known objects, including vessels, FADs, tethers, wakes, and associated shadows.

The remaining imagery is then evaluated for unusual surface patterns that could warrant further investigation.

• Image processing and masking: Identify and exclude known objects and their shadows from further analysis.

• Sea-surface characterization: Distinguish normal ocean texture and periodic wave patterns from potentially meaningful anomalies.

• Image quality assessment: Evaluate which areas of an image can be reliably analyzed despite sun glint or motion blur.

• Automated analysis: Use IDL to execute repeatable workflows and integrate image processing techniques.

The result is a structured screening process designed to narrow the search area and help researchers focus their attention on the most promising observations. Importantly, the workflow does not claim to identify schools of fish automatically at this stage. Instead, it produces candidates for expert review, providing a foundation for future detection capabilities.

“The sea surface is full of things that look more interesting than fish. Building the workflow in ENVI and IDL let us systematically rule those out. It showed us that the camera settings matter as much as the flight plan.”

Dr. Kuo-Wei Yen, Associate Researcher
Fisheries Research Institute, Ministry of Agriculture, Taiwan

Building a Flexible, Reproducible Workflow with ENVI and IDL

Another noteworthy aspect of the project is its emphasis on automation and reproducibility. The workflow runs without a graphical interface through IDL, allowing the researchers to automate processing while maintaining control over the methods and parameters used.

The team is also exploring additional capabilities in ENVI, including image-to-image registration, RX anomaly detection, and Random Forest classification, to improve the analysis of overlapping images and known object classes. Future work will investigate whether features that persist across multiple frames can help distinguish potential schools of fish from temporary surface disturbances.

This flexibility highlights an important advantage of combining ENVI's image analysis capabilities with IDL's scientific programming environment: researchers can develop customized workflows, test new methods, and adapt their analysis as new data becomes available.

From Drone Imagery to Better Marine Intelligence

Although the research is still evolving, it demonstrates how thoughtful image analysis can help address complex environmental monitoring challenges. By systematically eliminating known objects, assessing image quality, and investigating potential anomalies, the researchers are establishing a practical foundation for more effective drone-based fisheries monitoring.

The broader implications extend beyond fisheries research. Similar approaches could support marine environmental monitoring, coastal surveillance, floating debris detection, and other applications where analysts must identify subtle features against complex and dynamic backgrounds.

Click here to download the pdf.