Automatic Field-of-View Adjustment for a View-Expansive Microscope via LSTM-Based Gaze and Pipette Motion Interpretation

Intracytoplasmic sperm injection (ICSI) is a delicate micromanipulation procedure that requires frequent, manual adjustments to the microscope's field-of-view (FOV) and brightness. These manual interventions interrupt the workflow, increase procedure time, and necessitate extensive training. The authors propose an AI-based system that automatically adjusts the FOV of a view-expansive microscope—a system capable of providing both large-FOV and high-resolution imagery without physical lens changes.

To automate the FOV, the researchers developed a Long Short-Term Memory (LSTM) model that interprets the operator's intent in real-time. The model takes eight input parameters, including the holding pipette's position and velocity, as well as the operator's gaze position. By training the model on data from an experienced ICSI operator, the system learns to anticipate when to zoom in for high-resolution tasks (like injection) and when to zoom out for navigation (like moving oocytes between workspaces).

The researchers evaluated the system by having six novice operators perform ICSI-mimicking tasks using microbeads. The results showed that the automatic FOV adjustment system significantly improved performance, reducing the average task completion time from 60.5 seconds to 48.0 seconds (p < 0.001). Notably, this improvement allowed novice operators to achieve working speeds equivalent to those of an expert operator using a conventional microscope.

Manual microscope adjustments are a significant source of cognitive load and procedural delay in assisted reproductive technologies. By automating these adjustments through implicit intent recognition—using gaze and pipette motion rather than requiring the operator to perform explicit commands—this system reduces the technical barrier to entry for novice operators. This could potentially improve the consistency, reproducibility, and accessibility of ICSI procedures in clinical settings.