These findings underscore the potential of laser speckle pattern imaging in sensing the neural dynamics related to shape recognition. However, further research is needed to refine classification models, improve sensitivity to non-polygonal shapes, and explore how neural processing of complex visual stimuli influences speckle pattern response. Future work should also investigate the role of higher-order visual processing mechanisms and their effect on the speckle patterns to enhance the robustness and interpretability of this approach.
The blank screen stimuli provide a critical reference point for measuring neural responses and activity in the visual cortex, forming a baseline of visual stimulation. This allows us to isolate and investigate the specific effects of visual stimuli. For example, Olman et al.employed a white screen condition in the diffusion magnetic resonance imaging (MRI) experiments to explore the structural organization of the human visual cortex in individuals with and without visual experience. The findings demonstrated that the visual experience significantly shapes the functional connectivity of the visual cortex. Similarly, used a white screen condition in fMRI experiments to study neural dynamics associated with the visual hallucinations in schizophrenia. They observed abnormal neural responses in the visual cortex during hallucinations compared to the white-screen condition, offering insights into the neural mechanisms underlying these phenomena.
To strengthen our control conditions, we included both a white blank screen and a black (dark) blank screen. Blank black screen baselines are commonly used in vision neuroscience, studies show that V1 neurons often respond differently to dark versus bright stimuli, with OFF responses dominating for dark stimuli and ON responses for bright ones. Research on black versus white blank screen luminance in human psychophysics also shows that blank screen color (black vs. white) can affect masking and scene recognition thresholds, suggesting that baselines are not interchangeable.
In our study, the performance of the model in identifying both white and black blank-screen videos (as shown in Fig. 2) confirms its ability to reliably classify visual stimuli lacking complex features. The addition of black-screen controls helps to minimize, but not fully exclude, potential luminance and back-scatter effects. While these results increase confidence that observed dynamics arise primarily from neural responses rather than display light leakage, laser-off and phantom-head controls remain necessary for definitive validation.
This finding emphasizes the importance of using both white and black blank screen control conditions to elucidate the specific effects of structured visual stimuli on the neural activity captured via speckle imaging.
The manipulation of color and screen position stimuli is crucial for investigating how these visual features are processed and encoded by the visual cortex. Both features play an important role in object recognition, spatial awareness, and scene analysis. For example, researchhas revealed that distinct regions of the visual cortex are activated by different colors, suggesting specialized color-processing mechanisms. Additionally, found that the spatial position of a stimuli influences neural responses, with specific regions of the visual cortex exhibiting sensitivity to stimuli location. We systematically varied the stimuli like color, shape, size, and position (Fig. 3). The results (Fig. 2) demonstrate how these features influenced the model's classification accuracy. These variations provide a window into the neural mechanisms that integrate fundamental aspects of perception, advancing our understanding of visual processing.
Investigating the combined processing of mixed multiple shapes, colors, and positions, enables the study of how the visual cortex integrates complex visual information. It was found that selective neural responses in the visual cortex can arise from specific combinations of shape and color, as well as the spatial relationships between stimuli. The improved classification recall for mixed multi-shape videos (80%, as shown in Table 1) suggests that the model was able to classify better the underlying patterns of integration within these stimuli. The achieved high recall for rectangle videos (98%) and triangle videos (90%) further indicates potential biases or specialized processing of polygonal shapes.
Analyzing the processing of multiple different shapes and positions on the screen provides critical insights into the visual cortex response to naturalistic visual stimuli. Studies highlighted the role of context and spatial relationships in shaping neural responses to complex objects. By systematically presenting combinations of stimuli, our study provides preliminary evidence that the DNN model, despite its simplicity, reflects some aspects of the visual cortex's integration of complex visual scenes. This is particularly evident in its ability to distinguish mixed-shape and white-screen videos with 99% recall, as shown in Table 1, while the model struggled with other categories.
The performance of the speckle-based DNN model across different shapes and subjects has been evaluated using detailed metrics, as shown in Tables 1 and 2, and visualized in Fig. 4. To examine these results to understand the model's strengths and limitations, Fig. 4 presents two heatmaps that compare F1-Scores and Confidence levels across the eight subjects (S01-S08) for various shape classes; circle, multi-circle, multi-rectangle, multi-triangle, mix, rectangle, triangle, and white, including results from both white and black (dark) screen backgrounds. The F1-Score, which balances precision and recall, and the confidence, calculated as their average, provide a clear picture of how well the model performs. The Mix class shows high F1-Scores (0.855-0.920) and confidence levels (0.861-0.921) across all subjects, indicating strong ability to classify complex, multi-shape stimuli. Similarly, multi-rectangle performs well with F1-Scores (0.681-0.770) and confidence (0.718-0.786), suggesting effective recognition of multi-rectangle patterns.
In contrast, circle and multi-circle exhibit very low F1-Scores (up to 0.048) and confidence (up to 0.048), confirming the model's difficulty with circular shapes. This supports the idea that smooth contours may not produce enough distinct speckle dynamics at the V1 level for accurate classification. Intermediate performance was observed for Rectangle and Triangle classes, with F1-Scores ranging between 0.646 and 0.751 and confidence values between 0.668 and 0.768. The White screen condition yielded moderate F1-Scores (0.515-0.625) and confidence values (0.584-0.673). The addition of the Black screen background produced comparable results, with F1-Scores (0.523-0.627) and confidence (0.544-0.641), demonstrating that the model can differentiate between visual absence under dark versus white backgrounds. This confirms that performance is not simply driven by trivial luminance differences or leakage from the display. Nevertheless, these findings cannot substitute for direct physical controls, which will be included in future experimental refinements.
The multi-triangle class presents a challenge, with lower F1-Scores (0.028-0.204) and confidence (0.083-0.248), indicating difficulties in detecting multi-triangle patterns. This could be due to overlapping speckle signals or insufficient training data. Comparing these findings with the aggregated results in Table 1, most classes align well, though circle and multi-circle show some discrepancies, suggesting a need for refined aggregation methods or additional data.
The persistent inability to classify circular stimuli can be attributed to several interrelated factors; First, circular stimuli offer minimal edge-based transitions across frames, unlike polygons, and our SHAP saliency analysis confirms near‑zero temporal importance for circle sequences, whereas polygons show distinct high-SHAP peaks aligned with shape transitions. Second, numerous neurophysiological studies have demonstrated that V1 neurons (including simple, complex, and end‑stopped types) are strongly tuned to oriented edges, corners, and junctions, while responding weakly to smooth or isotropic contours (e.g. complex pattern selectivity in macaque V1; curvature analysis of human visual areas). Third, in low-signal experimental conditions a circle may appear more like a uniform white screen, which is consistent with the frequent misclassification of circle trials as White in the confusion matrix (Fig. 2a). Lastly, each stimulus class had equal trial counts, so class imbalance cannot account for the drop in circle performance.
Importantly, our choice to use a Leave-One-Subject-Out (LOSO) cross-validation protocol was motivated not only by the need to prevent data leakage, but also by the presence of natural inter-subject differences, such as scalp geometry, hair density, and subtle variations in laser-camera alignment. These physiological and experimental variations create meaningful differences in speckle signal characteristics between individuals. By holding out each participant entirely during testing, we ensure that our performance metrics reflect true inter-subject generalization, rather than overfitting to participant-specific signal idiosyncrasies.
While this study provides foundational insights into the visual processing using speckle-pattern AI analysis, there are several avenues for future exploration to extend and enhance the findings.
Future research could explore a more detailed analysis of the interactions between color and shape. By investigating how combinations of specific colors and geometric forms influence neural responses, researchers could gain deeper insights into the specialized processing mechanisms of the visual cortex. For example, examining the role of color contrast, saturation, and shape complexity might uncover nuanced patterns of neural activity that are not yet fully understood.
The use of more advanced machine learning architectures, such as transformers, could significantly improve the classification and analysis of visual stimuli. Transformers, known for their ability to capture long-range dependencies and complex patterns, may offer a more nuanced understanding of how the visual cortex integrates multiple visual features. While not adopted in the current study due to data limitations and overfitting risk, this remains a promising avenue for future work. By leveraging these models, researchers could potentially uncover subtle correlations between neural signals and visual stimuli that simpler models might overlook.
Incorporating temporal dynamics into the analysis, such as investigating how neural responses evolve over time in response to the moving stimuli, could provide valuable insights into the processing of dynamic visual environments. Motion analysis, coupled with speckle-pattern imaging, may help elucidate how the visual cortex integrates spatial and temporal information to construct a coherent perception of the environment.
Expanding the scope of research to include multisensory stimuli, such as visual stimuli combined with auditory or tactile inputs, could offer a broader perspective on how the brain integrates information across different sensory modalities. Understanding how the visual cortex interacts with other sensory regions may shed light on the complex interplay of neural networks during perception.
Taken together, these research findings suggest that our speckle‑based DNN approach inherently favours edge-derived neural encoding over smooth contour processing; circular shapes likely fail to produce sufficiently distinctive speckle dynamics at the V1 level. Future studies could explore radial‑frequency distorted circles or stimuli with higher curvature to probe whether they elicit detectable speckle signals, or whether higher visual areas (e.g., V4 curvature domains) would be necessary for accurate classification.
Incorporating more complex visual stimuli, such as natural scenes, textures, or three-dimensional shapes, could reveal how the visual cortex processes real-world stimuli. These experiments would bridge the gap between controlled laboratory tasks and naturalistic visual experiences, offering a more comprehensive understanding of neural processing.
To further affirm the robustness of the experimental setup and address potential concerns regarding signal contamination, future investigations should include additional control conditions. Specifically, these may encompass: (1) laser-off trials (with the screen active but the laser deactivated) to verify that ambient or screen light does not influence the speckle signal; and (2) phantom-head tests (employing a rubber or foam model approximating human head geometry) to exclude optical leakage from the monitor. These controls would reinforce that the observed speckle patterns originate exclusively from coherent backscattered dynamics within biological tissue, thereby enhancing the methodological reliability as outlined in the current study.
By pursuing these directions, future research could deepen our understanding of the visual cortex's function, enhance the capabilities of the AI-powered speckle-pattern analysis, and expand its applications in both fundamental neuroscience and clinical practice.