Labeled temperate hardwood tree stomatal image datasets from seven taxa of Populus and 17 hardwood species
Wang, Jiaxin; Renninger, Heidi J.; Ma, Qin
Abstract:
Machine learning (ML) algorithms have shown potential in automatically detecting and measuring stomata. However, ML algorithms require substantial data to efficiently train and optimize models, but their potential is restricted by the limited availability and quality of stomatal images. To overcome this obstacle, we have compiled a collection of around 11,000 unique images of temperate broadleaf angiosperm tree leaf stomata from various projects conducted between 2015 and 2022. The dataset includes over 7,000 images of 17 commonly encountered hardwood species, such as oak, maple, ash, elm, and hickory, and over 3,000 images of 55 genotypes from seven Populus taxa. Inner_guard_cell_walls and whole_stomata (stomatal aperture and guard cells) were labeled and had a corresponding YOLO label file that can be converted into other annotation formats. With the use of our dataset, users can (1) employ state-of-the-art machine learning models to identify, count, and quantify leaf stomata; (2) explore the diverse range of stomatal characteristics across different types of hardwood trees; and (3) develop new indices for measuring stomata.
Species: Hardwood, Poplar
Network: N/A
Annotation: N/A
Outputs: Dataset
2023
Rotating Stomata Measurement Based on Anchor-Free Object Detection and Stomata Conductance Calculation
Zhang, Fan; Wang, Bo; Lu, Fuhao; Zhang, Xinhong
Abstract:
Stomata play an essential role in regulating water and carbon dioxide levels in plant leaves, which is important for photosynthesis. Previous deep learning-based plant stomata detection methods are based on horizontal detection. The detection anchor boxes of deep learning model are horizontal, while the angle of stomata is randomized, so it is not possible to calculate stomata traits directly from the detection anchor boxes. Additional processing of image (e.g., rotating image) is required before detecting stomata and calculating stomata traits. This paper proposes a novel approach, named DeepRSD (deep learning-based rotating stomata detection), for detecting rotating stomata and calculating stomata basic traits at the same time. Simultaneously, the stomata conductance loss function is introduced in the DeepRSD model training, which improves the efficiency of stomata detection and conductance calculation. The experimental results demonstrate that the DeepRSD model reaches 94.3% recognition accuracy for stomata of maize leaf. The proposed method can help researchers conduct large-scale studies on stomata morphology, structure, and stomata conductance models.
Species: Lettuce, Maize
Network: Custom CNN
Annotation: Custom
Outputs: gs, stoma width, stoma length
2023
Automated plant species identification from the stomata images using deep neural network: A study of selected mangrove and freshwater swamp forest tree species of Bangladesh
Stomatal traits of leaves are critical for regulating the exchange of gases between plant tissues and the atmosphere, and thus play a crucial role in the physiological activities of plants. The hypothesis of this study is that distinct stomatal features among different species grown in diverse habitats can serve as a potential marker for species identification. Leaf samples were collected from the mangrove forests of Sundarbans and the freshwater swamp forests of Ratargul in Bangladesh. In total, we examined 11 species from eight different families. We used deep convolutional neural network (DCNN) to automatically identify tree species from microscopic stomatal imprints, as there is currently no established protocol for this task. For model training, 80% (866 images) of the data was used for training the models. Our study observed significant variations in stomatal attributes such as length, width, and density among different species, families, and habitats. These variations could help in accurate species identification by machine learning approaches used in the present study. An empirical comparison was conducted among EfficientNetV2, Xception, VGG16, VGG19, MobileNetV2, ResNet50V2, Resnet152, DenseNet201, and NasNetLarge. We propose a novel approach called the “Normalized Leverage Factor” that utilizes accuracy, precision, recall, and f1-score to select the optimal model. This approach eliminates the non-uniformity of the scores. Although MobileNetV2 achieved an accuracy of 99.06%, our findings indicate that EfficientNetV2 is the optimal model for species identification. This is due to its higher normalized leverage factor (1.92) compared to MobileNetV2 (1.88). The findings demonstrate that plants of diverse habitats show a unique footprint of stomata that offers an innovative method of species identification using DCNN. The study would help to develop a stomatal image-based user interface to identify species even without expert taxonomic knowledge and could be particularly useful in fields such as pharmacology, conservation biology, forestry, and environmental science.
Automated estimation of stomatal number and aperture in haskap (Lonicera caerulea L.)
Meng, Xiangji; Nakano, Arisa; Hoshino, Yoichiro
Abstract:
Main conclusion: This study developed the reliable Mask R-CNN model to detect stomata in Lonicera caerulea. The obtained data could be utilized for evaluating some characters such as stomatal number and aperture area. Abstract: The native distribution of haskap (Lonicera caerulea L.), a small-shrub species, extends through Northern Eurasia, Japan, and North America. Stomatal observation is important for plant research to evaluate the physiological status and to investigate the effect of ploidy levels on phenotypes. However, manual annotation of stomata using microscope software or ImageJ is time consuming. Therefore, an efficient method to phenotype stomata is needed. In this study, we used the Mask Regional Convolutional Neural Network (Mask R-CNN), a deep learning model, to analyze the stomata of haskap efficiently and accurately. We analyzed haskap plants (dwarf and giant phenotypes) with the same ploidy but different phenotypes, including leaf area, stomatal aperture area, stomatal density, and total number of stomata. The R-square value of the estimated stomatal aperture area was 0.92 and 0.93 for the dwarf and giant plants, respectively. The R-square value of the estimated stomatal number was 0.99 and 0.98 for the two phenotypes. The results showed that the measurements obtained using the models were as accurate as the manual measurements. Statistical analysis revealed that the stomatal density of the dwarf plants was higher than that of the giant plants, but the maximum stomatal aperture area, average stomatal aperture area, total number of stomata, and average leaf area were lower than those of the giant plants. A high-precision, rapid, and large-scale detection method was developed by training the Mask R-CNN model. This model can help save time and increase the volume of data.
Species: Haskap
Network: Mask R-CNN
Annotation: Semantic
Outputs: Stomata count, stomata density, pore area
2023
RotatedStomataNet: a deep rotated object detection network for directional stomata phenotype analysis
Stomata act as a pathway for air and water vapor during respiration, transpiration and other gas metabolism, so the stomata phenotype is important for plant growth and development. Intelligent detection of high throughput stoma is a key issue. However, current existing methods usually suffer from detection error or cumbersome operations when facing densely and unevenly arranged stomata. The proposed RotatedStomataNet innovatively regards stomata detection as rotated object detection, enabling an end-to-end, real-time and intelligent phenotype analysis of stomata and apertures. The system is constructed based on the Arabidopsis and maize stomatal data sets acquired in a destructive way, and the maize stomatal data set acquired in a nondestructive way, enabling one-stop automatic collection of phenotypic such as the location, density, length and width of stomata and apertures without step-by-step operations. The accuracy of this system to acquire stomata and apertures has been well demonstrated in monocotyledon and dicotyledon, such as Arabidopsis, soybean, wheat, and maize. And the experimental results showed that the prediction results of the method are consistent with those of manual labeled. The test sets, system code, and its usage are also given (https://github.com/AITAhenu/RotatedStomataNet).
The quantification of stomatal pore size has long been a fundamental approach to understand the physiological response of plants in the context of environmental adaptation. Automation of such methodologies not only alleviates human labor and bias but also realizes new experimental research methods through massive analysis. Here, we present an image analysis pipeline that automatically quantifies stomatal aperture of Arabidopsis thaliana leaves from bright-field microscopy images containing mesophyll tissue as noisy backgrounds. By combining a You Only Look Once X–based stomatal detection submodule and a U-Net-based pore segmentation submodule, we achieved a mean average precision with an intersection of union (IoU) threshold of 50% value of 0.875 (stomata detection performance) and an IoU of 0.745 (pore segmentation performance) against images of leaf discs taken with a bright-field microscope. Moreover, we designed a portable imaging device that allows easy acquisition of stomatal images from detached/undetached intact leaves on-site. We demonstrated that this device in combination with fine-tuned models of the pipeline we generated here provides robust measurements that can substitute for manual measurement of stomatal responses against pathogen inoculation. Utilization of our hardware and pipeline for automated stomatal aperture measurements is expected to accelerate research on stomatal biology of model dicots.
Species: Arabidopsis
Network: U-Net
Annotation: Semantic
Outputs: Stomata count, pore area
2023
StoManager1: An Enhanced, Automated, and High-throughput Tool to Measure Leaf Stomata and Guard Cell Metrics Using Empirical and Theoretical Algorithms
Automated stomata detection and measuring are vital for understanding plant physiological performance and ecological functioning in global water and carbon cycles. Current methods are laborious, time-consuming, prone to bias, and limited in scale. We developed StoManager1, a high-throughput tool utilizing empirical and theoretical algorithms and convolutional neural networks to automatically detect, count, and measure over 30 stomatal and guard cell metrics, including stomata and guard cell area, length, width, and orientation, stomatal evenness, divergence, and aggregation index. These metrics, combined with leaf functional traits, explained 78% and 93% of productivity and intrinsic water use efficiency (iWUE) variances in hardwoods, making them significant factors in leaf physiology and tree growth. StoManager1 demonstrates exceptional precision and recall (mAP@0.5 over 0.993), effectively capturing diverse stomatal properties across various species. StoManager1facilitates the automation of measuring leaf stomata, enabling broader exploration of stomatal control in plant growth and adaptation to environmental stress and climate change. This has implications for global gross primary productivity (GPP) modeling and estimation, as integrating stomatal metrics can enhance comprehension and predictions of plant growth and resource usage worldwide. StoManager1's source code and an online demonstration are available on GitHub (https://github.com/JiaxinWang123/StoManager.git), along with a user-friendly Windows application on Zenodo (https://doi.org/10.5281/zenodo.7686022).
An automatic plant leaf stoma detection method based on YOLOv5
Li, Xin; Guo, Siyu; Gong, Linrui; Lan, Yuan
Abstract:
The stomata on the leaf surface are mainly responsible for the material exchange between the internal and external environments of the plant, a large number of methods have been proposed to automatically measure the distribution position and number of stomatal, but few methods could achieve both stomatal count and open/closed-state judgment. Therefore, this study proposes an automatic detection method for leaf stomatal morphology analysis based on an attention mechanism and deep learning. In order to obtain more stomatal feature information and send it to the network for learning, the proposed method adds a coordinate attention (CA) mechanism to the YOLOV5 backbone part. At the same time, in order to avoid the overfitting of the model during the training process, the authors added the training trick of label smoothing. Finally, the detection ability of the proposed method for stomata is verified on the broad bean leaves stomata dataset. The experimental results show that our method achieves a detection accuracy of 0.934 and an mAP of 0.968. By comparing with other state-of-the-art algorithms, the detection capability of our method has been significantly improved. The generalization of the model is verified on the wheat leaf stomatal dataset. The experimental results show that our method can achieve a detection accuracy of 0.894 and an mAP of 0.907.
Rapid non-destructive method to phenotype stomatal traits
Pathoumthong, Phetdalaphone; Zhang, Zhen; Roy, Stuart J.; El Habti, Abdeljalil
Abstract:
Background: Stomata are tiny pores on the leaf surface that are central to gas exchange. Stomatal number, size and aperture are key determinants of plant transpiration and photosynthesis, and variation in these traits can affect plant growth and productivity. Current methods to screen for stomatal phenotypes are tedious and not high throughput. This impedes research on stomatal biology and hinders efforts to develop resilient crops with optimised stomatal patterning. We have developed a rapid non-destructive method to phenotype stomatal traits in three crop species: wheat, rice and tomato. Results: The method consists of two steps. The first is the non-destructive capture of images of the leaf surface from plants in their growing environment using a handheld microscope; a process that only takes a few seconds compared to minutes for other methods. The second is to analyse stomatal features using a machine learning model that automatically detects, counts and measures stomatal number, size and aperture. The accuracy of the machine learning model in detecting stomata ranged from 88 to 99%, depending on the species, with a high correlation between measures of number, size and aperture using the machine learning models and by measuring them manually. The rapid method was applied to quickly identify contrasting stomatal phenotypes. Conclusions: We developed a method that combines rapid non-destructive imaging of leaf surfaces with automated image analysis. The method provides accurate data on stomatal features while significantly reducing time for data acquisition and analysis. It can be readily used to phenotype stomata in large populations in the field and in controlled environments.
Species: Rice, Sundarbans, Tomato, Wheat
Network: YOLO
Annotation: Bounding Box
Outputs: Stomata density, pore area
2023
StomataTracker: Revealing circadian rhythms of wheat stomata with in-situ video and deep learning
Plant stomata are essential channels for gas exchange between plants and the environment. The infrared gas-exchange system has greatly accelerated the studies of stomatal conductance (gs). Nevertheless, due to the lack of in-situ monitoring techniques, the behavior of stomata themselves remains poorly understood, especially in nocturnal environmental conditions. Here, a deep-learning-based stoma tracking pipeline (StomataTracker) was first proposed to continuously monitor stoma traits from unprecedentedly long-term, continuous, and non-destructive video data. Compared to the semi-automatic method (ImageJ), the open-source StomataTracker could greatly improve the extraction efficiency from 207 s to 1.47 s of stomatal traits, including stomatal area, perimeter, length, and width. The R2 adjusted of the four stomatal traits ranged from 0.620 to 0.752. In addition, the rhythm of wheat stomata opening in a completely dark environment was first reported from long-term video data. The closed time of stoma at night was negatively correlated with stomatal traits, and the R ranged from −0.583 to −0.855. The heterogeneity of stomatal behavior also highlighted that smaller stomata have the rhythm pattern of longer closure time at night. Overall, our study provides a novel perspective for stomatal study, and it is conducive to accelerating the application of stomatal circadian rhythm in wheat breeding.
Species: Wheat
Network: U-Net
Annotation: Semantic
Outputs: Stoma width, stoma length, stoma area
2022
Automatic stomata recognition and measurement based on improved YOLO deep learning model and entropy rate superpixel algorithm
The traditional methods of analyzing stomatal traits are mostly manual observation and measurement. These methods are time-consuming, labor-intensive, and inefficient. Some methods have been proposed for the automatic recognition and counting of stomata, however most of those methods could not complete the automatic measurement of stomata parameters at the same time. Some non-deep learning methods could automatically measure the parameters of stomata, but they could not complete the automatic recognition and detection of stomata. In this paper, a deep learning-based method was proposed for automatically identifying, counting and measuring stomata of maize (Zea mays L.) leaves at the same time. An improved YOLO (You Only Look Once) deep learning model was proposed to identify stomata of maize leaves automatically, and an entropy rate superpixel algorithm was used for the accurate measurement of stomatal parameters. According to the characteristics of the stomata images data set, the network structure of YOLOv5 was modified, which greatly reduced the training time without affecting the recognition performance. The predictor in YOLO deep learning model was optimized, which reduced the false detection rate. At the same time, the 16-fold and 32-fold down-sampling layers were simplified according to the characteristics of stomatal objects, which improved the recognition efficiency. Experimental results showed that the recognition precision of the improved YOLO deep learning model reached 95.3% on the maize leaves stomatal data set, and the average accuracy of parameter measurement reached 90%. The proposed method could fully automatically complete the recognition, counting and measurement of stomata of plants, which can help agricultural scientists and botanists to conduct large-scale researches of stomatal morphology, structure and physiology, as well as the researches combined with genetic analysis or molecular-level analysis.
Species: Lettuce, Maize
Network: YOLO
Annotation: Bounding Box
Outputs: Stomata count, stoma width, stoma length, stoma area
2022
LeafNet: a tool for segmenting and quantifying stomata and pavement cells
Stomata play important roles in gas and water exchange in leaves. The morphological features of stomata and pavement cells are highly plastic and are regulated during development. However, it is very laborious and time-consuming to collect accurate quantitative data from the leaf surface by manual phenotyping. Here, we introduce LeafNet, a tool that automatically localizes stomata, segments pavement cells (to prepare them for quantification), and reports multiple morphological parameters for a variety of leaf epidermal images, especially bright-field microscopy images. LeafNet employs a hierarchical strategy to identify stomata using a deep convolutional network and then segments pavement cells on stomata-masked images using a region merging method. LeafNet achieved promising performance on test images for quantifying different phenotypes of individual stomata and pavement cells compared with six currently available tools, including StomataCounter, Cellpose, PlantSeg, and PaCeQuant. LeafNet shows great flexibility, and we improved its ability to analyze bright-field images from a broad range of species as well as confocal images using transfer learning. Large-scale images of leaves can be efficiently processed in batch mode and interactively inspected with a graphic user interface or a web server (https://leafnet.whu.edu.cn/). The functionalities of LeafNet could easily be extended and will enhance the efficiency and productivity of leaf phenotyping for many plant biologists.
Species:
Network:
Annotation:
Outputs:
2022
StomataScorer: a portable and high-throughput leaf stomata trait scorer combined with deep learning and an improved CV model
To measure stomatal traits automatically and nondestructively, a new method for detecting stomata and extracting stomatal traits was proposed. Two portable microscopes with different resolutions (TipScope with a 40× lens attached to a smartphone and ProScope HR2 with a 400× lens) are used to acquire images of living stomata in maize leaves. FPN model was used to detect stomata in the TipScope images and measure the stomata number and stomatal density. Faster RCNN model was used to detect opening and closing stomata in the ProScope HR2 images, and the number of opening and closing stomata was measured. An improved CV model was used to segment pores of opening stomata, and a total of 6 pore traits were measured. Compared to manual measurements, the square of the correlation coefficient (R2) of the 6 pore traits was higher than 0.85, and the mean absolute percentage error (MAPE) of these traits was 0.02%–6.34%. The dynamic stomata changes between wild-type B73 and mutant Zmfab1a were explored under drought and re-watering condition. The results showed that Zmfab1a had a higher resilience than B73 on leaf stomata. In addition, the proposed method was tested to measure the leaf stomatal traits of other nine species. In conclusion, a portable and low-cost stomata phenotyping method that could accurately and dynamically measure the characteristic parameters of living stomata was developed. An open-access and user-friendly web portal was also developed which has the potential to be used in the stomata phenotyping of large populations in the future.
Species: Maize
Network: R-CNN
Annotation: Bounding Box
Outputs: Pore width, pore length, pore area
2022
Microscopic image recognition method of stomata in living leaves based on improved YOLO-X
Dai, Tianhong; Zhang, Jingzong; Li, Kexin
Abstract:
Stomata are the main medium of water exchange in plants, regulating gas exchange and responsible for the processes of photosynthesis and transpiration. Stomata are surrounded by guard cells and the transpiration rate is controlled by opening and closing stomata. Stomatal state (open and close) plays an important role in describing the health of plants. In addition, counting the number of stomata is of great signicance for scientists to study the number of opening and closeing stomata and to measure their density and distribution on the leaf surface through different sampling techniques. Although some techniques for calculating the number of stomata have been proposed, these methods are used to produce samples in isolation and then to identify and classify the states in the sample leaves. We improved YOLO-X and then implemented a transfer learning method to count the number of stomata and identify the stomatal opening and closing status of live black poplar leaves. In the end, the average accuracy and recall of the method were 98.3% and 95.9%, which helped researchers to obtain accurate information on leaf stomatal opening and closing status in an ecient and simple way.
Species: Poplar
Network: YOLO
Annotation: Bounding Box
Outputs: Stomata count, pore classification
2022
SAI: Fast and automated quantification of stomatal parameters on microscope images
Sai, Na; Bockman, James Paul; Chen, Hao; Watson-Haigh, Nathan; Xu, Bo; Feng, Xueying; Piechatzek, Adriane; Shen, Chunhua; Gilliham, Matthew
Abstract:
Using microscopy to investigate stomatal behaviour is a common technique in plant physiology research. Manual inspection and measurement of stomatal features is a low throughput process in terms of time and human effort, which relies on expert knowledge to identify and measure stomata accurately. This process represents a significant bottleneck in research pipelines, adding significant researcher time to any project that requires it. To alleviate this, we introduce StomaAI (SAI): a reliable and user-friendly tool that measures stomata of the model plant Arabidopsis (dicot) and the crop plant barley (monocot grass) via the application of deep computer vision. We evaluated the reliability of predicted measurements: SAI is capable of producing measurements consistent with human experts and successfully reproduced conclusions of published datasets. Hence, SAI boosts the number of images that biologists can evaluate in a fraction of the time so is capable of obtaining more accurate and representative results.
### Competing Interest Statement
The authors have declared no competing interest.
Species: Arabidopsis, Barley
Network: Mask R-CNN
Annotation: Semantic
Outputs: Pore width, pore length, pore area
2021
Optical topometry and machine learning to rapidly phenotype stomatal patterning traits for maize QTL mapping
Xie, Jiayang; Fernandes, Samuel B.; Mayfield-Jones, Dustin; Erice, Gorka; Choi, Min; Lipka, Alexander E.; Leakey, Andrew D.B.
Abstract:
Stomata are adjustable pores on leaf surfaces that regulate the tradeoff of CO2 uptake with water vapor loss, thus having critical roles in controlling photosynthetic carbon gain and plant water use. The lack of easy, rapid methods for phenotyping epidermal cell traits have limited discoveries about the genetic basis of stomatal patterning. A high-throughput epidermal cell phenotyping pipeline is presented here and used for quantitative trait loci (QTL) mapping in field-grown maize (Zea mays). The locations and sizes of stomatal complexes and pavement cells on images acquired by an optical topometer from mature leaves were automatically determined. Computer estimated stomatal complex density (SCD; R2 0.97) and stomatal complex area (SCA; R2 0.71) were strongly correlated with human measurements. Leaf gas exchange traits were genetically correlated with the dimensions and proportions of stomatal complexes (rg 0.39-0.71) but did not correlate with SCD. Heritability of epidermal traits was moderate to high (h2 0.42-0.82) across two field seasons. Thirty-six QTL were consistently identified for a given trait in both years. Twenty-four clusters of overlapping QTL for multiple traits were identified, with univariate versus multivariate single marker analysis providing evidence consistent with pleiotropy in multiple cases. Putative orthologs of genes known to regulate stomatal patterning in Arabidopsis (Arabidopsis thaliana) were located within some, but not all, of these regions. This study demonstrates how discovery of the genetic basis for stomatal patterning can be accelerated in maize, a C4 model species where these processes are poorly understood.
Patchy stomata are a common and characteristic phenomenon in plants. Understanding and studying the regulation mechanism of patchy stomata are of great significance to further supplement and improve the stomatal theory. Currently, the common methods for stomatal behavior observation are based on static images, which makes it difficult to reflect dynamic changes of stomata. With the rapid development of portable microscopes and computer vision algorithms, it brings new chances for stomatal movement observation. In this study, a stomatal behavior observation system (SBOS) was proposed for real-time observation and automatic analysis of each single stoma in wheat leaf using object tracking and semantic segmentation methods. The SBOS includes two modules: the real-time observation module and the automatic analysis module. The real-time observation module can shoot videos of stomatal dynamic changes. In the automatic analysis module, object tracking locates every single stoma accurately to obtain stomatal pictures arranged in time-series; semantic segmentation can precisely quantify the stomatal opening area (SOA), with a mean pixel accuracy (MPA) of 0.8305 and a mean intersection over union (MIoU) of 0.5590 in the testing set. Moreover, we designed a graphical user interface (GUI) so that researchers could use this automatic analysis module smoothly. To verify the performance of the SBOS, the dynamic changes of stomata were observed and analyzed under chilling. Finally, we analyzed the correlation between gas exchange and SOA under drought stress, and the correlation coefficients between mean SOA and net photosynthetic rate (Pn), intercellular CO2 concentration (Ci), stomatal conductance (Gs), and transpiration rate (Tr) are 0.93, 0.96, 0.96, and 0.97.
Species: Wheat
Network: U-Net
Annotation: Semantic
Outputs: Stomata density, stomata count, gs,
2021
Determining leaf stomatal properties in citrus trees utilizing machine vision and artificial intelligence
Identifying and quantifying the number and size of stomata on leaf surfaces is useful for a wide range of plant ecophysiological studies, specifically those related to water-use efficiency of different plant species or agricultural crops. The time-consuming nature of manually counting and measuring stomata have limited the utility of manual methods for large-scale precision agriculture applications. A deep learning segmentation network was developed to automate the analysis of stomatal density and size and to distinguish between open and closed stomata using citrus trees grafted on different rootstocks as a model system. A novel method was developed utilizing the Mask-RCNN algorithm, which allows identification, quantification, and characterization of stomata from leaf epidermal peel microscopic images with an accuracy of up to 99%. Moreover, this method permits the differentiation of open and closed stomata with 98% precision and measurement of individual stomata size. In the citrus model system, significant differences in the size and density of stomata and diurnal regulation patterns were detected that were associated with the rootstock cultivar on which the trees were grafted. Nearly 9000 individual stomata were analyzed, which would have been impractical using manual methods. The novel automated method presented here is not only accurate, but also rapid and low-cost, and can be applied to a variety of crop and non-crop plant species.
Species: Orange
Network: R-CNN
Annotation: Bounding Box
Outputs: Stomata count, stomata density
2021
Deep Transfer Learning-Based Multi-Object Detection for Plant Stomata Phenotypic Traits Intelligent Recognition
Yang, Xiao Hui; Xi, Zi Jun; Li, Jie Ping; Feng, Xin Lei; Zhu, Xiao Hong; Guo, Si Yi; Song, Chun Peng
Abstract:
Plant stomata phenotypic traits can provide a basis for enhancing crop tolerance in adversity. Manually counting the number of stomata and measuring the height and width of stomata obviously cannot satisfy the high-throughput data. How to detect and recognize plant stomata quickly and accurately is the prerequisite and key for studying the physiological characteristics of stomata. In this research, we consider stomata recognition as a multi-object detection problem, and propose an end-to-end framework for intelligent detection and recognition of plant stomata based on feature weights transfer learning and YOLOv4 network. It is easy to operate and greatly facilitates the analysis of stomata phenotypic traits in high-throughput plant epidermal cell images. For different cultivars, multi-scales, rich background features, high density, and small stomata object images, the proposed method can precisely locate multiple stomata in microscope images and automatically give phenotypic traits of stomata. Users can also adjust the corresponding parameters to maximize the accuracy and scalability of automatic stomata detection and recognition. Experimental results on actual data provided by the National Maize Improvement Center show that the proposed method is superior to the existing methods in high stomata automatic detection and recognition accuracy, low training cost, strong generalization ability.
Species: Wheat, Maize
Network: YOLO
Annotation: Bounding Box
Outputs: Stomata count, stoma width, stoma length
2021
Stomata Detector: High-throughput automation of stomata counting in a population of African rice (Oryza glaberrima) using transfer learning
Cowling, Sophie B.; Soltani, Hamidreza; Mayes, Sean; Murchie, Erik H.
Abstract:
Author contributions: SBC project conception, acquisition of stomatal peels and microscopy images, image annotation for machine learning training set, statistical analysis and biological advice for HS. HS generation of automated stomatal counting algorithm and software. EHM and SM project conception, guidance and advice.. CC-BY-NC-ND 4.0 International license perpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for this this version posted December 3, 2021. ; https://doi. Abstract Stomata are dynamic structures that control the gaseous exchange of CO2 from the external to internal environment and water loss through transpiration. The density and morphology of stomata have important consequences in crop productivity and water use efficiency, both are integral considerations when breeding climate change resilient crops. The phenotyping of stomata is a slow manual process and provides a substantial bottleneck when characterising phenotypic and genetic variation for crop improvement. There are currently no open-source methods to automate stomatal counting. We used 380 human annotated micrographs of O. glaberrima and O. sativa at x20 and x40 objectives for testing and training. Training was completed using the transfer learning for deep neural networks method and R-CNN object detection model. At a x40 objective our method was able to accurately detect stomata (n = 540, r = 0.94, p<0.0001), with an overall similarity of 99% between human and automated counting methods. Our method can batch process large files of images. As proof of concept, characterised the stomatal density in a population of 155 O. glaberrima accessions, using 13,100 micrographs. Here, we present developed Stomata Detector; an open source, sophisticated piece of software for the plant science community that can accurately identify stomata in Oryza spp., and potentially other monocot species.. CC-BY-NC-ND 4.0 International license perpetuity. It is made available under a preprint (which was not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in The copyright holder for this this version posted December 3, 2021. ; https://doi.
Species: Rice
Network: R-CNN
Annotation: Bounding Box
Outputs: Stomata count, stomata density
2021
An Affordable Image-Analysis Platform to Accelerate Stomatal Phenotyping During Microscopic Observation
Recent technical advances in the computer-vision domain have facilitated the development of various methods for achieving image-based quantification of stomata-related traits. However, the installation cost of such a system and the difficulties of operating it on-site have been hurdles for experimental biologists. Here, we present a platform that allows real-time stomata detection during microscopic observation. The proposed system consists of a deep neural network model-based stomata detector and an upright microscope connected to a USB camera and a graphics processing unit (GPU)-supported single-board computer. All the hardware components are commercially available at common electronic commerce stores at a reasonable price. Moreover, the machine-learning model is prepared based on freely available cloud services. This approach allows users to set up a phenotyping platform at low cost. As a proof of concept, we trained our model to detect dumbbell-shaped stomata from wheat leaf imprints. Using this platform, we collected a comprehensive range of stomatal phenotypes from wheat leaves. We confirmed notable differences in stomatal density (SD) between adaxial and abaxial surfaces and in stomatal size (SS) between wheat-related species of different ploidy. Utilizing such a platform is expected to accelerate research that involves all aspects of stomata phenotyping.
Stomata are pores in the epidermal tissue of leaf plants formed by specialised cells called guard cells, which regulate the stomatal opening. Stomata facilitate gas exchange, being pivotal in the regulation of processes such as pho-tosynthesis and transpiration. The analysis of the number and behaviour of stomata is a task carried out by studying microscopic images; and, nowadays, this task is mainly conducted manually, or using programs that can count and determine the position of stomata but are not able to determine their morphology. In this paper, we have conducted a study of 10 deep learning algorithms to segment stom-ata from several species. The model that achieves the best Dice score, with a value of 96.06%, is obtained with the DeepLabV3+ algorithm, whereas the model that provides the best trade-off between inference time and Dice score was trained using the ContextNet architecture. This is a first step towards improving the measurements provided by stomata analysis tools, that will in turn help plant biologists to advance their understanding of dynamics in plants.
Species:
Network:
Annotation:
Outputs:
2021
Barley genotypes vary in stomatal responsiveness to light and CO2 conditions
Changes in stomatal conductance and density allow plants to acclimate to changing environmental conditions. In the present paper, the influence of atmospheric CO2 concentration and light intensity on stomata were investigated for two barley genotypes—Barke and Bojos, differing in their sensitivity to oxidative stress and phenolic acid profiles. A novel approach for stomatal density analysis was used—a pair of convolution neural networks were developed to automatically identify and count stomata on epidermal micrographs. Stomatal density in barley was influenced by genotype, as well as by light and CO2 conditions. Low CO2 conditions resulted in increased stomatal density, although differences between ambient and elevated CO2 were not significant. High light intensity increased stomatal density compared to low light intensity in both barley varieties and all CO2 treatments. Changes in stomatal conductance were also measured alongside the accumulation of pentoses, hexoses, disaccharides, and abscisic acid detected by liquid chromatography coupled with mass spectrometry. High light increased the accumulation of all sugars and reduced abscisic acid levels. Abscisic acid was influenced by all factors—light, CO2, and genotype—in combination. Differences were discovered between the two barley varieties: oxidative stress sensitive Barke demonstrated higher stomatal density, but lower conductance and better water use efficiency (WUE) than oxidative stress resistant Bojos at saturating light intensity. Barke also showed greater variability between treatments in measurements of stomatal density, sugar accumulation, and abscisic levels, implying that it may be more responsive to environmental drivers influencing water relations in the plant.
Species: Barley
Network: Custom CNN
Annotation: Custom
Outputs: Stomata density
2021
Optimizing the experimental method for stomata-profiling automation of soybean leaves based on deep learning
Sultana, Syada Nizer; Park, Halim; Choi, Sung Hoon; Jo, Hyun; Song, Jong Tae; Lee, Jeong Dong; Kang, Yang Jae
Abstract:
Stomatal observation and automatic stomatal detection are useful analyses of stomata for taxonomic, biological, physiological, and eco-physiological studies. We present a new clearing method for improved microscopic imaging of stomata in soybean followed by automated stomatal detection by deep learning. We tested eight clearing agent formulations based upon different ethanol and sodium hypochlorite (NaOCl) concentrations in order to improve the transparency in leaves. An optimal formulation—a 1:1 (v/v) mixture of 95% ethanol and NaOCl (6–14%)—produced better quality images of soybean stomata. Additionally, we evaluated fixatives and dehydrating agents and selected absolute ethanol for both fixation and dehydration. This is a good substitute for formaldehyde, which is more toxic to handle. Using imaging data from this clearing method, we developed an automatic stomatal detector using deep learning and improved a deep-learning algorithm that automatically analyzes stomata through an object detection model using YOLO. The YOLO deep-learning model successfully recognized stomata with high mAP (~0.99). A web-based interface is provided to apply the model of stomatal detection for any soybean data that makes use of the new clearing protocol.
Species: Soybean
Network: YOLO
Annotation: Bounding Box
Outputs: Stomata count, stomata density
2021
Automated stomata detection in oil palm with convolutional neural network
Kwong, Qi Bin; Wong, Yick Ching; Lee, Phei Ling; Sahaini, Muhammad Syafiq; Kon, Yee Thung; Kulaveerasingam, Harikrishna; Appleton, David Ross
Abstract:
Stomatal density is an important trait for breeding selection of drought tolerant oil palms; however, its measurement is extremely tedious. To accelerate this process, we developed an automated system. Leaf samples from 128 palms ranging from nursery (1 years old), juvenile (2–3 years old) and mature (> 10 years old) were collected to build an oil palm specific stomata detection model. Micrographs were split into tiles, then used to train a stomata object detection convolutional neural network model through transfer learning. The detection model was then tested on leaf samples acquired from three independent oil palm populations of young seedlings (A), juveniles (B) and productive adults (C). The detection accuracy, measured in precision and recall, was 98.00% and 99.50% for set A, 99.70% and 97.65% for set B, and 99.55% and 99.62% for set C, respectively. The detection model was cross-applied to another set of adult palms using stomata images taken with a different microscope and under different conditions (D), resulting in precision and recall accuracy of 99.72% and 96.88%, respectively. This indicates that the model built generalized well, in addition has high transferability. With the completion of this detection model, stomatal density measurement can be accelerated. This in turn will accelerate the breeding selection for drought tolerance.
Species: Oil Palm
Network: MobileNet
Annotation: Bounding Box
Outputs: Stomata count, stomata density
2021
A generalised approach for high-throughput instance segmentation of stomata in microscope images
Jayakody, Hiranya; Petrie, Paul; Boer, Hugo Jan de; Whitty, Mark
Abstract:
Background: Stomata analysis using microscope imagery provides important insight into plant physiology, health and the surrounding environmental conditions. Plant scientists are now able to conduct automated high-throughput analysis of stomata in microscope data, however, existing detection methods are sensitive to the appearance of stomata in the training images, thereby limiting general applicability. In addition, existing methods only generate bounding-boxes around detected stomata, which require users to implement additional image processing steps to study stomata morphology. In this paper, we develop a fully automated, robust stomata detection algorithm which can also identify individual stomata boundaries regardless of the plant species, sample collection method, imaging technique and magnification level. Results: The proposed solution consists of three stages. First, the input image is pre-processed to remove any colour space biases occurring from different sample collection and imaging techniques. Then, a Mask R-CNN is applied to estimate individual stomata boundaries. The feature pyramid network embedded in the Mask R-CNN is utilised to identify stomata at different scales. Finally, a statistical filter is implemented at the Mask R-CNN output to reduce the number of false positive generated by the network. The algorithm was tested using 16 datasets from 12 sources, containing over 60,000 stomata. For the first time in this domain, the proposed solution was tested against 7 microscope datasets never seen by the algorithm to show the generalisability of the solution. Results indicated that the proposed approach can detect stomata with a precision, recall, and F-score of 95.10%, 83.34%, and 88.61%, respectively. A separate test conducted by comparing estimated stomata boundary values with manually measured data showed that the proposed method has an IoU score of 0.70; a 7% improvement over the bounding-box approach. Conclusions: The proposed method shows robust performance across multiple microscope image datasets of different quality and scale. This generalised stomata detection algorithm allows plant scientists to conduct stomata analysis whilst eliminating the need to re-label and re-train for each new dataset. The open-source code shared with this project can be directly deployed in Google Colab or any other Tensorflow environment.
Species: Poplar, Gingko
Network: Mask R-CNN
Annotation: Semantic
Outputs: Stomata count, stoma area
2021
Deep learning-based high-throughput phenotyping accelerates gene discovery for stomatal traits
All authors contributed equally. Future food security in the face of climate change requires rapid, efficient, and flexible plant genetic improvement. For this, an integrated understanding of developmental and physiological mechanisms from DNA sequences (genotypes) to terminal traits (phenotypes) under different environmental conditions is indispensable. Stomatal traits influencing photosynthesis, gas exchange, and water use are crucial targets for plant improvement programs involving major crop C 4 plants sorghum (Sorghum bicolor) and maize (Zea mays; Leakey et al., 2019). However, the ability to select optimal plant genotypes is still challenged by the pace at which the acquisition and processing of stomatal phenotypic data can be accomplished. Artificial intelligence (AI) is revolutionizing the way in which problems are approached and solved across a wide range of disciplines. Machine learning, an AI subfield, is increasingly being used in agriculture to classify plants, identify pests, predict weather conditions, and track yield, among several other applications (van Dijk et al., 2021). In this issue of Plant Physiology, Bheemanahalli et al. (2021), Ferguson et al. (2021), and Xie et al. (2021) introduce the use of AI-enabled high-throughput stomatal phenotyp-ing platforms in combination with screening methods to identify specific genes and variations controlling stomatal-related traits in sorghum and maize. While considerable attempts have been made to address the bottlenecks associated with the phenotyping of stomatal traits through computer-aided image acquisition, previously developed methods suffered from issues of being time-and labor-intensive and of inaccurate stomata identification, classification, and quantification in C 4 grass species (Furbank and Tester, 2011). By addressing those issues, the studies by the three groups present end-to-end pipelines that use a deep learning algorithm to automatically identify, classify, and quantify stomatal traits associated with plant water use efficiency (WUE) and drought tolerance (Figure 1A). By integrating this pipeline with genomic studies, the authors further report the underlying genetic architecture of stoma-tal traits (Figure 1C). Traditional stomatal phenotyping involves plant tissue collection and preparation for imaging, image data acquisition under microscope, and manual phenotyping of traits of interest. To relieve the phenotyping bottleneck, Ferguson et al. (2021) and Xie et al. (2021) used optical topometry, a rapid and nondestructive method for measuring surface characteristics at the nanometer scale, and acquired images of leaves to extract morphology-related stomata traits. The three-dimensional topographic layer of the raw images was first filtered to capture the points of interest and then flattened to two dimensions in grayscale with luminosity optimization and contrast enhancement. After preprocessing, the authors trained a convolutional neural network model (Mask R-CNN) for automatic detection and counting of stomatal traits, such as number, density , and area (He et al., 2017). A typical deep learning framework for phenotyping starts with feeding and prepro-cessing of raw images, followed by several layers of automatic feature extraction during training, and ends up with the trained and validated model that can identify, classify, quantify, and predict the phenotypic traits of interest (Singh et al., 2018). Mask R-CNN detects and localizes objects of interest , such as stomata, using bounding boxes and generates precise segmentation masks (Figure 1A). The algorithm then uses several convolutional layers to identify and classify object region and then to predict object type. To train the Mask R-CNN model, the authors first labeled the input News and Views
Species:
Network:
Annotation:
Outputs:
2021
Classical phenotyping and deep learning concur on genetic control of stomatal density and area in sorghum
Stomatal density (SD) and stomatal complex area (SCA) are important traits that regulate gas exchange and abiotic stress response in plants. Despite sorghum (Sorghum bicolor) adaptation to arid conditions, the genetic potential of stomata-related traits remains unexplored due to challenges in available phenotyping methods. Hence, identifying loci that control stomatal traits is fundamental to designing strategies to breed sorghum with optimized stomatal regulation. We implemented both classical and deep learning methods to characterize genetic diversity in 311 grain sorghum accessions for stomatal traits at two different field environments. Nearly 12,000 images collected from abaxial (Ab) and adaxial (Ad) leaf surfaces revealed substantial variation in stomatal traits. Our study demonstrated significant accuracy between manual and deep learning methods in predicting SD and SCA. In sorghum, SD was 32%–39% greater on the Ab versus the Ad surface, while SCA on the Ab surface was 2%–5% smaller than on the Ad surface. Genome-Wide Association Study identified 71 genetic loci (38 were environment-specific) with significant genotype to phenotype associations for stomatal traits. Putative causal genes underlying the phenotypic variation were identified. Accessions with similar SCA but carrying contrasting haplotypes for SD were tested for stomatal conductance and carbon assimilation under field conditions. Our findings provide a foundation for further studies on the genetic and molecular mechanisms controlling stomata patterning and regulation in sorghum. An integrated physiological, deep learning, and genomic approach allowed us to unravel the genetic control of natural variation in stomata traits in sorghum, which can be applied to other plants.
Species: Orange
Network: Mask R-CNN
Annotation: Semantic
Outputs: Stomata count, stoma area
2021
A Deep Learning Method for Fully Automatic Stomatal Morphometry and Maximal Conductance Estimation
Gibbs, Jonathon A.; Mcausland, Lorna; Robles-Zazueta, Carlos A.; Murchie, Erik H.; Burgess, Alexandra J.
Abstract:
Stomata are integral to plant performance, enabling the exchange of gases between the atmosphere and the plant. The anatomy of stomata influences conductance properties with the maximal conductance rate, gsmax, calculated from density and size. However, current calculations of stomatal dimensions are performed manually, which are time-consuming and error prone. Here, we show how automated morphometry from leaf impressions can predict a functional property: the anatomical gsmax. A deep learning network was derived to preserve stomatal morphometry via semantic segmentation. This forms part of an automated pipeline to measure stomata traits for the estimation of anatomical gsmax. The proposed pipeline achieves accuracy of 100% for the distinction (wheat vs. poplar) and detection of stomata in both datasets. The automated deep learning-based method gave estimates for gsmax within 3.8 and 1.9% of those values manually calculated from an expert for a wheat and poplar dataset, respectively. Semantic segmentation provides a rapid and repeatable method for the estimation of anatomical gsmax from microscopic images of leaf impressions. This advanced method provides a step toward reducing the bottleneck associated with plant phenotyping approaches and will provide a rapid method to assess gas fluxes in plants based on stomata morphometry.
A stomata classification and detection system in microscope images of maize cultivars
Aono, Alexandre H.; Nagai, James S.; Dickel, Gabriella da S.M.; Marinho, Rafaela C.; de Oliveira, Paulo E.A.M.; Papa, João P.; Faria, Fabio A.
Abstract:
Plant stomata are essential structures (pores) that control the exchange of gases between plant leaves and the atmosphere, and also they influence plant adaptation to climate through photosynthesis and transpiration stream. Many works in literature aim for a better understanding of these structures and their role in the evolution process and the behavior of plants. Although stomata studies in dicots species have advanced considerably in the past years, even there is not much knowledge about the stomata of cereal grasses. Due to the high morphological variation of stomata traits intra- and inter-species, detecting and classifying stomata automatically becomes challenging. For this reason, in this work, we propose a new system for automatic stomata classification and detection in microscope images for maize cultivars based on transfer learning strategy of different deep convolution neural netwoks (DCNN). Our performed experiments show that our system achieves an approximated accuracy of 97.1% in identifying stomata regions using classifiers based on deep learning features, which figures out as a nearly perfect classification system. As the stomata are responsible for several plant functionalities, this work represents an important advance for maize research, providing an accurate system in replacing the current manual task of categorizing these pores on microscope images. Furthermore, this system can also be a reference for studies using images from different cereal grasses.
Species: Maize
Network: VGG
Annotation: Bounding Box
Outputs: Stomata count
2021
Identification of Plant Stomata Based on YOLO v5 Deep Learning Model
Stomata is an important structure in all terrestrial plants and is very vital in controlling plant photosynthesis and transpiration flow. Precise detection of plant stomata is the basis for studying stomata characteristics. Traditional detection methods are mostly manual operations, which is a tedious and inefficient process. Manually extracting features requires high image quality. Choosing appropriate features depends on certain prior knowledge, especially for the object with large morphological changes such as plant stomata. With the widespread use of deep learning technology, efficient solutions to this task have become possible. This article combines the characteristics of the corn leaf stomatal data sets to improve the latest object detection model YOLO v5)You Only Look Once(. By introducing the attention mechanism, that is, adding the SE module to the backbone network, the precision and recall of stoma detection are improved. At the same time, The loss function has been improved from to for avoiding some problems that may occur when selecting the best prediction box. Experimental results show that the precision and recall rates of the improved model on the corn leaf stomata data sets have reached 94.8% and 98.7% respectively, lay the foundation for the measurement of stomatal parameters. In addition, this paper also can help agriculturists and botanists to build their own data sets for stomatal research by explaining the methods of acquiring, pre-processing, and annotating data sets.
Species: Maize
Network: YOLO
Annotation: Bounding Box
Outputs: Stomata count
2021
Stomatal State Identification and Classification in Quinoa Microscopic Imprints through Deep Learning
Razzaq, Abdul; Shahid, Sharaiz; Akram, Muhammad; Ashraf, Muhammad; Iqbal, Shahid; Hussain, Aamir; Azam Zia, M.; Qadri, Sulman; Saher, Najia; Shahzad, Faisal; Shah, Ali Nawaz; Rehman, Aziz Ur; Jacobsen, Sven Erik
Abstract:
Stomata are the main medium of plants for the trade of water, regulate the gas exchange, and are responsible for the process of photosynthesis and transpiration. The stomata are surrounded by guard cells, which help to control the rate of transpiration by opening and closing the stomata. The stomata states (open and close) play a significant role in describing the plant's health. Moreover, stomata counting is important for scientists to investigate the numbers of stomata that are open and those that are closed to measure their density and distribution on the surface of leaves through different sampling techniques. Although a few techniques for stomata counting have been proposed, these approaches do not identify and classify the stomata based on their states in leaves. In this research, we have developed an automatic system for stomata state identification and counting in quinoa leaf images through the transformed learning (neural network model Single Shot Detector) approach. In leaf imprint, the state of stomata has been determined by measuring the correlation between the area of stomata and the aperture of each detected stoma in the image. The stomata states have been classified through the Support Vector Machine (SVM) algorithm. The overall identification and classification accuracy of the proposed system are 98.6% and 97%, respectively, helping researchers to obtain accurate stomatal state information for leaves in an efficient and simple way.
Species: Quinoa
Network: MobileNet
Annotation: Bounding Box
Outputs: Stomata count, pore classification
2021
A Deep Learning-Based Method for Automatic Assessment of Stomatal Index in Wheat Microscopic Images of Leaf Epidermis
The stomatal index of the leaf is the ratio of the number of stomata to the total number of stomata and epidermal cells. Comparing with the stomatal density, the stomatal index is relatively constant in environmental conditions and the age of the leaf and, therefore, of diagnostic characteristics for a given genotype or species. Traditional assessment methods involve manual counting of the number of stomata and epidermal cells in microphotographs, which is labor-intensive and time-consuming. Although several automatic measurement algorithms of stomatal density have been proposed, no stomatal index pipelines are currently available. The main aim of this research is to develop an automated stomatal index measurement pipeline. The proposed method employed Faster regions with convolutional neural networks (R-CNN) and U-Net and image-processing techniques to count stomata and epidermal cells, and subsequently calculate the stomatal index. To improve the labeling speed, a semi-automatic strategy was employed for epidermal cell annotation in each micrograph. Benchmarking the pipeline on 1,000 microscopic images of leaf epidermis in the wheat dataset (Triticum aestivum L.), the average counting accuracies of 98.03 and 95.03% for stomata and epidermal cells, respectively, and the final measurement accuracy of the stomatal index of 95.35% was achieved. R2 values between automatic and manual measurement of stomata, epidermal cells, and stomatal index were 0.995, 0.983, and 0.895, respectively. The average running time (ART) for the entire pipeline could be as short as 0.32 s per microphotograph. The proposed pipeline also achieved a good transferability on the other families of the plant using transfer learning, with the mean counting accuracies of 94.36 and 91.13% for stomata and epidermal cells and the stomatal index accuracy of 89.38% in seven families of the plant. The pipeline is an automatic, rapid, and accurate tool for the stomatal index measurement, enabling high-throughput phenotyping, and facilitating further understanding of the stomatal and epidermal development for the plant physiology community. To the best of our knowledge, this is the first deep learning-based microphotograph analysis pipeline for stomatal index assessment.
Species: Wheat
Network: R-CNN
Annotation: Bounding Box
Outputs: Stomata index, stomata count
2020
From leaf to label: A robust automated workflow for stomata detection
Meeus, Sofie; Van den Bulcke, Jan; wyffels, Francis
Abstract:
Plant leaf stomata are the gatekeepers of the atmosphere–plant interface and are essential building blocks of land surface models as they control transpiration and photosynthesis. Although more stomatal trait data are needed to significantly reduce the error in these model predictions, recording these traits is time-consuming, and no standardized protocol is currently available. Some attempts were made to automate stomatal detection from photomicrographs; however, these approaches have the disadvantage of using classic image processing or targeting a narrow taxonomic entity which makes these technologies less robust and generalizable to other plant species. We propose an easy-to-use and adaptable workflow from leaf to label. A methodology for automatic stomata detection was developed using deep neural networks according to the state of the art and its applicability demonstrated across the phylogeny of the angiosperms. We used a patch-based approach for training/tuning three different deep learning architectures. For training, we used 431 micrographs taken from leaf prints made according to the nail polish method from herbarium specimens of 19 species. The best-performing architecture was tested on 595 images of 16 additional species spread across the angiosperm phylogeny. The nail polish method was successfully applied in 78% of the species sampled here. The VGG19 architecture slightly outperformed the basic shallow and deep architectures, with a confidence threshold equal to 0.7 resulting in an optimal trade-off between precision and recall. Applying this threshold, the VGG19 architecture obtained an average F-score of 0.87, 0.89, and 0.67 on the training, validation, and unseen test set, respectively. The average accuracy was very high (94%) for computed stomatal counts on unseen images of species used for training. The leaf-to-label pipeline is an easy-to-use workflow for researchers of different areas of expertise interested in detecting stomata more efficiently. The described methodology was based on multiple species and well-established methods so that it can serve as a reference for future work.
Species: Herbarium samples
Network: VGG
Annotation: Bounding Box
Outputs: Stomata count
2020
The Implementation of Deep Learning Using Convolutional Neural Network to Classify Based on Stomata Microscopic Image of Curcuma Herbal Plants
Andayani, U.; Sumantri, I. B.; Pahala, Andes; Muchtar, M. A.
Abstract:
There are so many types of herbal plants that come from the same genus. The similarity of features in herbal plants makes it difficult to distinguish. Especially in the pharmaceutical field, which is very risky for making mistakes. The part of plants that is often used as ingredients for herbal medicines is the leaves, so research on leaves and their constituent organelles is very important for the pharmaceutical world. Therefore, an approach is needed to identify the types of organelles present in the leaves, where the organelles most frequently studied are the stomata. So, a neural network approach is needed to distinguish plant characteristics in the same genus. In this research there are 2 species of plants in the Curcuma genus, namely turmeric and ginger. This research was conducted by implementing Deep Learning using Convolutional Neural Network (CNN) with the help of the Gabor filter process and feature extraction using the Gray Level Co-occurrence Matrix (GLCM). The study was conducted using 160 microscopic images of Curcuma herbal plants as training data with training accuracy of 93.1% and test data of 40 images with an accuracy of 92.5%.
Species: Turmeric
Network: Custom CNN
Annotation: Image classification
Outputs: Image classification
2020
LabelStoma: A tool for stomata detection based on the YOLO algorithm
Stomata are pores in the epidermal tissue of leaf plants formed by specialised cells called guard cells, which regulate the stomatal opening. Stomata facilitate gas exchange, being pivotal in the regulation of processes such as photosynthesis and transpiration. The analysis of the number and behaviour of stomata is a task carried out by studying microscopic images, and that can serve, among other things, to better manage crops in agriculture or to better understand how plants fix CO2 and lose water under different conditions. However, quantifying the number of stomata in an image traditionally has been a labor intensive and thus expensive process since an image might contain dozens of stomata. Several automatic stomata detection models have been developed and presented in the literature, but they fail to generalise to images from species different to those employed to train the model; and, in addition, they lack a simple interface to employ them. In this work, we tackle these problems by training a YOLO model. Such a model achieves a F1-score of 0.91 in images from the species employed for training it, and similar F1-score for datasets containing images of different species. Moreover, in order to facilitate the use of the model, we have developed LabelStoma, an open-source and simple-to-use graphical user interface that employs the YOLO model. In addition, this tool provides a simple method to adapt the YOLO model to the users’ images, and, therefore, customising the model to the users’ needs. Thanks to this work, the analysis of plant stomata of different species will be more reliable and comparable; and, the developed tools will help to advance our understanding of CO2 and H2O dynamics in plants, such as photosynthesis and transpiration, and ecosystems related processes, such as carbon and water cycles.
Stomata are microscopic pores on the plant epidermis that regulate the water content and CO2 levels in leaves. Thus, they play an important role in plant growth and development. Currently, most of the common methods for the measurement of pore anatomy parameters involve manual measurement or semi-automatic analysis technology, which makes it difficult to achieve high-throughput and automated processing. This paper presents a method for the automatic segmentation and parameter calculation of stomatal pores in microscope images of plant leaves based on deep convolutional neural networks. The proposed method uses a type of convolutional neural network model (Mask R-CNN (region-based convolutional neural network)) to obtain the contour coordinates of the pore regions in microscope images of leaves. The anatomy parameters of pores are then obtained by ellipse fitting technology, and the quantitative analysis of pore parameters is implemented. Stomatal microscope image datasets for black poplar leaves were obtained using a large depth-of-field microscope observation system, the VHX-2000, from Keyence Corporation. The images used in the training, validation, and test sets were taken randomly from the datasets (562, 188, and 188 images, respectively). After 10-fold cross validation, the 188 test images were found to contain an average of 2278 pores (pore widths smaller than 0.34 μm (1.65 pixels) were considered to be closed stomata), and an average of 2201 pores were detected by our network with a detection accuracy of 96.6%, and the intersection of union (IoU) of the pores was 0.82. The segmentation results of 2201 stomatal pores of black poplar leaves showed that the average measurement accuracies of the (a) pore length, (b) pore width, (c) area, (d) eccentricity, and (e) degree of stomatal opening, with a ratio of width-to-maximum length of a stomatal pore, were (a) 94.66%, (b) 93.54%, (c) 90.73%, (d) 99.09%, and (e) 92.95%, respectively. The proposed stomatal pore detection and measurement method based on the Mask R-CNN can automatically measure the anatomy parameters of pores in plants, thus helping researchers to obtain accurate stomatal pore information for leaves in an efficient and simple way.
Species: Poplar
Network: Mask R-CNN
Annotation: Semantic
Outputs: Stomata density, stomata count
2020
Accelerating Automated Stomata Analysis Through Simplified Sample Collection and Imaging Techniques
Millstead, Luke; Jayakody, Hiranya; Patel, Harsh; Kaura, Vihaan; Petrie, Paul R.; Tomasetig, Florence; Whitty, Mark
Abstract:
Digital image processing is commonly used in plant health and growth analysis, aiming to improve research efficiency and repeatability. One focus is analysing the morphology of stomata, with the aim to better understand the regulation of gas exchange, its link to photosynthesis and water use and how they are influenced by climatic conditions. Despite the key role played by these cells, their microscopic analysis is largely manual, requiring intricate sample collection, laborious microscope application and the manual operation of a graphical user interface to identify and measure stomata. This research proposes a simple, end-to-end solution which enables automatic analysis of stomata by introducing key changes to imaging techniques, stomata detection as well as stomatal pore area calculation. An optimal procedure was developed for sample collection and imaging by investigating the suitability of using an automatic microscope slide scanner to image nail polish imprints. The use of the slide scanner allows the rapid collection of high-quality images from entire samples with minimal manual effort. A convolutional neural network was used to automatically detect stomata in the input image, achieving average precision, recall and F-score values of 0.79, 0.85, and 0.82 across four plant species. A novel binary segmentation and stomatal cross section analysis method is developed to estimate the pore boundary and calculate the associated area. The pore estimation algorithm correctly identifies stomata pores 73.72% of the time. Ultimately, this research presents a fast and simplified method of stomatal assay generation requiring minimal human intervention, enhancing the speed of acquiring plant health information.
Species: Apricot
Network: AlexNet
Annotation: Bounding Box
Outputs: Pore area
2019
Genetic Diversity in Stomatal Density among Soybeans Elucidated Using High-throughput Technique Based on an Algorithm for Object Detection
The stomatal density (SD) can be a promising target to improve the leaf photosynthesis in soybeans (Glycine max (L.) Merr). In a conventional SD evaluation, the counting process of the stomata during a manual operation can be time-consuming. We aimed to develop a high-throughput technique for evaluating the SD and elucidating the variation in the SD among various soybean accessions. The central leaflet of the first trifoliolate was sampled, and microscopic images of the leaflet replica were obtained among 90 soybean accessions. The Single Shot MultiBox Detector, an algorithm for an object detection based on deep learning, was introduced to develop an automatic detector of the stomata in the image. The developed detector successfully recognized the stomata in the microscopic image with high-throughput. Using this technique, the value of R2 reached 0.90 when the manually and automatically measured SDs were compared in the 150 images. This technique discovered a variation in SD from 93 ± 3 to 166 ± 4 mm−2 among the 90 accessions. Our detector can be a powerful tool for a SD evaluation with a large-scale population in crop species, accelerating the identification of useful alleles related to the SD in future breeding programs.
Species: Soybean
Network: VGG
Annotation: Bounding Box
Outputs: Stomata density, stomata count
2019
Deep convolutional neural networks based framework for estimation of stomata density and structure from microscopic images
Analysis of stomata density and its configuration based on scanning electron microscopic (SEM) image of a leaf surface, is an effective way to characterize the plant’s behaviour under various environmental stresses (drought, salinity etc.). Existing methods for phenotyping these stomatal traits are often based on manual or semi-automatic labeling and segmentation of SEM images. This is a low-throughput process when large number of SEM images is investigated for statistical analysis. To overcome this limitation, we propose a novel automated pipeline leveraging deep convolutional neural networks for stomata detection and its quantification. The proposed framework shows a superior performance in contrast to the existing stomata detection methods in terms of precision and recall, 0.91 and 0.89 respectively. Furthermore, the morphological traits (i.e. length & width) obtained at stomata quantification step shows a correlation of 0.95 and 0.91 with manually computed traits, resulting in an efficient and high-throughput solution for stomata phenotyping.
Species:
Network:
Annotation:
Outputs:
2019
Automatic segmentation and measurement methods of living stomata of plants based on the CV model
Background: The stomata of plants mainly regulate gas exchange and water dispersion between the interior and external environments of plants and play a major role in the plants' health. The existing methods of stomata segmentation and measurement are mostly for specialized plants. The purpose of this research is to develop a generic method for the fully automated segmentation and measurement of the living stomata of different plants. The proposed method utilizes level set theory and image processing technology and can outperform the existing stomata segmentation and measurement methods based on threshold and skeleton in terms of its versatility. Results: The single stomata images of different plants were the input of the method and a level set based on the Chan-Vese model was used for stomatal segmentation. This allowed the morphological features of the stomata to be measured. Contrary to existing methods, the proposed segmentation method does not need any prior information about the stomata and is independent of the plant types. The segmentation results of 692 living stomata of black poplars show that the average measurement accuracies of the major and minor axes, area, eccentricity and opening degree are 95.68%, 95.53%, 93.04%, 99.46% and 94.32%, respectively. A segmentation test on dayflower (Commelina benghalensis) stomata data available in the literature was completed. The results show that the proposed method can effectively segment the stomata images (181 stomata) of dayflowers using bright-field microscopy. The fitted slope of the manually and automatically measured aperture is 0.993, and the R2 value is 0.9828, which slightly outperforms the segmentation results that are given in the literature. Conclusions: The proposed automated segmentation and measurement method for living stomata is superior to the existing methods based on the threshold and skeletonization in terms of versatility. The method does not need any prior information about the stomata. It is an unconstrained segmentation method, which can accurately segment and measure the stomata for different types of plants (woody or herbs). The method can automatically discriminate whether the pore region is independent or not and perform pore region extraction. In addition, the segmentation accuracy of the method is positively correlated with the stomata's opening degree.
Species: Poplar
Network: R-CNN
Annotation: Bounding
Outputs: Stomata density, pore length, pore width, pore area
2019
StomataCounter: a neural network for automatic stomata identification and counting
Fetter, Karl C.; Eberhardt, Sven; Barclay, Rich S.; Wing, Scott; Keller, Stephen R.
Abstract:
Stomata regulate important physiological processes in plants and are often phenotyped by researchers in diverse fields of plant biology. Currently, there are no user-friendly, fully automated methods to perform the task of identifying and counting stomata, and stomata density is generally estimated by manually counting stomata. We introduce StomataCounter, an automated stomata counting system using a deep convolutional neural network to identify stomata in a variety of different microscopic images. We use a human-in-the-loop approach to train and refine a neural network on a taxonomically diverse collection of microscopic images. Our network achieves 98.1% identification accuracy on Ginkgo scanning electron microscropy micrographs, and 94.2% transfer accuracy when tested on untrained species. To facilitate adoption of the method, we provide the method in a publicly available website at http://www.stomata.science/.
Species: Gingko
Network: Custom CNN
Annotation: Custom
Outputs: Stomata count
2018
Automatic Quantification of Stomata for High-Throughput Plant Phenotyping
Stomatal morphology is a key phenotypic trait for plants' response analysis under various environmental stresses (e.g. Drought, salinity etc.). Stomata exhibit diverse characteristics with respect to orientation, size, shape and varying degree of papillae occlusion. Thus, the biologists currently rely on manual or semi-automatic approaches to accurately compute its morphological traits based on scanning electron microscopic (SEM) images of leaf surface. In contrast to these subjective and low-throughput methods, we propose a novel automated framework for stomata quantification. It is realized based on a hybrid approach where the candidate stomata region is first detected by a convolutional neural network (CNN) and the occlusion is dealt with an inpainting algorithm. In addition, we propose stomata segmentation based quantification framework to solve the problem of shape, scale and occlusion in an end-to-end manner. The performance of the proposed automated frameworks is evaluated by comparing the derived traits with manually computed morphological traits of stomata. With no prior information about its size and location, the hybrid and end-to-end machine learning frameworks shows a correlation of 0.94 and 0.93, respectively on rice stomata images. Furthermore, they successfully enable wheat stomata quantification showing generalizability in terms of cultivars.
Species: Rice
Network: Custom CNN
Annotation: Custom
Outputs: Stomata count, stomata density
2018
DeepStomata: Facial Recognition Technology for Automated Stomatal Aperture Measurement
Stomata are an attractive model for studying the physiological responses of plants to various environmental stimuli[1][1]–[3][2]. Of the morphological parameters that represent the degree of stomatal opening, the length of the minor axis of the stomatal pore (the stomatal aperture) has been most commonly used to dissect the molecular basis of its regulation. Measuring stomatal apertures is time consuming and labour intensive, preventing their use in large-scale studies. Here, we completely automated this process by developing a program called DeepStomata , which combines stomatal region detection and pore isolation by image segmentation. The former, which comprises histograms of oriented gradients (HOG)-based stomatal detection and the convolutional neural network (CNN)-based classification of open/closed-state stomata, acts as an efficient conditional branch in the workflow to selectively quantify the pores of open stomata. An analysis of batches of images showed that the accuracy of our automated aperture measurements was equivalent to that of manual measurements, however had higher sensitivity (i,e., lower false negative rate) and the process speed was at least 80 times faster. The outstanding performance of our proposed method for automating a laborious and repetitive task will allow researchers to focus on deciphering complex phenomena.
[1]: #ref-1
[2]: #ref-3
Species: Dayflower
Network: SSD
Annotation: Bounding Box
Outputs: Pore length, pore width, pore area
2017
Microscope image based fully automated stomata detection and pore measurement method for grapevines
Jayakody, Hiranya; Liu, Scarlett; Whitty, Mark; Petrie, Paul
Abstract:
Background: Stomatal behavior in grapevines has been identified as a good indicator of the water stress level and overall health of the plant. Microscope images are often used to analyze stomatal behavior in plants. However, most of the current approaches involve manual measurement of stomatal features. The main aim of this research is to develop a fully automated stomata detection and pore measurement method for grapevines, taking microscope images as the input. The proposed approach, which employs machine learning and image processing techniques, can outperform available manual and semi-automatic methods used to identify and estimate stomatal morphological features. Results: First, a cascade object detection learning algorithm is developed to correctly identify multiple stomata in a large microscopic image. Once the regions of interest which contain stomata are identified and extracted, a combination of image processing techniques are applied to estimate the pore dimensions of the stomata. The stomata detection approach was compared with an existing fully automated template matching technique and a semi-automatic maximum stable extremal regions approach, with the proposed method clearly surpassing the performance of the existing techniques with a precision of 91.68% and an F1-score of 0.85. Next, the morphological features of the detected stomata were measured. Contrary to existing approaches, the proposed image segmentation and skeletonization method allows us to estimate the pore dimensions even in cases where the stomatal pore boundary is only partially visible in the microscope image. A test conducted using 1267 images of stomata showed that the segmentation and skeletonization approach was able to correctly identify the stoma opening 86.27% of the time. Further comparisons made with manually traced stoma openings indicated that the proposed method is able to estimate stomata morphological features with accuracies of 89.03% for area, 94.06% for major axis length, 93.31% for minor axis length and 99.43% for eccentricity. Conclusions: The proposed fully automated solution for stomata detection and measurement is able to produce results far superior to existing automatic and semi-automatic methods. This method not only produces a low number of false positives in the stomata detection stage, it can also accurately estimate the pore dimensions of partially incomplete stomata images. In addition, it can process thousands of stomata in minutes, eliminating the need for researchers to manually measure stomata, thereby accelerating the process of analysing plant health.
Species: Grapevine
Network: Mask R-CNN
Annotation: Semantic
Outputs: Stomata count, pore length, pore width, pore area
Stomatal Feature Extraction of Lettuce Leaves Using Improved U-Net Network