Table 1 presents the existing review articles covering DL applications in different sections of agriculture [22,23,24,25,26,27,28]. From the table, it is evident that the reported studies (based on the authors’ knowledge) lacks a critical overview of recent advancements in DL methodologies for CEA. Thus, a need to review the recent works in CEA is consequential to determine state of the art, identify current challenges, and provide future recommendations. Figure 1 shows the bibliometric network and co-occurrence map of the author-supplied keywords.

The article’s organization is as follows: Section 2 features the methodology of the review process, including establishing review protocol, keywords selection, research questions formation, and data extraction. Section 3 presents the results of the review, including data synthesis and answers to the core research questions. Existing challenges and future recommendations are discussed in Section 4. The overall conclusions of the review is presented in Section 5.

In this research, we adhered to the SLR standard approach as described by Chitu Okoli and Kira Schabram [29]. Using this approach, we identified, specified, and analyzed all the publications in DL for CEA applications from 2019 to date, in order to present a response to each research question (RQ) and identify any gaps. Planning, conducting, and reporting the review are the three parts we divided the SLR process into. Figure 2 depicts the actions taken at each level of the SLR. During the planning phase we identified RQs, relevant keywords, and databases. After the RQs were prepared, the search protocol was created, along with which databases and search strings should be used. Search string for each database was generated using selected keywords. Wiley, Web of Science, IEEEXplore Springer Link, Google Scholar, Scopus, and Science Direct are the databases used in this study. The databases were chosen to ensure adequate coverage of the target sector and to increase the scope of the assessment. By going through all the eligible studies, pertinent studies were chosen for the conducting review stage. Significant information was retrieved from the publications that met the selection/inclusion criteria in response to the RQs. Extracted data from selected publications were used to answer the RQs during the reporting stage, and the outcomes were presented using accompanying visuals and summary tables. This type of literature analysis demonstrates the most recent findings of DL research in CEA.

Identifying RQs is essential to the systematic review. At the start of the study, we set the RQs up to adhere to the review procedure. The searched articles were examined from a variety of aspects, and the following RQs were established.

In order to focus the search results on papers that were specifically relevant to the SLR’s scope, a methodical approach was taken. The original search was conducted using a generalized search equation that included the necessary keywords “deep learning” AND “controlled environment agriculture” OR “greenhouse” OR “plant factory” OR “vertical farm” to obtain the expanded search results. From the search results, a few studies were selected to extract the author supplied keywords, and synonyms. The discovered keywords produced the general search string/equation: (“controlled environment agriculture” OR “greenhouse” OR “plant factory” OR “vertical farm” OR “indoor farm”) AND (“deep learning” OR “deep neural network”). All seven databases were searched using the same keywords. Following search strings were used for different databases:

Science Direct: (“controlled environment agriculture” OR “greenhouse” OR “plant factory” OR “vertical farm”) AND (“Deep Learning”) NOT (“Internet of Things” OR “GREENHOUSE GAS” OR “gas emissions” OR “Machine learning”)

After all the results were processed, a total of 751 studies were found using the aforementioned search strings.

To establish the limits for the SLR, the inclusion Criteria (IC) and exclusion Criteria (EC) were defined. To choose the pertinent research based on the IC and EC, the studies that were obtained from all databases were carefully examined. The search outcomes from several databases were combined in a spreadsheet and compared to all of the IC and EC. A study must meet all of the ICs and ECs in order to be considered for the review. Upon passing the IC and EC, all studies that could respond to the RQs were deemed pertinent and chosen. The ICs and ECs are presented below:

IC.1: Peer-reviewed journal publications and conference papers.

Applying the ICs and ECs produced a total of 72 eligible articles were selected, which were then shortlisted for additional examination. An overview of article search and selection procedure is shown Figure 3. The distribution of selected papers from different databases is shown in Table 2.

Table 3 and Table 4 presents the summary of studies that fulfilled the selection criteria. The necessary data required to answer the RQs, were extracted from the selected studies. The extracted data were summarized using a spreadsheet application. In the spreadsheet, each study was assigned to separate row, and the column was assigned to different parameters. Tasks, DL model, training networks, imaging system, optimizer, pre-processing augmentation, application domain, performance parameters, growing medium, and publication year, journal, and country, as well as challenges were retrieved from the selected studies. To properly respond to the RQs, all of the extracted data were categorized and synthesized into various classifications. The following sections present the results of this SLR.

RQ.2: What are the main application domains of DL in CEA?

In this subsection, we present the DL models in greenhouse production for diverse applications. Table 3 present the application domain, tasks, DL model, network, optimizer, datasets, pre-processing augmentation, imaging method, growing medium and performance of DL in greenhouse.

This subsection presents the main applications of the reviewed works that utilized DL in indoor farms (vertical farms, shipping containers, plant factories, etc.,). Table 4 present the application domain, tasks, DL model, network, optimizer, datasets, preprocessing augmentation, imaging method, growing medium, and performance of DL in indoor farms.

We observed a rapid advancement in CEA using DL techniques between 2019 and 2022, as demonstrated in Figure 6. With rising work since 2019, this illustrates the relevance of DL in CEA. In Figure 7, we showed the distribution of published articles by various journals. The figure shows that the journal Computers and Electronics in Agriculture published the most DL for CEA articles (19). We also presented the country-by-country distribution of the evaluated articles, with China accounting for 40% of the total, indicating the highest number of publications, as shown in Figure 8. Korea and the Netherlands each contain 10% and 7% of the papers, respectively.

Our survey found that various evaluation parameters were employed in the selected publications (RQ.3). Precision, recall, intersection-over-union (IoU), root mean square error (RMSE), mean average precision (mAP), F1-Score, root mean square error (RMSE), R-Square, peak signal noise ratio (PSNR), Jaccard index, success rate, sensitivity, specificity, accuracy, structural similarity index measure (SSIM), errors, standard error of prediction (SEP), and inference time were the most commonly used evaluation parameters for the DL analysis in CEA. Figure 9 depicts the frequency with which the assessment parameters are used. With 29 times, accuracy was the most frequently utilized as an evaluation measure. Precision, recall, mAP, F1-Score, and RMSE were used at least 10 times; IoU and R-Square were used 5 times, while the rest were used fewer than 5 times. We noticed that RMSE and R-Square were utilized as evaluation metrics in all microclimate prediction studies. Success rate and accuracy were used as evaluation measures for robotic harvesting applications. With the exception of a few cases of recall, precision, mAP, and F1-score, works related to growth monitoring applications used accuracy, RMSE, R-Square, and accuracy. RMSE, precision, recall, mAP, F1-Score, and accuracy were commonly utilized in other applications in the examined studies.

RQ.4: What are the DL backbone networks used in CEA applications?

There are many backbone networks, but this article will only focus on the backbone networks used in the reviewed papers, which include ResNet, EfficientNet, DarkNet, Xception, InceptionResNet, MobileNet, VGG, GoogleNet, PRPNet. These network structures are fine-tuned or combined with other backbone structures.

ResNet was the most often utilized network in CEA applications, according to the survey, as illustrated in Figure 10. The ResNet architecture can overcome the vanishing/exploding gradient problem [104]. When using gradient-based learning and backpropagation to train a deep neural network, the number of n hidden layers is multiplied by the n number of derivatives. The vanishing gradient problem occurs when the derivatives are modest, and the gradient rapidly diminishes as it spreads throughout the model until it vanishes. The gradient increases exponentially as the derivatives grow, resulting in the exploding gradient problem. A skip connection strategy is utilized in the ResNet to skip some training layers and connect directly to the output. The benefit of utilizing the skipping approach is that if any layer degrades the performance of the network, regularization will skip it, preventing exploding/vanishing gradient problems.

The main feature of MobileNet [105] is that it uses depth-wise separable convolutions to replace the standard convolutions of traditional network structures. Its significant advantages are high computational efficiency and small parameters of convolutional networks. MobileNet v1 and v2 are used in the reviewed articles, with v2 performing faster than v1. ResNet, on the other hand, adds a structure made up of multiple layers of networks that feature a shortcut connection known as a residual block. ResNet and FPN are used by Mask R-CNN to combine and extract multi-layer information. Many variants of ResNet architecture were discovered in reviewed articles, i.e., the same concept but with a different number of layers. A ResNeXt replicates a building block that combines a number of transformations with the same topology. It exposes a new dimension in comparison to ResNet, and requires minimal extra effort in designing each path.

Inception network [106] uses many tricks to push performance, both in terms of speed and accuracy, such as in dimension reduction. The versions of the inception network used in these reviewed papers are InceptionV2, InceptionV3, Inception-ResNetV2, and SSD InceptionV2. Each version is an upgrade to increase the accuracy and reduce the computational complexity. InceptionResNetV2 can achieve higher accuracies at a lower epoch. With the advantage of expanding network depth while using a small convolution filter size, VGG [107] can significantly boost model performance. VGGNet inherits some of its framework from AlexNet [108]. GoogleNet [109] has an inception module inspired by sparse matrices, which can be clustered into dense sub-matrices to boost computation speed, which is in contrast to AlexNet and VGGNet, which increases the network depth to improve training results. Contrary to VGG-nets, the Inception model family has shown that correctly constructed topologies can produce compelling accuracy with minimal theoretical complexity.

The backbone network for You Only Look Once (YOLO), DarkNet, has been enhanced in its most recent edition. YOLOv2 and YOLOv3 introduce DarkNet19 and DarkNet53, respectively, while YOLOv4 proposes CSPDarkNet [110]. CSPNet [111] is proposed to mitigate the problem of heavy inference computations from the network architecture perspective and has been seen to be used in the recent YOLO structure, i.e., SE-YOLOv5 [56]. Other backbone network structures include Xception [112] with different layers of 65 and 71, EfficientNet [113], and PRPNet [55].

RQ.5: What are the optimization methods used for CEA applications?

In contrast to the increasing complexity of neural network topologies [114], the training methods remain very straightforward. In order to make a neural network efficient, it must first be trained, as most neural networks produce random outputs without it. Optimizers, which modify the properties of the neural network, such as weights and learning rate, have long been recognized as a primordial component of DL, and a robust optimizer can dramatically increase the performance of a given architecture.

Stochastic gradient descent (SGD) is an optimization approach and one of the variants of gradient descent that is also commonly used in neural networks. It updates the parameters for each training one at a time, eliminating redundancy. As a hyper-parameter, the learning rate of SGD is often difficult to tune because the magnitudes of multiple parameters change greatly, and adjustment is required during the training process. Several adaptive gradient descent variants have been created to address this problem, including Adaptive Moment Estimation (Adam) [115], RMSprop [116], Ranger [117], Momentum [118], and Nesterov [119]. These algorithms automatically adapt the learning rate to different parameters, based on the statistics of gradient leading to faster convergence, simplifying learning strategies, and have been seen in many neural networks applied to CEA applications, as demonstrated in Figure 11.

RQ.6: What are the primary growing media and plants used for DL in the CEA?

We note that the most common growing medium used in the evaluated studies is soil-based (78%), as shown in Figure 12. There are 14 publications on hydroponics, one on aquaponics, and none on aeroponics for soil-less growing media. This insinuates that these soilless growing media are still in their infancy. We also showed the distribution of the plants used in the evaluated papers, with tomatoes representing 39% of all plants grown in the CEA and corresponding to the highest number of publications, as shown in Figure 13. The percentages of papers that planted lettuce, pepper, and cucumber are 16%, 9%, and 8%, respectively. According to the reviewed publications, it was also discovered that indoor farms used soil-less techniques (hydroponics and aquaponics) more frequently than greenhouse systems, which frequently used soil-based growing medium.

To the best of our knowledge, the paragraphs below provide a brief description of some specific aspects on the challenges and potential directions of DL applications in CEA.

For DL models to be effective, learning typically needs a lot of data. Such huge training datasets are difficult to gather, not publicly available for some CEA applications, and may even be problematic owing to privacy laws. Even while data augmentation and massive training datasets methods can somewhat make up for the shortage of huge labeled datasets, it is difficult to completely meet the demand for hundreds or thousands, if not less, high-quality data points. When utilized with validated data, DL models may not be able to generalize in situations where the data is insufficient. However, we discovered a number of studies that used smaller datasets and attained great accuracy, as shown in [40,45,56,59,82]. The studies demonstrated various strategies for handling this circumstance by carefully choosing the features that ensure the method will perform at its peak. Additionally, in order to ensure optimal performance and streamline the processing of the learning algorithms, the dimensionality of the input vectors for the classification and detection algorithms must be reduced.

DL algorithms are also susceptible to the caliber of the data utilized to train them. Overfitting can occur when an algorithm “learns” about noise and excessive details in the input set, which has a detrimental effect on the created model’s ability to generalize. The model in this instance performs admirably on the training dataset but poorly on new data. To combat the overfitting model, regularization techniques include weight decay/regularization, altering the network’s complexity (i.e., the amount of weights and their values), early halting, and activity regularization.

We expect in the future to see more combinations of two-time series models for temporal sequence processing as demonstrated in [31]. It is also anticipated that more methods would use LSTM or other RNN models in the future, utilizing the time dimension to make more accurate predictions, especially in climatic condition prediction.Additionally, it helps to gauge the reliability of time series prediction by offering an explicable result. As a result, improving interpretability will receive a lot of attention in the future [120].

The majority of the evaluated studies focused on supervised learning, while just a small number used semi-supervised learning. Future works that include unsupervised learning into CEA applications will be heavily reliant on tools like the generative adversarial network (GAN). A generative modeling method known as GAN learns to replicate a specific data distribution. The lack of data is a major barrier to creating effective deep neural network models, but GANs are the solution [121]. In order to lessen model overfitting, the realistic images created by GAN that differ from the original training data are appealing in data augmentation of DL-computer vision.

Another area worth noting is the clear interest in the use of AI and computer vision in CEA applications. With the use of DL-computer vision, a number of difficult CEA issues are being resolved. However, DL-computer vision does face significant difficulties, one of which is the enormous processing power. Adopting cloud-based solutions with auto scaling, load balancing, and high availability characteristics is one way to deal with this issue. Real-time video input analysis and real-time inferences are some of the limitations of cloud solutions, but edge devices with features like GPU accelerators can do it. Utilizing computer vision solutions on edge hardware helps lessen latency restrictions. Few works have addressed the need for proper security to ensure data integrity and dependability in the rapidly expanding field of computer vision in CEA; additional research into this area is needed in subsequent works.

There is an imperative need where deep learning needs to be applied in the next few years such as developing more microclimate models for monitoring and maintaining the microclimatic parameters to the desired range for optimal plant growth and development, thus helping in irrigation and fertigation management of the crops. The need for AI, particularly DL, to derive an empirical and non-linear “growth response function” that maps microclimate conditions to crop growth stages is critical because, according to the reviewed papers, this has not been extensively studied. This calls for the optimization of microclimate control set points at various growth stages of crops. There are currently very few publications that have developed prediction models for the microclimate parameters in CEA. In addition to the microclimate prediction models, the need to also develop more microclimate control systems such as (1) developing automatic shading system to prevent crops from harsh sunlight in greenhouses, (2) developing pad-fan systems and fogging systems based on vapor pressure deficit (VPD) control, which is an effective way to simultaneously maintain ideal ranges of temperature and relative humidity, thus significantly enhancing plant photosynthesis and productivity in greenhouse production, (3) developing photoperiod control systems based on light spectrum and intensity control. Despite the paucity of studies on microclimate prediction and control, extensive research is needed in the use of edge-AI systems for precise monitoring at various phases of crop growth. Lastly, it is crucial to investigate the use of DL for nutrient solution management in soilless cultures (influenced by both microclimate conditions and crop growth). We anticipate that further research that considers monitoring, predicting, controlling, and optimizing microclimate factors in CEA will become available in the near future as advancements in accuracy, efficiency, and architectures are put forth. Additionally, the labor availability and associated costs, are a growing concern for the sustainability and profitability of CEA industry. Some research has been reported for developing robotic systems, but majority of it is focused on field production. However, the CEA is a unique production environment and the indoor grown crops have different requirements for automation based on the production technology employed (greenhouse, vertical tower, vertical tier, hydroponic, dutch bucket, pot/tray, etc., ). Further, the CEA crops are more dense (plants per unit area), which makes robotics applications more challenging. Thus, extensive efforts are required to develop DL-driven automation and robotic systems for different production environments, to address these challenges.