research paper on deep ecology

• Search Menu
• Advance articles
• Editor's Choice
• Food for Thought
• Food for Thought: Luminaries
• Food for Thought: Rising Tides
• Quo Vadimus
• Stories from the Front Lines
• Symposium Issues
• Themed Sets
• Introductions
• Hidden Gems
• Author Guidelines
• Submission Site
• Open Access
• Why Publish with us?
• About ICES Journal of Marine Science
• About the International Council for the Exploration of the Sea
• Editorial Board
• Advertising and Corporate Services
• Self-Archiving Policy
• Dispatch Dates
• Terms and Conditions
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, a non-comprehensive review of dl, established cases: identification and quantification of marine biodiversity, emerging cases, conclusions and future directions, data availability statement, acknowledgements.

• < Previous

Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook †

• Article contents
• Figures & tables
• Supplementary Data

Morten Goodwin, Kim Tallaksen Halvorsen, Lei Jiao, Kristian Muri Knausgård, Angela Helen Martin, Marta Moyano, Rebekah A Oomen, Jeppe Have Rasmussen, Tonje Knutsen Sørdalen, Susanna Huneide Thorbjørnsen, Unlocking the potential of deep learning for marine ecology: overview, applications, and outlook, ICES Journal of Marine Science , Volume 79, Issue 2, March 2022, Pages 319–336, https://doi.org/10.1093/icesjms/fsab255

• Permissions Icon Permissions

The deep learning (DL) revolution is touching all scientific disciplines and corners of our lives as a means of harnessing the power of big data. Marine ecology is no exception. New methods provide analysis of data from sensors, cameras, and acoustic recorders, even in real time, in ways that are reproducible and rapid. Off-the-shelf algorithms find, count, and classify species from digital images or video and detect cryptic patterns in noisy data. These endeavours require collaboration across ecological and data science disciplines, which can be challenging to initiate. To promote the use of DL towards ecosystem-based management of the sea, this paper aims to bridge the gap between marine ecologists and computer scientists. We provide insight into popular DL approaches for ecological data analysis, focusing on supervised learning techniques with deep neural networks, and illustrate challenges and opportunities through established and emerging applications of DL to marine ecology. We present case studies on plankton, fish, marine mammals, pollution, and nutrient cycling that involve object detection, classification, tracking, and segmentation of visualized data. We conclude with a broad outlook of the field’s opportunities and challenges, including potential technological advances and issues with managing complex data sets.

Marine ecosystems are complex, highly diverse, and productive, providing renewable resources to a growing human population. At the same time, the oceans are particularly sensitive to and impacted by anthropogenic stressors (Antão et al ., 2020 ). As such, the scientific community strives to deliver up-to-date information about the state of marine ecosystems so that management decisions are well-informed. Ideally, such decisions use ecosystem-based management (EBM) approaches to preserve ecosystem health and productivity while allowing appropriate human use. EBM is especially relevant in densely populated coastal areas. During this period of rapid environmental change, EBM requires researchers to track ecological change and critical events when, and not well after, they occur. Fortunately, technological developments in observation methods over the last couple of decades have provided ecologists with a range of new tools for obtaining vast amounts of data from marine ecosystems. These include high-end cameras, echo sounders, and hydrophones, combined with various sensors to measure environmental parameters. Researchers can attach such technologies to cabled observatories or static rigs to assess temporal dynamics, or remotely or autonomously operated vehicles to evaluate spatial variability. However, because these technologies can produce an unprecedented amount of data, which has traditionally required manual processing, ecologists may be reluctant to adopt them as an alternative or supplement to traditional sampling techniques. For example, using traditional gear (e.g. nets and traps) to assess the abundance of fish has been an established sampling technique for centuries and is still used today. These methods are efficient for manual data handling and straightforward: as soon as the fish are caught, counted, and the data entered, it can be analysed by the researchers. On the other hand, detecting and counting fish with cameras is less destructive to animals and habitat, offers high-resolution temporal data, allows researchers to observe behaviour of animals and habitat use, and often provides a more representative estimate of species diversity and relative abundance (Bacheler et al ., 2017 ). However, extracting all of this information from videos manually is a laborious task. Thus, automating this step would undoubtedly encourage more fish biologists to use cameras for data collection.

Many diverse fields of research are undergoing rapid change due to advances in the use of artificial intelligence (AI) for data interpretation. AI offers fast and accurate analysis of the large volumes of data collected by sensors, cameras, and other observation technologies. Off-the-shelf algorithms can now, with high precision, find, count, and classify organisms from digital images and real-time video, (Lopez-Guede et al ., 2020 ; Knausgård et al ., 2021 ; Li and Du, 2021 ) and detect cryptic patterns in noisy images or acoustic data (Weinstein, 2018 ). An increasing number of marine ecologists embrace this opportunity, yet initiating collaborations across ecological and data science disciplines can be challenging for several reasons. First, transferring the necessary information to start a project between an ecologist and a computer scientist can be a steep learning curve because knowledge barriers and field-specific jargon can cloud otherwise fruitful discussions and halt progression. Second, ecologists unfamiliar with AI may not be aware of the opportunities available to address a particular problem. Before an ecologist approaches an AI expert, they may need to know about the possibilities and limitations of AI, how to prepare and annotate data sets, and what information to provide the computer scientist to identify the best AI method for the task at hand. Meanwhile, before advising on the possibilities, the computer scientist may find it challenging to understand the underlying ecological question, the data and its inherent variability/noisiness, how it is categorized, and what level of accuracy is needed. Thus, substantial investment in the interdisciplinary partnership is required in order to achieve a common understanding.

This paper aims to bridge the gap between marine ecologists and computer scientists to expedite the initial stages of collaboration. To provide common ground, we describe the most popular and suitable AI techniques for ecological data analysis, including technical concepts. AI is a general term referring to any AI technique that can solve a complicated problem (Goodwin, 2020 ; Russell and Norvig, 2002 ). We focus on applicable and well-established methods, namely deep neural networks (DNNs), synonymous with “deep learning” (DL), and learning with supervision (supervised machine learning). Supervised learning requires algorithms to be presented with datasets that have been labelled with accurate information on the region of interest, for example the presence or location of known species, objects, or sound. The algorithms learn to associate the labels with the examples (Christin et al ., 2019 ). With enough training material, the algorithms can produce models that automatically recognize and identify new and unseen examples in other datasets without the need for new labels (LeCun et al ., 2015 ). One of the biggest challenges for supervised learning is the demand for a large, labelled training dataset of sufficient quality to achieve high accuracy (Malde et al ., 2020 ; Beyan and Browman, 2020 ). Close collaboration between ecologists and computer scientists would likely facilitate and accelerate the dedicated effort required to collect and label representative datasets (Weinstein, 2018 ; Schneider et al ., 2019 ; Beyan and Browman, 2020 ).

This paper is organized as follows: In "A non-comprehensive review of DL", we summarize popular DL tools relevant for ecologists and explain standard AI terms. We then provide an overview of machine learning approaches as applied to a series of marine ecology case studies ( Table 1 ). The section "Established cases: identification and quantification of marine biodiversity" describes three cases where AI has been applied to ecological data, namely: fish detection, classification, and tracking in underwater videos; image-based analysis for plankton monitoring; and acoustic monitoring of whales. These applications are generally focused on species and higher-order taxonomic classification for biomonitoring purposes. Yet, DL in ecology research is not limited to these cases and we are confident that the DL toolset will further impact emerging research areas in marine ecology at additional levels of biological organization. Therefore, the section "Emerging cases" continues with four case studies where we see the potential for DL to make a major impact. At the individual level, we show the potential for DL to enable individual visual re-identification of fish using unique patterns (similar to facial recognition) and analysis of fish vocal communication to identify individuals (i.e. vocal recognition) to better understand mating behaviour. At the ecosystem level, we show how DL can aid in ghost fishing gear detection and determining the ecological functions of fish in the carbon cycle. We conclude by discussing technological advances, complexity in data, and acceleration of data collection and labelling through open-source approaches.

Machine learning approaches to ecological data applied (green) or explored (blue) in the case studies (C1–C7), and some alternatives (orange). Grey cells indicate no added benefit to using that approach for the task. Approaches: (Ap A): One label per region of interest, (Ap B): One label per image, (Ap C): Pixel-wise segmentation, (Ap D): Ground truth spectrograms with labelled region of interest, (Ap E): Labelled spectrograms with regions of interest, and (Ap F): Segmented time series data.

AI is a broad concept, but the most commonly applied technique is machine learning. Machine learning is a set of algorithms that learn from an environment containing data such as images. The most common AI approach used in biology is supervised learning, which is when the data are labelled or categorized so that the algorithms can learn from the data. Conversely, unsupervised learning is when algorithms do not use labelled data but, instead, learn data structures that are reinforced when the algorithms continuously interact with an environment, such as playing a board game. Figure 1 illustrates the overall procedure for training and application of AI with supervised learning.

The workflow of AI based strategies. (1) The (yellow) column illustrates the training phase, in which labelled data is used to train the AI algorithm. (2) The first row (blue) shows that the performance of the trained AI is evaluated using a validation data set and the AI algorithm may be updated and refined in this process. (3) The bottom row (green) shows the application phase, using the AI on a test data set once the training and validation are completed.

Among the most popular and widely used AI algorithms are the family of artificial neural networks. A neural network is a set of human brain-inspired networks with artificial neurons and synapses that are trained to approximate an external function, typically mapping from input data (e.g. images) to labelled values or categories (e.g. classes). A neural network consists of a layer of input neurons connected to the input data and a layer of output neurons mapping to the values or categories to be predicted. It is common to have layers between the input and output, which are referred to as hidden layers. When a network has more than one hidden layer, it is referred to as DL or a DNN.

Neural networks, especially DL, are the go-to machine learning approach for categorizing and recognizing images and sound data. These techniques have won numerous pattern recognition and machine learning competitions for image and sound analytics (Schmidhuber, 2015 ; Tessler et al ., 2017 ). In recent years, DL has become the predominant analytical technology in many domains, including health (Esteva et al ., 2019 ), customer evaluation (Lessmann et al ., 2019 ), and crisis management (Ben Lazreg et al ., 2019 ; Ben Lazreg, Noori, Comes and Goodwin, 2019 ). Aquatic ecology has experienced the early stages of the same shift, where object detection and semantic segmentation are being used to identify and locate marine species in raw images, videos, and audio recordings for the purpose of species (Knausgård et al ., 2021 ) and individual (Bogucki et al ., 2019 ) classification, and to quantify abundance. Despite the domination of deeper over more shallow neural networks, there is no need to employ DL models exclusively. Depending on the complexity and the nature of the problem, various models with different depths can be utilized. For example, Kohonen networks, which consist of only one layer, are shallow but useful for biology-related classifications and visualization (Suryanarayana et al ., 2008 ). In addition to identifying and counting fish and other marine animals, there is enormous potential to apply DL to a wide range of data in coastal ecology (Grasso et al ., 2019 ; Marre et al ., 2020 ). In the following subsections, we will briefly go through the basics of DNN. A glossary of AI terms is summarized in Table 2 .

Glossary table.

All neural networks are function approximators; they mimic the function presented in the training data and adapt to this function through an optimization process. During training, the neural networks’ weights, which are many real-valued and connected neurons followed by activations, are updated to match the training data. In more detail, the real-valued difference between the predicted output, |$\hat{Y}$|⁠ , and the expected output, Y , is referred to as the loss, which guides the training. For example, Y can be a list of image categories where each value in the vector relates a category to an image, and |$\hat{Y}$| is then the neural network’s predicted image categories. If the neural network is able to correctly predict image categories, |$\hat{Y}$| will be identical to Y and the loss will be zero. The goal of the training process is, generally speaking, to minimize the loss. However, the loss minimization should be done with care since a small loss may indicate that DL has learned specific patterns for each example rather than general trends in the data (i.e. overfitting). To check for overfitting, a separate validation data set is normally employed to independently evaluate the algorithm’s performance.

A properly trained network has active or inactive neurons that jointly match the training data and minimize the loss. This is analogous to a series of virtual dials that can be turned completely on, completely off, or somewhere in between, indicating the relevance for each feature. During training, the loss for each neuron is propagated backward through the network so that each neuron’s contribution matches the product of the weight and a hyper-parameterized learning rate. Hence, each neuron’s influence of the loss is matched with a corresponding adjustment of weights, and its adjustment is kept small by the learning rate. When the loss is propagated backwards, the dials are turned slightly in the direction that decreases the loss.

A neural network is considered shallow if it has one layer of input neurons, one layer of hidden neurons, and one output layer. The same network would be considered deep if it had more than one hidden layer, and very deep if it had more than ten hidden layers. Any neuron that is not at the input layer combines a weighted sum from active neurons in the previous layer. The sum is then followed by an activation function for the next layer of neurons. Despite popular belief, the depth of the DL may not be proportional to the difficulty of the problem that it can solve. It is not always true that deeper networks solve more complicated issues than shallower networks. Some problems can be solved with shallow networks, but in many cases very deep models empirically outperform the shallow ones for image and sound categorizations. For example, a type of neural network called Residual Networks (sometimes abbreviated to ResNets) often has 18, 34, 50, or 101 layers. Usually, the deeper networks perform better image classification, but occasionally the most shallow network, with 18 layers, is sufficient and even more accurate than the deeper networks (Aloysius and Geetha, 2017 ).

A poorly trained network is said to ’overfit’ when it performs significantly better on the training data compared to the testing data and increased training improves the training results but, at the same time, worsens the testing results. Hence, overfitting is observable by increased training accuracy and decreased testing accuracy. A potential mitigation is to increase the complexity of the network or increase the amount of training data.

A notable limitation of DL is its dependency on vast amounts of training data. The data requirement typically becomes a significant problem in supervised learning, as a successful application in most cases depends on large quantities of human-classified training examples. This challenge is extensively presented in the marine biology domain, as the limited capacity of trained experts makes extensive and quality-assured labelled training databases hard to acquire. A beneficial property of deep unsupervised learning is its independence of labelled data. However, due to the unsupervised nature, the application area is rather limited in the marine biology domain and has mostly been confined to finding anomalies through re-identification (Dargan et al ., 2019 ; Ferreira et al ., 2020a ) and data clustering.

Deep semi-supervised learning has emerged in recent years to mitigate the limitations of supervised and unsupervised learning. Semi-supervised approaches combine training on a small amount of labelled data with a subsequent training phase using large amounts of unlabelled data. In applications where there is often a lack of human-classified training data, semi-supervised learning is especially useful.

In the paragraphs below, we summarize typical problems relevant to marine ecology where DNN can be utilized as a promising solution.

Image classification

DNN is the de facto standard for machine vision, such as the categorization of images and video files. The most prominent approach among various DNNs is Convolutional Neural Networks (CNNs), which extract relevant features of an image for subsequent classification by a neural network through a series of two-dimensional mathematical convolutional operations with learnable filters of typical sizes 3 × 3, 5 × 5, and 7 × 7 applied in the image pixels. A CNN trained for classification of images finds the function that best maps the input of pixels to a class, e.g. presence of a fish, plankton, or a rope in the photo ( Figure 2 ). Note that the CNN generates small image blocks from the convolutionals of overlapped data within each image. CNN categorizes the image but does not output in which part of the image the object is located.

Examples of classification, object detection, and pixel-wise segmentation with illustrations of the techniques applied to fish images or audio files.

The first popularized CNN models were LeNet-1–LeNet-5  (LeCun et al ., 1995 ), which contain all the basic building blocks still used today. A major advancement, in terms of both architecture and performance, came in 2012 with AlexNet (Krizhevsky et al ., 2012 ). AlexNet achieved an error rate of 15%, which was better than all non-neural network architectures, for which the previous best error rate was 26%. These early models suffered from vanishing gradients, meaning that the input data was gradually lost when additional layers were added. This limitation hindered the development of DNN and the performance of the DL models suffered. Later, major innovations included: (1) inception networks (Szegedy et al ., 2015 ), which utilized parallel convolutions of different sizes, (2) residual architecture (He et al ., 2016 ), which added skip connections to allow for an image to both be processed by convolutions and skipped through the network, and (3) squeeze-and-excitation networks (Hu et al ., 2018 ), which introduced a method to add additional parameters to each convolutional block so that the model could adjust the weight of each block. Each of these innovations has enabled larger, more complex networks. Therefore, the rapid advancement of the image classification field indicates that the newest techniques are, in general, much better than earlier ones. Unless there is specific evidence to the contrary, practitioners are advised to choose a more recent approach.

Object detection and semantic segmentation

Object detection extends CNN models by detecting regions of interest in the image ( Figure 2 ). In addition to classification, a network trained for object detection can output the x - and y -location, width, and height of the object of interest. This information is then used to draw a boundary box around the object to be classified, e.g. a fish. In this way, a single image can be divided into multiple regions by generating several boundary boxes, allowing for many classes to be classified within a single image. In practice, this means that we can detect and count objects in an image or a video, e.g. the number of fish. The approach has been extended even further by pixel-wise detection and classification of the entire image. This approach scales down the image with convolutions and pooling operations, followed by reverse order scaling-up of the same image. This is known as an encoder–decoder architecture (Girshick et al ., 2014 ) and allows for categorization of every region in the image at a high level of detail. A commonly used method for object detection is You Only Look Once (YOLO; Redmon et al ., 2016 ; Yu et al ., 2021 ). Variants of Region-based CNN (R-CNN), including Fast R-CNN (Girshick, 2015 ), and Mask R-CNN (He et al ., 2017 ), are used for pixel-wise segmentation.

Individual identification

A Siamese Neural Network (SNN; Koch et al ., 2015 ) is a type of DL model that contains two identical sub-networks with the same layers, hyper parameters, and weights. The neuron weight updates are mirrored and so can be used to find the similarity of the inputs by comparing vector features. An SNN allows us to detect if two images are the same, e.g. two faces are of the same person or two fish photos are of the same fish taken at a different time. Hence, an SNN can classify a new class without re-training the entire network. Other features include robustness to class imbalance (i.e. data is unequally distributed between classes) and learning efficiency in the semantic similarities between images. However, SNNs need more training data and longer computational time than competing networks.

When training an SNN, a typical loss function used to detect differences in input is a so-called triplet loss, in which the baseline input is compared both with a positive and a negative example. A perfectly trained SNN should have a zero loss for the positive example and a loss for the negative example. For example, when detecting individual marine animals, a comparison between pictures of the same animal should have a smaller loss than a comparison between pictures of two different individuals of the same species. This approach can be used as a method for classification to identify if two pictures include the same individual and verify whether an image consists of an individual that is not part of the training data. Figure 2 provides examples of classification, object detection, and segmentation, and how they are typically evaluated.

Audio signal classification

Audio signal classification is a classic yet challenging field of audio signal processing. In brief, it comprises capturing appropriate features from an audio sequence and employing these features to distinguish the class that the sequence is most likely to fit. Depending on the application domain, one may predict a global signal class with a unique label or a subset of the possible classes with multiple labels. Traditionally, finding appropriate features and designing a suitable classifier are configured as separate procedures. This approach has several drawbacks. The extracted features might not be optimal for the classification objective. Further, certain features may require prior human knowledge, be difficult to describe precisely, or be subjective and unstable. As mitigation, DNN-based approaches are developed to perform feature extraction jointly with classification.

In contrast to feed-forward neural networks, recurrent neural networks (RNNs) contain feedback loops. These loops allow RNNs to use their reasoning from previous data to influence upcoming data, hence lending themselves to process a series of data. This feature is useful when working with data that changes over time, so-called time series, including audio signals. RNNs vary in complexity, from standard RNNs, often called vanilla RNNs, to models with more complex memory elements, including gated recurrent units (GRUs) and long short-term memory (LSTM). The more complex a module is, the more likely it is to learn intricate patterns in the data. However, increasing the complexity also increases training time and the chance of overfitting. Therefore, the choice of RNN will depend heavily on the level of complexity of the pattern(s) of interest.

A recent trend is to combine RNNs with new variants of feed-forward methods such as attentions (Phan et al ., 2019 ; Chaudhari et al ., 2021 ), combine them with multiple attentions coupled together into the collective concept of transformers (Moritz et al ., 2020 ; Tay et al ., 2020 ), or avoid RNNs altogether and only use transformers. Attention and transformers weigh the significance of input data and, like RNNs, they are designed to handle sequential input data such as audio signals and other time series. However, unlike RNNs, there is no feedback loop, which means that it learns the context for any position in the input sequence. Despite similarities, attentions and transformers are technically not RNNs since they do not rely on recurrent feedback loops but rather a more straightforward feed forward mechanism. However, a transformer has more parameters to learn and is, therefore, more computationally intensive. In practice, developers must balance the complexity of the models with available training time and resources. RNNs, CNNs, and attention modules can be combined to improve the performance of the system. For example, an attention-based convolutional RNN model is utilized for environmental sound classification (Zhang et al ., 2020 ).

Another approach for audio classification is to convert the audio into a visual representation and use image classification as described above. Spectrograms are such a visual representation and can be classified using a DNN or CNN. These mechanisms, alone or in combination, can be utilized for audio classification tasks in marine ecology-related applications. Figure 2 includes examples of classification, object detection, and data point segmentation with CNN and RNN networks for audio categorization.

Evaluation criteria

To evaluate the performance of a trained model, different parameters are utilized by the different approaches, such as accuracy, precision, and recall ( Figure 3 ). Accuracy is the ratio of correct classifications to the total number of classifications. Precision for the positive predictions is the ratio of true positive predictions over the sum of true positive and false positive predictions. The same concept applies to the precision of negative prediction. Recall is the ratio of true positive predictions over the sum of true positive and false negative predictions. A result of a DL algorithm may be precise but not accurate when results are biased but with small variance. A DL algorithm is considered valid if it is both accurate and precise.

Evaluation metrics accuracy, precision, recall, F1-score, and Intersection over Union (IoU) for classifications, predictions, object detections, and segmentations.

For example, if the expected output Y is five images of cod and five images of trout, and the predicted output |$\hat{Y}$| correctly identifies all cod and only four of the trout, with one trout wrongly identified as cod, the algorithm is correct nine out of ten times, yielding an overall accuracy of 90%. In this example, the precision for trout is 100%, i.e. all trout were predicted as trout, but only |$\frac{5}{6}=83\%$| for cod, i.e. for all the predicted cod, only 83% are actually cod. The recall for cod will be 100%, i.e. the algorithm identifies all cod, but the recall for trout will be |$\frac{4}{5}=80\%$|⁠ , i.e. the algorithm only identifies 80% of the trout. Table 3 illustrates this example as a Confusion Matrix.

Example of a Confusion Matrix with five cod and five trout.

The parameter used for performance evaluation depends on the data. Accuracy is most suitable if the data set is balanced, meaning an approximately equal number of examples in each class, and where false positives and false negatives have similar implications. But if the data set is imbalanced, which is typical for ecological data (e.g. some species are more common than others), precision or recall are better. A high precision relates to a low false-positive rate, whereas a high recall relates to how well the model detects the class in the total data set.

Closely related to precision and recall is the Receiver Operated Characteristics (ROC) curve, where the true positive rate is on the y axis and the false positive rate is on the x- axis. Each AI output includes a score that represents certainty. Changing the threshold of acceptable scores affects how conservative the output will be. Only accepting output with a high classification score will result in few false positives but many false negatives. Conversely, accepting output with a lower score will result in more false positives but fewer false negatives. The normalized Area Under the ROC Curve (AUC) describes how well the algorithm works across this range of score thresholds. The value of AUC is a number between 0 and 1, with the latter describing a perfect network.

The F1 score, a unified metric, is a weighted average of precision and recall and, therefore, encompasses both the false positives and false negatives. A general rule of thumb is to use the F1 score for evaluation when unsure based on the other metrics.

For object detection and semantic segmentation, the evaluation should depict how much pixel-wise overlap there is between the predicted and actual objects. The metric used is called ’intersection over union’ (IoU). Using fish identification as an example, an IoU of 0 means that there is no overlap between the areas of the predicted fish and actual areas of the fish. Conversely, an IoU of 1 means a perfect pixel-by-pixel overlap between the predicted and actual areas of the fish.

There is no universally right answer as to how much data is needed—generally, the more data, the better. Learning an intricate pattern requires more data than learning a simpler one. For example, for a DL to classify an image as either a sea trout or another fish species with clear morphological differences, such as a cod, it may achieve a near-perfect separator with relatively few samples. However, more data are likely to be required for a model to learn to distinguish sea trout from a closely related species with similarities in appearance, such as salmon, simply because that is a more complex task to learn.

Mitigation for the lack of data means using an existing model with weights pre-trained using other data sources, such as the ImageNet database (Deng et al ., 2009a ). The typical approach is to first train with an available, sizeable dataset and subsequently train with a smaller but more relevant dataset. In this way, the learning algorithms find the general image patterns from a big dataset (e.g. shapes, species patterns, and face patterns) and the individual differences from the smaller dataset. This process, known as transfer learning, allows researchers to use readily available large data sets like ImageNet to be used on data that seems highly unrelated to the data set of interest. For example, ImageNet has been annotated in categories like ‘balloon,’ ‘tiger,’ and ‘cat,’ yet can be used to train a network to classify fish vocalizations in the Mexican Gulf (Waddell et al ., 2021 ). However, transfer learning has a greater advantage when the domain difference between the data is small. Because distribution in real-world ecological data sets is particularly prominent, undesirable variations can result in misclassifications.

For a classification or object detection task, the dataset needs to be labelled (sometimes referred to as annotated), usually by a human expert (e.g. an ecologist). The labelled data is often referred to as the Y vector. An accurate classifier algorithm should correctly map the input, known as the X vectors (e.g. images) to the appropriate Y vector (the labels). These predicted labels are often referred to as the |$\hat{Y}$| vector, regardless of whether the predictions are correct ( ⁠|$\hat{Y}$| matches Y ), or incorrect ( ⁠|$\hat{Y}$| does not match Y ).

The labels for a classification task are distinct for each input variable, such as a species of fish for each image. This requires manual categorization and labelling of a large set of images. For object detection and semantic segmentation, the labels must also indicate where in the image the object of interest is located. In the case of audio input for RNN and CNN classification, the start and stop times of all events of interest must be labelled in order to segment the data into relevant categories. If object detection is used on spectrograms of audio, the frequency bands must also be labelled, encasing the contours of interest in the spectrogram. As is the case with images, existing labelled datasets also exist for audio, which can be used for training when data is otherwise limited (e.g. the DCLDE 2015 data set for baleen whale social calls; Huang et al ., 2016 ).

A labelled dataset is divided into three separate datasets, as illustrated in Figure 1 : training, validation, and testing. The training set is used to train the model, meaning that it tries to find an approach to map the training set’s input vector X (images) with the training set’s correct labels Y . The validation set follows and is first used to check whether the algorithm can map the validation set’s input vector X with the validation set’s correct labels Y , which is separate from the training set vectors. This validation set then provides a prediction/classification that can be used to evaluate the performance of the model. After this evaluation, further fine-tuning of the model hyperparameters can be done and the model retrained with the training set if needed ( Figure 1 ). Note that because the model has used the validation set as part of the training process, a new set is needed for the final check of how well the algorithm can classify. This is called the test set. Hence, the training data set is used for the actual optimization process, while the validation data set is used for performance feedback after each training step (epoch). This means that the validation data steers the tuning of hyperparameters and will therefore heavily influence the weights of the final model. In contrast, the testing data is used for verification of the final model only and not for the optimization and selection process while training. Because the test set is kept out of the entire training process, it serves as an independent verification of the resulting model.

If the trained network performs poorly, a possible cause is that the dataset is too small. A simple mitigation is data augmentation, which is used to artificially enlarge the data set and essentially duplicate the training data with modifications. It is important to note that only the training data should be augmented and any augmented image should not have a counterpart in the validation or test sets. Such an inappropriate use of augmentation would falsely increase performance. Typical augmentation techniques include flipping horizontally and vertically, rotating, scaling, cropping, translating the x- and y-coordinate systems, and adding noise. More advanced techniques include using Generative Adversarial Networks for generating new images.

The application of DNN provides an alternative to laborious or repetitive manual tasks, such as processing data from underwater recording equipment. The following section presents three cases in ecological research where DL is already used to alleviate data processing and is likely to become the method of choice. These cases exemplify the methods described in the section "A non-comprehensive review of DL":

Case 1: detection, classification, and tracking of fish in images and videos

Monitoring of fish populations and communities is a central activity within marine management and conservation. Traditional sampling methods to track population trends, estimate abundance, and to infer movement patterns of fish have relied on studies that involve animal handling (i.e. fishing gears, individual tags, and biologgers). These methods are not only invasive, but also time consuming. Developing and applying passive ways to both obtain the necessary data and to speed up analysis are therefore imperative. Today, automated detection, classification, and tracking of small-scale movements of fish through images and video are made possible with DL, an application well-suited to this task.

When selecting AI approaches for monitoring, consider that a real-life underwater scenario typically involves multiple fish present in the same image, which precludes the use of standard classification techniques. A solution to this problem is to introduce object detection before classification. The object detection step discriminates between individuals within an image and separates them, and in this way, prepares the image data for classification. Object detection and classification can be two completely separate steps in a pipeline (Knausgård et al ., 2021 ; Connolly et al ., 2021 ), or integrated as part of an object detector, such as YOLOv1-YOLOv5 (Redmon et al ., 2016 ; Bochkovskiy et al ., 2020 ; Jalal et al ., 2020 ; Yang et al ., 2021 ; Shin et al ., 2021 ; Yu et al ., 2021 ).

Detecting and counting species from still images and videos is relatively straightforward using standard DL object detection algorithms, as described in "A non-comprehensive review of DL". However, a challenge with setting up a detection algorithm is that well-established object detection training datasets, such as Coco (Lin et al ., 2014 ) and ImageNet (Deng et al ., 2009b ), include few or no images within the category of each species of fish and with very little variation in the background. Thus, the applicability of such data sets is limited to the first part of a transfer-learning process, in which object detection in general is learned. To increase the precision of detection for a specific use-case in marine ecology, one must then train the DNN with more relevant images (e.g. of fish in their natural environment). Collecting and labelling relevant image and video data is therefore central to building a high-performance and robust fish detector. Public datasets are currently an integral part of this research, particularly for fish detection and species identification (e.g. Fish4Knowlege; Fisher et al ., 2016 , datasets of temperate fish species; Knausgård et al ., 2021 , and across species, location, and depths, as in NOAA fishery datasets; Link et al ., 2015 , and the OzFish dataset; Ditria et al ., 2021 ). The best performance by AI in species identification (i.e. classification) is achieved with a specialized CNN that only classifies species without detecting at the same time. The squeeze-and-excitation-based CNN presented in (Knausgård et al ., 2021 ) reached classification accuracy of 99.27% on the Fish4Knowledge dataset (Fisher et al ., 2016 ) and 87.74% on a second temperate species dataset.

Marine researchers often collect videos rather than still images and are interested in tracking the same animal across consecutive frames to obtain information on behaviour (e.g. to estimate swimming speed; Beyan et al ., 2015 ), or to ensure that the same fish is not counted multiple times (Lopez-Marcano et al ., 2021 ). To continuously follow a moving object’s position in a video sequence, such as a swimming fish, object tracking can be used. One way of implementing tracking is to use a detection algorithm that feeds another tracking algorithm with position data. When tracking multiple objects (e.g. a school of fish), a track association decision needs to be made for each object (e.g. each individual fish). Thus, a complete tracking system typically consists of a detection algorithm, association of detection with tracks, and the actual tracking algorithm. In practice, tracking commonly involves Kalman filters or other recursive estimators to enable efficient dynamic tracking of objects  (Ristic et al ., 2004 ), including specific fish (Barreiros et al ., 2021 ). Another emerging approach is to let DL solve the entire multi-class tracking problem in one step (Ciaparrone et al ., 2020 ). This one-step approach typically results in a more homogeneous system, but with less fine-scale control than when applying well-understood recursive estimators. Further, a fully integrated CNN-tracking approach leaves less room for the user to include a priori information on expected fish dynamics and behaviour. A CNN-only approach will, however, completely avoid the meticulous tuning requirement of mathematical models and Kalman filter parameters.

We see DL as an essential building block for automating image and video analysis where the goal is to quantify, classify, and track fish. DL can either be used in a modular pipeline with separate steps for detection (Knausgård et al ., 2021 ), association, and track building, or as a complete solution to a multi-object tracking problem (Jalal et al ., 2020 ; Yang et al ., 2021 ; Shin et al ., 2021 ). As these DL tools are adaptable for use with different ecosystems or species by virtue of the training datasets used, the potential for AI in monitoring is great.

Case 2: image-based analysis for plankton monitoring

Plankton is a highly diverse group with very different morphologies and sizes ranging from submicrons to a few centimeters, or even a few meters (Lombard et al ., 2019 ). Plankton are responsible for about 50% of global primary production (Field et al ., 1998 ) and constitute the base of many marine food webs. Some species serve as bioindicators of ecosystem health, while others can form toxic blooms with adverse impacts on other marine life, including commercially important fish. Therefore, tracking seasonal, interannual, and spatial changes in plankton composition and abundance is central to coastal monitoring. Image-based monitoring is now an established tool in many regions and it generates an ever-increasing volume of plankton images each year. Various AI approaches have been developed to analyse this data and reduce manual processing. Plankton identification and counting are arguably some of the most useful examples of DL in marine biology. The ultimate goal is fully automated plankton classification without human biases (Culverhouse, 2007 ). This bias is not trivial, as human experts can only achieve 67–83 % self-consistency during a difficult classification task (Culverhouse et al ., 2003 ), although accuracy is much higher (> 90%) when working with natural plankton samples with many taxa which have variable classification difficulty (Luo et al ., 2018 ).

Several systems for image acquisition and AI analysis of plankton are commercially available  (Lombard et al ., 2019 ), including in situ  (e.g. Imaging FlowCytobot, VPR, and IISIS) and those that image samples, fixed or fresh, on research vessels or in the laboratory (e.g. ZooCam and FlowCam). All approaches share the same basic principles: pictures are taken of the sampling volume and the objects are segmented (i.e. into individual organisms). Each segment is then classified into one of several pre-defined classes, typically taxonomic or functional groups, but living organisms are always separated from non-living particles. Besides the predicted classification, the algorithms can extract object features (e.g. length, width, and equivalent spherical diameter) and, therefore, provide information on plankton community structure and function (e.g. normalized biomass size spectra; Wang et al ., 2020 ). Seasonal and interannual variability in plankton abundance and composition obtained using these image-based DL methods is comparable with traditional microscopy (e.g. FlowCam; Alvarez et al ., 2014 ).

Initial plankton classification models were based on statistical approaches but soon transitioned into machine learning solutions (Luo et al ., 2018 ; Kerr et al ., 2020 ), including algorithms that classified plankton based on object features such as size or edge, for example Support-Vector Machine (SVM) and Random Forest (RF) algorithms  (Faillettaz et al ., 2016 ; Fischer et al ., 2020 ). SVM and RF algorithms reach 70–90% accuracy in classification for the most abundant plankton groups, but rare or cryptic species can still be a problem. These classifiers also cannot extract the object features from the raw data and instead require these to be manually defined by ecologists, a cumbersome process. In order to overcome these issues, CNNs are being proposed, such as collaborative CNNs with configurations to deal with class imbalance (e.g. where one type of plankton is much more frequent than another;  Kerr et al ., 2020 ) or when the environment dynamically changes (dataset shift) using a supervised quantification scheme  (Orenstein et al ., 2020 ). These CNNs achieve state-of-the-art  90% classification accuracy when classifying independent test sets (e.g. 97% accuracy classifying 0.1 million FlowCam images; Kerr et al ., 2020 ), although accuracy decreases with very many diverse images (e.g. 83% accuracy for 52 million zooplankton images from IISIS; Briseño-Avena et al ., 2020 ). Other approaches to improve accuracy of conventional CNNs are through inclusion of context data (e.g. sampling location and time) in the classifier (Ellen et al ., 2019 ), using unsupervised clustering of data (Schroeder et al ., 2020 ), or combining CNNs with SVM classifiers (Cheng et al ., 2020 ).

DL enables a whole new approach to plankton coastal monitoring by (semi-) automatic analysis of samples either in situ or in the lab (Wang et al ., 2019 ). DL is used to monitor long-term, seasonal, and spatial changes in taxonomical groups (Briseño-Avena et al ., 2020 ) and size spectra  (Yu et al ., 2016 ; Wang et al ., 2020 ), to track plankton that serve as bioindicators of ecosystem health (Uusitalo et al ., 2016 ), or as an early-warning system for harmful algal blooms that impact higher trophic levels and, ultimately, humans (Gorocs et al ., 2018 ; Orenstein et al ., 2020 ). However, DL cannot yet replace a taxonomist for difficult identification tasks (e.g. identification of certain species or life stages of zooplankton or larval fish), and as such are not yet adequate for studies that require high taxonomic resolution. Experts are also required to create training sets and validate the results. However, manual hours can be reduced if training sets and analysis pipelines are made publicly available (Li et al ., 2020 ; Chen, 2021 ; Schmid et al ., 2021 ), as well as through the creation of global databases and training sets (e.g. Ecotaxa; Picheral et al ., 2017 ). Ultimately, the combination of traditional physical plankton sampling with autonomous platforms that combine image-based data with data from other sources (e.g. genomics, acoustics, and pigments) appears to be the best way forward for coastal plankton monitoring studies (Gorsky et al ., 2019 ; Lombard et al ., 2019 ).

Case 3: passive acoustic monitoring of whales

The use of long-term underwater passive acoustic monitoring (PAM) recording has grown in the last couple of decades to become an indispensable tool for investigating relative population trends and temporal and spatial migration patterns of a wide range of whale species (Wiggins and Hildebrand, 2016 ).

For many years, the standard procedure for detecting and classifying whale calls from PAM recording has been to retrieve the sound recording, use a software package like Triton (Wiggins and Hildebrand, 2007 ) to create spectrograms lasting 1–2 min, then have the spectrograms manually scanned for call contours by a trained data analyst. This method is not only highly labor-intensive, as PAM recording can cover months, if not years, but the results are also subjective (Baumgartner and Mussoline, 2011 ). As many whale calls are highly stereotypical, algorithms like matched filtering (Giannakis and Tsatsanis, 1990 ) and spectrogram correlation  (Mellinger and Clark, 1997 ) have successfully been developed for automated call detection. However, these methods tend to work poorly on calls with more variability in frequency modulation. Hence, manually scanning spectrograms continues to be used for many call types. The manual procedure of visually scanning spectrograms for known call contours is very similar to the image classification process. Further, sound classification using DL is becoming well established outside of marine bioacoustics (Piczak, 2015 ; Sharma et al ., 2020 ; Mushtaq et al ., 2021 ), which has led to significant interest in using CNN for automated whale call detection.

Among the whale calls recently being investigated using CNN are those of the beluga whale ( Delphinapterus leucas ) with an AUC of 0.9906 (Zhong et al ., 2020 ), North Atlantic right whale ( Eubalaena glacialis ) with an AUC of 0.902  (Shiu et al ., 2020 ), killer whales ( Orcinus orca ) with an AUC of 0.9523  (Bergler et al ., 2019 ), and sperm whales ( Physeter macrocephalus ) with 99.5% accuracy in detecting sperm whale clicks in 650 spectrograms  (Bermant et al ., 2019 ). A drawback of CNN classification without object detection is that it does not relay information about where in the image an object is located. For example, when examining spectrograms where the x -axis is the timeline, no information is included about the call’s specific time, nor the number of calls, thus the CNN serves as a “presence” identification tool only. A work-around for this issue has been to make the spectrograms very small, covering only a short timeline (e.g. 2 s; Bergler et al ., 2019 ). When creating a spectrogram, there needs to be an overlap between two consecutive spectrograms. Otherwise, a call located at the intersection of two spectrograms might be missed. Using short spectrograms combined with these overlaps can increase the redundant data up to 20% (Bergler et al ., 2019 ) and thereby increase the computational cost at a similar level. Object detection can solve these issues for whale call detection. For example, a custom-made region-based CNN for detecting regions of interest in combination with a transformed pre-trained CNN for further classifying the regions of interest was successfully trained and tested on the highly variable D call emitted by blue whales and 40 Hz calls emitted by fin whales ( Balaenoptera physalus ; Rasmussen and Širović, 2021 ).

Looking to the future, use of AI generally, and DL specifically, in automated detection of whale calls in PAM recordings will undoubtedly benefit from the recent developments in neural architecture search (NAS) algorithms  (Sun et al ., 2019 ). This new technique of automatically developing network architecture from prefabricated blocks will cut down significantly on the work needed to adapt networks to fit specific species and calls, and make CNN more accessible for whale researchers. A general move from using CNNs to perform image recognition on spectrograms extracted from the PAM to using DL directly on the PAM is also anticipated. This can be done via recurrent networks like long short-time memory networks  (Hochreiter and Schmidhuber, 1997 ) or a recently developed type of network called the transformer (Vaswani et al ., 2017 ).

A common theme of the established cases mentioned above is that they replace tasks currently conducted by humans—where using DL can reduce costs, labour, and sometimes improved accuracy compared to human analysts. However, DL has the capacity to be applied to solve more complex tasks, detecting patterns in visual and acoustic data that are difficult for humans to reliably detect or discriminate. In this section, we illustrate novel research avenues in which we predict DL will be successfully applied in the near future.

Identifying and characterizing individual phenotypes

Case 4: visual re-identification of individuals in wild fish populations.

Methods for individual identification are needed to answer many questions in animal behaviour and ecology, such as growth, movement, and survival inferred from capture–recapture studies (Clutton-Brock and Sheldon, 2010 ). Currently, the most common approach is to mark animals with various physical identifiers to recognize individuals upon re-sight or re-capture, such as leg rings on birds, number scratching or paint on reptiles, or lip tattoos on larger carnivores. In marine and freshwater systems, capture–recapture studies on fish are most often performed using external number tags or radio-frequency identification (RFID) tags (Pine et al ., 2003 ). However, trapping and tagging surveys are often costly, logistically challenging to conduct, and are intrusive to the animals.

A less invasive and more practical way forward for data collection is to use images or videos from wildlife cameras and perform DL image analysis by taking advantage of natural markings that make individuals identifiable (Schneider et al ., 2019 ). Like humans, many animals have unique features about their individual appearance, such as intricate patterns of spots and stripes on the skin, fur, or feathers. A trained computer vision algorithm can distinguish between individuals as different classes, even when the identifying features are highly complex. CNN networks have been trained to recognize individuals (individual re-identification [Re-ID]) from photos of animals across many taxa, including birds (e.g., 93.6% accuracy; Ferreira et al ., 2020b ), turtles (e.g., 95% accuracy; Carter et al ., 2014 ), and terrestrial and marine mammals (e.g., 92.5% accuracy; Schofield et al ., 2019 ). Many fish species also have solid visual pigmentation; stripes, spots, or mosaic in contrasting colours that can be clearly seen in images and video surveys  (dala Corte and Moschetta, 2016 ; Hau and Sadovy de Mitcheson, 2019 ; Mucientes et al ., 2019 ), particularly coastal fish like the corkwing wrasse ( Symphodus melops ; Figure 4 ). Therefore, development of Re-ID has potential to replace physical tagging for individual identification of teleost fish, and would also be of great value for monitoring, as it could be used to assess individual movement, behaviour, and growth. Re-ID could also solve the problem of double counting when individuals re-enter the field of view, thus improving video-based monitoring of abundance  (Aguzzi et al ., 2015 ; Campos-Candela et al ., 2018 ; Perry et al ., 2018 )

Established and emerging cases for DL in marine biology, from individuals, to species, to ecosystems. Data input type icons represent images/video (cases 1, 2, 4, and 6), audio (cases 3 and 5), and large-scale environmental monitoring data that is often stored on remote servers (i.e. ”the cloud”; case 7). Photo credits: Geir Eliassen (ghost fishing gear), Frithjof Moy/Havforskningsinstituttet (kelp forest).

As far as we are aware, Re-ID by CNN has not been tested in wild populations. One of the challenges preventing the widespread development of AI-based Re-ID is the need for photos or videos of known individuals, independently validated with high certainty, for the training and validation of the algorithm. One solution to this problem is collecting data by using remote detection systems, such as RFID technology, to identify individuals tagged with passive integrated transponders (PITs). By combining PIT-tagging with RFID and synchronized underwater cameras, a large, automatically labelled dataset of many individuals could be created over a relatively short time (Schneider et al ., 2019 ; Ferreira et al ., 2020b ).

Case 5: inter- and intra-individual variability in fish vocal communications

Acoustic communication is a fundamental component of animal life, especially for aquatic species for which visual cues are not as effective (Tessler et al ., 2017 ). For example, many fish hear their species mating choruses from several kilometers away (Winn, 1964 ). Subtle variation in complex acoustic signals is challenging for humans to detect or interpret. Furthermore, using algorithms to detect patterns that defy human perception has technological limitations, including processing high volumes of noisy, real-time acoustic data. Using algorithms to detect acoustic signaling presents the additional challenge of source identification in moving animals. However, advances both in audio recording technologies and in DL algorithms that can detect and classify acoustic signals in natural settings have opened up new systems for study, both on land and at sea (Parsons et al ., 2009 ). These technologies unlock the potential for understanding inter- and intra-individual variation in acoustic communication of fish.

Marine mammals are relatively well studied in this respect, as vocalizations can be classified at the species, population, and even individual levels (e.g., Case 3). However, understanding of the diversity of fish vocalizations and how these vary within species is poorly understood. Moving beyond species-level to population- and individual-level classification of vocalizations is necessary to understand the ecological and evolutionary consequences of acoustic communication in fish and the potential impacts of anthropogenic noise pollution on them. Further, individual-level classification permits a better understanding of intra-individual variation in communication, which is necessary for understanding the role of vocalizations in fish behaviour and personality.

A prime example is Atlantic cod ( Gadus morhua ), which use drumming vocalizations during social interactions, particularly during mating (Brawn, 1961 ). Yet, our understanding of inter-and intra-individual variation in drumming is limited. There is potential to catalog individual variation in sound production using DL algorithms (Deng and Yu, 2014 ). Fine-scale individual variation in fish sounds, especially without a priori knowledge, is beyond human perception. Thus, automating this task requires DL approaches that do not rely on labelled training sets. Specifically, CNNs can detect and classify fish sounds by implementing a transformer network (Deng and Yu, 2014 ). Transformer networks work solely on optimized self attention (looking at other positions in the input sequence for clues that can help lead to a better encoding of each input element) and are currently state-of-the-art in translation tasks. The transformer network is rapidly replacing RNNs previously used for this kind of task, as it solves two of the problems inherent in RNNs: 1) long computing times due to serial processing and 2) vanishing gradients.

Case 6: ghost fishing gear detection

When fishing gear is lost, the continued mortality of fish, crustaceans, and other species caught in the gear is termed ghost fishing (Brown and Macfadyen, 2007 ). The problem is widespread and high rates of fish trap loss are reported (Vadziutsina and Riera, 2020 ). Using DL to detect and locate lost gear can greatly increase the efficiency of clean-up efforts, as human effort could then focus on retrieving gear (e.g., using remotely operated vehicles). Detection of ghost fishing gear has been achieved using side-scan sonar for data acquisition followed by feature cloud generation, which involves looking for objects in an image by identifying areas of high entropy, then clustering and noise reduction to separate the objects from noise by looking for clusters of the identified areas  (Labbe-Morissette and Gauthier, 2020 ).

The next step is using autonomous object detection to extract the location of lost fishing gear. The detection of lost fishing nets using a towed underwater camera followed by automatic object detection has been achieved with a region-based CNN (R-CNN; Politikos et al ., 2021 ). In that study, fishing nets were detected with higher precision than any other type of marine litter. Detection of more types and features of fishing gear is of interest to researchers and clean-up efforts (e.g. whether the feature detected is a trap, fyke net, or ropes). Image classification may be an effective approach to provide this level of detail, where low resolution images are not usually a hindrance for successful image classification. As well as video, side-scan sonar on autonomously operated vehicles could provide the data needed for this approach. Towed underwater cameras may represent a low-cost option for data collection, whereas autonomously operated vehicles equipped with side-scan sonar represent a high-cost option.

Case 7: carbon cycling by fish

The ocean sinks approximately one-third of greenhouse gas emissions out of the atmosphere, including carbon dioxide. The ocean carbon sink is driven by a physical and a biological pump. As well as plankton and bacteria, fish contribute to the biological pump, with recent estimates suggesting 16 percent of sinking carbon could be due to fish (Saba et al ., 2021 ). However, the role of fish in the biological pump is not well understood (Martin et al ., 2021 ). The data on fish required to improve our understanding relates to metabolic use and excretion of consumed carbon and other nutrients; properties of carbon and nutrient outputs and their fate in the environment; habitat use and connectivity of ecosystems; and physical interactions with extrinsic carbon and nutrients in the environment. As well as advancing knowledge of the role of fish, this knowledge could inform effective management approaches to maintaining or restoring ecosystem carbon function. As an emerging field, zoogeochemistry has the advantage that much of the relevant data are already published for other purposes. For example, metabolic rates and behavioural data is already published for many commercially important species through fisheries and climate change research. Using AI in this field has the potential to expedite a better understanding of fish ecological functions, effects of human disturbance, and therefore potential management of important carbon sink habitats. Here, we present a few of the options available to apply DL to zoogeochemistry research.

In habitats where visual sampling is possible, video images could be used with object detection, classification, and tracking to identify the presence or absence, behaviour, and features of particles from fish and their short-term fate  (e.g., defecation, spawning, and whether material reaches and settles on the sea floor). This could inform estimates of the volume of carbon transferred into or out of a habitat by fish, and the short-term fate of the carbon or nutrient they release. Methods that use AI computer vision to determine the connectivity of fish populations can also be of value in estimating carbon flow  (Lopez-Marcano et al ., 2021 ). The long-term fate of carbon and nutrients depends on physical, chemical, and biological conditions of the environment. Graph networks, which map out a physical system using nodes to form a graph, have recently been used to simulate the physical behaviour of materials (Sanchez-Gonzalez et al ., 2020 ). This technology has potential application to estimating the probable fate of carbon and nutrient outputs through simulations that combine oceanographic data with features of the carbon released by fish. With many variables to consider, recent approaches to assessing carbon contained in sediments in different habitats include a combination of survey (acoustic and image-based) and bathymetry data, modelling, and remote ground-truthing (Wilson et al ., 2018 ; Hunt et al ., 2020 ). The current approach is manual, but there is potential for AI application to link habitats to carbon fates and make spatial and temporal estimates on cycling and sinking of carbon and nutrients. Graph networks could be applied to generate probable long-term fates of carbon and nutrient outputs based on isotopes (Lyubchich and Woodland, 2019 ), or where simulations can be informed from video observations and environmental parameters such as season, temperature, currents, and maps of habitat type (Sanchez-Gonzalez et al ., 2020 ).

As has been mentioned in earlier cases, biological data for fish is partially or fully available for commercially targeted species in online databases (e.g. Fishbase). Such databases have been used to generate estimates of nutrient output from fish, such as nitrogen and phosphorous (Schiettekatte et al ., 2020 ). AI can be trained on these databases to estimate ecological and behavioural carbon flows, including on food webs and habitat use  (Bohan et al ., 2011 ). This training could then be applied to generate estimates for species where ecological data is limited, such as deep-sea fish. The research needs for deep-sea fish are urgent as commercial interest is increasing at the same time as the significance of these species in moving carbon from surface waters to the deep sea is beginning to be explored, but data collection methods are expensive, time consuming, and patchy (Martin et al ., 2020 ). In this instance, DL could be used to detect probable carbon flows by using logic-based machine learning (i.e. techniques that incorporate background knowledge or rules; Bohan et al ., 2011 ).

We are entering a new era in ocean research and management thanks to new technological developments in observational methods combined with AI-supported data analysis. Data collection, processing, and interpretation are at the core of ecological studies and biodiversity monitoring. Scientists are increasingly relying on indirect observations from various sensors generating large and complex data sets, especially in the aquatic environment. Thus, we envision that within a decade, marine researchers will firmly integrate AI and DL in data collection and analysis within most sub-fields of applied marine biology. This development will only continue to accelerate with new generations of biologists better educated in computer science and informatics (Weinstein, 2018 ).

Non-human, autonomous, and remote platforms such as cabled observatories, autonomous underwater vehicles or gliders, and ships of opportunity will have a pivotal role in ocean monitoring (Whitt et al ., 2020 ). These platforms will record continuous, real-time information on water physics, chemistry, community composition, and biomass of plankton, fish, and other marine species. For example, long-term monitoring of harmful bloom-forming plankton species can be achieved using inexpensive image technology anchored to piers (Gorocs et al ., 2018 ; Orenstein et al ., 2020 ). Similarly, changes in whale population trends and migrations can be investigated using PAM (Szesciorka et al ., 2020 ). These methods are likely to decrease reliance on manual analysis or direct sampling through more invasive, expensive, time-consuming, or labour-intensive traditional approaches. This new way of observing the ocean will generate large volumes of data that will only be feasible to analyse with the help of AI. Therefore, AI will play a key role in making routine processes more time-efficient and alleviate the manual work required. For example, a trained data analyst currently needs 50–350 workdays to manually scan 1 year’s worth of PAM recordings for whale calls (Woods and Sirovic, pers. comm.). In contrast, the same task can be accomplished by a trained neural network in approximately 4 workdays (Bergler et al ., 2019 ). Fully automated coastal monitoring systems will be faster and more efficient at detecting changes of interest, such as necessitating warnings to the public where toxic algae are abundant and enabling redirection of boat traffic where whales are moving across shipping routes. Altogether, this monitoring information will be valuable in the development of biological indicators and in integrated assessments to support EBM (Tam et al ., 2017 ).

It is important to emphasize that expert work will always be needed to create and correctly label training sets and revise automated analyses, such as when new species enter a system. A model’s accuracy performance is likely to decrease significantly when new species that are not part of the training data are introduced. There is no established automated approach to detect when models need retraining. Repeated validation of the models is required to ensure up-to-date performance. This anticipated demand emphasizes the need to develop multidisciplinary skills in researchers at all career stages, as well as the skills required to form fruitful interdisciplinary collaborations (McDonald et al ., 2018 ).

Collaborative work based on open access and sharing culture (from model configurations to training sets) will be essential to advance this future. While this is a common practice within AI communities, the culture of marine science is not as open. However, funding agencies, publishers, and institutions are increasingly enforcing open access for data generated via public funds. The FAIR Principles for scientific data management and stewardship are now widely adopted (Wilkinson et al ., 2019 ). These emphasize improving the access, utility, and reuse of data by machines in addition to individual researchers. As such, they may play a vital role in applying AI to the marine domain. Some collaborative initiatives are underway to create global databases for plankton and benthic images and training sets (e.g., EcoTaxa; Picheral et al ., 2017 and BIIGLE; Langenkämper et al ., 2017 ), as well as pipelines (Chen, 2021 ). Ultimately, we envision libraries of images, videos, and metadata available globally, similarly to the open access GenBank database for sequence information and associated metadata for genetic material hosted by the National Center for Biotechnology Information (NCBI) in the United States.

We have provided examples of how image and audio analysis are already used to analyse marine biodiversity distribution and dynamics in non-invasive ways, emerging applications of AI, and a look at what the future of AI in marine ecology requires. The United Nations Decade of the Ocean has just started, with the aim of achieving “a healthy, safe, and resilient ocean for sustainable development by 2030 and beyond”. We have shown that AI will be key to achieve this goal by developing new technology to uncover new aspects of and potential threats to marine ecosystems’ structures and functions, thereby informing EBM. This new knowledge will directly address several of the key challenges identified for the Decade, from effective EBM and biodiversity conservation, to creating a digital representation of the ocean and delivering data, knowledge, and technology to all. The Decade of the Ocean initiative promotes global cooperation and interdisciplinary efforts at all levels, which are at the core of how AI-linked marine studies will progress. Where researchers have the opportunity to gather large amounts of complex ecological data, unfamiliarity with AI jargon and the latest developments should not prevent collaborations with data and computer scientists to support EBM of ocean resources during this time of rapid change.

Morten Goodwin is supported by the Norwegian Research Council HAVBRUK2 innovation project CreateView Project (no. 309784). Rebekah A. Oomen is supported by the James S. McDonnell Foundation 21st Century Postdoctoral Fellowship (no. 220020556). Susanna Huneide Thorbjørnsen is supported by Handelens Miljøfond.

No original data are presented.

We thank four anonymous reviewers for their feedback which improved the manuscript.

All authors have an equal contribution.

Aguzzi   J. , Doya   C. , Tecchio   S. , De Leo   F. C. , Azzurro   E. , Costa   C. , Sbragaglia   V.  et al.    2015 . Coastal observatories for monitoring of fish behaviour and their responses to environmental changes . Reviews in Fish Biology and Fisheries , 25 : 463 – 483 .

Google Scholar

Aloysius   N. , Geetha   M.   2017 . A review on deep convolutional neural networks . In   Proceedings of the 2017 International Conference on Communication and Signal Processing (ICCSP)’ , IEEE , pp. 0588 – 0592 .

Google Preview

Alvarez   E. , Moyano   M. , Lopez-Urrutia   A. , Nogueira   E. , Scharek   R.   2014 . Routine determination of plankton community composition and size structure: a comparison between FlowCAM and light microscopy . Journal of Plankton Research , 36 : 170 – 184 .

Antão   L. H. , Bates   A. E. , Blowes   S. A. , Waldock   C. , Supp   S. R. , Magurran   A. E. , Dornelas   M.  et al.    2020 . Temperature-related biodiversity change across temperate marine and terrestrial systems . Nature Ecology and Evolution , 4 : 927 – 933 .

Bacheler   N. M. , Geraldi   N. R. , Burton   M. L. , Muñoz   R. C. , Kellison   G. T.   2017 . Comparing relative abundance, lengths, and habitat of temperate reef fish using simultaneous underwater visual census, video, and trap sampling . Marine Ecology Progress Series , 574 : 141 – 155 .

Barreiros   M. d. O. , Dantas   D. d. O. , Silva   L. C. d. O. , Ribeiro   S. , Barros   A. K.   2021 . Zebrafish tracking using YOLOv2 and Kalman filter . Scientific Reports , 11 : 3219 .

Baumgartner   M. F. , Mussoline   S. E.   2011 . A generalized baleen whale call detection and classification system . The Journal of the Acoustical Society of America , 129 : 2889 – 2902 .

Ben Lazreg   M. , Goodwin   M. , Granmo   O.-C.   2019 . An iterative information retrieval approach from social media in crisis situations . In   Proceedings of the 2019 International Conference on Information and Communication Technologies for Disaster Management (ICT-DM) , IEEE , pp. 1 – 8 .

Ben Lazreg   M. , Noori   N. , Comes   T. , Goodwin   M.   2019 . Not a target. a deep learning approach for a warning and decision support system to improve safety and security of humanitarian aid workers . In   Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence , pp. 378 – 382 .

Bergler   C. , Schröter   H. , Cheng   R. X. , Barth   V. , Weber   M. , Nöth   E. , Hofer   H.  et al.    2019 . Orca-spot: an automatic killer whale sound detection toolkit using deep learning . Scientific reports , 9 : 1 – 17 .

Bermant   P. C. , Bronstein   M. M. , Wood   R. J. , Gero   S. , Gruber   D. F.   2019 . Deep machine learning techniques for the detection and classification of sperm whale bioacoustics . Scientific Reports . 9 : 1 – 10 .

Beyan   C. , Boom   B. J. , Liefhebber   J. M. P. , Shao   K.-T. , Fisher   R. B.   2015 . Natural swimming speed of Dascyllus reticulatus increases with water temperature . ICES Journal of Marine Science , 72 : 2506 – 2511 .

Beyan   C. , Browman   H. I.   2020 . Setting the stage for the machine intelligence era in marine science . ICES Journal of Marine Science . 77 : 1267 – 1273 .

Bochkovskiy   A. , Wang   C.-Y. , Liao   H.-Y. M.   2020 . YOLOv4: optimal speed and accuracy of object detection . arXiv preprint not peer reviewed arXiv:2004.10934 .

Bogucki   R. , Cygan   M. , Khan   C. B. , Klimek   M. , Milczek   J. K. , Mucha   M.   2019 . Applying deep learning to right whale photo identification . Conservation Biology , 33 : 676 – 684 .

Bohan   D. A. , Caron-Lormier   G. , Muggleton   S. , Raybould   A. , Tamaddoni-Nezhad   A.   2011 . Automated discovery of food webs from ecological data using logic-based machine learning . PLoS ONE , 6 : e29028 .

Brawn   V. M.   1961 . Reproductive behaviour of the Cod (Gadus Callarias L.) . Behaviour , 18 : 177 – 197 .

Briseño-Avena   C. , Schmid   M. S. , Swieca   K. , Sponaugle   S. , Brodeur   R. D. , Cowen   R. K.   2020 . Three-dimensional cross-shelf zooplankton distributions off the Central Oregon coast during anomalous oceanographic conditions . Progress in Oceanography , 188 : 102436 . .

Brown   J. , Macfadyen   G.   2007 . Ghost fishing in European waters: impacts and management responses . Marine Policy , 31 : 488 – 504 .

Campos-Candela   A. , Palmer   M. , Balle   S. , Alós   J.   2018 . A camera-based method for estimating absolute density in animals displaying home range behaviour . Journal of Animal Ecology , 87 : 825 – 837 .

Carter   S. J. , Bell   I. P. , Miller   J. J. , Gash   P. P.   2014 . Automated marine turtle photograph identification using artificial neural networks, with application to green turtles . Journal of Experimental Marine Biology and Ecology , 452 : 105 – 110 .

Chaudhari   S. , Mithal   V. , Polatkan   G. , Ramanath   R. , 2021 . An attentive survey of attention models . ACM Transactions on Intelligent Systems and Technology (TIST) . 12 : 1 – 32 ., doi:10.1145/3465055 .

Cheng   X. , Ren   Y. , Cheng   K. , Cao   J. , Hao   Q.   2020 . Method for training convolutional neural networks for in situ plankton image recognition and classification based on the mechanisms of the human eye . Sensors (Basel) , 20 : 2592 .

Christin   S. , Hervet   E. , Lecomte   N.   2019 . Applications for deep learning in ecology . Methods in Ecology and Evolution , 10 : 1632 – 1644 .

Ciaparrone   G. , Sánchez   F. L. , Tabik   S. , Troiano   L. , Tagliaferri   R. , Herrera   F.   2020 . Deep learning in video multi-object tracking: a survey . Neurocomputing , 381 : 61 – 88 .

Clutton-Brock   T. , Sheldon   B. C.   2010 . Individuals and populations: the role of long-term, individual-based studies of animals in ecology and evolutionary biology . Trends in Ecology and Evolution , 25 : 562 – 573 .

Connolly   R. , Fairclough   D. , Jinks   E. , Ditria   E. , Jackson   G. , Lopez-Marcano   S. , Olds   A.  et al.    2021 . Improved accuracy for automated counting of a fish in baited underwater videos for stock assessment . bioRxiv . doi: 10.1101/2021.02.01.429285 .

Culverhouse   P. F.   2007 . Natural object categorization: man versus machine . Automated Taxon Identification in Systematics: Theory, Approaches and Applications, MacLeod , CRC Press , pp. 25 – 46 .

Culverhouse   P. F. , Williams   R. , Reguera   B. , Herry   V. , González-Gil   S.   2003 . Do experts make mistakes? A comparison of human and machine indentification of dinoflagellates . Marine Ecology Progress Series , 247 : 17 – 25 .

dala Corte   R. B. , Moschetta   J. B. , Becker   F. G.   2016 . Photo-identification as a technique for recognition of individual fish: a test with the freshwater armored catfish Rineloricaria aequalicuspis Reis & Cardoso, . . 2001 (Siluriformes: Loricariidae) . Neotropical Ichthyology , 01 : e150074 .

Dargan   S. , Kumar   M. , Ayyagari   M. R. , Kumar   G.   2019 . A survey of deep learning and its applications: a new paradigm to machine learning . Archives of Computational Methods in Engineering , 27 : 1071 – 1092 .

Deng   J. , Dong   W. , Socher   R. , Li   L.-J. , Li   K. , Fei-Fei   L.   2009a . ImageNet: a large-scale hierarchical image satabase . In   Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition ‘CVPR09’ . IEEE Computer Society .

Deng   J. , Dong   W. , Socher   R. , Li   L.-J. , Li   K. , Fei-Fei   L.   2009b . Imagenet: a large-scale hierarchical image database . In   Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition , IEEE , pp. 248 – 255 .

Deng   L. , Yu   D.   2014 . Deep learning: methods and applications . Foundations and Trends in Signal Processing . 7 : 197 – 387 .

Ditria   E. M. , Connolly   R. M. , Jinks   E. L. , Lopez-Marcano   S.   2021 . Annotated video footage for automated identification and counting of fish in unconstrained seagrass habitats . Frontiers in Marine Science . 8 : 629485 .

Ellen   J. S. , Graff   C. A. , Ohman   M. D.   2019 . Improving plankton image classification using context metadata . Limnology and Oceanography: Methods , 17 : 439 – 461 .. https://aslopubs.onlinelibrary.wiley.com/doi/full/10.1002/lom3.10234

Esteva   A. , Robicquet   A. , Ramsundar   B. , Kuleshov   V. , DePristo   M. , Chou   K. , Cui   C.  et al.    2019 . A guide to deep learning in healthcare . Nature Medicine , 25 : 24 – 29 .

Faillettaz   R. , Picheral   M. , Luo   J. Y. , Guigand   C. , Cowen   R. K. , Irisson   J.-O.   2016 . Imperfect automatic image classification successfully describes plankton distribution patterns . Methods in Oceanography , 15 : 60 – 77 .

Ferreira   A. C. , Silva   L. R. , Renna   F. , Brandl   H. B. , Renoult   J. P. , Farine   D. R. , Covas   R.  et al.    2020a . Deep learning-based methods for individual recognition in small birds . Methods in Ecology and Evolution , 11 : 1072 – 1085 .

Ferreira   A. C. , Silva   L. R. , Renna   F. , Brandl   H. B. , Renoult   J. P. , Farine   D. R. , Covas   R.  et al.    2020b . Deep learning-based methods for individual recognition in small birds . Methods in Ecology and Evolution , 11 : 225300989 .

Field   C. B. , Behrenfeld   M. J. , Randerson   J. T. , Falkowski   P.   1998 . Primary production of the biosphere: integrating terrestrial and oceanic components . Science , 281 : 237 – 240 .

Fischer   A. D. , Hayashi   K. , McGaraghan   A. , Kudela   R. M.   2020 . Return of the “age of dinoflagellates” in Monterey Bay: drivers of dinoflagellate dominance examined using automated imaging flow cytometry and long-term time series analysis . Limnology and Oceanography , 65 : 2125 – 2141 .. .

Fisher   R. B. , Chen-Burger   Y.-H. , Giordano   D. , Hardman   L. , Lin   F.-P.  et al.    2016 . Fish4Knowledge: Collecting and Analyzing Massive Coral Reef Fish Video Data , 104 , Springer .

Giannakis   G. B. , Tsatsanis   M. K.   1990 . Signal detection and classification using matched filtering and higher order statistics . IEEE Transactions on Acoustics, Speech, and Signal Processing , 38 : 1284 – 1296 .

Girshick   R.   2015 . Fast r-cnn . In   Proceedings of the IEEE International Conference on Computer Vision , pp. 1440 – 1448 .

Girshick   R. , Donahue   J. , Darrell   T. , Malik   J.   2014 . Rich feature hierarchies for accurate object detection and semantic segmentation . In   Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 580 – 587 .

Goodwin   M.   2020 . AI: Myten om maskinene , Humanist forlag .

Gorocs   Z. , Tamamitsu   M. , Bianco   V. , Wolf   P. , Roy   S. , Shindo   K. , Yanny   K.  et al.    2018 . A deep learning-enabled portable imaging flow cytometer for cost-effective, high-throughput, and label-free analysis of natural water samples . Light: Science and Applications , 7 : 52312417 .

Gorsky   G. , Bourdin   G. , Lombard   F. , Pedrotti   M. L. , Audrain   S. , Bin   N. , Boss   E.  et al.    2019 . Expanding Tara oceans protocols for underway, ecosystemic sampling of the ocean-atmosphere interface during Tara Pacific expedition (2016–2018) . Frontiers in Marine Science , 6 : 750 . https://www.frontiersin.org/article/10.3389/fmars.2019.00750 .

Grasso   I. , Archer   S. D. , Burnell   C. , Tupper   B. , Rauschenberg   C. , Kanwit   K. , Record   N. R.   2019 . The hunt for red tides: deep learning algorithm forecasts shellfish toxicity at site scales in coastal maine . Ecosphere , 10 : e02960 .

Hau   C. Y. , Sadovy de Mitcheson   Y.   2019 . A facial recognition tool and legislative changes for improved enforcement of the cites appendix ii listing of the humphead wrasse, Cheilinus undulatus . Aquatic Conservation: Marine and Freshwater Ecosystems , 29 : 2071 – 2091 .

He   K. , Gkioxari   G. , Dollár   P. , Girshick   R.   2017 . Mask r-cnn . In   Proceedings of the IEEE International Conference on Computer Vision , pp. 2961 – 2969 .

He   K. , Zhang   X. , Ren   S. , Sun   J.   2016 . Deep residual learning for image recognition . In   Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 770 – 778 .

Hochreiter   S. , Schmidhuber   J.   1997 . Long short-term memory . Neural Computation , 9 : 1735 – 1780 .

Hu   J. , Shen   L. , Sun   G.   2018 . Squeeze-and-excitation networks . In   Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 7132 – 7141 .

Huang   H. C. , Joseph   J. , Huang   M. J. , Margolina   T.   2016 . Automated detection and identification of blue and fin whale foraging calls by combining pattern recognition and machine learning techniques . In   Proceedings of the OCEANS 2016 MTS/IEEE Monterey . IEEE , pp. 1 – 7 .

Hunt   C. , Demšar   U. , Dove   D. , Smeaton   C. , Cooper   R. , Austin   W. E.   2020 . Quantifying marine sedimentary carbon: a new spatial analysis approach using seafloor acoustics, imagery, and ground-truthing data in Scotland . Frontiers in Marine Science , 7 : 588 .

Jalal   A. , Salman   A. , Mian   A. , Shortis   M. , Shafait   F.   2020 . Fish detection and species classification in underwater environments using deep learning with temporal information . Ecological Informatics , 57 : 101088 .

Kerr   T. , Clark   J. R. , Fileman   E. S. , Widdicombe   C. E. , Pugeault   N.   2020 . Collaborative deep learning models to handle class imbalance in flowcam plankton imagery . IEEE Access , 8 : 170013 – 170032 .

Knausgård   K. M. , Wiklund   A. , Sørdalen   T. K. , Halvorsen   K. T. , Kleiven   A. R. , Jiao   L. , Goodwin   M.   2021 . Temperate fish detection and classification: a deep learning based approach . Applied Intelligence , 1 – 14 .

Koch   G. , Zemel   R. , Salakhutdinov   R.   2015 . Siamese neural networks for one-shot image recognition . In   Proceedings of the ICML Deep Learning Workshop , 2 , Lille, France .

Krizhevsky   A. , Sutskever   I. , Hinton   G. E.   2012 . Imagenet classification with deep convolutional neural networks . in Pereira   F. , Burges   C. J. C. , Bottou   L. , Weinberger   K. Q. , eds, Advances in Neural Information Processing Systems , 25 , Curran Associates, Inc , pp. 1097 – 1105 .. https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf .

Labbe-Morissette   G. , Gauthier   S.   2020 . Unsupervised extraction of underwater regions of interest in side scan sonar imagery . Journal of Ocean Technology , 15 , pp. 96 – 108 .

Langenkämper   D. , Zurowietz   M. , Schoening   T. , Nattkemper   T. W.   2017 . Biigle 2.0 – browsing and annotating large marine image collections . Frontiers in Marine Science , 4 : 83 .

LeCun   Y. , Bengio   Y . et al.    1995 . Convolutional networks for images, speech, and time series . The Handbook of Brain Theory and Neural Networks , 3361 : 1995 .

LeCun   Y. , Bengio   Y. , Hinton   G.   2015 . Deep learning . Nature , 521 : 436 – 444 .

Lessmann   S. , Haupt   J. , Coussement   K. , De Bock   K. W.   2019 . Targeting customers for profit: an ensemble learning framework to support marketing decision-making . Information Sciences , 557 : 286 – 301 .

Li   D. , Du   L.   2021 . Recent advances of deep learning algorithms for aquacultural machine vision systems with emphasis on fish . Artificial Intelligence Review , 1 : 1 – 40 .

Li   J. , Yang   Z. , Chen   T.   2021 . ‘Dyb-planktonnet’ . doi: 10.21227/875n-f104 .

Li   Q. , Sun   X. , Dong   J. , Song   S. , Zhang   T. , Liu   D. , Zhang   H.  et al.    2020 . Developing a microscopic image dataset in support of intelligent phytoplankton detection using deep learning . ICES Journal of Marine Science , 77 : 1427 – 1439 .

Lin   T.-Y. , Maire   M. , Belongie   S. , Hays   J. , Perona   P. , Ramanan   D. , Dollár   P.  et al.    2014 . Microsoft coco: common objects in context . In   Proceedings of the European Conference on Computer Vision , Springer , pp. 740 – 755 .

Link   J. S. , Griffis   R. B. , Busch   D. S.   2015 . NOAA fisheries climate science strategy . NOAA Technical Memorandum NMFS-F/SPO-155 . U.S.Department of Commerce .

Lombard   F. , Boss   E. , Waite   A. M. , Vogt   M. , Uitz   J. , Stemmann   L. , Sosik   H. M.  et al.    2019 . Globally consistent quantitative observations of planktonic ecosystems . Frontiers in Marine Science , 6 : 196 .

Lopez-Guede   L.-V. , Marini   F. , Johnsen   A.   2020 . Video image enhancement and machine learning pipeline for underwater animal detection and classification at cabled observatories . Sensors , 20 : 726 .

Lopez-Marcano   S. , Brown   C. J. , Sievers   M. , Connolly   R. M.   2021 . The slow rise of technology: computer vision techniques in fish population connectivity . Aquatic Conservation Marine and Freshwater Ecosystems , 31 : 210 – 217 .

Luo   J. Y. , Irisson   J.-O. , Graham   B. , Guigand   C. , Sarafraz   A. , Mader   C. , Cowen   R. K.   2018 . Automated plankton image analysis using convolutional neural networks . Limnology and Oceanography Methods , 16 : 814 – 827 .

Lyubchich   V. , Woodland   R. J.   2019 . Using isotope composition and other node attributes to predict edges in fish trophic networks . Statistics and Probability Letters , 144 : 63 – 68 .

Malde   K. , Handegard   N. O. , Eikvil   L. , Salberg   A.-B.   2020 . Machine intelligence and the data-driven future of marine science . ICES Journal of Marine Science , 77 : 1274 – 1285 .

Marre   G. , Braga   C. D. A. , Ienco   D. , Luque   S. , Holon   F. , Deter   J.   2020 . Deep convolutional neural networks to monitor coralligenous reefs: operationalizing biodiversity and ecological assessment . Ecological Informatics , 59 : 101110 .

Martin   A. H. , Pearson   H. C. , Saba   G. K. , Olsen   E. M.   2021 . Integral functions of marine vertebrates in the ocean carbon cycle and climate change mitigation . One Earth , 4 : 680 – 693 .

Martin   A. , Boyd   P. , Buesseler   K. , Cetinic   I. , Claustre   H. , Giering   S. , Henson   S.  et al.    2020 . The oceans’ twilight zone must be studied now, before it is too late . Nature , 580 : 26 – 28 .

McDonald   K. S. , Hobday   A. J. , Fulton   E. A. , Thompson   P. A.   2018 . Interdisciplinary knowledge exchange across scales in a globally changing marine environment . Global Change Biology , 24 : 3039 – 3054 .

Mellinger   D. K. , Clark   C. W.   1997 . Methods for automatic detection of mysticete sounds . Marine and Freshwater Behaviour and Physiology , 29 : 163 – 181 .

Moritz   N. , Wichern   G. , Hori   T. , Le Roux   J.   2020 . All-in-one transformer: unifying speech recognition, audio tagging, and event detection . In   Proceedings of  INTERSPEECH . 3112 – 3116 .

Mucientes   G. , Irisarri   J. , Villegas-Ríos   D.   2019 . Interannual fine-scale site fidelity of male ballan wrasse labrus bergylta revealed by photo-identification and tagging . Journal of Fish Biology , 95 : 1151 – 1155 .

Mushtaq   Z. , Su   S.-F. , Tran   Q.-V.   2021 . Spectral images based environmental sound classification using CNN with meaningful data augmentation . Applied Acoustics , 172 : 107581 .

Orenstein   E. C. , Kenitz   K. M. , Roberts   P. L. D. , Franks   P. J. S. , Jaffe   J. S. , Barton   A. D.   2020 . Semi- and fully supervised quantification techniques to improve population estimates from machine classifiers . Limnology and Oceanography, Methods , 18 : 739 – 753 ..   https://aslopubs.onlinelibrary.wiley.com/doi/full/10.1002/lom3.10399

Orenstein   E. C. , Ratelle   D. , Briseno-Avena   C. , Carter   M. L. , Franks   P. J. S. , Jaffe   J. S. , Roberts   P. L. D.   2020 . The scripps plankton camera system: a framework and platform for in situ microscopy . Limnology and Oceanography, Methods , 18 : 681 – 695 .

Parsons   M. J. , McCauley   R. D. , Mackie   M. C. , Siwabessy   P. , Duncan   A. J.   2009 . Localization of individual mulloway ( Argyrosomus japonicus ) within a spawning aggregation and their behaviour throughout a diel spawning period . ICES Journal of Marine science , 66 : 1007 – 1014 .

Perry   D. , Staveley   T. A. , Gullström   M.   2018 . Habitat connectivity of fish in temperate shallow-water seascapes . Frontiers in Marine Science , 4 : 1 – 12 .

Phan   H. , Chén   O. Y. , Pham   L. , Koch   P. , De Vos   M. , McLoughlin   I. , Mertins   A.   2019 . Spatio-temporal attention pooling for audio scene classification . In   Proceedings of INTERSPEECH .

Picheral   M. , Colin   S. , Irisson   J.-O.   2017 . EcoTaxa’a tool for the taxonomic classification of images . http://ecotaxa.obs-vlfr.fr/ . ( Last accessed on October 2021 ).

Piczak   K. J.   2015 . Environmental sound classification with convolutional neural networks . In   Proceedings of the 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP) , IEEE , pp. 1 – 6 .

Pine   W. E. , Pollock   K. H. , Hightower   J. E. , Kwak   T. J. , Rice   J. A.   2003 . A review of tagging methods for estimating fish population size and components of mortality . Fisheries , 28 : 10 – 23 .

Politikos   D. V. , Fakiris   E. , Davvetas   A. , Klampanos   I. A. , Papatheodorou   G.   2021 . Automatic detection of seafloor marine litter using towed camera images and deep learning . Marine Pollution Bulletin , 164 : 111974 .

Rasmussen   J. H. , Širović   A.   2021 . Automatic detection and classification of baleen whale social calls using convolutional neural networks . The Journal of the Acoustical Society of America , 149 : 3635 – 3644 .

Redmon   J. , Divvala   S. , Girshick   R. , Farhadi   A.   2016 . You only look once: unified, real-time object detection . In   Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 779 – 788 .

Ristic   B. , Arulampalam   S. , Gordon   N.   2004 . Beyond the Kalman Filter: Particle Filters for Tracking Applications , Artech House Radar Library, Artech House . https://books.google.no/books?id=cjFDngEACAAJ .

Russell   S. , Norvig   P.   2002 . Artificial intelligence: a modern approach . Englewood Cliffs, N.J: Prentice Hall .

Saba   G. K. , Burd   A. B. , Dunne   J. P. , Hernández-León   S. , Martin   A. H. , Rose   K. A. , Salisbury   J . et al.    2021 . Toward a better understanding of fish-based contribution to ocean carbon flux . Limnology and Oceanography , 66 : 1639 – 1664 .

Sanchez-Gonzalez   A. , Godwin   J. , Pfaff   T. , Ying   R. , Leskovec   J. , Battaglia   P.   2020 . Learning to simulate complex physics with graph networks . In   Proceedings of the International Conference on Machine Learning . PMLR , pp. 8459 – 8468 .

Schiettekatte   N. M. , Barneche   D. R. , Villéger   S. , Allgeier   J. E. , Burkepile   D. E. , Brandl   S. J. , Casey   J. M . et al.    2020 . Nutrient limitation, bioenergetics and stoichiometry: a new model to predict elemental fluxes mediated by fish . Functional Ecology , 34 : 1857 – 1869 .

Schmid   M. S. , Daprano   D. , Jacobson   K. M. , Sullivan   C. , Briseño-Avena   C. , Luo   J. Y. , Cowen   R. K.   2021 . ‘A Convolutional Neural Network based high- throughput image classification pipeline – code and documentation to process plankton underwater imagery using local HPC infrastructure and NSF’s XSEDE’ . National Science Foundation, National Aeronautics and Space Administration, Belmont Forum, Extreme Science and Engineering Discovery Environment . doi: 10.5281/zenodo.4641158 .

Schmidhuber   J.   2015 . Deep learning in neural networks: an overview . Neural Networks , 61 : 85 – 117 .

Schneider   S. , Taylor   G. W. , Linquist   S. , Kremer   S. C.   2019 . Past, present and future approaches using computer vision for animal re-identification from camera trap data . Methods in Ecology and Evolution , 10 : 1151 – 1155 .

Schofield   D. , Nagrani   A. , Zisserman   A. , Hayashi   M. , Matsuzawa   T. , Biro   D. , Carvalho   S.   2019 . Chimpanzee face recognition from videos in the wild using deep learning . Science Advances , 5 : 1 – 10 .

Schroeder   S.-M. , Kiko   R. , Koch   R.   2020 . MorphoCluster: efficient annotation of plankton images by clustering . Sensors , 20 : 3060 .

Sharma   J. , Granmo   O.-C. , Goodwin   M.   2020 . Environment sound classification using multiple feature channels and attention based deep convolutional neural network . In   Proceedings of the INTERSPEECH . 1186 – 1190 .. doi: 10.21437/Interspeech.2020-1303 .

Shin   Y. , Choi   J. H. , Choi   H. S.   2021 . Deep learning based fish object detection and tracking for smart aqua farm . The Journal of the Korea Contents Association , 21 : 552 – 560 .

Shiu   Y. , Palmer   K. , Roch   M. A. , Fleishman   E. , Liu   X. , Nosal   E.-M. , Helble   T.  et al.    2020 . Deep neural networks for automated detection of marine mammal species . Scientific Reports , 10 : 1 – 12 .

Sun   Y. , Xue   B. , Zhang   M. , Yen   G. G.   2019 . Completely automated cnn architecture design based on blocks . IEEE Transactions on Neural Networks and Learning Systems , 31 : 1242 – 1254 .

Suryanarayana   I. , Braibanti   A. , Rao   R. S. , Ramam   V. A. , Sudarsan   D. , Rao   G. N.   2008 . Neural networks in fisheries research . Fisheries Research , 92 : 115 – 139 .

Szegedy   C. , Liu   W. , Jia   Y. , Sermanet   P. , Reed   S. , Anguelov   D. , Erhan   D.  et al.    2015 . Going deeper with convolutions . In   Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . pp. 1 – 9 .

Szesciorka   A. R. , Ballance   L. T. , Širović   A. , Rice   A. , Ohman   M. D. , Hildebrand   J. A. , Franks   P. J.   2020 . Timing is everything: drivers of interannual variability in blue whale migration . Scientific Reports , 10 : 1 – 9 .

Tam   J. C. , Link   J. S. , Rossberg   A. G. , Rogers   S. I. , Levin   P. S. , Rochet   M.-J. , Bundy   A.  et al.    2017 . Towards ecosystem-based management: identifying operational food-web indicators for marine ecosystems . ICES Journal of Marine Science , 74 : 2040 – 2052 .

Tay   Y. , Dehghani   M. , Bahri   D. , Metzler   D.   2020 . Efficient transformers: a survey . a rXiv:2009.06732 .

Tessler   C. , Givony   S. , Zahavy   T. , Mankowitz   D. , Mannor   S.   2017 . A deep hierarchical approach to lifelong learning in minecraft . In   Proceedings of the AAAI Conference on Artificial Intelligence , 31 .

Uusitalo   L. , Fernandes   J. A. , Bachiller   E. , Tasala   S. , Lehtiniemi   M.   2016 . Semi-automated classification method addressing marine strategy framework directive (MSFD) zooplankton indicators . Ecological Indicators , 71 : 398 – 405 .

Vadziutsina   M. , Riera   R.   2020 . Review of fish trap fisheries from tropical and subtropical reefs: main features, threats and management solutions . Fisheries Research , 223 : 105432 .

Vaswani   A. , Shazeer   N. , Parmar   N. , Uszkoreit   J. , Jones   L. , Gomez   A. N. , Kaiser   L. , Polosukhin   I.   2017 . Attention is all you need . In Advances in neural information processing systems , pp. 5998 – 6008 .

Waddell   E. E. , Rasmussen   J. H. , Širović   A.   2021 . Applying artificial intelligence methods to detect and classify fish calls from the Northern Gulf of Mexico . Journal of Marine Science and Engineering , 9 : 1128 .

Wang   N. , Yu   J. , Yang   B. , Zheng   H. , Zheng   B.   2020 . Vision-based in situ monitoring of plankton size spectra via a convolutional neural network . IEEE Journal of Oceanic Engineering , 45 : 511 – 520 .

Wang   Z. A. , Moustahfid   H. , Mueller   A. V. , Michel   A. P. , Mowlem   M. , Glazer   B. T. , Mooney   T. A.  et al.    2019 . Advancing observation of ocean biogeochemistry, biology, and ecosystems with cost-effective in situ sensing technologies . Frontiers in Marine Science , 6 : 519 .

Weinstein   B. G.   2018 . A computer vision for animal ecology . Journal of Animal Ecology , 87 : 533 – 545 .

Whitt   C. , Pearlman   J. , Polagye   B. , Caimi   F. , Muller-Karger   F. , Copping   A. , Spence   H.  et al.    2020 . Future vision for autonomous ocean observations . Frontiers in Marine Science , 7 : 697 .

Wiggins   S. M. , Hildebrand   J. A.   2007 . High-frequency acoustic recording package (HARP) for broad-band, long-term marine mammal monitoring . In   Proceedings od the 2007 Symposium on Underwater Technology and Workshop on Scientific Use of Submarine Cables and Related Technologies . IEEE , pp. 551 – 557 .

Wiggins   S. M. , Hildebrand   J. A.   2016 . Long-term monitoring of cetaceans using autonomous acoustic recording packages . In   Listening in the Ocean , Springer , pp. 35 – 59 .

Wilkinson   M. D. , Dumontier   M. , Aalbersberg   I. J. , Appleton   G. , Axton   M. , Baak   A. , Blomberg   N.  et al.    2019 . The FAIR Guiding Principles for scientific data management and stewardship . Scientific Data , 6 : 160018 .

Wilson   R. J. , Speirs   D. C. , Sabatino   A. , Heath   M. R.   2018 . A synthetic map of the North-West European shelf sedimentary environment for applications in marine science . Earth System Science Data , 10 : 109 – 130 .

Winn   H. E.   1964 . The biological significance offish sounds. In Ed. by Tavolga   W. N. , Marine bio-acoustics , pp. 213 – 231 .. Pergamon Press , Oxford .

Yang   X. , Zhang   S. , Liu   J. , Gao   Q. , Dong   S. , Zhou   C.   2021 . Deep learning for smart fish farming: applications, opportunities and challenges . Reviews in Aquaculture , 13 : 66 – 90 .

Yu   X. , Wei   Y. , Zhu   M. , Zhou   Z.   2016 . Automated classification of zooplankton for a towed imaging system . In   Proceedings of the OCEANS 2016 . IEEE , Shanghai .

Yu   Y. , Zhao   J. , Gong   Q. , Huang   C. , Zheng   G. , Ma   J.   2021 . Real-time underwater maritime object detection in side-scan sonar images based on transformer-YOLOv5 . Remote Sensing , 13 : 3555 .

Zhang   Z. , Xu   S. , Zhang   S. , Qiao   T. , Cao   S.   2020 . Attention based convolutional recurrent neural network for environmental sound classification . Neurocomputing . 453 : 896 – 903 .

Zhong   M. , Castellote   M. , Dodhia   R. , Lavista Ferres   J. , Keogh   M. , Brewer   A.   2020 . Beluga whale acoustic signal classification using deep learning neural network models . The Journal of the Acoustical Society of America , 147 : 1834 – 1841 .

Supplementary data

Email alerts, citing articles via.

• Contact ICES
• Recommend to your Library

Affiliations

• Online ISSN 1095-9289
• Print ISSN 1054-3139
• Copyright © 2024 ICES/CIEM
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

deep ecology Recently Published Documents

Total documents.

• Latest Documents
• Most Cited Documents
• Contributed Authors
• Related Sources
• Related Keywords

Eco-Initiatives in Municipal Cultural Institutions as Examples of Activities for Sustainable Development: A Case Study of Poznan

The aim of the study is to demonstrate which pro-environmental initiatives consistent with the idea of sustainable development are undertaken by municipal cultural institutions in Poznan—the largest city in the Greater Poland region in Poland. The diagnostic survey was conducted by means of interviews with directors of municipal cultural institutions or their representatives. A total of 12 (out of 15) municipal cultural institutions from the city of Poznan were surveyed. All were asked to cite specific examples of their activities, visualising their institution’s approach to ecology. The results indicated that the institutions undertake numerous eco-initiatives, which very often fit into the discourse on so-called “deep ecology” and address the sources of the existing environmental crisis. Their activities are also aimed at stimulating a “return to nature” of the city’s inhabitants. In addition to initiatives aimed at the recipients of the offer of individual institutions, many institutions also pay attention to pro-environmental internal activities. In effect, the results could be divided into three general groups, which include internal institutional activities, the implementation of projects on environmental themes, and educational activities in the field of ecology. However, these activities were undertaken individually, as the survey revealed a lack of inter-institutional cooperation on ongoing projects.

Samkhya Philosophy, Deep Ecology and Sustainable Development

Samkhya philosophy is one of the oldest philosophies in the Indian philosophical system. This philosophy is independent in origin and mainly known for its evolution theory. Samkhya philosophy has accepted the two ultimate and independent realities, Nature and pure Consciousness. This paper is an attempt to comprehend the notion of deep ecology from the Samkhya’s evolution theory perspective. In this paper, firstly, we have elucidated the Samkhya philosophy of suffering and the solution to the problem. In the second part of the article, we have argued that how Samkhya’s philosophical notion of Nature demonstrates our embeddedness in Nature. Therefore, the idea of deep ecology is always there in one or another form in Samkhya philosophy. It has also connections to the idea of sustainable development.

Offline: The origins story—towards a Deep Ecology

A study on the characteristic of deep ecology through the hwaeom ideology ― focused on on seon-woo kim’s poems, nature · animals · non-human: a study of community consciousness in the night watchman.

The Night Watchman written by Louis Erdrich won the 2021 Pulitzer Prize for fiction. The core characters running through the novel are not only Thomas Wazhashk, but Patrice who bears the burden of narrating the natural world of the Turtle Mountain reserve. Louis Erdrich not only noticed the connection between females and nature with keen eyes, but also human and non-humans. The interaction among them reflects the author’s thoughts on the ecological environment, human survival, and indigenous tradition conflicted with modern appeal. Therefore, this article, using the Biocentric Equality of deep ecology, explores the Community Consciousness in the novel.

Hugh MacDiarmid’s ‘On Raised Beach’: ‘Geopoetics’ in a Time of Catastrophic Crisis’

The poet Hugh MacDiarmid (1892–1978) was the major driving force behind the twentieth-century Scottish literary renaissance and was also a passionate Scottish nationalist. His poem ‘On a Raised Beach’ (1934) has been understood in theological and philosophical terms as a metaphysical exploration, albeit one grounded in an immediate experience of nature that took place on Shetland. In this paper, MacDiarmid’s epic is placed in the context of the present environmental crisis and the ongoing consequences of the COVID-19 pandemic. ‘On a Raised Beach’ can now be re-located within the hermeneutical tradition of ‘Geopoetics’, a Scottish genre that is articulated and asserted by the poet Kenneth White (1938–). Whilst, however, White draws upon the highly contested and polyvalent concept of ‘shamanism’ in elaborating his standpoint, we shall argue that it is also appropriate to look for affinities between this dynamic poem and the ethos and mysticism of ‘deep ecology’, a perspective that invokes the equally contested mythology of ‘Gaia’.

A Universe of Subjects: Process Cosmology and Deep Ecology

Two perspectives on ecological art.

In the paper “Two perspectives on ecological art”, we will compare two perspectives on sustainability and the practicality of an ecological artwork. One perspective is from Latvian media artist Maija Demitere, researching slow media art, deep sustainability, and food production. Demitere uses micro-gardening prototypes as an instrument to inform the public on the problems of food production (local food, biodiversity, pesticides, herbicides, pollution caused by agriculture). Demitere uses gardening in combination with DIY (Do it Yourself) technologies to talk about slow living, ecology of the mind, and mindfulness. The second perspective is offered by Jan Glöckner. Glöckner is a German artist and researcher. His research interests are collaborations between fungi and Hominidae. Glöckner reaches out with diplomatic gestures towards fungi to re-localise humans within the larger domain of living entities. He is working on an ethical framework that draws from deep ecology and Tibetian Buddhism to ensure the rights of microorganisms and macroorganisms in artistic, industrial, and research setups. The first part of the paper will focus on recycling, waste management, waste produced by households, and the artists’ perspective on the problem. The second part will focus on a specific case of the exhibition “Life” by Olafur Eliasson at the Foundation Beyeler in Riehen, Switzerland. The second part will also look at the idea of “artistic greenwashing”. The last part of the article will attempt to conclude what can be considered an actual sustainable artwork and propose possible key points that describe a (deep) ecological artwork. The paper uses such methods as case studies, literature analysis, and autoethnography.

Deep Ecology, Business Ethics and Personal Responsibility

Kedudukan dan peran manusia dalam alam: tanggapan atas kritik al gore terhadap arne naess.

<em><em>This articleaims to respond Al Gore’s critics onArne Naess’ concept of deep ecology. ForAl Gore, deep ecology of ArneNaesshas reduced the position and the role of human being in nature. The reason is becauseNaess’s deep ecologyis assumed to see human being as the source of destruction on earth and alien which is not part of nature; it grasps man as creature without ability to think and to have free will; and it has no solution for current ecological crisis. Naess, in constrast, comprehends human beings as good in himself, part of nature, and a unique creature. Because of this uniqueness,human being has responsibility to protect and to preserve nature.Thus, although they have differences, both deep ecology of Naess and the ecology of Al Gore havemany similarities. These similarities can be used ascontribution to any effort to save the earth today.<br /></em></em><p><strong><em>Key words</em></strong><em>: </em>ekologi dalam, menusia, alam, antroposentrisme, ekosentrisme</p>

Export Citation Format

Share document.

A platform of deep ecology

• Published: September 1987
• Volume 7 , pages 185–190, ( 1987 )

Cite this article

• David Rothenberg 1  

285 Accesses

6 Citations

Explore all metrics

What is the movement known as ‘deep ecology’? A point form summary of some of its more basic intuitions is presented, followed by some suggestions on how these basic beliefs can be elaborated and put into action. Basic references in the field are listed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Conclusion: The Way Forward

Jean-Marc Coicaud

The Foundations of Deep Learning with a Path Towards General Intelligence

Exploring, expounding & ersatzing: a three-level account of deep learning models in cognitive neuroscience.

Vanja Subotić

Devall, B. and Sessions, G. (1985) Deep Ecology: Living as if Nature Mattered , Peregrine Smith Books, Salt Lake City.

Google Scholar  

Fox, W. (1984) Deep Ecology: A New Philosophy for Our Time? The Ecologist 14 (5/6) 194–200.

Glacken, C. (1967) Traces on the Rhodian Shore . University of California Press, Berkeley.

Gullvag, I. and Wetleson, J. (eds.) (1982) In Sceptical Wonder: Inquiries into the Philosophies of Arne Naess on the Occasion of His 70th Birthday . Universitetsforlaget, Oslo.

Lovejoy, A. (1936) The Great Chain of Being . Harvard University Press, Cambridge, MA.

Naess, A. (1973) The Shallow and the Deep, Long Range Ecology Movements: A Summary. Inquiry , 16, p. 95–100.

Naess, A. (1974) Gandhi and Group Conflict: An Exploration of Satyagraha — theoretical background . Universitetsforlaget, Oslo

Naess, A. (1975) Freedom, Emotion, and Self-Subsistence: A systematization of a part of Spinoza's Ethics . Universitetsforlaget, Oslo.

Naess, A. (1979) Self-realization in Mixed Communities, of Humans, Bears, Sheep, and Wolves. Inquiry , 22, p. 231–241.

Naess, A. (1984) A Defense of the Deep Ecology Movement. Environmental Ethics 6 (3) 265–70.

Naess, A. (1985) Identification as a Source for Deep Ecological Attitudes. In: Tobias, M. (ed.) Deep Ecology . Avant Books, San Diego.

Naess, A. (1986) Intrinsic Value — Will the Defenders of Nature Please Rise? In: Soule, M. (ed.) Conservation Biology: Science and Scarcity and Diversity . Sinauer Associates, Sunderland, MA.

Pepper, D. (1984) The Roots of Modern Environmentalism . Croom Helm, London.

Reed, P. and Rothenberg, D. (eds.) (1987) Wisdom and the Open Air: Selections from Norwegian Ecophilosophy . Council for Environmental Studies, Univ. of Oslo, Norway.

Seamon, D. and Mugerauer, R. (eds.) (1986) Dwelling Place, and Environment , Martinus Nijhoff, The Hague.

Sessions, G. (1981) Shallow and Deep Ecology: A Review of the Philosophical Literature. In: Schulz and Hughes (eds) Ecological Consciousness . University Press of America, Washington.

Sylvan, R. (1985) A Critique of Deep Ecology, Discussion Paper in Environmental Philosophy No. 12, Philosophy Department, the Australian National University, Canberra.

Watson, R. (1983) A Critique of Anti-Anthropocentric Biocentrism. Environmental Ethics , Vol. 5, pp. 245–256.

Watson, R. (1985) Challenging the Underlying Dogmas of Environmentalism, Whole Earth Review , p. 5–13.

Download references

Author information

Authors and affiliations.

Boston University, 02215, Boston, Massachusetts, USA

David Rothenberg

You can also search for this author in PubMed   Google Scholar

Additional information

David Rothenberg was a founding member of the Ecophilosophy Group at the University of Oslo. Although he is currently at the Department of Philosophy in Boston University. As the author states, this article is intended as a discussion piece and it is hoped that it will stimulate responses.

Rights and permissions

Reprints and permissions

About this article

Rothenberg, D. A platform of deep ecology. Environmentalist 7 , 185–190 (1987). https://doi.org/10.1007/BF02233590

Download citation

Issue Date : September 1987

DOI : https://doi.org/10.1007/BF02233590

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Environmental Management
• Nature Conservation
• Economic Geology
• Basic Reference
• Basic Belief
• Find a journal
• Publish with us
• Track your research

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

•  We're Hiring!
•  Help Center

Deep Ecology

• Most Cited Papers
• Most Downloaded Papers
• Newest Papers
• Save to Library
• Last »
• Environmental Philosophy Follow Following
• Environmental Ethics Follow Following
• Anthropocentrism Follow Following
• Ecocentrism Follow Following
• Ecofeminism Follow Following
• Environmental Justice Follow Following
• Religion and Ecology Follow Following
• Environmentalism Follow Following
• Ecology Follow Following
• Ecosophy Follow Following

Enter the email address you signed up with and we'll email you a reset link.

• Academia.edu Publishing
•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Perspective
• Open access
• Published: 07 February 2024

Towards (better) fluvial meta-ecosystem ecology: a research perspective

• Matthew Talluto 1 ,
• Rubén del Campo 1 ,
• Edurne Estévez 1 ,
• Florian Altermatt 2 , 3 ,
• Thibault Datry 4 &
• Gabriel Singer 1  

npj Biodiversity volume  3 , Article number:  3 ( 2024 ) Cite this article

2732 Accesses

23 Altmetric

Metrics details

• Carbon cycle
• Community ecology
• Ecosystem ecology

Rivers are an important component of the global carbon cycle and contribute to atmospheric carbon exchange disproportionately to their total surface area. Largely, this is because rivers efficiently mobilize, transport and metabolize terrigenous organic matter (OM). Notably, our knowledge about the magnitude of globally relevant carbon fluxes strongly contrasts with our lack of understanding of the underlying processes that transform OM. Ultimately, OM processing en route to the oceans results from a diverse assemblage of consumers interacting with an equally diverse pool of resources in a spatially complex network of heterogeneous riverine habitats. To understand this interaction between consumers and OM, we must therefore account for spatial configuration, connectivity, and landscape context at scales ranging from local ecosystems to entire networks. Building such a spatially explicit framework of fluvial OM processing across scales may also help us to better predict poorly understood anthropogenic impacts on fluvial carbon cycling, for instance human-induced fragmentation and changes to flow regimes, including intermittence. Moreover, this framework must also account for the current unprecedented human-driven loss of biodiversity. This loss is at least partly due to mechanisms operating across spatial scales, such as interference with migration and habitat homogenization, and comes with largely unknown functional consequences. We advocate here for a comprehensive framework for fluvial networks connecting two spatially aware but disparate lines of research on (i) riverine metacommunities and biodiversity, and (ii) the biogeochemistry of rivers and their contribution to the global carbon cycle. We argue for a research agenda focusing on the regional scale—that is, of the entire river network—to enable a deeper mechanistic understanding of naturally arising biodiversity–ecosystem functioning coupling as a major driver of biogeochemically relevant riverine carbon fluxes.

Similar content being viewed by others

Revealing biogeochemical signatures of Arctic landscapes with river chemistry

Arial J. Shogren, Jay P. Zarnetske, … William B. Bowden

New insights into the food web of an Australian tropical river to inform water resource management

Leah S. Beesley, Bradley J. Pusey, … Samantha A. Setterfield

River ecosystem metabolism and carbon biogeochemistry in a changing world

Tom J. Battin, Ronny Lauerwald, … Pierre Regnier

Introduction

River networks are unique: they are characterized by a hierarchical dendritic structure with unidirectional water flow that imposes constraints on the fluxes of energy, limiting resources, and organisms 1 , 2 . These meta-ecosystems (i.e., collections of local ecosystems connected by flows of material and organisms; detailed definitions of bolded terms can be found in the glossary in Box 1), can only be properly understood in a spatial context. The spatial structure of rivers contributes to their role in the global carbon cycle. Rivers metabolize large amounts of terrigenous OM, producing globally significant carbon fluxes to the atmosphere and ocean 3 . Moreover, river networks can never be treated as a closed system, even as a first approximation. Material and carbon inputs from the terrestrial environment and from groundwater, and outflows to the ocean or the atmosphere are likely to be large relative to flows among locations within the network, especially among those that are not connected by water flow 4 . Thus, understanding terrigenous OM processing in rivers requires considering both the spatial structure of the river network as well as the surrounding landscape context .

Respiration in rivers is a heterotrophic ecosystem function (EF) that results from consumer biodiversity interacting with OM as the main resource. Theory and empirical work have both postulated and demonstrated that biodiversity is a key driver of EF, and that its effects are pervasive and strong 5 , 6 . However, OM in rivers originates from a range of sources and follows a multitude of transformation pathways while simultaneously being transported downstream 7 , 8 , 9 . For consumers, this results in a chemically complex resource space that changes markedly along a river network 10 , 11 , 12 . The importance of the spatial distribution of consumer biodiversity for the efficiency of OM processing is thus difficult to assess when either consumers or resources are considered alone. Indeed, high OM diversity may need considerable (functional) biodiversity of consumers for its efficient processing. Thus, the resource use efficiency for OM, which is a critical measure of EF underlying biodiversity–ecosystem function (BEF) relationships 13 , will likely be tied to the capability of locally present organisms to metabolize the locally available OM. Consequently, we expect that while the potential for positive effects of biodiversity on EF exists in most river networks, the degree to which this potential is realized will depend on local conditions, including the abundance and composition of relevant species, competitive equilibria, and the spatio-temporal match between the chemical and physical traits of OM and trophic traits of consumer communities 14 , 15 , 16 , 17 . Understanding such river network-wide BEF relationships may be the key to unlocking when and where rivers metabolize OM intensively, thereby supporting fluvial food webs and potentially feeding back to biodiversity, and when inefficient processing rather results in the transport of most OM further downstream (Fig. 1 ).

A The spatial configuration of the river network interacts with B the abiotic and physical characteristics of the network, C the spatial context in which the network is embedded and, potentially, with D topological interruptions to the connectivity of the network. These factors control E the distribution and dispersal of consumer species as well as the input and transport of organic matter. Taken together at the network scale, F patterns of consumer and resource diversity will emerge. G The degree to which these two types of diversity are spatially congruous (and the extent to which human influence disrupts this congruity) will determine meta-ecosystem functioning, and the shape of the BEF relationship.

OM sources are strongly tied to catchment properties, and OM processing is often viewed through a biogeochemical lens, where transport is a purely physical process and specific knowledge of the local biological community governing OM metabolism is abstracted away or unavailable 18 , 19 , 20 , 21 . In contrast, the distribution of organisms in rivers is well-described using metacommunity theory, which explains how dispersal interacts with local conditions to produce species assemblages and aquatic food-webs 22 , 23 . Indeed, catchment properties can have a strong imprint on the dispersal processes shaping metacommunities, since many organisms disperse with water flow, suggesting that the physical template of the river network acts as a strong underlying control on both community composition and OM distribution. Classically, the river continuum concept 24 integrates both of these components, describing the dynamics of environmental and biological changes along a longitudinal continuum. It specifically proposes that biological communities are structured by changes in the river’s physical structure and energy sources and availability moving from upstream to downstream, which in turn shapes how these communities use available resources. Yet, in many analyses, a substantial part of the aquatic food web remains unexplained 25 , and the distribution of OM can depend on biological in addition to physical processes 11 , 26 . Moreover, recent work has shown that a more detailed representation of resource flux in a spatial (i.e., network) context is key for explaining consumer distributions 17 , revealing the limitations of a purely longitudinal view. Importantly, these concepts rarely recognize the importance of the (e.g., chemical) diversity of OM, even though OM represents a critical limiting resource 27 , 28 , 29 .

The spatial configuration of river networks thus constrains the transport of OM and much of the dispersal of organisms, likely leaving a strong imprint on both OM diversity 30 , 31 and biodiversity 32 , 33 , 34 , 35 (Fig. 1 ) because of fundamentally different rules. For instance, organisms can disperse up- and/or downstream depending on dispersal traits, whereas resources are largely transported only downstream. For both, downstream movement depends on both space and water flow; a distant location may be variously more or less accessible depending on flow conditions (e.g., high flow supports fish migration 36 and accelerates OM transport 20 ). Further, the landscape context is key; the composition of the surrounding terrestrial matrix has a large impact on the state of the meta-ecosystem, resulting in multi-scale variation in OM sources, consumer community composition, and the ability of organisms to disperse laterally 37 . Headwaters may be especially sensitive due to their isolation, limited connectivity to other ecosystems and low flow of water relative to more downstream locations 2 , 38 , 39 . Downstream regions will further integrate resource and biological inputs from all upstream regions, and so will be sensitive both to the surrounding matrix as well as to disruptions in hydrological connectivity. Anthropogenic influence itself may have a distinct spatial character: for example, point source pollution providing unusually high concentrations of OM and other resources, or fragmentation of the river network by damming and altered flow regimes that may even include non-perennial river network sections.

Here, we advocate for unifying research on carbon biogeochemistry in river networks with that on fluvial metacommunities, focusing on the scale of entire river networks as the primary spatial unit of interest. We briefly review some key research on how both OM biogeochemistry and metacommunities are organized in space, and then argue that future research must consider both simultaneously, especially at the scale of entire river networks, if we are to better understand the coupling between biodiversity and EF and the role of rivers in the global carbon cycle.

OM as a spatio-temporally dynamic resource space within river networks

Describing how OM forms a multidimensional resource space for consumers requires advanced capability to describe relevant OM traits. Conceptually, we can divide OM into two pools consisting of dissolved organic matter (DOM) and particulate organic matter (POM), where DOM is <0.45 μm 40 . Indeed, the integration of various OM pools across river networks strongly depends on OM size. DOM moves freely with the water, while for POM particle size controls retention and thus transport behaviour 2 (Fig. 2 ). Thus, small particles may have origin points across a larger proportion of the upstream network than large particles (and may also therefore represent OM conditions across a greater portion of the upstream network) 2 , 41 . Indeed, particle size must be considered as a master trait interacting strongly with other dimensions of the resource space. All environmental, chemical or biological factors involved in OM processing are conditioned by OM particle size 42 , 43 . Particle size not only mediates how well OM is retained, but also constrains the target consumer community. Since the degradation process of POM also continuously provides new resources (i.e., the formation of smaller detritus fragments), particle size is intrinsically related to the chemical composition and degradability of OM 44 . Thus, the sharp distinction between DOM and POM is somewhat arbitrary; we can more accurately view these two pools as extreme ends of an OM continuum defined by particle size 40 . Here, we consider OM generally as representing the entire continuum, but continue to use the DOM and POM distinction when it is useful for describing different behaviours at opposing ends of this continuum.

An evident example for this is the confluence of the rivers Sarantaporos (left) and Aoos (right) in the Vjosa river network on the border of Albania and Greece (photo left side, taken in April 2018, credit G. Singer). The huge variation in turbidity reflects upstream differences in geology, and contributes to the diversity in DOM characteristics between the two tributaries. On the right side, a principal component analysis represents the spatial variation in DOM chemistry in the Vjosa river network based on spectroscopic indices. Each polygon represents the differentiation of DOM composition among tributaries and their confluence at various points across the network, with the Aoos–Sarantaporos confluence highlighted in red. The comparison of polygon size gives an idea of the degree of DOM differentiation at every confluence; the differentiation at this confluence of the Aoos and Sarataporos is among the largest across the entire network.

Going beyond particle size to understand OM traits requires an understanding of OM composition and diversity. Many studies consider only relatively indirect measures of OM quality. These measures are derived mostly from absorbance or fluorescence spectroscopic analyses of DOM 45 , microscopical identification of fine POM (e.g., relative proportion of animals, diatoms or vascular plant residues) 46 and selected chemical traits for coarse POM (e.g., C, N, P, lignin, tannins and fibre content) 47 . Novel technology, however, allows us to move beyond such proxies. Indeed, a description of OM diversity should rival the resolution of biodiversity available through molecular biological means and move towards describing actual functional interactions between a consumer and a resource unit (i.e., an organic molecule or particle). For instance, DOM can be described at the level of molecular species by size-exclusion, liquid or ion chromatography coupled to mass spectrometry 48 , 49 , 50 , and POM can be described on a per-particle basis with regard to physical features and macromolecular composition using tools like infrared microspectroscopy 51 , 52 , 53 (Box 2).

To unravel the role of such highly resolved OM diversity on its processing, we will then need to understand how, where and when OM diversity arises in river networks. DOM molecules may travel for long distances with subsurface soil and groundwater until reaching a headwater stream. DOM begins to degrade along these paths, making it more recalcitrant, but also diversifying the OM pool entering river systems through integration of a larger part of the catchment 54 . In contrast, abscised leaves, likely the most important POM fraction, are sourced locally from riparian vegetation 8 , 55 . Local sourcing, resulting in a shorter terrestrial path, implies a lower OM diversity, yet a fresher stage when entering the river. Merely given the differences in sourcing pathways from terrestrial systems, DOM and POM diversities at the moment of entering the river network likely depend on the heterogeneity of land cover in the catchment across different spatial scales. Local OM diversity will reflect a combination of these inputs with an integration of similar inputs (and processing) from upstream.

En-route transformation of OM is the final key step to consider in the diversification of the resource space. Discontinuities in the transport of OM can be hot moments (and/or hot spots) of OM processing and therefore, of diversification 3 . Transport discontinuities can happen in space due to fluvial landscape complexity (e.g., in different fluvial habitats of a braided river section or across a floodplain with wetlands), but also in time due to hydrological variability, ranging from the interruption of transport by drying, to fast flushing by flooding. When OM is retained under contrasting environmental conditions, for instance across a transient aquatic-terrestrial habitat mosaic of an intermittent river, it is subjected to different degradation pathways; the result is a diversification of OM without transport, with potential implications for later downstream decomposition when flow and transport are resumed 9 , 56 , 57 . The final consequence of these “pulsed” processing-transport dynamics is that OM diversity is not continuously transferred across the river network, but subjected to discrete step changes at confluences.

At large scales this is well conceptualized by the pulse-shunt concept, a transport-dominated view, which is dynamic but ignorant to the level of resource-consumer interaction that we postulate here. In a recent review, Kothawala et al. 58 point out that OM decomposition requires sufficient water residence time to allow for the interaction of OM and the consumer community 59 , but also the absence of environmental constraints hindering this interaction (e.g., low temperatures or even water availability in drying rivers). We suggest that the lack of a consumer community that is well matched to the resource pool can be as important as the environment for OM decomposition to happen 60 .

Re-evaluating niche paradigms in fluvial consumer metacommunities: Sorting in the resource space drives consumption

The organization of consumer diversity—that of macroinvertebrates and heterotrophic microbes (bacteria, fungi and protists)—is paramount for structuring OM processing patterns at the regional scale. While river-network scale macroinvertebrate diversity is becoming well documented 32 , 61 , 62 , 63 , the spatial distribution of microbial diversity is less explored (but see 64 , 65 , 66 ). These studies suggest important differences in the mechanisms structuring the consumer metacommunities across river networks. Local-scale consumer diversity is shaped by the interplay of regional processes (e.g., dispersal) and local processes that define the niche space and sort consumer species from the regional species pool 63 , 67 (Box 3). The relative importance of regional vs. local processes is highly context dependent, with potential disparities depending the river network structure (including the fragmentation level, dendritic structure, and geology; 17 , 22 , 68 , 69 ) and the type of consumer 70 . For example, while microbes generally disperse downstream with river flow, some macroinvertebrates can additionally disperse upstream or overland, depending on their dispersal traits. Notably, metacommunity studies generally suffer from incomplete assessment of the local factors that define niche space 71 , potentially leading to a systematic underestimation of the sorting-driven fraction of community composition. Much research has focused on habitat hydrodynamics, sediment properties, and temperature as key local factors that define niche space, while resources (e.g., OM properties) have remained under-explored. Novel techniques enabling highly resolved characterization of molecular OM diversity and POM size diversity (Box 2) may improve niche space characterization and, ultimately, advance the assessment of the importance of resource diversity in determining consumer community sorting and diversity.

For sorting to be a dominant driver of the consumer metacommunity structure in which OM properties may play a relevant role, a stable environment and low to intermediate levels of dispersal are needed. At high levels of dispersal, regional competition is too strong for local sorting to overcome inputs from dispersal, leading to a community structured by “ mass effects ” (i.e., the incoming mass of upstream migrants overwhelms niche-based selection, and the most common migrants dominate the community) 72 , 73 , 74 . At low to intermediate levels of dispersal the community sorting by local properties is favoured, which can promote greater local consumer specialization, resulting in a mosaic of locally well adapted consumers (Box 3). When confronted with hyper-diverse resources, a poorly adapted consumer community may be simply unable to process the majority of OM, resulting in inefficient OM processing and subsequent OM transport downstream. In contrast, a well-adapted community might be highly efficient at OM processing due to the improved match between resource and consumer traits. Nonetheless, BEF relationships can be saturating 75 ; that is, beyond a certain threshold, higher consumer diversity may not lead to higher functionality, as some consumers will have similar functional roles.

Efficient decomposition of chemically diverse OM will only happen where and when there is a proper spatio-temporal match between the features of OM and the traits of local consumers 76 , 77 . Spatially, the idea of OM “spiralling” 78 downstream along a flowing watercourse, i.e., the concept of OM of various lability being partly locally consumed and also transformed while simultaneously experiencing downstream transport 79 , 80 , 81 , suggests a pre-programmed mismatch: a community passes on a qualitatively reshaped resource pool that forms the niche space for downstream communities while at the same time sends now less optimally adapted propagules for this consumer community. However, this pre-programmed mismatch may be less relevant for communities that are more sorted by OM derived from local in-stream primary production (i.e., autochthonous OM). In particular for POM, terrigenous inputs can be of low quality, and thus autochthonous POM can be an important resource for invertebrates even when it is not the dominant POM source 82 , 83 . However, this resource might not be relevant for all invertebrate taxa; shredders (invertebrates that feed by shredding plant material), for example, often exclusively feed on terrigenous POM (see, e.g., ref. 84 ), and are therefore not likely sorted by autochthonous OM.

Temporally, seasonality in OM inputs (i.e., leaf fall) results in changes to the nature of the resource space (i.e., OM properties) over the year. For example, in temperate regions, the majority of allochthonous OM is derived in autumn from the adjacent terrestrial ecosystem because of the high input of POM in form of senescent leaves and the peak of terrestrial DOM washed from the nearby catchment with increasing flows. In contrast, autochthonous OM increases in summer, when temperatures and light availability are highest 85 . Additionally, hydrological variability plays a key role in decoupling resources and consumers. On the one hand, floods enhance terrestrial DOM by washing the surrounding terrestrial ecosystem and mobilizing in-stream retained POM (changing the nature of the resource space) while at the same time scour the communities. In these conditions, most of the OM is transported downstream (i.e., not processed locally), a process known as OM “shunting” 20 . As a result, the OM profile is “flattened” along the river network (i.e, the profile becomes similar across all locations, rather than showing different compositions reflecting variable processing at each location). On the other hand, drying stops the longitudinal flow of resources and consumers, promotes the accumulation of poorly processed OM and impoverishes consumer communities, thereby disconnecting the resource from their potential consumers and enhancing a resource–consumer mismatch. These variations in OM properties interact with the life cycle length/generation time of consumers, which have time scales of days to weeks for microbes but months to years for invertebrates. This timing aligns with the faster variation of DOM, which mostly “goes with the flow”, compared to the “retained” POM properties; but still suggests that consumers always lag behind the dynamics of their resource space. Empirical studies have thus far largely neglected these dynamic resource–consumer interactions 37 (but see ref. 17 ) and accounting for this asymmetric spatial and temporal variation of OM and consumers will advance the understanding of the dynamics of OM processing at the meta-ecosystem level.

Research outline and prospectus

We close with a call for research across all scales in fluvial meta-ecosystems, designed to integrate knowledge from the local to the regional scale. First, laboratory experiments will provide the most direct way of manipulating key drivers, such as OM inputs, the available biodiversity, transport and dispersal. Future experiments should go further, both by manipulating aspects of OM and biodiversity in tandem, and by increasing the scale (e.g., by using larger mesocosm facilities, or making manipulations with hyper-diverse communities). Importantly, such experiments are not sufficient to fully characterize regional EF, but they can be used to rule out (or de-emphasize) certain mechanisms in favour of others and can provide estimates about the effect size of various mechanisms.

Second, field studies with adequate regional-scale replication are needed along important natural gradients, especially in understudied regions (e.g., the global south). These gradients can be either spatial (e.g., covering a range of natural network structures, land use gradients, and degrees of human influence), temporal (e.g., pre- and post-damming, time series capturing variable discharge, and intermittence), or ideally both. At even larger scales, we need better efforts to coordinate such studies in multiple river networks to capture continental- and global-scale gradients. This can include the obvious climatic gradients, but should also capture (for example) gradients in human influence and overall degree of landscape heterogeneity, and gradients of fragmentation by damming, water abstraction or flow regimes including drying. Coordinated distributed experiments or global research networks (e.g., eLTER, GLEON) could facilitate the implementation of such large research projects where great sampling effort is unavoidable but unreachable for small research groups or low-funded institutions or countries.

Finally, models will be necessary to bridge the gap between what is possible with laboratory and field studies. In many cases, mechanisms that can be precisely quantified in the laboratory will be infeasible to study in the field, and the degree to which (and mechanisms by which) laboratory-scale effects scale to local ecosystems and regional meta-ecosystems will be unknown or not quantifiable. An added benefit of models is that, when properly constructed, they can themselves provide information back to field studies by generating testable hypotheses and guiding experimental design (Box 4). Ideally, this crosstalk between modelling and empirical work is iterative, where testable hypotheses generated by theory and modelling suggests empirical work, the results of which are used to refine the models.

Conclusion: Fluvial meta-ecosystems in the Anthropocene

Globally, freshwater ecosystems represent a small fraction of total land area, but support a disproportionately high fraction of total biodiversity and are increasingly under threat 86 , 87 . We have argued here that threats to biodiversity in rivers represent threats to (meta-) ecosystem function that are potentially much greater than would be predicted under existing frameworks due to the combination of the unique spatial structure of rivers and the potential for biodiversity and resource diversity to become de-coupled in space. These spatial processes include not only the “classic” spatial effects such as land use, network topology, connectivity, and flow, but also the complex spatio-temporal dynamics of organic matter, the biological community, and the spatio-temporal matching of both. It follows that anthropogenic changes will have multi-faceted impacts on fluvial meta-ecosystem functioning, potentially disrupting both consumer and resource stability in the entire meta-ecosystem 88 , 89 . Therefore, it is essential to understand and integrate mechanisms from local to river network scales. Without this understanding, neither the generation of reliable predictions nor the management for biodiversity and EF in river networks will be possible 1 .

Thus, we call for research that specifically integrates the interactions between consumer communities and OM at all relevant scales in order to properly inform management and assess potential impacts. In particular, it is not enough to consider how a proposed development project will impact local ecosystems, especially when the project will impact connectivity (e.g., diversion projects, dams). Rather, impact assessments must properly consider how work will affect the natural movement of organisms and resources both down- and upstream of the proposed location, and how these regional impacts will propagate to functioning of the entire meta-ecosystem. Moreover, human-induced changes in ecosystem temporal dynamics have the potential to greatly change how consumer–resource interactions unfold in space. For example, changes to resource phenology (e.g., changing timing of leaf fall) will certainly influence the timing of OM availability in the network; such changes will likely propagate to consumer community composition in both space and time and may interfere with consumer phenology 90 , 91 , 92 . Phenological changes could also interact with other temporal anthropogenic changes. Increasing intermittency can increase the accumulation of terrigenous carbon in dry riverbeds, shifting systems away from steady or predictable OM availability and more towards OM pulses 93 , 94 . The effects of such changes on consumer communities in space and time is understudied. Thus, we argue that only by fully considering spatio-temporal context, feedbacks, and the inter-connected nature of river networks can we close the scale gap and come to a more complete understanding of how and why river networks function the way they do.

Box 1. Glossary of bolded terms

Biodiversity : The variety of organisms within a specified location. Biodiversity can be with respect to taxonomy (e.g., the number of species), but can also refer to phylogenetic or functional diversity. In the context of this manuscript, biodiversity of feeding traits is particularly relevant for understanding OM processing.

Biodiversity–ecosystem functioning relationships : Correlations between the diversity of organisms and the magnitude of ecosystem functions at a location.

Ecosystem function : Biophysical processes that contribute to the quantities and fluxes of organisms and materials in an ecosystem. Examples include dispersal rates, primary and secondary production, and nutrient mineralization.

Landscape context : A description of the setting in the landscape in which a river network is embedded; for example, the proportion of various land use types, or the geological substrate within a river’s catchment.

Mass effects : The influence of species due to large population sizes and/or dispersal fluxes, such that the large number of propagules overcomes other mechanisms structuring a community.

Metacommunity : A series of biological communities (i.e., collections of organisms in a particular place in time) that are linked together by dispersal.

Meta-ecosystem : Similar to metacommunities; a collection of interconnected ecosystems that exchange material (e.g., nutrients), energy, and organisms via spatial linkages.

Niche space : A multidimensional description of the conditions under which a particular organism is able to maintain positive population grown.

Organic matter (OM) diversity : The variety of OM in a particular place and time; includes all aspects of OM that might be relevant for consumers, including chemical composition, reactivity, and differences in molecule or particle size.

Regional species pool : The collection of species that can potentially occur across an entire metacommunity.

Sorting : Community ecology process by which a list of organisms that arrive at a site are selected, generally based on having traits matching the local environmental conditions that allow them to establish and better compete for space and resources.

Box 2. Measuring relevant niche dimensions: OM diversity defines the resource space for consumers

Chemical composition of OM has been traditionally considered the main driver of decomposition. Most common OM characterization techniques used so far provide only limited biochemical information; either because they only inform about a certain fraction of the OM pool (e.g., chromophoric DOM by spectroscopic measurements), or because the necessity of combining a great variety of laborious chemical analyses to quantify different elements or molecules. This incomplete characterization of OM has resulted in certain knowledge gaps regarding OM processing, such as the lack of understanding of OM molecule interactions controlling priming reactions or non-additive effects of chemical diversity on decomposition 58 . Resolving these knowledge gaps requires that we widen our understanding of OM as forming various niche dimensions. To this end, we must increase the analytical resolution of OM characterization, but also consider additional niche dimensions, e.g., OM physical properties such as particle size).

Novel techniques enable an in-depth chemical molecular characterization of complex OM mixtures, and can be easily applied to OM size gradients. These include nuclear magnetic resonance spectroscopy (NMR) 49 , Fourier transform ion cyclotron mass spectrometry (FT-ICR-MS) 48 and Fourier transform infrared spectroscopy (FTIR) 50 . However, each of these techniques describes a different chemical structural level of OM. For example, FTIR can only characterize functional groups (e.g., COOH, O–H, C=O); NMR describes chemical bonds (e.g., CH, CH 2 , NH 2 ), functional groups and molecules (e.g., carbohydrates, proteins, lipids); and FT-ICR-MS informs about elemental composition (e.g., C, N, O, S) as well as likely macromolecules (e.g., biopolymers, polysaccharides). This includes also chemical bonds, functional groups and simple molecules in OM, but has deficits regarding structural resolution 95 .

Some techniques, such as fluorescence measurements and size exclusion chromatography coupled with organic and/or nitrogen organic detector(s), describe individual molecules and macromolecules and are mostly used to characterize DOM 51 . Although they can also be used in the POM fraction, they require a liquid sample and therefore can only be performed on DOM extracted from the POM, resulting in highly tedious sample processing. On the contrary, pyrolysis gas chromatography mass spectrometry 96 , which describes molecules and macromolecules, is mostly applied to characterize POM. Although it can also be used for DOM, it requires large amounts of freeze-dried sample material.

In the case of POM, techniques such as laser diffractometry, flow cytometry and imaging/photometry are used to additionally characterize the particle physical properties, particularly size. Laser diffractometry is based on the fact that particles, when hit with a light beam, diffract light in a given angle that depends on particle size (i.e., the angle increases with decreasing particle size) 97 . Similarly, flow cytometry detects light scattering after particles pass one by one through incident light beams from one or more lasers 98 . Flow cytometry can be used to measure not only size, but also surface roughness/granularity or volume. Moreover, it can be coupled with fluorescence detectors, allowing automatic differentiation of particle types when some particles are fluorescently stained (e.g., laser in situ scattering and transmissiometry) 99 . Imaging/photometrical techniques, which process particle pictures (obtained, e.g., from microscopes, cameras, etc.) using image processing and particle analysis software (e.g., ImageJ) 100 , can obtain multiple parameters including area (a proxy for size), perimeter, and major and minor axes. While laser diffractometry and flow cytometry are much faster than imaging/photometry techniques when many particles need to be measured, the main advantage of imaging/photometrical techniques is that it can be combined with other techniques (e.g., FTIR) to assess both physical and chemical molecular properties of individual particles.

Box 3. Metacommunities in river networks: species sorting by environmental factors, dispersal limitation, and chance events

Understanding the distribution, abundance and eventual function of communities is one of the core goals of ecology 67 , 101 . While much work had been done around well-mixed populations or communities (i.e., single or multi-species assemblages at one site/locality), most natural systems are spatially structured. Spatial structure, and spatial heterogeneity in particular, means that not all species are at all sites all the time, but that there are differences in species distribution and abundance. Presence or absence of species can be due to deterministic factors (e.g., local environmental conditions not allowing a species to survive/persist), species not yet having colonized a site, or chance events (e.g., a species going locally extinct due to a stochastic reason) 102 . In reality, these different processes concur and can generate feedbacks, as the presence or absence of a species can for example trigger the dispersal or survival of another species.

The metacommunity concept describes the processes governing the emergence and persistence of spatially structure multi-species assemblages 67 , 101 . While environmental factors generally generate the envelope within which each species can persist, colonization of specific locations by species is mostly governed by the interplay of dispersal, persistence, and extinction (due to deterministic or stochastic reasons). Recent work in metacommunity ecology has shown that non-trivial patterns of species distribution and community structure in heterogeneous landscapes can emerge solely due to a combination of dispersal and stochasticity, particularly in river networks 70 , 103 , 104 , 105 . Importantly, which species is living where in a network of habitats (e.g., along a river network) can be the outcome of the network’s structure and properties, yet also result in feedbacks on community function, for example the complexity or structure of food-webs 17 , 22 .

Understanding the structure and function of ecological communities requires assessing the abundance and traits of many (potentially thousands) of species across multiple spatial and temporal scales. For example, for a coherent study of realistic food webs that fully mechanistically characterizes the transport and processing of resources, organismal groups including bacteria, invertebrates and vertebrates need to be covered. Until very recently, such monitoring has been largely impossible due to methodological constraints. However, recent advances in the use of environmental DNA (eDNA) techniques allow the study of many organisms at a time by the analysis of their DNA present in water 106 , 107 , and the reconstruction of communities and food webs across space 108 , 109 , 110 , 111 . By doing so, ecology has paralleled (bio)geochemistry with a tool for high-throughput analysis of ecological communities, creating the potential to link organismal composition and diversity to the occurrence and diversity of chemical molecules covering nutrients to complex organic compounds. Ultimately, the integration of these two fields in a spatially explicit perspective may allow understanding of how the abiotic world, shaped by chemical compounds and physical properties, is cross-linked to the biological world, generating feedbacks and ultimately driving the state and properties of both.

Box 4. Building models of fluvial meta-ecosystem functioning

Models should be an essential component of any research programme studying fluvial meta-ecosystem functioning. In particular, process-based models can be used to explore the intersection between theory and empirical observation and allow for easy scaling from local to regional processes. Models also allow for manipulations that are impractical or impossible in the field, and they can be used as a tool for hypothesis generation, suggesting useful avenues for future field studies. Here, we elaborate a general framework for how such models could be formulated.

We propose that meta-ecosystem models begin from two coupled differential equations, one describing physical state variables, and one biological. As an example, for the physical side, we model the change in the concentration of dissolved organic carbon ( \(\left[{\rm {DOM}}\right]\) ) in a stream reach i using a transport-reaction equation 112 :

where input is the combination of transport of DOM from upstream reaches and contributions from lateral (e.g., overland or groundwater) inputs, and output is DOM transported to the next reach downstream. These terms will necessarily scale with discharge, and may also be modelled as a function of other characteristics (e.g., streambed area, leaf production) depending on the level of detail required. The final term is reaction, summed across all s species present in the reach, and combines the consumption rate of a species j and the abundance of that species in the reach. This equation operates at the local scale; regional dynamics emerge from the interactions between transport (driven by connectivity and water flow in the river network) and the consumption of resources, which is itself determined by metacommunity dynamics.

For the community side, we can describe the rate of change in the number of reaches occupied by species j :

This is a simple metapopulation model 113 , but extended to include all species in a community. The first term in the model describes colonizations; P j is the number of reaches occupied by species j and serves as a dispersal term (i.e., more occupied reaches results in more colonization). N is the total number of reaches; we multiply by ( N − P j ) because sites must be unoccupied to be colonized. Finally, c is the colonization rate, which is a function of the DOM concentration, thereby linking it to the physical equation. The second term describes extinctions and follows similar logic; a reach must be occupied to experience extinction, and the extinction rate m is a function of the DOM concentration.

Importantly, colonization and extinction rates are heterogeneous in space, shifting the focus of the model from the regional scale to the local scale. Other processes of interest can also be easily incorporated; for example, competition with other species can be incorporated in the extinction term 114 , dispersal rates can influence colonization 115 , and habitat/niche dimensions can be made a part of the colonization and extinction functions 116 , 117 .

These two models are strongly linked; the processing term in the resource equation is dependent on the traits of locally available species and will be higher when species possess the ability to process the resource. Conversely, species with high affinity for the resource will be more likely to colonize a reach where the resource concentration is high, and more likely to go extinct when it is low. Meta-ecosystem functioning is an emergent property in this model that results from the interplay of species presence–absence across all reaches and the DOM consumption rates of these species. It can easily be estimated by summing the resource processing term over all local ecosystems and over a desired time interval.

Cid, N. et al. From meta-system theory to the sustainable management of rivers in the Anthropocene. Front. Ecol. Environment. 20 , 49–57 (2022).

Article   Google Scholar  

Battin, T. J. et al. Biophysical controls on organic carbon fluxes in fluvial networks. Nat. Geoscience. 1 , 95–100 (2008).

Article   CAS   Google Scholar  

Battin, T. J. et al. River ecosystem metabolism and carbon biogeochemistry in a changing world. Nature 613 , 449–459 (2023).

Article   CAS   PubMed   Google Scholar  

Gounand, I., Little, C. J., Harvey, E. & Altermatt, F. Cross-ecosystem carbon flows connecting ecosystems worldwide. Nat. Commun. 9 , 4825 (2018).

Article   PubMed   PubMed Central   Google Scholar  

Cardinale, B. J. et al. Biodiversity loss and its impact on humanity. Nature 486 , 59–67 (2012).

Duffy, J. E., Godwin, C. M. & Cardinale, B. J. Biodiversity effects in the wild are common and as strong as key drivers of productivity. Nature 549 , 261–264 (2017).

Hotchkiss, E. R. et al. Sources of and processes controlling CO 2 emissions change with the size of streams and rivers. Nat. Geosci. 8 , 696–699 (2015).

Datry, T. et al. A global analysis of terrestrial plant litter dynamics in non-perennial waterways. Nat. Geosci. 11 , 497–503 (2018).

del Campo, R., Corti, R. & Singer, G. Flow intermittence alters carbon processing in rivers through chemical diversification of leaf litter. Limnol. Oceanogr. Lett. 6 , 232–242 (2021).

Bhattacharya, R. & Osburn, C. L. Spatial patterns in dissolved organic matter composition controlled by watershed characteristics in a coastal river network: the Neuse River Basin, USA. Water Res. 169 , 115248 (2020).

Casas-Ruiz, J. P. et al. Delineating the continuum of dissolved organic matter in temperate river networks. Global Biogeochem. Cycles 34 , e2019GB006495 (2020).

Roebuck, J. A. J., Seidel, M., Dittmar, T. & Jaffé, R. Controls of land use and the river continuum concept on dissolved organic matter composition in an anthropogenically disturbed subtropical watershed. Environ. Sci. Technol. 54 , 195–206 (2020).

Hodapp, D., Hillebrand, H. & Striebel, M. "Unifying” the concept of resource use efficiency in ecology. Front. Ecol. Evol. 6 , 233 (2019).

Andersen, T., Elser, J. J. & Hessen, D. O. Stoichiometry and population dynamics. Ecol. Lett. 7 , 884–900 (2004).

Barnes, A. D. et al. Energy flux: the link between multitrophic biodiversity and ecosystem functioning. Trends Ecol. Evol. 33 , 186–197 (2018).

Little, C. J., Fronhofer, E. A. & Altermatt, F. Nonlinear effects of intraspecific competition alter landscape-wide scaling up of ecosystem function. Am. Nat. 195 , 432–444 (2020).

Article   PubMed   Google Scholar  

Jacquet, C., Carraro, L. & Altermatt, F. Meta-ecosystem dynamics drive the spatial distribution of functional groups in river networks. Oikos 2022 , e09372 (2022).

Wagner, S. et al. Linking the molecular signature of heteroatomic dissolved organic matter to watershed characteristics in world rivers. Environ. Sci. Technol. 49 , 13798–13806 (2015).

Bertuzzo, E., Helton, A. M., Hall, R. O. & Battin, T. J. Scaling of dissolved organic carbon removal in river networks. Adv. Water Resour. 110 , 136–146 (2017).

Raymond, P. A., Saiers, J. E. & Sobczak, W. V. Hydrological and biogeochemical controls on watershed dissolved organic matter transport: pulse-shunt concept. Ecology 97 , 5–16 (2016).

Wohl, E., Hall Jr, R. O., Lininger, K. B., Sutfin, N. A. & Walters, D. M. Carbon dynamics of river corridors and the effects of human alterations. Ecol. Monogr. 87 , 379–409 (2017).

Harvey, E. & Altermatt, F. Regulation of the functional structure of aquatic communities across spatial scales in a major river network. Ecology 100 , e02633 (2019).

Ho, H.-C., Altermatt, F. & Carraro, L. Coupled biological and hydrological processes shape spatial food-web structures in riverine metacommunities. Front. Ecol. Evol. 11 , https://doi.org/10.3389/fevo.2023.1147834 (2023).

Vannote, R. L. & Sweeney, B. W. Geographic analysis of thermal equilibria: a conceptual model for evaluating the effect of natural and modified thermal regimes on aquatic insect communities. Am. Nat. 115 , 667–695 (1980).

Heino, J. et al. A comparative analysis reveals weak relationships between ecological factors and beta diversity of stream insect metacommunities at two spatial levels. Ecol. Evol. 5 , 1235–1248 (2015).

Lambert, T., Bouillon, S., Darchambeau, F., Massicotte, P. & Borges, A. V. Shift in the chemical composition of dissolved organic matter in the Congo River network. Biogeosciences 13 , 5405–5420 (2016).

Rosi-Marshall, E. J. & Wallace, J. B. Invertebrate food webs along a stream resource gradient. Freshw. Biol. 47 , 129–141 (2002).

Sobczak, W. V., Cloern, J. E., Jassby, A. D. & Müller-Solger, A. B. Bioavailability of organic matter in a highly disturbed estuary: the role of detrital and algal resources. Proc. Natl Acad. Sci. USA 99 , 8101–8105 (2002).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Subalusky, A. L., Dutton, C. L., Njoroge, L., Rosi, E. J. & Post, D. M. Organic matter and nutrient inputs from large wildlife influence ecosystem function in the Mara River, Africa. Ecology 99 , 2558–2574 (2018).

Anderson, K. E. & Hayes, S. M. The effects of dispersal and river spatial structure on asynchrony in consumer–resource metacommunities. Freshw. Biol. 63 , 100–113 (2018).

Helton, A. M., Hall Jr, R. O. & Bertuzzo, E. How network structure can affect nitrogen removal by streams. Freshw. Biol. 63 , 128–140 (2018).

Altermatt, F., Seymour, M. & Martinez, N. River network properties shape α -diversity and community similarity patterns of aquatic insect communities across major drainage basins. J. Biogeogr. 40 , 2249–2260 (2013).

Alther, R. & Altermatt, F. Fluvial network topology shapes communities of native and non-native amphipods. Ecosphere 9 , e02102 (2018).

Carrara, F., Altermatt, F., Rodriguez-Iturbe, I. & Rinaldo, A. Dendritic connectivity controls biodiversity patterns in experimental metacommunities. Proc. Natl Acad. Sci. USA 109 , 5761–5766 (2012).

Harvey, E., Gounand, I., Fronhofer, E. A. & Altermatt, F. Metaecosystem dynamics drive community composition in experimental, multi-layered spatial networks. Oikos 129 , 402–412 (2020).

Taylor, M. K. & Cooke, S. J. Meta-analyses of the effects of river flow on fish movement and activity. Environ. Rev. 20 , 211–219 (2012).

Gounand, I., Harvey, E., Little, C. J. & Altermatt, F. Meta-Ecosystems 2.0: rooting the theory into the field. Trends Ecol. Evol. 33 , 36–46 (2018).

Marx, A. et al. A review of CO 2 and associated carbon dynamics in headwater streams: a global perspective. Rev. Geophys. 55 , 560–585 (2017).

Carraro, L. & Altermatt, F. Optimal Channel Networks accurately model ecologically-relevant geomorphological features of branching river networks. Commun. Earth Environ. 3 , 1–10 (2022).

Google Scholar  

Wotton, R. S. In The Classification of Particulate and Dissolved Matter 2 edn, (ed. Wotton, R. S.) The Biology of Particles in Aquatic Systems (Taylor and Francis, Boca Raton, 1994).

Wallace, J. B. et al. Long-term dynamics of coarse particulate organic matter in three Appalachian Mountain streams. J. N. Am. Benthol. Soc. 14 , 217–232 (1995).

Suberkropp, K. F. In Microorganisms and organic matter decomposition (eds Naiman, R. J. & Bilby, R. E.) River Ecology and Management: Lessons from the Pacific Coastal Ecoregion 120–143 (Springer-Verlag, New York, 1998).

Allan, J. D. & Castillio, M. M. In Detrital Energy Sources (eds Allan, J. D. & Castillio, M. M.) Stream Ecology: Structure and Function of Running Waters 135–161 (Springer Netherlands, Dordrecht, 2007).

Wipfli, M. S., Richardson, J. S. & Naiman, R. J. Ecological linkages between headwaters and downstream ecosystems: transport of organic matter, invertebrates, and wood down headwater channels1. J. Am. Water Resour. Assoc. 43 , 72–85 (2007).

Stedmon, C. A., Markager, S. & Bro, R. Tracing dissolved organic matter in aquatic environments using a new approach to fluorescence spectroscopy. Mar. Chem. 82 , 239–254 (2003).

Voshell Jr, J. R. & Parker, C. R. Quantity and quality of seston in an impounded and a free-flowing river in Virginia, USA. Hydrobiologia 122 , 271–280 (1985).

Zhang, M. et al. Leaf litter traits predominantly control litter decomposition in streams worldwide. Global Ecol. Biogeogr. 28 , 1469–1486 (2019).

Sleighter, R. L. & Hatcher, P. G. The application of electrospray ionization coupled to ultrahigh resolution mass spectrometry for the molecular characterization of natural organic matter. J. Mass Spectrom. 42 , 559–574 (2007).

Hertkorn, N., Harir, M., Koch, B. P., Michalke, B. & Schmitt-Kopplin, P. High-field NMR spectroscopy and FTICR mass spectrometry: powerful discovery tools for the molecular level characterization of marine dissolved organic matter. Biogeosciences 10 , 1583–1624 (2013).

Matilainen, A. et al. An overview of the methods used in the characterisation of natural organic matter (NOM) in relation to drinking water treatment. Chemosphere 83 , 1431–1442 (2011).

Huber, S. A., Balz, A., Abert, M. & Pronk, W. Characterisation of aquatic humic and non-humic matter with size-exclusion chromatography–organic carbon detection– organic nitrogen detection (LC-OCD-OND). Water Res. 45 , 879–885 (2011).

Sleighter, R. L. & Hatcher, P. G. Molecular characterization of dissolved organic matter (DOM) along a river to ocean transect of the lower Chesapeake Bay by ultrahigh resolution electrospray ionization Fourier transform ion cyclotron resonance mass spectrometry. Mar. Chem. 110 , 140–152 (2008).

Tremblay, L., Alaoui, G. & Léger, M. N. Characterization of aquatic particles by direct FTIR analysis of filters and quantification of elemental and molecular compositions. Environ. Sci. Technol. 45 , 9671–9679 (2011).

McDonough, L. K. et al. A new conceptual framework for the transformation of groundwater dissolved organic matter. Nat. Commun. 13 , 2153 (2022).

Little, C. J. & Altermatt, F. Landscape configuration alters spatial arrangement of terrestrial-aquatic subsidies in headwater streams. Landsc. Ecol. 33 , 1519–1531 (2018).

Larned, S. T., Datry, T., Arscott, D. B. & Tockner, K. Emerging concepts in temporary-river ecology. Freshw. Biol. 55 , 717–738 (2010).

Lynch, L. M. et al. River channel connectivity shifts metabolite composition and dissolved organic matter chemistry. Nat. Commun. 10 , 459 (2019).

Kothawala, D. N., Kellerman, A. M., Catalán, N. & Tranvik, L. J. Organic matter degradation across ecosystem boundaries: the need for a unified conceptualization. Trends Ecol. Evol. 36 , 113–122 (2021).

Catalán, N., Marcé, R., Kothawala, D. N. & Tranvik, Lars J. Organic carbon decomposition rates controlled by water retention time across inland waters. Nat. Geosci. 9 , 501–504 (2016).

Graham, E. B. et al. Microbes as engines of ecosystem function: when does community structure enhance predictions of ecosystem processes? Front. Microbiol. 7 , https://doi.org/10.3389/fmicb.2016.00214 (2016).

Brown, B. L. & Swan, C. M. Dendritic network structure constrains metacommunity properties in riverine ecosystems. J. Anim. Ecol. 79 , 571–580 (2010).

Göthe, E., Angeler, D. G. & Sandin, L. Metacommunity structure in a small boreal stream network. J. Anim. Ecol. 82 , 449–458 (2013).

Gauthier, M. et al. Fragmentation promotes the role of dispersal in determining 10 intermittent headwater stream metacommunities. Freshw. Biol. 65 , 2169–2185 (2020).

Besemer, K. et al. Headwaters are critical reservoirs of microbial diversity for fluvial networks. Proc. R. Soc. B: Biol. Sci. 280 , 20131760 (2013).

Widder, S. et al. Fluvial network organization imprints on microbial co-occurrence networks. Proc. Natl Acad. Sci. USA 111 , 12799–12804 (2014).

Read, D. S. et al. Catchment-scale biogeography of riverine bacterioplankton. ISME J. 9 , 516–526 (2015).

Leibold, M. A. et al. The metacommunity concept: a framework for multi-scale community ecology. Ecol. Lett. 7 , 601–613 (2004).

Shao, X., Fang, Y., Jawitz, J. W., Yan, J. & Cui, B. River network connectivity and fish diversity. Sci. Total Environ. 689 , 21–30 (2019).

Witteveen, N. H., Freixa, A. & Sabater, S. Local and regional environmental factors drive the spatial distribution of phototrophic biofilm assemblages in Mediterranean streams. Hydrobiologia 847 , 2321–2336 (2020).

Heino, J. et al. Metacommunity organisation, spatial extent and dispersal in aquatic systems: Patterns, processes and prospects. Freshw. Biol. 60 , 845–869 (2015).

Vorste, R. V., McElmurray, P., Bell, S., Eliason, K. M. & Brown, B. L. Does stream size really explain biodiversity patterns in lotic systems? A call for mechanistic explanations. Diversity 9 , 26 (2017).

Mouquet, N. & Loreau, M. Community patterns in source–sink metacommunities. Am. Nat. 162 , 544–557 (2003).

Declerck, S. A. J., Winter, C., Shurin, J. B., Suttle, C. A. & Matthews, B. Effects of patch connectivity and heterogeneity on metacommunity structure of planktonic bacteria and viruses. ISME J. 7 , 533–542 (2013).

Horváth, Z., Vad, C. F. & Ptacnik, R. Wind dispersal results in a gradient of dispersal limitation and environmental match among discrete aquatic habitats. Ecography 39 , 726–732 (2016).

Schwartz, M. W. et al. Linking biodiversity to ecosystem function: Implications for conservation ecology. Oecologia 122 , 297–305 (2000).

Logue, J. B. et al. Experimental insights into the importance of aquatic bacterial community composition to the degradation of dissolved organic matter. ISME J. 10 , 533–545 (2016).

Ruiz-González, C., Niño García, J. P., Lapierre, J.-F. & del Giorgio, P. A. The quality of organic matter shapes the functional biogeography of bacterioplankton across boreal freshwater ecosystems: the functional biogeography of bacteria. Global Ecol. Biogeogr. 24 , 1487–1498 (2015).

Stream Solute Workshop. Concepts and methods for assessing solute dynamics in stream ecosystems. J. N. Am. Benthol. Soc. 9 , 95–119 (1990).

Kaplan, L. A. & Newbold, J. D. In 4 - The Role of Monomers in Stream Ecosystem Metabolism (eds. Findlay, S. E. G. & Sinsabaugh, R. L.) Aquatic Ecosystems Aquatic Ecology, 97–119 (Academic Press, Burlington, 2003).

Newbold, J. D., Mulholland, P. J., Elwood, J. W. & O’Neill, R. V. Organic carbon spiralling in stream ecosystems. Oikos 38 , 266–272 (1982).

Wiegner, T. N., Kaplan, L. A., Newbold, J. D. & Ostrom, P. H. Contribution of dissolved organic C to stream metabolism: a mesocosm study using 13C-enriched tree-tissue leachate. J. N. Am. Benthol. Soc. 24 , 48–67 (2005).

Brett, M. T. et al. How important are terrestrial organic carbon inputs for secondary production in freshwater ecosystems? Freshw. Biol. 62 , 833–853 (2017).

Leal, J. S. et al. Global and local drivers of the relative importance of allochthonous and autochthonous energy sources to freshwater food webs. Ecography 2023 , e06612 (2023).

Estévez, E. et al. Catchment land cover influences macroinvertebrate food-web structure and energy flow pathways in mountain streams. Freshw. Biol. 64 , 1557–1571 (2019).

Wagner, K., Bengtsson, M. M., Findlay, R. H., Battin, T. J. & Ulseth, A. J. High light intensity mediates a shift from allochthonous to autochthonous carbon use in phototrophic stream biofilms. J. Geophys. Res.: Biogeosci. 122 , 1806–1820 (2017).

Albert, J. S. et al. Scientists’ warning to humanity on the freshwater biodiversity crisis. Ambio 50 , 85–94 (2021).

Tickner, D. et al. Bending the curve of global freshwater biodiversity loss: an emergency recovery plan. BioScience 70 , 330–342 (2020).

Gravel, D., Massol, F. & Leibold, M. A. Stability and complexity in model meta-ecosystems. Nat. Commun. 7 , 12457 (2016).

Mougi, A. & Kondoh, M. Food-web complexity, meta-community complexity and community stability. Sci. Rep. 6 , 24478 (2016).

Rüegg, J. et al. Thinking like a consumer: linking aquatic basal metabolism and consumer dynamics. Limnol. Oceanogr. Lett. 6 , 1–17 (2021).

Kominoski, J. S. & Rosemond, A. D. Conservation from the bottom up: Forecasting effects of global change on dynamics of organic matter and management needs for river networks. Freshw. Sci. 31 , 51–68 (2012).

Woods, T., Kaz, A. & Giam, X. Phenology in freshwaters: a review and recommendations for future research. Ecography 2022 , e05564 (2022).

Catalàn, N. et al. Pulse, shunt and storage: hydrological contraction shapes processing and export of particulate organic matter in river networks. Ecosystems 26 , 873–892 (2023).

Hale, R. L. & Godsey, S. E. Dynamic stream network intermittence explains emergent dissolved organic carbon chemostasis in headwaters. Hydrol. Process. 33 , 1926–1936 (2019).

Derrien, M., Brogi, S. R. & Gonçalves-Araujo, R. Characterization of aquatic organic matter: assessment, perspectives and research priorities. Water Res. 163 , 114908 (2019).

Jeanneau, L. et al. Sources of dissolved organic matter during storm and inter-storm conditions in a lowland headwater catchment: constraints from high-frequency molecular data. Biogeosciences 12 , 4333–4343 (2015).

Beuselinck, L., Govers, G., Poesen, J., Degraer, G. & Froyen, L. Grain-size analysis by laser diffractometry: comparison with the sieve-pipette method. CATENA 32 , 193–208 (1998).

Agagliate, J., Röttgers, R., Twardowski, M. S. & McKee, D. Evaluation of a flow cytometry method to determine size and real refractive index distributions in natural marine particle populations. Appl. Opt. 57 , 1705–1716 (2018).

Boss, E., Haëntjens, N., Westberry, T. K., Karp-Boss, L. & Slade, W. H. Validation of the particle size distribution obtained with the laser in-situ scattering and transmission (LISST) meter in flow-through mode. Opt. Express 26 , 11125–11136 (2018).

Igathinathane, C., Pordesimo, L. O., Columbus, E. P., Batchelor, W. D. & Methuku, S. R. Shape identification and particles size distribution from basic shape parameters using ImageJ. Comput. Electron. Agric. 63 , 168–182 (2008).

Holyoak, M., Leibold, M. A. & Holt, R. D. (eds.) Metacommunities: Spatial Dynamics and Ecological Communities (University of Chicago Press, Chicago, 2005).

Vellend, M. Conceptual synthesis in community ecology. Q. Rev. Biol. 85 , 183–206 (2010).

Altermatt, F. Diversity in riverine metacommunities: a network perspective. Aquat. Ecol. 47 , 365–377 (2013).

Bertuzzo, E. et al. Geomorphic controls on elevational gradients of species richness. Proc. Natl Acad. Sci. 113 , 1737–1742 (2016).

Sarremejane, R., Mykrä, H., Bonada, N., Aroviita, J. & Muotka, T. Habitat connectivity and dispersal ability drive the assembly mechanisms of macroinvertebrate communities in river networks. Freshw. Biol. 62 , 1073–1082 (2017).

Deiner, K. et al. Environmental DNA metabarcoding: transforming how we survey animal and plant communities. Mol. Ecol. 26 , 5872–5895 (2017).

Taberlet, P., Coissac, E., Hajibabaei, M. & Rieseberg, L. H. Environmental DNA. Mol. Ecol. 21 , 1789–1793 (2012).

Altermatt, F. et al. Uncovering the complete biodiversity structure in spatial networks: the example of riverine systems. Oikos 129 , 607–618 (2020).

Blackman, R. C., Ho, H.-C., Walser, J.-C. & Altermatt, F. Spatio-temporal patterns of multi-trophic biodiversity and food-web characteristics uncovered across a river catchment using environmental DNA. Commun. Biol. 5 , 1–11 (2022).

Carraro, L., Mächler, E., Wüthrich, R. & Altermatt, F. Environmental DNA allows upscaling spatial patterns of biodiversity in freshwater ecosystems. Nat. Commun. 11 , 3585 (2020).

Mächler, E. et al. Assessing different components of diversity across a river network using eDNA. Environ. DNA 1 , 290–301 (2019).

Soetaert, K. & Herman, P. M. A Practical Guide to Ecological Modelling (Springer, Berlin, 2009).

Levins, R. Some demographic and genetic consequences of environmental heterogeneity for biological control1. Bull. Entomol. Soc. Am. 15 , 237–240 (1969).

Hunt, J. J. & Bonsall, M. B. The effects of colonization, extinction and competition on co-existence in metacommunities. J. Anim. Ecol. 78 , 866–879 (2009).

Kneitel, J. M. & Miller, T. E. Dispersal rates affect species composition in metacommunities of Sarracenia purpurea inquilines. Am. Nat. 162 , 165–171 (2003).

Holt, R. & Keitt, T. Alternative causes for range limits: a metapopulation perspective. Ecol. Lett. 3 , 41–47 (2000).

Talluto, M. V., Boulangeat, I., Vissault, S., Thuiller, W. & Gravel, D. Extinction debt and colonization credit delay range shifts of eastern North American trees. Nat. Ecol. Evol. 1 , 1–6 (2017).

Download references

Acknowledgements

M.T. and G.S. acknowledge funding from the European Research Council Starting Grant number ERC-STG 716196 (FLUFLUX). E.E. was supported by a postdoctoral grant from the Basque Government. Jan Martini aided with illustrations for Fig. 1 . Support for publication fees was provided by the Vice Rector for Research at the University of Innsbruck.

Author information

Authors and affiliations.

Department of Ecology, University of Innsbruck, Technikerstrasse 25, 6020, Innsbruck, Austria

Matthew Talluto, Rubén del Campo, Edurne Estévez & Gabriel Singer

Department of Aquatic Ecology, Eawag: Swiss Federal Institute of Aquatic Science and Technology, Überlandstrasse 133, 8600, Dübendorf, Switzerland

Florian Altermatt

Department of Evolutionary Biology and Environmental Studies, University of Zürich, Winterthurerstrasse 190, 8057, Zürich, Switzerland

National Research Institute for Agriculture, Food and Environment (INRAE), 5 Rue de la Doua, 69100, Villeurbanne, France

Thibault Datry

You can also search for this author in PubMed   Google Scholar

Contributions

M.T., G.S., T.D., and F.A. developed the initial manuscript outline. All authors contributed text to the main manuscript, and M.T. and R.d.C. prepared the figures. M.T. led revisions with input from all authors.

Corresponding author

Correspondence to Matthew Talluto .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Talluto, M., del Campo, R., Estévez, E. et al. Towards (better) fluvial meta-ecosystem ecology: a research perspective. npj biodivers 3 , 3 (2024). https://doi.org/10.1038/s44185-023-00036-0

Download citation

Received : 28 June 2023

Accepted : 22 December 2023

Published : 07 February 2024

DOI : https://doi.org/10.1038/s44185-023-00036-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Anthropocene newsletter — what matters in anthropocene research, free to your inbox weekly.

Deep Ecology Research Paper

Sample Deep Ecology Research Paper. Browse other research paper examples and check the list of environmental research paper topics for more inspiration. iResearchNet offers academic assignment help for students all over the world: writing from scratch, editing, proofreading, problem solving, from essays to dissertations, from humanities to STEM. We offer full confidentiality, safe payment, originality, and money-back guarantee. Secure your academic success with our risk-free services.

The term deep ecology was introduced by Norwegian philosopher and mountaineer Arne Naess (1912–) in 1972 at the Third World Future Research Conference in Bucharest. After a distinguished academic career, Naess chose early retirement to focus his expertise on addressing the global ecological crisis. Naess coined the terms deep ecology and shallow ecology to juxtapose what he regarded as two radically different approaches for problematizing (Problematisieren) and responding to the ecological crisis (Naess 1973). Deep ecology posits that along with humans’ special capacities for reason and moral consciousness come equally special responsibilities, particularly in relation to the flourishing of nonhuman life and the ecological sustainability of the planet. In the years since introducing the term, Naess and other supporters of the deep ecology movement have written extensively, elaborating and popularizing the term in various directions. Considerable controversy and confusion has ensued, in part because deep ecology calls for a radical rethinking of our relationship to each other and nature, because provocative terminology and philosophical vagueness have been employed, and because deep ecology has unfolded over time, with substantial emendation. As a result, deep ecology has come to mean many things to many people. Regrettably, the term is now often used without discrimination to refer to three distinct entities: (a) the particular ecophilosophical approach advanced by Naess and other theoreticians of the deep ecology movement; (b) the international, grassroots, often activist oriented deep ecology movement; and (c) Naess’ personal systematization of a philosophy of ecological harmony, Ecosophy T. After introducing deep ecology as a unique approach to ecophilosophy, the evolution of deep ecology will be reviewed and the remaining two entities will be briefly considered.

Academic Writing, Editing, Proofreading, And Problem Solving Services

Get 10% off with 24start discount code, 1. deep ecology as ecophilosophy.

The purpose of deep ecology as an ecophilosophical approach is to assist individuals in the process of weaving descriptive and prescriptive premises about the world, ecological science, and their ultimate beliefs into a cohesive framework for guiding decisions involving society and nature—ecologically inspired total views or ecosophies. This focus on praxis (responsibility and action) separates the deep ecology approach from more descriptive inquiries into environmental philosophy that focus on axiological questions, such as extending ‘rights’ to certain nonhumans or grading intrinsic value. The ontologically inspired deep ecology approach emphasizes eliminating the perception of fundamental people/environment and spiritual/physical cleavages. Its primary strategy for overcoming the ecological crisis is to help individuals avoid pseudorational thinking.

Naess argues that many regrettable environmental decisions are made in a state of ‘philosophical stupor,’ where narrow concerns are confused with, and then substituted for, more fundamental ones. In proposing the deep/shallow contrast, Naess applies his research on empirical semantics, philosophy of science, Spinoza, the inquiring skepticism of Sextus Empiricus, and Gandhi on nonviolent communication. His technical semantic distinction is directed at the individual’s level of problematizing—the extent to which they can and do coherently and consistently trace their views, practices, and actions back to their ultimate beliefs or bedrock assumptions.

In relating this notion of persistently asking deeper questions to the ecological crisis, Naess broadens his concept of ‘depth.’ In the context of deep ecology as an ecophilosophical approach, depth refers to both the general level of problematizing one employs in seeking out the underlying, coevolving causes of the ecological crisis and the extent of one’s willingness to consider a broad array of social and policy responses, even if they necessitate changes that represent a radical departure from the status quo.

The ‘shallow,’ currently more influential, approach is identified with treating the symptoms of environmental degradation, such as pollution and resource depletion. Its central concern is the health and prosperity of people in the economically privileged countries. This reform-oriented approach is grounded not in ultimate premises that plumb the relationship between humans and nature, but in technological optimism, economic progress, and scientific management. A core premise is that all environmental problems are manageable—nature is a puzzle to be deciphered by human ingenuity and manipulated, albeit more efficiently, for human benefit. From this perspective, remedy for environmental problems is limited to economic, technological, and managerial reforms. This effort to palliate human impacts, rather than probe and address their underlying causes, favors a search for ‘technical’ solutions to what are more likely ethical, social, and political problems. By truncating the realm of problematizing, the shallow approach, perhaps inadvertently, prunes the set of conceivable social changes to a narrow incrementalism.

The ‘deep’ approach, as the other hand, while in no way discounting the exigency of addressing pollution and resource depletion, adopts a broader, long-term, more skeptical stance. Doubtful about technological optimism, critical of limitless economic growth, and decidedly against valuing nature in purely instrumental terms, it asks if the shallow approach’s proposed solutions take into consideration the pervasiveness and severity of the problems they hope to rectify. Drawing on a wide diversity of philosophical or religious ultimate premises, which acknowledge that every living being has value in itself, the deep approach sees the flourishing of nature and culture as fundamentally intertwined. Nature is viewed as mentor, standard, and partner rather than vassal. A key premise is that environmental management is much more about managing the habits and desires of humans than managing nature. Remedy for environmental problems is sought by identifying and responding to the complex ‘root’ causes of the ecological crisis, dedicating special attention to protecting the wild and free from thoughtless human interference. Taking less for granted, the deep approach calls for the public questioning of every practice, assumption, and value that propels the ecological crisis.

By juxtaposing these two, almost caricatured, perspectives, Naess employs a technique of Gandhian nonviolent communication designed to confront core disagreements. The premise is that society’s potential to overcome the ecological crisis rests on maneuvering discussion to its root causes. One of the primary root causes, Naess asserts, is the widespread disjunction between people’s core beliefs and actions. People, in general, neither comprehend how their practices and everyday lifestyle choices harm the environment, nor recognize how these consequences may be in direct conflict with their core beliefs—this is the primary weakness of the shallow approach. A crucial, underlying hypothesis of the deep approach is that teasing out the presumed inconsistencies between an individual’s actions and their fundamental beliefs will effect constructive change instead of generating more serious ancillary conflict.

Rather than simply calling for a new environmental ethic or a change in fundamental values, Naess’ approach to ecophilosophy centers on transforming practice and policy by challenging individuals to develop more thoroughly reasoned, consistent, and ecologically inspired total views. Some will take issue with the core premise underlying this goal, namely that thoroughly reasoned and consistent positions do generally lead to improved policies and actions. There are, however, important indirect procedural benefits that can result from attempting to couple reason and values in decision making.

In any case, Naess intended his distinction between the shallow and deep approaches to environmentalism to be restricted to argumentation patterns and the diversity of policy and lifestyle changes that are given consideration. The distinction was never intended to shed light on the ‘depth’ of particular individuals or their values. Nevertheless, as an expert in semantics and communication theory, Naess cannot be exonerated for failing to anticipate the unfortunate derogatory connotations of his terminology.

2. The Unfolding Of Deep Ecology

Naess’ (1973) inaugural article, The Shallow and the Deep, Long-Range Ecology Movement: A Summary, gave both an appellation and a rudimentary philosophical framework to a movement that had been nascent for well over a century. Deep ecology as both a liberatory social movement and an ecophilosophical approach draws on at least five roots: (a) the nonanthropocentrism and reverence for wildness given voice by Henry David Thoreau, John Muir, and John Stuart Mill; (b) the ecological perspective gleaned from a scientific understanding of humans’ role in creating environmental degradation, articulated by George Perkins Marsh, Aldo Leopold, Rachael Carson, and Paul Ehrlich; (c) the ecocentric vision and social criticism brought to life by the writing and activism of Paul Shepard, Gary Snyder, Edward Abbey, and Dave Brower; (d) the belief in the ultimate unity of all life, communicated through the nonviolent conflict resolution work of Gandhi; and (e) the call for individuals to be conscious of their value priorities and to counteract passivity and despair by actively making them manifest in their daily life, explicated in the philosophical systematization of Spinoza.

Naess’ brief initial article, along with establishing the deep/shallow contrast, outlined a seven-point survey, which was intended to summarize the shared norms and perspectives of the deep ecology movement. This survey introduced terms and slogans, such as ‘rejection of the human-in-environment image in favor of the relational, total field image,’ ‘biospherical egalitarianism—in principle,’ and ‘complexity, not complication.’ While Naess employed vagueness here intentionally to encourage widespread acceptance, these often enigmatic generalizations acted as a lightening rod for controversy.

Apart from the continuous emphasis on maintaining the ‘diversity’ of both human cultures and nature and the reincarnation of the ‘relational, total field image’ notion as Naess’ subtle and rich concept of ‘gestalt ontology,’ these terms neither figure in Naess’ more seasoned renditions of the significant tenets of the deep ecology movement nor his more mature descriptions of deep ecology as an ecophilosophical approach. Nevertheless, they do continue to resurface in many popular discussions and critiques of deep ecology. Most of the terms Naess introduces in the 1973 article, such as diversity, complexity, autonomy, decentralization, symbiosis, egalitarianism, and classlessness, do, however, figure prominently as key norms in his subsequent derivations of Ecosophy T (1977, 1989).

Naess’ only book length discussion of deep ecology began with a short mimeograph, Økologi og Filosfi (Ecology and Philosophy), published in 1971. The fifth revision and expansion of this initial work resulted in the 1976 classic, Økologi, Samfunn, og Livsstil (Ecology, Community, and Lifestyle), which has been translated into Swedish, English (1989), Italian, Japanese, and Czech. Perhaps the earliest intimation that deep ecology should be separated into a social movement, an ecophilosophical approach, and Ecosophy T occurs in 1986, with Naess’ article, ‘The deep ecology movement: Some philosophical aspects.’ This research paper offered an early presentation of the ‘deep ecology platform,’ which was prepared in collaboration with philosopher George Sessions in 1984.

In the late 1970s Naess’ project was taken up and popularized by the California-based team of Sessions and sociologist Bill Devall. Sessions had earlier initiated the process of popularizing deep ecology with his Ecophilosophy Newsletter, published in six volumes, intermittently from 1976 to 1984. Devall and Sessions’ collaboration culminated in the first English language, book length discussion of deep ecology (1985). Subsequently, the two deep ecology theorists worked independently. Devall published extensively on deep ecology practice (1988). Sessions continued to write on the historical foundations and theory of deep ecology, producing the most comprehensive deep ecology anthology to date (1995).

Two additional anthologies are dedicated to the exploration of deep ecology, Reed and Rothenberg (1993) focus on the Norwegian roots of deep ecology and Drengson and Inoue (1995) present an introduction to deep ecology theory and practice. Deep ecology essays and responses to critiques also figure prominently, along with ecofeminism and political ecology, in Zimmerman et al.’s leading environmental philosophy anthology (2001). In addition, Witoszek and Brennan (1999) produced one volume on deep ecology criticism, which includes dialogic responses by both Naess and his interlocutors. All of these collections, however, are necessarily highly selective. As the primary architect of deep ecology, Naess has more than 75 published articles and book chapters and more than 60 unpublished pieces to his credit. The most comprehensive selection of these articles appears in volume 10 of the 11 volume Selected Works of Arne Naess (2002) .

The most significant intellectual history of deep ecology and exhaustive effort to characterize what is ‘distinct’ about it was offered by Fox (1990). Subsequently, Glasser (1997) argued that Fox’s distilling of deep ecology into ‘Self-realization!’ results in a vitiated version of deep ecology. McLaughlin (1993) takes up a discussion of deep ecology as a response to industrialism’s role in creating and perpetuating the ecological crisis. The relationship between Naess’ earlier philosophical work and deep ecology is explored by Glasser (1996), who also mines the policy implications of deep ecology. Cramer (1998) measures the degree of influence that deep ecology principles have had on American environmental politics.

While discussions of deep ecology appear in many popular and academic journals, only the Trumpeter Journal of Ecosophy has been devoted to advancing deep ecology theory and practice. The Trumpeter was under the editorship of Alan Drengson from 1983 to 1997. Subsequently, it has continued as an on-line journal, under the editorship of Bruce Morito.

3. The Deep Ecology Movement

The deep ecology movement is an informal, global affiliation of individuals who believe that overcoming the ecological crisis will require radical changes in the way humans relate to each other and nature. Its supporters are united not by their commitment to deep ecology as an academic ecophilosophy (which they may not recognize or embrace), but by their common commitment to ultimate premises that value nature for its own sake and their general rejection of anthropocentrism. What distinguishes the deep ecology movement from deep ecology as ecophilosophy is that it embodies praxis rather than merely philosophizes about it. The intuitions of deep ecology are not articulated as professional philosophy, but as art, poetry, ritual, conservation biology, grassroots activism, simple lifestyles, bioregionalism, and ecologically sustainable design, farming, fishing, and forestry.

The movement is perhaps best characterized as the group of individuals that do (or would) endorse the eight point Deep Ecology Platform. The ‘Eight Points,’ which summarize 10 years of thinking about deep ecology, were prepared to supersede the problematic seven point (1973) characterization. The first three points outline a high-level norm for protecting the planet’s full diversity and richness of life forms— species, cultures, watersheds, landscapes, and ecosystems. Because the scale and character of human interference with the biosphere are currently excessive and thus incompatible with this norm and the thriving of human life and cultures (points 4 and 5), the next two principles outline a series of changes in practice and policy that can help bring them in line. The final point, without calling for specific actions or imposing particular priorities, urges those who embrace the prior seven points to help effect change.

The Eight Points are intended to express at a generic and abstract level the central views that supporters of the deep ecology movement hold in common. The Platform principles are not intended to characterize common views in particular situations. Since 1984, the Eight Points have been republished extensively along with detailed commentary and gained wide currency. Although they are frequently taken as the ‘heart’ of deep ecology, the eight points alone cannot effect the breadth and complexity of deep ecology as an ecophilosophical approach.

While the Platform can clearly serve as an ‘ecological pillar’ for Green politics, it has been criticized for not integrating the other three pillars. These concerns have been fueled by the sometimes inflammatory and ostensibly misanthropic statements that have been made by ‘Earth Firsters’ and others who have taken inspiration from deep ecology. While the Platform principles do not directly address social justice, grassroots democracy, and nonviolence, they are in no way inimical to them. Quite the contrary, the practical policy changes required by point six leave open immeasurable opportunities for collaboration with the peace and social justice movements. For instance, how can society characterize what constitutes a bundle of ‘vital needs’ for particular communities in particular situations (Point 3); how can society create new accounting systems that operationalize the distinction between ‘quality of life’ and ‘standard of living’ (Point 7); and how can society develop policies to support long-term population reduction through noncoercive and democratic means (Point 4).

4. Ecosophy T

Naess’ personal application of the deep ecology approach results in a normative-derivational systematization akin to that of Spinoza or Aristotle. He refers to his own deep ecological total view as Ecosophy T. Naess’ systematization is normative because it calls for a particular set of prescriptive premises. It is derivational by virtue of being a logical systematization, where premise-conclusion chains generate successively more specific norms (general ecological and ecopolitical principles and concrete decisions), which ‘follow’ from previously accepted premises and hypotheses.

Naess’ survey of Ecosophy T is built on a single, highly aggregated and complex ultimate norm, ‘Selfrealization!’ He likens it to ‘the universal self,’ ‘the absolute,’ and the Sanskrit atman. Naess uses a capital ‘S’ to distinguish the norm from ego-realization and an exclamation mark to identify it as an imperative. ‘Self-realization!’ is an ecocentric-ontological analogue of the pareto rule in economics. The premise is that increased realization of any individual or species rests on advancing (or at least not hindering) the realization potential of all other species and individuals. While working from a single ultimate norm enables Naess to eliminate the potential for norm conflicts, it also embodies a certain artificiality that necessitates considerable word magic.

Naess draws his support for the Deep Ecology Platform from his Ecosophy T. Other supporters of the deep ecology movement, however, need not and should not draw directly on Naess’ systematization. The Platform can be ‘derived’ from a broad variety of sometimes dissonant, ultimate norms, including those inspired by all of the world’s major religions, ecofeminism, creation myths, Leopold’s ‘Land Ethic,’ and E. O. Wilson’s ‘Biophilia Hypothesis’ to name a few. Ultimate premises are essential building blocks for developing an ecosophy, but their breadth and abstractness insure that they only serve as general guides.

5. Methodological Issues And Future Directions

Deep ecology has been criticized for being too critical of pragmatic efforts to address acute environmental problems. It has also been accused of being inherently fascist, antihumanist, anti-feminist, and against having indigenous peoples share wilderness areas with nonhumans. None of these criticisms are borne out by a sophisticated reading of the literature on deep ecology as an ecophilosophical approach. This is not to suggest that deep ecology as an ecophilosophical approach has all the answers. In inspiring many people to reconsider their value priorities and more consistently relate them to their lifestyles and everyday actions, deep ecology opens more questions than it answers. Three key issues for future research loom on the horizon. First, how can society motivate value priorities for ecological sustainability when individuals do not accept the premise that nonhumans have intrinsic value? Second, does taking a deep ecological perspective over a shallow ecological perspective actually result in better policy—policies that result in long-term protection of cultural and biological diversity, using democratic, noncoercive, and ethically unobjectionable, means. Third, how can deep ecology help to resolve conflicts in environmental decision making when they result not from pseudorational thinking, but real value conflicts.

Bibliography:

• Cramer P F 1998 Deep Environmental Politics: The Role of Radical En ironmentalism in Crafting American Environmental Policy. Praeger, Westport, CT
• Devall B 1988 Simple in Means, Rich in Ends: Practicing Deep Ecology. Gibbs Smith, Salt Lake City, UT
• Devall B, Sessions G 1985 Deep Ecology: Living as if Nature Mattered. Gibbs Smith, Salt Lake City, UT
• Drengson A, Inoue Y (eds.) 1995 The Deep Ecology Movement: An Introductory Anthology. North Atlantic Books, Berkeley, CA
• Fox W 1990 Toward a Transpersonal Ecology: De eloping New Foundations for Environmentalism. Shambhala, Boston
• Glasser H 1996 Naess’ deep ecology approach and environmental policy. Inquiry 39: 157–87
• Glasser H 1997 On Warwick Fox’s assessment of deep ecology. Environmental Ethics 19: 69–85
• McLaughlin A 1993 Regarding Nature: Industrialism and Deep Ecology. State University of New York, Albany, NY
• Naess A 1973 The shallow and the deep, long-range ecology movement. A summary. Inquiry 16: 95–100
• Naess A 1986 The deep ecology movement: Some philosophical aspects. Philosophical Inquiry 8: 10–31
• Naess A 1989 Ecology, Community, and Lifestyle: Outline of an Ecosophy. [Translated and revised by Rothenberg D]. Cambridge University Press, Cambridge, UK
• Naess A 2002 Selected Papers: Deep Ecology, Vol. X of the selected works of Arne Naess
• Reed P, Rothenberg D (eds.) 1993 Wisdom and the Open Air: The Norwegian Roots of Deep Ecology. University of Minnesota Press, Minneapolis, MN
• Sessions G (ed.) 1995 Deep Ecology for the Twenty-First Century. Shambhala, Boston
• Witoszek N, Brennan A (eds.) 1999 Philosophical Dialogues: Arne Naess and the Progress of Ecophilosophy. Rowman and Littlefield, Lanham, MD
• Zimmerman M E, Callicott J B, Sessions G, Warren K J, Clark J (eds.) 2001 Environmental Philosophy: From Animal Rights to Radical Ecology, 3rd edn. Prentice-Hall, Englewood Cliffs, NJ

ORDER HIGH QUALITY CUSTOM PAPER

Help | Advanced Search

Computer Science > Computer Vision and Pattern Recognition

Title: zero-shot building age classification from facade image using gpt-4.

Abstract: A building's age of construction is crucial for supporting many geospatial applications. Much current research focuses on estimating building age from facade images using deep learning. However, building an accurate deep learning model requires a considerable amount of labelled training data, and the trained models often have geographical constraints. Recently, large pre-trained vision language models (VLMs) such as GPT-4 Vision, which demonstrate significant generalisation capabilities, have emerged as potential training-free tools for dealing with specific vision tasks, but their applicability and reliability for building information remain unexplored. In this study, a zero-shot building age classifier for facade images is developed using prompts that include logical instructions. Taking London as a test case, we introduce a new dataset, FI-London, comprising facade images and building age epochs. Although the training-free classifier achieved a modest accuracy of 39.69%, the mean absolute error of 0.85 decades indicates that the model can predict building age epochs successfully albeit with a small bias. The ensuing discussion reveals that the classifier struggles to predict the age of very old buildings and is challenged by fine-grained predictions within 2 decades. Overall, the classifier utilising GPT-4 Vision is capable of predicting the rough age epoch of a building from a single facade image without any training.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• Campus Crime Stats
• About the College
• Administration & Staff
• Message from the Dean
• Faculty & Staff Resources
• Fall 2023 Fact Sheet

Strategic Planning

• 2023 - 2030 Strategic Plan
• Student Services Resources
• Undergraduate Students
• Prospective Students
• Graduate Programs
• Apply for Scholarships
• Schedule an Advising Appointment

Pre-medical and Pre-dental

• Course Requirements
• Join the PHASE Moodle Group
• Review Committee

Other Opportunities

• Science Residential College
• Undergraduate Research
• Ogden Honors College
• GeauxTeach Math and Science
• Distinguished Communicator Program
• Biological Sciences
• Geology & Geophysics
• Mathematics
• Physics & Astronomy
• Museum of Natural Science
• Media Center
• Geaux Science Explorations Events
• Annual Report
• Pursuit for Kids
• Science Next Blog
• LSU Experimental Podcast

Office of Academic Innovation & Engagement

• Geaux Science Explorations
• STEM Story Time
• Kids' Day at the Museum
• Our Earth, Our Laboratory
• Passport to Explore
• Geology Summer Camp

For Students

• Peer Mentors
• Beckman Scholars
• Information About Giving to the College
• Ways to Give
• Priority Funds
• Contact a Development Team Member
• Dean's Circle
• Hall of Distinction
• Update Your Information

Join the Dean's Circle

Give To Other College of Science Priorities

Much More Than a Grain of Salt: LSU Research Group Manipulates Sodium to Understand Ecology and Evolution of Plant-Animal Interactions

April 12, 2024

When you think of sodium, you may think of table salt or the delicious Cajun cuisine found in southern Louisiana. From the gumbo, jambalaya and étouffée, there's no shortage of sodium in the Deep South. But outside of the kitchen, an LSU research group is exploring sodium's unusual role in food webs.

– Photo taken by Luis Y. Santiago-Rosario

Dr. Luis Santiago-Rosario (currently a National Science Foundation postdoctoral fellow at the University of Minnesota), current LSU LAGNiAppE post-doc Ana Salgado, current LSU PhD student Diego Paredes-Burneo and Professor Kyle Harms published in November 2023 results from some carefully controlled hydroponic experiments to better understand the interrelationships among animals, plants and the substrates from which the plants obtain nutrients.

During Santiago-Rosario's time as a PhD student at LSU, he manipulated sodium concentrations in sunflower host plants and found that butterfly larvae that consumed low-sodium plant tissue cannibalized their siblings to get the sodium they needed but otherwise failed to obtain from their diets.

"In terms of sodium, there is a mismatch between what plants and animals need," he said. "Our plants took up and incorporated into their tissues however much sodium was in the substrate, which meant that caterpillars experienced a limitation when reared on the low-sodium plants. They then overcame the limitation by eating their siblings!"

Santiago-Rosario defended his PhD two years ago under Harms. The published paper was a part of his dissertation as he took a deep dive into the ecology of sodium and how it affects animal and plant development and their interactions.

"With some hardcore searching and working to figure it out, he found just the right butterfly species, figured out how to collect it in the wild in Texas, how to rear it in the lab, and then how to very carefully manipulate the amount of sodium in the plants," said Harms.

The group found that sodium uptake can have important consequences for herbivores and their population demographics, an important message as researchers continue to study the ecology and evolution of plant-animal interactions.

"Getting a little bit of knowledge in a small controlled system like we've done, sets the stage for many next steps to learn about these interactions in other species and in the field, especially on our human-dominated and changing planet," said Santiago-Rosario.

Harms added that his former student made a lasting legacy in the Biological Sciences Department.

"Luis in many ways was very much a model PhD student while he was here," said Harms. "I mean he did all sorts of things way beyond the outstanding research for his PhD. For just one example, he had a lasting influence as a two-term BioGrads President who seriously emphasized diversity, equity, and inclusion in our department."

Cali Stuckey

Marketing/Communications Specialist Department of Biological Sciences 225-578-1889

POPULAR SEARCHES:

Video Modal

This paper is in the following e-collection/theme issue:

Published on 16.4.2024 in Vol 26 (2024)

Adverse Event Signal Detection Using Patients’ Concerns in Pharmaceutical Care Records: Evaluation of Deep Learning Models

Authors of this article:

Original Paper

• Satoshi Nishioka 1 , PhD   ; 
• Satoshi Watabe 1 , BSc   ; 
• Yuki Yanagisawa 1 , PhD   ; 
• Kyoko Sayama 1 , MSc   ; 
• Hayato Kizaki 1 , MSc   ; 
• Shungo Imai 1 , PhD   ; 
• Mitsuhiro Someya 2 , BSc   ; 
• Ryoo Taniguchi 2 , PhD   ; 
• Shuntaro Yada 3 , PhD   ; 
• Eiji Aramaki 3 , PhD   ; 
• Satoko Hori 1 , PhD  

1 Division of Drug Informatics, Keio University Faculty of Pharmacy, Tokyo, Japan

2 Nakajima Pharmacy, Hokkaido, Japan

3 Nara Institute of Science and Technology, Nara, Japan

Corresponding Author:

Satoko Hori, PhD

Division of Drug Informatics

Keio University Faculty of Pharmacy

1-5-30 Shibakoen

Tokyo, 105-8512

Phone: 81 3 5400 2650

Email: [email protected]

Background: Early detection of adverse events and their management are crucial to improving anticancer treatment outcomes, and listening to patients’ subjective opinions (patients’ voices) can make a major contribution to improving safety management. Recent progress in deep learning technologies has enabled various new approaches for the evaluation of safety-related events based on patient-generated text data, but few studies have focused on the improvement of real-time safety monitoring for individual patients. In addition, no study has yet been performed to validate deep learning models for screening patients’ narratives for clinically important adverse event signals that require medical intervention. In our previous work, novel deep learning models have been developed to detect adverse event signals for hand-foot syndrome or adverse events limiting patients’ daily lives from the authored narratives of patients with cancer, aiming ultimately to use them as safety monitoring support tools for individual patients.

Objective: This study was designed to evaluate whether our deep learning models can screen clinically important adverse event signals that require intervention by health care professionals. The applicability of our deep learning models to data on patients’ concerns at pharmacies was also assessed.

Methods: Pharmaceutical care records at community pharmacies were used for the evaluation of our deep learning models. The records followed the SOAP format, consisting of subjective (S), objective (O), assessment (A), and plan (P) columns. Because of the unique combination of patients’ concerns in the S column and the professional records of the pharmacists, this was considered a suitable data for the present purpose. Our deep learning models were applied to the S records of patients with cancer, and the extracted adverse event signals were assessed in relation to medical actions and prescribed drugs.

Results: From 30,784 S records of 2479 patients with at least 1 prescription of anticancer drugs, our deep learning models extracted true adverse event signals with more than 80% accuracy for both hand-foot syndrome (n=152, 91%) and adverse events limiting patients’ daily lives (n=157, 80.1%). The deep learning models were also able to screen adverse event signals that require medical intervention by health care providers. The extracted adverse event signals could reflect the side effects of anticancer drugs used by the patients based on analysis of prescribed anticancer drugs. “Pain or numbness” (n=57, 36.3%), “fever” (n=46, 29.3%), and “nausea” (n=40, 25.5%) were common symptoms out of the true adverse event signals identified by the model for adverse events limiting patients’ daily lives.

Conclusions: Our deep learning models were able to screen clinically important adverse event signals that require intervention for symptoms. It was also confirmed that these deep learning models could be applied to patients’ subjective information recorded in pharmaceutical care records accumulated during pharmacists’ daily work.

Introduction

Increasing numbers of people are expected to develop cancers in our aging society [ 1 - 3 ]. Thus, there is increasing interest in how to detect and manage the side effects of anticancer therapies in order to improve treatment regimens and patients’ quality of life [ 4 - 8 ]. The primary approaches for side effect management are “early signal detection and early intervention” [ 9 - 11 ]. Thus, more efficient approaches for this purpose are needed.

It has been recognized that patients’ voices concerning adverse events represent an important source of information. Several studies have indicated that the number, severity, and time of occurrence of adverse events might be underevaluated by physicians [ 12 - 15 ]. Thus, patient-reported outcomes (PROs) have recently received more attention in the drug evaluation process, reflecting patients’ real voices. Various kinds of PRO measures have been developed and investigated in different disease populations [ 16 , 17 ]. Health care authorities have also encouraged the pharmaceutical industry to use PROs for drug evaluation [ 18 , 19 ], and it is becoming more common to take PRO assessment results into consideration for drug marketing approval [ 20 , 21 ]. Similar trends can be seen in the clinical management of individual patients. Thus, health care professionals have an interest in understanding how to appropriately gather patients’ concerns in order to improve safety management and clinical decisions [ 22 - 24 ].

The applications of deep learning for natural language processing have expanded dramatically in recent years [ 25 ]. Since the development of a high-performance deep learning model in 2018 [ 26 ], attempts to apply cutting-edge deep learning models to various kinds of patient-generated text data for the evaluation of safety events or the analysis of unscalable subjective information from patients have been accelerating [ 27 - 31 ]. Most studies have been conducted to use patients’ narrative data for pharmacovigilance [ 27 , 32 - 35 ], while few have been aimed at improvement of real-time safety monitoring for individual patients. In addition, there have been some studies on adverse event severity grading based on health care records [ 36 - 39 ], but none has yet aimed to extract clinically important adverse event signals that require medical intervention from patients’ narratives. It is important to know whether deep learning models could contribute to the detection of such important adverse event signals from concern texts generated by individual patients.

To address this question, we have developed deep learning models to detect adverse event signals from individual patients with cancer based on patients’ blog articles in online communities, following other types of natural language processing–related previous work [ 40 , 41 ]. One deep learning model focused on the specific symptom of hand-foot syndrome (HFS), which is one of the typical side effects of anticancer treatments [ 42 ], and another focused on a broad range of adverse events that impact patients’ activities of daily living [ 43 ]. We showed that our models can provide good performance scores in targeting adverse event signals. However, the evaluation relied on patients’ narratives from the patients’ blog data used for deep learning model training, so further evaluation is needed to ensure the validity and applicability of the models to other texts regarding patients’ concerns. In addition, the blog data source did not contain medical information, so it was not feasible to assess whether the models could contribute to the extraction of clinically important adverse event signals.

To address these challenges, we focused on pharmaceutical care records written by pharmacists at community pharmacies. The gold standard format for pharmaceutical care records in Japan is the SOAP (subjective, objective, assessment, plan)-based document that follows the “problem-oriented system” concept proposed by Weed [ 44 ] in 1968. Pharmacists track patients’ subjective concerns in the S column, provide objective information or observations in the O column, give their assessment from the pharmacist perspective in the A column, and suggest a plan for moving forward in the P column [ 45 , 46 ]. We considered that SOAP-based pharmaceutical care records could be a unique data source suitable for further evaluation of our deep learning models because they contain both patients’ concerns and professional health care records by pharmacists, including the medication prescription history with time stamps. Therefore, this study was designed to assess whether our deep learning models could extract clinically important adverse event signals that require intervention by medical professionals from these records. We also aimed to evaluate the characteristics of the models when applied to patients’ subjective information noted in the pharmaceutical care records, as there have been only a few studies on the application of deep learning models to patients’ concerns recorded during pharmacists’ daily work [ 47 - 49 ].

Here, we report the results of applying our deep learning models to patients’ concern text data in pharmaceutical care records, focusing on patients receiving anticancer treatment.

Data Source

The original data source was 2,276,494 pharmaceutical care records for 303,179 patients, created from April 2020 to December 2021 at community pharmacies belonging to the Nakajima Pharmacy Group in Japan [ 50 ]. To focus on patients with cancer, records of patients with at least 1 prescription for an anticancer drug were retrieved by sorting individual drug codes (YJ codes) used in Japan (YJ codes starting with 42 refer to anticancer drugs). Records in the S column (ie, S records) were collected from the patients with cancer as the text data of patients’ concerns for deep learning model analysis.

Deep Learning Models

The deep learning models used for this research were those that we constructed based on patients’ narratives in blog articles posted in an online community and that showed the best performance score in each task in our previous work (ie, a Bidirectional Encoder Representations From Transformers [BERT]–based model for HFS signal extraction [ 42 ] and a T5-based model for adverse event signal extraction [ 43 ]). BERT [ 26 ] and T5 [ 51 ] both belong to a type of deep learning model that has recently shown high performance in several studies [ 29 , 52 ]. Hereafter, we refer to the deep learning model for HFS signals as the HFS model, the model for any adverse event signals as All AE (ie, all or any adverse events) model, and the model for adverse event signals limited to patients’ activities of daily living as the AE-L (adverse events limiting patients’ daily lives) model. It was also confirmed that these deep learning models showed similar or higher performance scores for the HFS, All AE, or AE-L identification tasks using 1000 S records randomly extracted from the data source of this study compared to the values obtained in our previous work [ 42 , 43 ] (the performance scores of sentence-level tasks from our previous work are comparable, as the mean number of words in the sentences in the data source in our previous work was 32.7 [SD 33.9], which is close to that of the S records used in this study, 38.8 [SD 29.4]). The method and results of the performance-level check are described in detail in Multimedia Appendix 1 [ 42 , 43 ]. We applied the deep learning models to all text data in this study without any adjustment in setting parameters from those used in constructing them based on patient-authored texts in our previous work [ 42 , 43 ].

Evaluation of Extracted S Records by the Deep Learning Models

In this study, we focused on the evaluation of S records that our deep learning models extracted as HFS or AE-L positive. Each positive S record was assessed as if it was a true adverse event signal, a sort of adverse event symptom, whether or not an intervention was made by health care professionals. We also investigated the kind of anticancer treatment prescription in connection with each adverse event signal identified in S records.

To assess whether an extracted positive S record was a true adverse event signal, we used the same annotation guidelines as in our previous work [ 43 ]. In brief, each S record was treated as an “adverse event signal” if any untoward medical occurrence happened to the patient, regardless of the cause. For the AE-L model only, if a positive S record was confirmed as an adverse event signal, it was further categorized into 1 or more of the following adverse event symptoms: “fatigue,” “nausea,” “vomiting,” “diarrhea,” “constipation,” “appetite loss,” “pain or numbness,” “rash or itchy,” “hair loss,” “menstrual irregularity,” “fever,” “taste disorder,” “dizziness,” “sleep disorder,” “edema,” or “others.”

For the assessment of interventions by health care professionals and anticancer treatment prescriptions, information from the O, A, and P columns and drug prescription history in the data source were investigated for the extracted positive S records. The interventions by health care professionals were categorized in any of the following: “adding symptomatic treatment for the adverse event signal,” “dose reduction or discontinuation of causative anticancer treatment,” “consultation with physician,” “others,” or “no intervention (ie, just following up the adverse event signal).” The actions categorized in “others” were further evaluated individually. For this assessment, we also randomly extracted 200 S records and evaluated them in the same way for comparison with the results from the deep learning model. Prescription history of anticancer treatment was analyzed by primary category of mechanism of action (MoA) with subcategories if applicable (eg, target molecule for kinase inhibitors).

Applicability Check to Other Text Data Including Patients’ Concerns

To check the applicability of our deep learning models to data from a different source, interview transcripts from patients with cancer were also evaluated. The interview transcripts were created by the Database of Individual Patient Experiences-Japan (DIPEx-Japan) [ 53 ]. DIPEx-Japan divides the interview transcripts into sections for each topic, such as “onset of disease” and “treatment,” and posts the processed texts on its website. Processing is conducted by accredited researchers based on qualitative research methods established by the University of Oxford [ 54 ]. In this study, interview text data created from interviews with 52 patients with breast cancer conducted from January 2008 to October 2018 were used to assess whether our deep learning models can extract adverse event signals from this source. In total, 508 interview transcripts were included with the approval of DIPEx-Japan.

Ethical Considerations

This study was conducted with anonymized data following approval by the ethics committee of the Keio University Faculty of Pharmacy (210914-1 and 230217-1) and in accordance with relevant guidelines and regulations and the Declaration of Helsinki. Informed consent specific to this study was waived due to the retrospective observational design of the study with the approval of the ethics committee of the Keio University Faculty of Pharmacy. To respect the will of each individual stakeholder, however, we provided patients and pharmacists of the pharmacy group with an opportunity to refuse the sharing of their pharmaceutical care records by posting an overview of this study at each pharmacy store or on their web page regarding the analysis using pharmaceutical care records. Interview transcripts from DIPEx-Japan were provided through a data sharing arrangement for using narrative data for research and education. Consent for interview transcription and its sharing from DIPEx-Japan was obtained from the participants when the interviews were recorded.

From the original data source of 2,180,902 pharmaceutical care records for 291,150 patients, S records written by pharmacists for patients with a history of at least 1 prescription of an anticancer drug were extracted. This yielded 30,784 S records for 2479 patients with cancer ( Table 1 ). The mean and median number of words in the S records were 38.8 (SD 29.4) and 32 (IQR 20-50), respectively. We applied our deep learning models, HFS, All AE, and AE-L, to these 30,784 S records for the evaluation of the deep learning models for adverse event signal detection.

For interview transcripts created by DIPEx-Japan, the mean and median number of words were 428.9 (SD 160.9) and 416 (IQR 308-526), respectively, in the 508 transcripts for 52 patients with breast cancer.

a SOAP: subjective, objective, assessment, plan.

b S: subjective.

Application of the HFS Model

First, we applied the HFS model to the S records for patients with cancer. The BERT-based model was used for this research as it showed the best performance score in our previous work [ 42 ].

S Records Extracted as HFS Positive

The S records extracted as HFS positive by the HFS model ( Table 2 ) amounted to 167 (0.5%) records for 119 (4.8%) patients. A majority of the patients had 1 HFS-positive record in their S records (n=91, 76.5%), while 2 patients had as many as 6 (1.7%) HFS-positive records. When we examined whether the extracted S records were true adverse event signals or not, 152 records were confirmed to be adverse event signals, while the other 15 records were false-positives. All the false-positive S records were descriptions about the absence of symptoms or confirmation of improving condition (eg, “no diarrhea, mouth ulcers, or limb pain so far” or “the skin on the soles of my feet has calmed down a lot with this ointment”). Some examples of S records that were predicted as HFS positive by the model are shown in Table S1 in Multimedia Appendix 2 .

The same examination was conducted with interview transcripts from DIPEx-Japan. Only 1 (0.2%) transcript was extracted as HFS positive by the HFS model, and it was a true adverse event signal (100%). The actual transcript extracted as HFS positive is shown in Table S2 in Multimedia Appendix 2 .

a S: subjective.

b HFS: hand-foot syndrome.

c All false-positive S records were denial of symptoms or confirmation of improving condition.

Interventions by Health Care Professionals

The 167 S records extracted as HFS positive as well as 200 randomly selected records were checked for interventions by health care professionals ( Figure 1 ). The proportion showing any action by health care professionals was 64.1% for 167 HFS-positive S records compared to 13% for the 200 random S records. Among the actions taken for HFS positives, “adding symptomatic treatment” was the most common, accounting for around half (n=79, 47.3%), followed by “other” (n=18, 10.8%). Most “other” actions were educational guidance from pharmacists, such as instructions on moisturizing, nail care, or application of ointment and advice on daily living (eg, “avoid tight socks”).

Anticancer Drugs Prescribed

The types of anticancer drugs prescribed for HFS-positive patients are summarized based on the prescription histories in Table 3 . For the 152 adverse event signals identified by the HFS model in the previous section, the most common MoA class of anticancer drugs used for the patients was antimetabolite (n=62, 40.8%), specifically fluoropyrimidines (n=59, 38.8%). Kinase inhibitors were next (n=49, 32.2%), with epidermal growth factor receptor (EGFR) inhibitors and multikinase inhibitors as major subgroups (n=28, 18.4% and n=14, 9.2%, respectively). The third and fourth most common MoAs were aromatase inhibitors (n=24, 15.8%) and antiandrogen or estrogen drugs (n=7, 4.6% each) for hormone therapy.

a EGFR: epidermal growth factor receptor.

b VEGF: vascular endothelial growth factor.

c HER2: human epidermal growth factor receptor-2.

d CDK4/6: cyclin-dependent kinase 4/6.

Application of the All AE or AE-L model

The All AE and AE-L models were also applied to the same S records for patients with cancer. The T5-based model was used for this research as it gave the best performance score in our previous work [ 43 ].

S Records Extracted as All AE or AE-L positive

The numbers of S records extracted as positive were 7604 (24.7%) for 1797 patients and 196 (0.6%) for 142 patients for All AE and AE-L, respectively. In the case of All AE, patients tended to have multiple adverse event positives in their S records (n=1315, 73.2% of patients had at least 2 positives). In the case of AE-L, most patients had only 1 AE-L positive (n=104, 73.2%), and the largest number of AE-L positives for 1 patient was 4 (2.8%; Table 4 ).

We focused on AE-L evaluation due to its greater importance from a medical viewpoint and lower workload for manual assessment, considering the number of positive S records. Of the 197 AE-L–positive S records, it was confirmed that 157 (80.1%) records accurately extracted adverse event signals, while 39 (19.9%) records were false-positives that did not include any adverse event signals ( Table 4 ). The contents of the 39 false-positives were all descriptions about the absence of symptoms or confirmation of improving condition, showing a similar tendency to the HFS false-positives (eg, “The diarrhea has calmed down so far. Symptoms in hands and feet are currently fine” and “No symptoms for the following: upset in stomach, diarrhea, nausea, abdominal pain, abdominal pain or stomach cramps, constipation”). Examples of S records that were predicted as AE-L positive are shown in Table S3 in Multimedia Appendix 2 .

The deep learning models were also applied to interview transcripts from DIPEx-Japan in the same manner. The deep learning models identified 84 (16.5%) and 18 (3.5%) transcripts as All AE or AE-L positive, respectively. Of the 84 All AE–positive transcripts, 73 (86.9%) were true adverse event signals. The false-positives of All AE (n=11, 13.1%) were categorized into any of the following 3 types: explanations about the disease or its prognosis, stories when their cancer was discovered, or emotional changes that did not include clear adverse event mentions. With regard to AE-L, all the 18 (100%) positives were true adverse event signals (Table S4 in Multimedia Appendix 2 ). Examples of actual transcripts extracted as All AE or AE-L positive are shown in Table S5 in Multimedia Appendix 2 .

b All AE: all (or any of) adverse event.

c AE-L: adverse events limiting patients’ daily lives.

d All false-positive S records were denial of symptoms or confirmation of improving condition.

Whether or not interventions were made by health care professionals was investigated for the 196 AE-L–positive S records. As in the HFS model evaluation, data from 200 randomly selected S records were used for comparison ( Figure 2 ). In total, 91 (46.4%) records in the 196 AE-L–positive records were accompanied by an intervention, while the corresponding figure in the 200 random records was 26 (13%) records. The most common action in response to adverse event signals identified by the AE-L model was “adding symptomatic treatment” (n=71, 36.2%), followed by “other” (n=11, 5.6%). “Other” included educational guidance from pharmacists, inquiries from pharmacists to physicians, or recommendations for patients to visit a doctor.

The types of anticancer drugs prescribed for patients with adverse event signals identified by the AE-L model were summarized based on the prescription histories ( Table 5 ). In connection with the 157 adverse event signals, the most common MoA of the prescribed anticancer drug was antimetabolite (n=62, 39.5%) and fluoropyrimidine (n=53, 33.8%), which accounted for the majority. Kinase inhibitor (n=31, 19.7%) was the next largest category with multikinase inhibitor (n=14, 8.9%) as the major subgroup. These were followed by antiandrogen (n=27, 17.2%), antiestrogen (n=10, 6.4%), and aromatase inhibitor (n=10, 6.4%) for hormone therapy.

b JAK: janus kinase.

c VEGF: vascular endothelial growth factor.

d BTK: bruton tyrosine kinase.

e FLT3: FMS-like tyrosine kinase-3.

f PARP: poly-ADP ribose polymerase.

g CDK4/6: cyclin-dependent kinase 4/6.

h CD20: cluster of differentiation 20.

Adverse Event Symptoms

For the 157 adverse event signals identified by the AE-L model, the symptoms were categorized according to the predefined guideline in our previous work [ 43 ]. “Pain or numbness” (n=57, 36.3%) accounted for the largest proportion followed by “fever” (n=46, 29.3%) and “nausea” (n=40, 25.5%; Table 6 ). Symptoms classified as “others” included chills, tinnitus, running tears, dry or peeling skin, and frequent urination. When comparing the proportion of the symptoms associated with or without interventions by health care professionals, a trend toward a greater proportion of interventions was observed in “fever,” “nausea,” “diarrhea,” “constipation,” “vomiting,” and “edema” ( Figure 3 , black boxes). On the other hand, a smaller proportion was observed in “pain or numbness,” “fatigue,” “appetite loss,” “rash or itchy,” “taste disorder,” and “dizziness” ( Figure 3 , gray boxes).

This study was designed to evaluate our deep learning models, previously constructed based on patient-authored texts posted in an online community, by applying them to pharmaceutical care records that contain both patients’ subjective concerns and medical information created by pharmacists. Based on the results, we discuss whether these deep learning models can extract clinically important adverse event signals that require medical intervention, and what characteristics they show when applied to data on patients’ concerns in pharmaceutical care records.

Performance for Adverse Event Signal Extraction

The first requirement for the deep learning models is to extract adverse event signals from patients’ narratives precisely. In this study, we evaluated the proportion of true adverse event signals in positive S records extracted by the HFS or AE-L model. True adverse event signals amounted to 152 (91%) and 157 (80.1%) for the HFS and AE-L models, respectively ( Tables 2 and 4 ). Given that the proportion of true adverse event signals in 200 randomly extracted S records without deep learning models was 54 (27%; categories other than “no adverse event” in Figures 1 and 2 ), the HFS and AE-L models were able to concentrate S records with adverse event mentions. Although 15 (9%) for the HFS model and 39 (19.9%) for the AE-L model were false-positives, it was confirmed all of the false-positive records described a lack of symptoms or confirmation of improving condition. We considered that such false-positives are due to the unique feature of pharmaceutical care records, where pharmacists might proactively interview patients about potential side effects of their medications. As the data set of blog articles we used to construct the deep learning models included few such cases (especially comments on lack of symptoms), our models seemed unable to exclude them correctly. Even though we confirmed that the proportion of true “adverse event” signals extracted from the S records by the HFS or AE-L model was more than 80%, the performance scores to extract true “HFS” or “AE-L” signals were not so high based on the performance check using 1000 randomly extracted S records ( F 1 -scores were 0.50 and 0.22 for true HFS and AE-L signals, respectively; Table S1 in Multimedia Appendix 1 ). It is considered that the performance to extract true HFS and AE-L signals was relatively low due to the short length of texts in the S records, providing less context to judge the impact on patients’ daily lives, especially for the AE-L model (the mean word number of the S records was 38.8 [SD 29.4; Table 1 ], similar to the sentence-level tasks in our previous work [ 42 , 43 ]). However, we consider a true adverse event signal proportion of more than 80% in this study represents a promising outcome, as this is the first attempt to apply our deep learning models to a different source of patients’ concern data, and the extracted positive cases would be worthy of evaluation by a medical professional, as the potential adverse events could be caused by drugs taken by the patients.

When the deep learning models were applied to DIPEx-Japan interview transcripts, including patients’ concerns, the proportion of true adverse event signals was also more than 80% (for All AE: n=73, 86.9% and for HFS and AE-L: n=18, 100%). The difference in the results between pharmaceutical care S records and DIPEx-Japan interview transcripts was the features of false-positives, descriptions about lack of symptoms or confirmation of improving condition in S records versus explanations about disease or its prognosis, stories about when their cancer was discovered, or emotional changes in interview transcripts. This is considered due to the difference in the nature of the data source; the pharmaceutical care records were generated in a real-time manner by pharmacists through their daily work, where adverse event signals are proactively monitored, while the interview transcripts were purely based on patients’ retrospective memories. Our deep learning models were able to extract true adverse event signals with an accuracy of more than 80% from both text data sources in spite of the difference in their nature. When looking at future implementation of the deep learning models in society (discussed in the Potential for Deep Learning Model Implementation in Society section), it may be desirable to further adjust deep learning models to reduce false-positives depending upon the features of the data source.

Identification of Important Adverse Events Requiring Medical Intervention

To assess whether the models could extract clinically important adverse event signals, we investigated interventions by health care professionals connected with the adverse event signals that are identified by our deep learning models. In the 200 randomly extracted S records, only 26 (13%) consisted of adverse event signals, leading to any intervention by health care professionals. On the other hand, the proportion of signals associated with interventions was increased to 107 (64.1%) and 91 (46.4%) in the S records extracted as positive by the HFS and AE-L models, respectively ( Figures 1 and 2 ). These results suggest that both deep learning models can screen clinically important adverse event signals that require intervention from health care professionals. The performance level in screening adverse event signals requiring medical intervention was higher in the HFS model than in the AE-L model (n=107, 64.1% vs n=91, 46.4%; Figures 1 and 2 ). Since the target events were specific and adverse event signals of HFS were narrowly defined, which is one of the typical side effects of some anticancer drugs, we consider that health care providers paid special attention to HFS-related signals and took action proactively. In both deep learning models, similar trends were observed in actions taken by health care professionals in response to extracted adverse event signals; common actions were attempts to manage adverse event symptoms by symptomatic treatment or other mild interventions, including educational guidance from pharmacists or recommendations for patients to visit a doctor. More direct interventions focused on the causative drugs (ie, “dose reduction or discontinuation of anticancer treatment”) amounted to less than 5%; 7 (4.2%) for the HFS model and 6 (3.1%) for the AE-L model ( Figures 1 and 2 ). Thus, it appears that our deep learning models can contribute to screening mild to moderate adverse event signals that require preventive actions such as symptomatic treatments or professional advice from health care providers, especially for patients with less sensitivity to adverse event signals or who have few opportunities to visit clinics and pharmacies.

Ability to Catch Real Side Effect Signals of Anticancer Drugs

Based on the drug prescription history associated with S records extracted as HFS or AE-L positive, the type and duration of anticancer drugs taken by patients experiencing the adverse event signals were investigated. For the HFS model, the most common MoA of anticancer drug was antimetabolite (fluoropyrimidine: n=59, 38.8%), followed by kinase inhibitors (n=49, 32.2%, of which EGFR inhibitors and multikinase inhibitors accounted for n=28, 18.4% and n=14, 9.2%, respectively) and aromatase inhibitors (n=24, 15.8%; Table 3 ). It is known that fluoropyrimidine and multikinase inhibitors are typical HFS-inducing drugs [ 55 - 58 ], suggesting that the HFS model accurately extracted HFS side effect signals derived from these drugs. Note that symptoms such as acneiform rash, xerosis, eczema, paronychia, changes in the nails, arthralgia, or stiffness of limb joints, which are common side effects of EGFR inhibitors or aromatase inhibitors [ 59 , 60 ], might be extracted as closely related expressions to those of HFS signals. When looking at the MoA of anticancer drugs for patients with adverse event signals identified by the AE-L model, antimetabolite (fluoropyrimidine) was the most common one (n=53, 33.8%), as in the case of those identified by the HFS model, followed by kinase inhibitors (n=31, 19.7%) and antiandrogens (n=27, 17.2%; Table 5 ). Since the AE-L model targets a broad range of adverse event symptoms, it is difficult to rationalize the relationship between the adverse event signals and types of anticancer drugs. However, the type of anticancer drugs would presumably closely correspond to the standard treatments of the cancer types of the patients. Based on the prescribed anticancer drugs, we can infer that a large percentage of the patients had breast or lung cancer, indicating that our study results were based on data from such a population. Thus, a possible direction for the expansion of this research would be adjusting the deep learning models by additional training with expressions for typical side effects associated with standard treatments of other cancer types. To interpret these results correctly, it should be noted that we could not investigate anticancer treatments conducted outside of the pharmacies (eg, the time-course relationship with intravenously administered drugs would be missed, as the administration will be done at hospitals). To further evaluate how useful this model is in side effect signal monitoring for patients with cancer, comprehensive medical information for the eligible patients would be required.

Suitability of the Deep Learning Models for Specific Adverse Event Symptoms

Among the adverse event signals identified by the AE-L model, the type of symptom was categorized according to a predefined annotation guideline that we previously developed [ 43 ]. The most frequently recorded adverse event signals identified by the AE-L model were “pain or numbness” (n=57, 36.3%), “fever” (n=46, 29.3%), and “nausea” (n=40, 25.5%; Table 6 ). Since the pharmaceutical care records had information about interventions by health care professionals, the frequency of the presence or absence of the interventions for each symptom was examined. A trend toward a greater proportion of interventions was observed in “fever,” “nausea,” “diarrhea,” “constipation,” “vomiting,” and “edema” ( Figure 3 , black boxes). There seem to be 2 possible explanations for this: these symptoms are of high importance and require early medical intervention or effective symptomatic treatments are available for these symptoms in clinical practice so that medical intervention is an easy option. On the other hand, a trend for a smaller proportion of adverse event signals to result in interventions was observed for “pain or numbness,” “fatigue,” “appetite loss,” “rash or itchy,” “taste disorder,” and “dizziness” ( Figure 3 , gray boxes). The reason for this may be the lack of effective symptomatic treatments or the difficulty of judging whether the severity of these symptoms justifies medical intervention by health care providers. In either case, there may be room for improvement in the quality of medical care for these symptoms. We expect that our research will contribute to a quality improvement in safety monitoring in clinical practice by supporting adverse event signal detection in a cost-effective manner.

Potential for Deep Learning Model Implementation in Society

Although we evaluated our deep learning models using pharmaceutical care records in this study, the main target of future implementation of our deep learning models in society would be narrative texts that patients directly write to record their daily experiences. For example, the application of these deep learning models to electronic media where patients record their daily experiences in their lives with disease (eg, health care–related e-communities and disease diary applications) could enable information about adverse event signal onset that patients experience to be provided to health care providers in a timely manner. Adverse event signals can automatically be identified and shared with health care providers based on the concern texts that patients post to any platform. This system will have the advantage that health care providers can efficiently grasp safety-related events that patients experience outside of clinic visits so that they can conduct more focused or personalized interactions with patients at their clinic visits. However, consideration should be given to avoid an excessive burden on health care providers. For instance, limiting the sharing of adverse event signals to those of high severity or summarizing adverse event signals over a week rather than sharing each one in a real-time manner may be reasonable approaches for medical staff. We also need to think about how to encourage patients to record their daily experiences using electronic tools. Not only technical progress and support but also the establishment of an ecosystem where both patients and medical staff can feel benefit will be required. Prospective studies with deep learning models to follow up patients in the long term and evaluate outcomes will be needed. We primarily looked at patient-authored texts as targets of implementation, but our deep learning models may also be worth using medical data including patients’ subjective concerns, such as pharmaceutical care S records. As this study confirmed that our deep learning models are applicable to patients’ concern texts tracked by pharmacists, it should be possible to use them to analyze other “patient voice-like” medical text data that have not been actively investigated so far.

Limitations

First, the major limitation of this study was that we were not able to collect complete medical information of the patients. Although we designed this study to analyze patients’ concerns extracted by the deep learning models and their relationship with medical information contained in the pharmaceutical care records, some information could not be tracked (eg, missing history of medical interventions or anticancer treatment at hospitals as well as diagnosis of patients’ primary cancers). Second, there might be a data creation bias in S records for patients’ concerns by pharmacists. For example, symptoms that have little impact on intervention decisions might less likely be recorded by them. It should be also noted that the characteristics of S records may not be consistent at different community pharmacies.

Conclusions

Our deep learning models were able to screen clinically important adverse event signals that require intervention by health care professionals from patients’ concerns in pharmaceutical care records. Thus, these models have the potential to support real-time adverse event monitoring of individual patients taking anticancer treatments in an efficient manner. We also confirmed that these deep learning models constructed based on patient-authored texts could be applied to patients’ subjective information recorded by pharmacists through their daily work. Further research may help to expand the applicability of the deep learning models for implementation in society or for analysis of data on patients’ concerns accumulated in professional records at pharmacies or hospitals.

Acknowledgments

This work was supported by Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research (KAKENHI; grant 21H03170) and Japan Science and Technology Agency, Core Research for Evolutional Science and Technology (CREST; grant JPMJCR22N1), Japan. Mr Yuki Yokokawa and Ms Sakura Yokoyama at our laboratory advised SN about the structure of pharmaceutical care records. This study would not have been feasible without the high quality of pharmaceutical care records created by many individual pharmacists at Nakajima Pharmacy Group through their daily work.

Data Availability

The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.

Authors' Contributions

SN and SH designed the study. SN retrieved the subjective records of patients with cancer from the data source for the application of deep learning models and organized other data for subsequent evaluations. SN ran the deep learning models with the support of SW. SN, YY, and KS checked the adverse event signals for each subjective record that was extracted as positive by the models for hand-foot syndrome or adverse events limiting patients’ daily lives and evaluated the adverse event signal symptoms, details of interventions taken by health care professionals, and types of anticancer drugs prescribed for patients based on available data from the data source. HK and SI advised on the study concept and process. MS and RT provided pharmaceutical records at their community pharmacies along with advice on how to use and interpret them. SY and EA supervised the natural language processing research as specialists. SH supervised the study overall. SN drafted and finalized the paper. All authors reviewed and approved the paper.

Conflicts of Interest

SN is an employee of Daiichi Sankyo Co, Ltd. All other authors declare no conflicts of interest.

Performance evaluation of deep learning models.

Examples of S records and sample interview transcripts.

• Global cancer observatory: cancer over time. World Health Organization. URL: https://gco.iarc.fr/overtime/en [accessed 2023-07-02]
• Mattiuzzi C, Lippi G. Current cancer epidemiology. J Epidemiol Glob Health. 2019;9(4):217-222. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Montazeri F, Komaki H, Mohebi F, Mohajer B, Mansournia MA, Shahraz S, et al. Editorial: disparities in cancer prevention and epidemiology. Front Oncol. 2022;12:872051. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lasala R, Santoleri F. Association between adherence to oral therapies in cancer patients and clinical outcome: a systematic review of the literature. Br J Clin Pharmacol. 2022;88(5):1999-2018. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pudkasam S, Polman R, Pitcher M, Fisher M, Chinlumprasert N, Stojanovska L, et al. Physical activity and breast cancer survivors: importance of adherence, motivational interviewing and psychological health. Maturitas. 2018;116:66-72. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Markman M. Chemotherapy-associated neurotoxicity: an important side effect-impacting on quality, rather than quantity, of life. J Cancer Res Clin Oncol. 1996;122(9):511-512. [ CrossRef ] [ Medline ]
• Jitender S, Mahajan R, Rathore V, Choudhary R. Quality of life of cancer patients. J Exp Ther Oncol. 2018;12(3):217-221. [ Medline ]
• Di Nardo P, Lisanti C, Garutti M, Buriolla S, Alberti M, Mazzeo R, et al. Chemotherapy in patients with early breast cancer: clinical overview and management of long-term side effects. Expert Opin Drug Saf. 2022;21(11):1341-1355. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cuomo RE. Improving cancer patient outcomes and cost-effectiveness: a Markov simulation of improved early detection, side effect management, and palliative care. Cancer Invest. 2023;41(10):858-862. [ CrossRef ] [ Medline ]
• Pulito C, Cristaudo A, La Porta C, Zapperi S, Blandino G, Morrone A, et al. Oral mucositis: the hidden side of cancer therapy. J Exp Clin Cancer Res. 2020;39(1):210. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bartal A, Mátrai Z, Szûcs A, Liszkay G. Main treatment and preventive measures for hand-foot syndrome, a dermatologic side effect of cancer therapy. Magy Onkol. 2011;55(2):91-98. [ FREE Full text ] [ Medline ]
• Basch E, Jia X, Heller G, Barz A, Sit L, Fruscione M, et al. Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. J Natl Cancer Inst. 2009;101(23):1624-1632. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Basch E. The missing voice of patients in drug-safety reporting. N Engl J Med. 2010;362(10):865-869. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fromme EK, Eilers KM, Mori M, Hsieh YC, Beer TM. How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. J Clin Oncol. 2004;22(17):3485-3490. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu L, Suo T, Shen Y, Geng C, Song Z, Liu F, et al. Clinicians versus patients subjective adverse events assessment: based on patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Qual Life Res. 2020;29(11):3009-3015. [ CrossRef ] [ Medline ]
• Churruca K, Pomare C, Ellis LA, Long JC, Henderson SB, Murphy LED, et al. Patient-reported outcome measures (PROMs): a review of generic and condition-specific measures and a discussion of trends and issues. Health Expect. 2021;24(4):1015-1024. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Pérez-Alfonso KE, Sánchez-Martínez V. Electronic patient-reported outcome measures evaluating cancer symptoms: a systematic review. Semin Oncol Nurs. 2021;37(2):151145. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Patient-reported outcome measures: use in medical product development to support labeling claims. U.S. Food & Drug Administration. 2009. URL: https:/​/www.​fda.gov/​regulatory-information/​search-fda-guidance-documents/​patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims [accessed 2023-11-26]
• Appendix 2 to the guideline on the evaluation of anticancer medicinal products in man—the use of patient-reported outcome (PRO) measures in oncology studies—scientific guideline. European Medicines Agency. 2016. URL: https:/​/www.​ema.europa.eu/​en/​appendix-2-guideline-evaluation-anticancer-medicinal-products-man-use-patient-reported-outcome-pro [accessed 2023-11-26]
• Weber SC. The evolution and use of patient-reported outcomes in regulatory decision making. RF Q. 2023;3(1):4-9. [ FREE Full text ]
• Teixeira MM, Borges FC, Ferreira PS, Rocha J, Sepodes B, Torre C. A review of patient-reported outcomes used for regulatory approval of oncology medicinal products in the European Union between 2017 and 2020. Front Med (Lausanne). 2022;9:968272. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Newell S, Jordan Z. The patient experience of patient-centered communication with nurses in the hospital setting: a qualitative systematic review protocol. JBI Database System Rev Implement Rep. 2015;13(1):76-87. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yagasaki K, Takahashi H, Ouchi T, Yamagami J, Hamamoto Y, Amagai M, et al. Patient voice on management of facial dermatological adverse events with targeted therapies: a qualitative study. J Patient Rep Outcomes. 2019;3(1):27. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Giardina TD, Korukonda S, Shahid U, Vaghani V, Upadhyay DK, Burke GF, et al. Use of patient complaints to identify diagnosis-related safety concerns: a mixed-method evaluation. BMJ Qual Saf. 2021;30(12):996-1001. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lauriola I, Lavelli A, Aiolli F. An introduction to deep learning in natural language processing: models, techniques, and tools. Neurocomputing. 2022;470:443-456. [ CrossRef ]
• Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. 2019. Presented at: 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; June 2-7, 2019;4171-4186; Minneapolis, MN, USA.
• Dreisbach C, Koleck TA, Bourne PE, Bakken S. A systematic review of natural language processing and text mining of symptoms from electronic patient-authored text data. Int J Med Inform. 2019;125:37-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sim JA, Huang X, Horan MR, Stewart CM, Robison LL, Hudson MM, et al. Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: a systematic review. Artif Intell Med. 2023;146:102701. [ CrossRef ] [ Medline ]
• Weissenbacher D, Banda JM, Davydova V, Estrada-Zavala D, Gascó Sánchez L, Ge Y, et al. Overview of the seventh social media mining for health applications (#SMM4H) shared tasks at COLING 2022. 2022. Presented at: Proceedings of the Seventh Workshop on Social Media Mining for Health Applications, Workshop & Shared Task; October 12-17, 2022;221-241; Gyeongju, Republic of Korea. URL: https://aclanthology.org/2022.smm4h-1.54/
• Matsuda S, Ohtomo T, Okuyama M, Miyake H, Aoki K. Estimating patient satisfaction through a language processing model: model development and evaluation. JMIR Form Res. 2023;7(1):e48534. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yu D, Vydiswaran VGV. An assessment of mentions of adverse drug events on social media with natural language processing: model development and analysis. JMIR Med Inform. 2022;10(9):e38140. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu X, Chen H. A research framework for pharmacovigilance in health social media: identification and evaluation of patient adverse drug event reports. J Biomed Inform. 2015;58:268-279. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nikfarjam A, Sarker A, O'Connor K, Ginn R, Gonzalez G. Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features. J Am Med Inform Assoc. 2015;22(3):671-681. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kakalou C, Dimitsaki S, Dimitriadis VK, Natsiavas P. Exploiting social media for active pharmacovigilance: the PVClinical social media workspace. Stud Health Technol Inform. 2022;290:739-743. [ CrossRef ] [ Medline ]
• Bousquet C, Dahamna B, Guillemin-Lanne S, Darmoni SJ, Faviez C, Huot C, et al. The adverse drug reactions from patient reports in social media project: five major challenges to overcome to operationalize analysis and efficiently support pharmacovigilance process. JMIR Res Protoc. 2017;6(9):e179. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Young IJB, Luz S, Lone N. A systematic review of natural language processing for classification tasks in the field of incident reporting and adverse event analysis. Int J Med Inform. 2019;132:103971. [ CrossRef ] [ Medline ]
• Jacobsson R, Bergvall T, Sandberg L, Ellenius J. Extraction of adverse event severity information from clinical narratives using natural language processing. Pharmacoepidemiol Drug Saf. 2017;26(S2):37. [ FREE Full text ]
• Liang C, Gong Y. Predicting harm scores from patient safety event reports. Stud Health Technol Inform. 2017;245:1075-1079. [ CrossRef ] [ Medline ]
• Jiang G, Wang L, Liu H, Solbrig HR, Chute CG. Building a knowledge base of severe adverse drug events based on AERS reporting data using semantic web technologies. Stud Health Technol Inform. 2013;192(1-2):496-500. [ CrossRef ] [ Medline ]
• Usui M, Aramaki E, Iwao T, Wakamiya S, Sakamoto T, Mochizuki M. Extraction and standardization of patient complaints from electronic medication histories for pharmacovigilance: natural language processing analysis in Japanese. JMIR Med Inform. 2018;6(3):e11021. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Watanabe T, Yada S, Aramaki E, Yajima H, Kizaki H, Hori S. Extracting multiple worries from breast cancer patient blogs using multilabel classification with the natural language processing model bidirectional encoder representations from transformers: infodemiology study of blogs. JMIR Cancer. 2022;8(2):e37840. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nishioka S, Watanabe T, Asano M, Yamamoto T, Kawakami K, Yada S, et al. Identification of hand-foot syndrome from cancer patients' blog posts: BERT-based deep-learning approach to detect potential adverse drug reaction symptoms. PLoS One. 2022;17(5):e0267901. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nishioka S, Asano M, Yada S, Aramaki E, Yajima H, Yanagisawa Y, et al. Adverse event signal extraction from cancer patients' narratives focusing on impact on their daily-life activities. Sci Rep. 2023;13(1):15516. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Weed LL. Medical records that guide and teach. N Engl J Med. 1968;278(11):593-600. [ CrossRef ] [ Medline ]
• Podder V, Lew V, Ghassemzadeh S. SOAP Notes. Treasure Island, FL. StatPearls Publishing; 2023.
• Shenavar Masooleh I, Ramezanzadeh E, Yaseri M, Sahere Mortazavi Khatibani S, Sadat Fayazi H, Ali Balou H, et al. The effectiveness of training on daily progress note writing by medical interns. J Adv Med Educ Prof. 2021;9(3):168-175. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grothen AE, Tennant B, Wang C, Torres A, Sheppard BB, Abastillas G, et al. Application of artificial intelligence methods to pharmacy data for cancer surveillance and epidemiology research: a systematic review. JCO Clin Cancer Inform. 2020;4:1051-1058. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ohno Y, Kato R, Ishikawa H, Nishiyama T, Isawa M, Mochizuki M, et al. Using the natural language processing system MedNER-J to analyze pharmaceutical care records. medRxiv. Preprint posted online on October 2, 2023. [ CrossRef ]
• Ranchon F, Chanoine S, Lambert-Lacroix S, Bosson JL, Moreau-Gaudry A, Bedouch P. Development of artificial intelligence powered apps and tools for clinical pharmacy services: a systematic review. Int J Med Inform. 2023;172:104983. [ CrossRef ] [ Medline ]
• Nakajima Pharmacy. URL: https://www.nakajima-phar.co.jp/ [accessed 2023-12-07]
• Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res. 2020;21(140):1-67. [ FREE Full text ]
• Pathak A. Comparative analysis of transformer based language models. Comput Sci Inf Technol. 2021.:165-176. [ FREE Full text ] [ CrossRef ]
• DIPEx Japan. URL: https://www.dipex-j.org/ [accessed 2024-02-04]
• Herxheimer A, McPherson A, Miller R, Shepperd S, Yaphe J, Ziebland S. Database of patients' experiences (DIPEx): a multi-media approach to sharing experiences and information. Lancet. 2000;355(9214):1540-1543. [ CrossRef ] [ Medline ]
• Lara PE, Muiño CB, de Spéville BD, Reyes JJ. Hand-foot skin reaction to regorafenib. Actas Dermosifiliogr. 2016;107(1):71-73. [ CrossRef ]
• Zaiem A, Hammamia SB, Aouinti I, Charfi O, Ladhari W, Kastalli S, et al. Hand-foot syndrome induced by chemotherapy drug: case series study and literature review. Indian J Pharmacol. 2022;54(3):208-215. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McLellan B, Ciardiello F, Lacouture ME, Segaert S, Van Cutsem E. Regorafenib-associated hand-foot skin reaction: practical advice on diagnosis, prevention, and management. Ann Oncol. 2015;26(10):2017-2026. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ai L, Xu Z, Yang B, He Q, Luo P. Sorafenib-associated hand-foot skin reaction: practical advice on diagnosis, mechanism, prevention, and management. Expert Rev Clin Pharmacol. 2019;12(12):1121-1127. [ CrossRef ] [ Medline ]
• Tenti S, Correale P, Cheleschi S, Fioravanti A, Pirtoli L. Aromatase inhibitors-induced musculoskeletal disorders: current knowledge on clinical and molecular aspects. Int J Mol Sci. 2020;21(16):5625. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lacouture ME, Melosky BL. Cutaneous reactions to anticancer agents targeting the epidermal growth factor receptor: a dermatology-oncology perspective. Skin Therapy Lett. 2007;12(6):1-5. [ FREE Full text ] [ Medline ]

Abbreviations

Edited by G Eysenbach; submitted 25.12.23; peer-reviewed by CY Wang, L Guo; comments to author 24.01.24; revised version received 14.02.24; accepted 09.03.24; published 16.04.24.

©Satoshi Nishioka, Satoshi Watabe, Yuki Yanagisawa, Kyoko Sayama, Hayato Kizaki, Shungo Imai, Mitsuhiro Someya, Ryoo Taniguchi, Shuntaro Yada, Eiji Aramaki, Satoko Hori. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 16.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.