Abstract
Large pre-trained models (LPMs) have the capabilities to understand natural language, code, and diverse data including images; e.g., large language models (LLMs), code-generative models, and large vision models (LVMs) as well as combined as multi-modal models. We outlines the main applications of LPMs and multi-modal LPMs for ecology and biodiversity conservation. These applications include generating ecological data, generating code, providing insights into public opinion and sentiment. We highlighted the significant potential of LPMs and the potential use of Ecology-specialized LPMs for ecology and biodiversity conservation. They offer unprecedented opportunities for analyzing diverse data, extracting meaningful insights, and informing conservation decisions.
Recent advancements in artificial intelligence (AI) have led to the emergence of sophisticated large pre-trained models (LPMs). These LPMs represent a significant leap forward in AI technology, offering unparalleled capabilities in understanding natural language, generating human-like text, and performing complex tasks across various domains. LPMs have shown impressive proficiency in various applications, including natural language processing, code generation, and data/vision analysis.
For natural language processing, large language models (LLMs) like Generative Pre-trained Transformer (GPT) (Brown et al. 2020), such as ChatGPT (GPT-3 and 4), Llama, and Bard, leading the revolution in natural language processing. These models possess the ability to generate human-like text, decipher complex language structures, and even translate languages with remarkable accuracy. LLMs prove invaluable for sentiment analysis, text summarization, and cross-language communication (e.g., Rane et al. 2023; Patil et al.2024a). Also, code-generative LLM, like GitHub Copilot (https://github.com/features/copilot) and CodeT5 (https://github.com/salesforce/CodeT5), contribute to the programming of the codes for analysis and simulation by understanding complex code and aiding developers in debugging. Large vision models (LVMs) have the ability to process and interpret vast amounts of visual data with exceptional accuracy and efficiency. LVMs utilize deep learning techniques and extensive pre-training on massive image datasets.
Multi-modal LPMs is a combined model that integrates diverse data types, such as text, images, and audio, broadening their scope. Using LLM+LVM, the most developed multi-modal LPM, text-to-image (T2I) synthesis has undergone significant advancements, particularly with the emergence of LLM and their enhancement in LVM (Cheng et al. 2024). These models excel in tasks like image captioning, visual question answering, and audio transcription, enabling more context-rich AI systems. The potential applications of LPMs span from chatbots and virtual assistants to content creation and language learning. A significant advantage lies in their capacity to produce text indistinguishable from human writing, paving the way for more natural interactions with chatbots and virtual assistants (e.g., Rane et al. 2023 in medical). Furthermore, their language translation prowess holds the potential to reshape global communication (Patilet al. 2024a). Therefore, LPMs are now emerging as promising tools to tackle ecological challenges by analyzing vast data extracting meaningful insights, and forming a foundation for well-informed conservation decisions. The integration of LPMs in biodiversity conservation holds immense potential to revolutionize research practices and bolster conservation efforts. The subsequent sections delineate the applications of LPMs in ecology and biodiversity conservation.
Ecological data generation by LPMs
LPMs play a pivotal role in processing extensive documents on ecology, including scientific papers, reports, and online news articles, extracting pertinent information. For instance, researchers leverage LLMs to extract the description to identify novel or endangered species, monitor changes in population size, and discern emerging threats to biodiversity (e.g., Fabian et al. 2024). Many LLMs exhibit the ability to translate across languages, enabling researchers to access crucial information from non-English resources. Some studies suggested the potential of non-English resources in filling conservation and ecological knowledge gaps (e.g., Amano et al. 2021; Hannahet al. 2024). Despite biodiversity information being foundational to ecology, evolution, and conservation, Amano (et al. 2023) reported that non-English-language literature constituted 65% of the references cited as biodiversity information. Therefore, numerous natural history resources in non-English languages remain unshared due to the language barriers. LLMs facilitate the integration of natural history knowledge across languages, mitigating obstacles for the researchers in selecting the language for publication.
LPMs excel in analyzing the data, such as satellite information, to formulate statistical models that elucidate intricate relationships between species and their surroundings by code generation. A multi-modal LPM, LLM-LVM, can generate text from image analysis by automatically recognizing and classifying species names based on visual data (e.g., Jain et al. 2023; Parasharet al. 2023). This feature can enhance biodiversity monitoring and species identification efforts, particularly in large-scale surveys (e.g., Maurer et al. 2020; Wang et al. 2020). Therefore, LLM-LVM can process satellite imagery and extract valuable environmental data, such as land cover types, vegetation indices, and habitat connectivity. The combined use of text mining, image analysis, and satellite data processing by LLM-LVM has great potential for generating ecological data, which can enable researchers and conservation practitioners to gain valuable insights for effective biodiversity conservation and management strategies.
Code-generative LLMs foe ecolgical modeling and simulation
Code-generative LLMs play a crucial role in simulation research by promoting reproducibility and interpretability, ensuring that research findings can be validated and comprehended by the broader scientific community. For example, PyRates (Gast et al. 2023), a code-generation tool for dynamical systems modeling applied to biological systems, can be used for the dynamical models on ecological insights and ecosystem management. Therefore, LLMs present a unique opportunity for the standardization and commoditization of various research aspects, streamlining processes from SQL and GBIF data to R or Python codes. Researchers, by harnessing LLMs, can redirect their focus toward leveraging the rich and diverse field data accessible through platforms like GBIF, fostering a more comprehensive understanding of biodiversity patterns and trends (Anderson et al. 2023). A research approach with code-generative LLMs establishes a robust foundation for integrating logical support and employing case studies to address intricate conservation challenges, providing a radical yet promising avenue to advance biodiversity understanding and guide effective conservation strategies.
However, the utilization of code-generative LLMs in ecological modeling poses potential limitations and challenges. A major concern revolves around the risk of bias in the generated code, potentially leading to inaccurate or misleading predictions (Shah et al. 2020; Weidingeret al. 2021; Albrecht et al. 2022). LLMs, being trained on large datasets, may inherently contain biases and limitations that are not always apparent to users. Additionally, LLMs may generate code that proves challenging to understand or modify, restricting the flexibility and adaptability of ecological models. To surmount these challenges, it is imperative to rigorously evaluate the performance and accuracy of LLM-generated code in ecological modeling applications.
Providing insights into public opinion and sentiment
LPMs have the potential to provide valuable insights into public opinion and sentiment regarding biodiversity conservation. One application involves using AI to generate summaries of complex environmental reports, making them easier for the general public to understand. This approach could enhance communication and promote wider engagement with ecological and conservation issues.
LLMs can analyze a wide range of sources, including social media data, which has increasingly become a significant platform for public discourse. For instance, Lee et al. (2023) tried to capture public opinion on climate change by utilizing two nationally representative climate change surveys. They found that LLMs (GPT-3) could effectively capture presidential voting behaviors to decide the climate change policies. Also, GPT-4 exhibits improved performance when conditioned on both demographics and covariates. Several studies used LLMs as a natural language processing approach to investigate public attitudes and sentiment analysis (Koonchanok et al.2023) for public attitudes of news topic by Twitter, (Ahmadet al. 2024) for COVID-19 vaccination sentiment analysis).
LLMs have the potential to be exploited for creating fake news, spreading propaganda, and manipulating public opinion (Chen & Shu 2023), while their capacity to process extensive volumes of text also enables the identification of instances involving fake news, propaganda, and the manipulation of public opinion. This analytical capability proves crucial in illuminating the narratives being disseminated and their potential impact on public perception and policy-making. Recognizing the influence of false narratives, particularly concerning issues like climate change, becomes imperative to prevent a lack of action or misguided policies. However, Patil et al. (2024b) provided the LLM approach for recognizing false news using the LamaIndex (https://www.llamaindex.ai). So, multi-modal LLMs using LLM with false-news detected LLM would be useful to summarize the public opinion.
A significant advantage of utilizing LLMs for such analyses lies in their immediacy. While a systematic review stands as a foundational method for summarizing knowledge on a particular subject, it demands considerable effort and essentially overlooks contemporaneous sources such as news articles and social media content. Conversely, the analysis of recent reports and information can alleviate the inherent limitations of the systematic review approach. However, the application of LLMs in ecology and biodiversity conservation presents its share of challenges. The accuracy and reliability of LLMs hinge on the quality of the data used for their training. Moreover, LLMs have the potential to perpetuate biases and inequalities present in the data, posing a risk to the effectiveness of conservation management efforts. Responsible use of LLMs is paramount, considering not only their accuracy but also the environmental impact of deploying these models for conservation activities. LLMs should complement human expertise in conservation rather than replacing it, acknowledging the unique insights, intuition, and contextual understanding that models may not fully grasp. The incorporation of LLMs in ecology and conservation necessitates navigating intricate legal landscapes, ensuring compliance with various international and national regulations.
Development of Ecology-specialized LPMs
Recently, there have been developments in specialized LPMs for specific purposes (Ling et al. 2023), such as specialized LLMs, BioMedLM, and agriGPT, which focus on biomedicine and agriculture. It is worth considering the potential benefits of developing specialized LLMs in ecology and biodiversity science, as they could potentially accelerate the advancement of the field. Specialized LPMs in ecology have the potential to enhance our understanding of complex ecological systems and contribute to decision-making processes related to biodiversity and ecosystem conservation. Developing such a model requires a multifaceted approach to create an AI system that can comprehend, generate, and analyze ecological data and language at a high level of sophistication. The development of LLMs specialized in Ecology is considered a significant advancement in ecological research and biodiversity conservation.
These models, when combined with LVM offer unparalleled opportunities to revolutionize our understanding of complex ecological systems. There are some models for specializing LLMs such as knowledge-updated domain specialization (KUDS) with fine-tune step (Ling et al. 2023). KUDS needs more cost and less real-time use with tuning the LLMs, but for ecological knowledge based on long-term history would be suitable to use KUDS for the tuning. Through the use of LLMs with KUDS encompassing ecological literature and field observations, researchers can leverage their significant computational power to reveal complex relationships, forecast ecosystem dynamics, and provide support for conservation strategies based on evidence. The integration of LVMs and LLM-LPM into ecological studies enables the synthesis of diverse data sources, including satellite imagery, remote sensing data, and environmental sensor networks, thereby providing comprehensive insights into ecosystem health and resilience. In fact, (Jain et al. 2023) developed a LLM-LVM for text generation such as landscape category using satellite imagery. The combination of Ecology-specialized LPMs, including LLM, LVM, and multi-modal LPM, has the potential to advance ecological research and promote interdisciplinary collaboration. This could lead to more effective conservation initiatives aimed at protecting biodiversity and ecosystem services on a global scale.
Negative impacts of using LPMs on the environment
Rillig et al. (2023) noted that the energy consumption of LLMs during training and operation has significant adverse effects on the environment. The environmental impact of LLMs is closely linked to their immense computational requirements, which translate into extensive electricity usage. For instance, the training process of GPT-3 alone has been estimated to consume over 1287 MWh of energy (Rillig et al.2023). According to the studies conducted by Strubell et al.(2019) and Patterson et al. (2021), the estimated carbon footprint associated with this is 552.1 t of CO2. It is worth noting that as LLMs continue to evolve and their demand for larger datasets increases, their energy requirements are expected to escalate even further (e.g., Luccioni et al. 2023). Therefore, it is important to evaluate and address the environmental impact of LPMs in a conscientious manner. It is crucial to align with responsible environmental practices and emphasize the necessity of ethical and eco-friendly advancements in the field of LPMs.
Conclusion
We examine here the potential of LPMs to address challenges in ecology and biodiversity conservation. The development and integration of specialized LPMs, particularly those tailored for ecology and biodiversity conservation, represent a significant advancement in ecological research and conservation practices. Moreover, multi-modal LPMs with LVM have the potential to offer extensive insights into ecology and biodiversity conservation by integrating various data sources and texts in different languages. LPMs are considered to substantially improve methods for ecology and biodiversity conservation. Nonetheless, it is imperative to collaborate among researchers, practitioners, and stakeholders to achieve a successful integration of LPM applications in ecological research and conservation practices.
Acknowledgments
We would like to thank Akira S. Mori for his comments on our manuscript. DeepL Write has been valuable in helping us to improve our English writing of this paper.
References
Ahmad, I.S., Aliyu, L.J., Khalid, A.A., Aliyu, S.M., Muhammad, S.H., Abdulmumin, I., et al. (2024). Analyzing COVID-19 Vaccination Sentiments in Nigerian Cyberspace: Insights from a Manually Annotated Twitter Dataset. arXiv [cs.CL] .
Albrecht, J., Kitanidis, E. & Fetterman, A.J. (2022). Despite “super-human” performance, current LLMs are unsuited for decisions about ethics and safety. arXiv [cs.CL] .
Amano, T., Berdejo-Espinola, V., Akasaka, M., de Andrade Junior, M.A.U., Blaise, N., Checco, J., et al. (2023). The role of non-English-language science in informing national biodiversity assessments. Nature Sustainability , 6, 845–854.
Amano, T., Berdejo-Espinola, V., Christie, A.P., Willott, K., Akasaka, M., Báldi, A., et al. (2021). Tapping into non-English-language science for the conservation of global biodiversity. PLoS Biol. , 19, e3001296.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P.,et al. (2020). Language models are few-shot learners. Adv. Neural Inf. Process. Syst. , 33, 1877–1901.
Chen, C. & Shu, K. (2023). Combating Misinformation in the Age of LLMs: Opportunities and Challenges. arXiv [cs.CY] .
Cheng, W., Liu, J., Deng, J. & Ren, F. (2024). SSP: A Simple and Safe automatic Prompt engineering method towards realistic image synthesis on LVM. arXiv [cs.CV] .
Gast, R., Knösche, T.R. & Kennedy, A. (2023). PyRates-A code-generation tool for modeling dynamical systems in biology and beyond. PLoS Comput. Biol. , 19, e1011761.
Hannah, K., Haddaway, N.R., Fuller, R.A. & Amano, T. (2024). Language inclusion in ecological systematic reviews and maps: Barriers and perspectives. Res Synth Methods .
Jain, H., Verma, S. & Gupta, S. (2023). Investigating Large Vision Model Training Challenges on Satellite Datasets. In: InGARSS 2023 - India Geoscience and Remote Sensing Symposium .
Koonchanok, R., Pan, Y. & Jang, H. (2023). Tracking public attitudes toward ChatGPT on Twitter using sentiment analysis and topic modeling.arXiv [cs.CL] .
Lee, S., Peng, T.Q., Goldberg, M.H., Rosenthal, S.A., Kotcher, J.E., Maibach, E.W., et al. (2023). Can Large Language Models Capture Public Opinion about Global Warming? An Empirical Assessment of Algorithmic Fidelity and Bias. arXiv [cs.AI] .
Ling, C., Zhao, X., Lu, J., Deng, C., Zheng, C., Wang, J., et al.(2023). Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey. arXiv [cs.CL] .
Luccioni, A.S., Viguier, S. & Ligozat, A.-L. (2023). Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model. J. Mach. Learn. Res. , 24, 1–15.
Maurer, G.E., Hallmark, A.J., Brown, R.F., Sala, O.E. & Collins, S.L. (2020). Sensitivity of primary production to precipitation across the United States. Ecol. Lett. , 23, 527–536.
Parashar, S., Lin, Z., Li, Y. & Kong, S. (2023). Prompting Scientific Names for Zero-Shot Species Recognition. arXiv [cs.CV] .
Patil, D.D., Dhotre, D.R., Gawande, G.S., Mate, D.S., Shelke, M.V. & Bhoye, T.S. (2024a). Transformative Trends in Generative AI: Harnessing Large Language Models for Natural Language Understanding and Generation.Int J Intell Syst Appl Eng , 12, 309–319.
Patil, M., Yadav, H., Gawali, M., Suryawanshi, J., Patil, J., Yeole, A.,et al. (2024b). A Novel Approach to Fake News Detection Using Generative AI. Int J Intell Syst Appl Eng , 12, 343–354.
Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., et al. (2021). Carbon Emissions and Large Neural Network Training. arXiv [cs.LG] .
Rane, N.L., Tawde, A., Choudhary, S.P. & Rane, J. (2023). Contribution and performance of ChatGPT and other Large Language Models (LLM) for scientific and research advancements: a double-edged sword.International Research Journal of Modernization in Engineering Technology and Science .
Rillig, M.C., Ågerstrand, M., Bi, M., Gould, K.A. & Sauerland, U. (2023). Risks and Benefits of Large Language Models for the Environment.Environ. Sci. Technol. , 57, 3464–3466.
Shah, D.S., Schwartz, H.A. & Hovy, D. (2020). Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, Online, pp. 5248–5264.
Strubell, E., Ganesh, A. & McCallum, A. (2019). Energy and Policy Considerations for Deep Learning in NLP. arXiv [cs.CL] .
Wang, H., Liu, H., Cao, G., Ma, Z., Li, Y., Zhang, F., et al.(2020). Alpine grassland plants grow earlier and faster but biomass remains unchanged over 35 years of climate change. Ecol. Lett. , 23, 701–710.
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., et al. (2021). Ethical and social risks of harm from Language Models. arXiv [cs.CL] .