The undergraduate student participatory research project titled “A Comparative Analysis of the Reliability of Climate Information, Resilience to Misinformation, and Framing Tendencies of Generative Artificial Intelligence Systems,” prepared under the leadership of Research Assistant Dr. Ayten Bengisu Cansever Bayhan from the Visual Communication Design Department (VCD) of the Faculty of Fine Arts (FFA) at Istanbul Gelişim University (IGU), by a research team consisting of academics from the Faculty of Fine Arts and undergraduate student Bilge Kaan Alagöz, has been awarded funding. The research team also includes Research Assistant Hatice Çoban and Research Assistant Mustafa Dallı from the Turkish and English Departments of Interior Architecture and Environmental Design, and Research Assistant Okan Kırbacı from the Department of Radio, Television and Cinema.
Research project led by research assistant Dr. Ayten Bengisu Cansever Bayhan and her team receives funding
The undergraduate student participatory research project titled “A Comparative Analysis of the Reliability of Climate Information, Resilience to Misinformation, and Framing Tendencies of Generative Artificial Intelligence Systems,” prepared under the leadership of Research Assistant Dr. Ayten Bengisu Cansever Bayhan from the Visual Communication Design Department (VCD) of the Faculty of Fine Arts (FFA) at Istanbul Gelişim University (IGU), by a research team consisting of academics from the Faculty of Fine Arts and undergraduate student Bilge Kaan Alagöz, has been awarded funding. The research team also includes Research Assistant Hatice Çoban and Research Assistant Mustafa Dallı from the Turkish and English Departments of Interior Architecture and Environmental Design, and Research Assistant Okan Kırbacı from the Department of Radio, Television and Cinema.
Generative artificial intelligence and the reliability of climate information will be examined
Today, generative artificial intelligence assistants are increasingly among the tools consulted by users in the process of obtaining information on scientific topics. The reliability of information provided by artificial intelligence systems is particularly important in subjects such as climate change, which require scientific consensus, uncertainties, and policy dimensions to be evaluated together. The project will examine the possible tendencies of these systems to generate inaccurate information, validate false premises, provide sources with low verifiability, and highlight certain solution frameworks through the experience of Turkish-speaking users.
Four generative artificial intelligence assistants will be compared
Within the scope of the research, the climate information generation performance of four widely accessible generative artificial intelligence assistants will be comparatively examined. A 25-item claim bank, validated by experts and based on classifications in the climate misinformation literature, will be created. Each claim will be tested in four different artificial intelligence systems and in two independent repetitions under two different conditions: neutral evaluation and a leading prompt that reinforces a false premise.
As a result of this experimental design, a total of 400 AI outputs are planned to be obtained. The outputs will be coded by two independent evaluators according to categories including scientific accuracy, quality of reasoning, resistance to false premises, uncertainty calibration, source reliability, and framing.
Evaluation based on scientific sources
The reference answer key to be used in the evaluation process of the project will be prepared based on IPCC reports, peer-reviewed scientific publications, and reliable scientific and institutional sources. Thus, the study will examine not only whether artificial intelligence systems produce correct or incorrect answers, but also how they respond to misleading prompts, the verifiability of the sources they provide, and the solution and responsibility frameworks through which they address the climate problem.
The originality of the research lies in its comparative examination of general-purpose artificial intelligence assistants used in Turkish within a controlled and reproducible experimental design in the context of climate misinformation. The project aims to move the evaluation of artificial intelligence systems beyond a one-dimensional assessment of accuracy by addressing resistance to misinformation, source reliability, and framing tendencies together.
Research outputs will be transformed into user awareness
At the end of the project, a test set consisting of Turkish climate claims, a coding guideline, and a comparative performance report are planned to be prepared. In addition, a draft scientific publication will be developed based on the research findings. In order to ensure that the project's outputs extend beyond the academic field, a short user guide titled “What Should We Pay Attention to When Getting Climate Information from Artificial Intelligence?”, infographics, and an awareness event will also be prepared.
As the Visual Communication Design Department of the Faculty of Fine Arts at Istanbul Gelişim University, we hope that the research will contribute to the fields of artificial intelligence literacy and climate communication, and we congratulate our students on their participation in this important study.
The undergraduate student participatory research project titled “A Comparative Analysis of the Reliability of Climate Information, Resilience to Misinformation, and Framing Tendencies of Generative Artificial Intelligence Systems,” prepared under the leadership of Research Assistant Dr. Ayten Bengisu Cansever Bayhan from the Visual Communication Design Department (VCD) of the Faculty of Fine Arts (FFA) at Istanbul Gelişim University (IGU), by a research team consisting of academics from the Faculty of Fine Arts and undergraduate student Bilge Kaan Alagöz, has been awarded funding. The research team also includes Research Assistant Hatice Çoban and Research Assistant Mustafa Dallı from the Turkish and English Departments of Interior Architecture and Environmental Design, and Research Assistant Okan Kırbacı from the Department of Radio, Television and Cinema.
Generative artificial intelligence and the reliability of climate information will be examined
Today, generative artificial intelligence assistants are increasingly among the tools consulted by users in the process of obtaining information on scientific topics. The reliability of information provided by artificial intelligence systems is particularly important in subjects such as climate change, which require scientific consensus, uncertainties, and policy dimensions to be evaluated together. The project will examine the possible tendencies of these systems to generate inaccurate information, validate false premises, provide sources with low verifiability, and highlight certain solution frameworks through the experience of Turkish-speaking users.
Four generative artificial intelligence assistants will be compared
Within the scope of the research, the climate information generation performance of four widely accessible generative artificial intelligence assistants will be comparatively examined. A 25-item claim bank, validated by experts and based on classifications in the climate misinformation literature, will be created. Each claim will be tested in four different artificial intelligence systems and in two independent repetitions under two different conditions: neutral evaluation and a leading prompt that reinforces a false premise.
As a result of this experimental design, a total of 400 AI outputs are planned to be obtained. The outputs will be coded by two independent evaluators according to categories including scientific accuracy, quality of reasoning, resistance to false premises, uncertainty calibration, source reliability, and framing.
Evaluation based on scientific sources
The reference answer key to be used in the evaluation process of the project will be prepared based on IPCC reports, peer-reviewed scientific publications, and reliable scientific and institutional sources. Thus, the study will examine not only whether artificial intelligence systems produce correct or incorrect answers, but also how they respond to misleading prompts, the verifiability of the sources they provide, and the solution and responsibility frameworks through which they address the climate problem.
The originality of the research lies in its comparative examination of general-purpose artificial intelligence assistants used in Turkish within a controlled and reproducible experimental design in the context of climate misinformation. The project aims to move the evaluation of artificial intelligence systems beyond a one-dimensional assessment of accuracy by addressing resistance to misinformation, source reliability, and framing tendencies together.
Research outputs will be transformed into user awareness
At the end of the project, a test set consisting of Turkish climate claims, a coding guideline, and a comparative performance report are planned to be prepared. In addition, a draft scientific publication will be developed based on the research findings. In order to ensure that the project's outputs extend beyond the academic field, a short user guide titled “What Should We Pay Attention to When Getting Climate Information from Artificial Intelligence?”, infographics, and an awareness event will also be prepared.
As the Visual Communication Design Department of the Faculty of Fine Arts at Istanbul Gelişim University, we hope that the research will contribute to the fields of artificial intelligence literacy and climate communication, and we congratulate our students on their participation in this important study.