Introduction

Jens Dierkes

During the planning of the festival, we, the organizers, were looking for new formats for the celebration of open science and we came across Ignite.

What is an Ignite talk? It is a short and fun format to put an idea that is of importance to you and that you would like to share into a five-minute story. Stories are the connecting fabric between people in societies throughout history. And stories are likely to be easier to absorb than the abstract talks you often hear at conferences.

Through stories, you can learn to better understand each other, you can relate to each other, and you can even form bonds. And that is exactly what Ignite talks aim to achieve.

Each five-minute story is supported by 20 slides that auto advance every 15 seconds. That all means that you have to come to your points quickly and concisely and that you have to focus on the two to three main messages of your idea.
This is quite a challenge. What is very important here is the audience, because the setting should be such that the audience is on our side, eager and open to learn, and forgiving if you stumble a bit in your presentation.

What’s attractive about an Ignite event from the perspective of the audience is that your curiosity is entertained by a number of short stories and a range of perspectives. This seems to be a very good complement to longer talks. For the speaker, this format helps them to concentrate. And the good news is that you can practice your speech as many as 10 times per hour.

Stories are at the center of Ignite and these are also important in the context of open science, since the digital transformation is also driving a cultural change in the way we do science. Think of the climate crisis. Good stories help bring about change. They appeal to the unconscious emotional mind rather than the rational.

Cultural change is not a straight and smooth road, but a bumpy one, with potholes, where we have to pave the way, build bridges, decide which junction to take, and possibly explore uncharted territory. This was all part of the festival. What you need is a kind of sense-making and intelligent experimentation.

And in this sense, for us, the organisers, this first Ignite session is an experiment to give the audience different perspectives on Open Science in a condensed way, to spark your imagination / increase your interest in Open Science.

Ignite Talks at the festival – Open Science Success and Horror Stories

Talk 1: It takes good research conditions to foster good research practice

Rima-Maria Rahal

The first talk was centered around the relation of working conditions in academia and good research practice. At its core, open science movement dedicated to ensuring good research practice and improving existing practices. Therefore, open science also relates to the environments in which research takes place, given that they influence how research is done.

It follows that research environments need to be designed to promote, not hinder, good research practice. Increasing the propotion of permant research positions has foreseeable positive effects, ensuring that good research practice can become the norm. Redesigning academic systems to put an end to the end-ofcareer threat, hypercompetiviteness and the primacy of publication counts and impact factors – in short: misaligned incentives – would allow researcher to invest the time it takes to work collaboratively, to test boundary conditions of their research, to make materials openly available, introduce Open Science in teaching and mentoring and to work towards open and transparent communication with the public throughout the research process.

Detailed arguments for this approach, as well as an international perspective on working conditions in academia and their relation to good research practice can be found in Rahal et al (2023).

You can watch the recording of Rima-Maria Rahals Ignite Talk here.

Talk 2: Open Science Movement in Psychology

Marc Jekel

How did the Open Science movement start in psychology?

1. In a prominent psychology journal, Bem (2011) reported a series of experiments involving over 1000 participants. These experiments suggested that unforeseen future events can have an influence on the present. The findings indicated that various well-known effects in psychology can occur in reverse chronological order. For example, participants were presented with 48 nouns (such as different types of foods, animals, etc.) and were later asked to recall as many as they could in an unexpected memory test. Surprisingly, the nouns that were practiced after the recall test were more likely to be recalled. These time-reversed effects cannot be explained by existing psychological theory or, more fundamentally, the laws of physics. This makes it more likely that there are flaws in the studies rather than time-reversed effects being a reality. Consequently, these studies sparked a discussion within the research community regarding the weaknesses of the methodology employed in psychology and other empirical disciplines.
2. One commonly discussed reason is the impact of research practices on inferential statistics. Inferential statistics is a branch of statistics that provides tools, such as significance tests, which researchers use to determine if a finding observed in an experiment is inconsistent with chance. A null-hypothesis significance test calculates the likelihood of the observed findings, or more extreme findings, assuming it is purely due to chance. Typically, the null-hypothesis is rejected if this probability is below 5%, indicating that the observed finding is highly unlikely under the chance assumption. In the past, and to some extent even nowadays, statistical significance has been a prerequisite for publishing research in prestigious journals, which can lead to various problems (refer to Talk I).
3. Unfortunately, the accepted probability of incorrectly rejecting a chance finding, which is 5%, can increase substantially when researchers engage in questionable research practices (Simmons et al., 2011). A survey (John et al., 2012) also revealed that researchers have indeed employed questionable research practices in the past. Therefore, the crucial question was to what extent the findings reported in journals are genuine or simply a coincidence resulting from questionable research practices and other issues in the research process. While there is no definitive answer to this important question, the reproducibility project (Open Science Collaboration, 2015) provided the first systematic empirical evidence by repeating 100 studies that had been published in three prestigious psychology journals. Out of the 100 findings, only about one-third could be replicated.

The Open Science movement emerged in response to these events, with the aim of developing tools that address questionable research practices and other flaws. Key elements of this movement include transparency in the research process, including sharing data and materials, implementing procedures, and creating space in journals for reports of replication studies. Although progress has been made, challenges still exist, such as establishing shared norms for documenting data and analysis scripts, which would allow psychology to evolve as a truly cumulative science.

Talk 3: FAIR Open data using Github and Zenodo

In his Ignite Talk Hannes Wünsche from the Robert Koch Institute (RKI) explains how they were able to build a complete new infrastructure and pipeline to publish, update and handle the huge amounts of the data right in the beginning of the Corona pandemic using the existing and established platforms Github and Zenodo:

Talk 4: Advancing Open Peer Review

Franciska Heenes, University and City Library Cologne

Open Peer Review (OPR) is becoming a popular alternative to traditional peer review because it is transparent and inclusive. Disclosing reviewers’ identities and comments offers numerous benefits, but it also presents challenges that need to be addressed. This talk looked at the advantages and obstacles associated with OPR and gave recommendations for a success story.

Benefits of Open Peer Review

OPR offers numerous benefits that significantly contribute to the overall quality and impact of scientific research. One key advantage is its promotion of transparency. Advocates of OPR argue that transparency enhances the review process by encouraging reviewers to provide detailed and well-reasoned feedback. Additionally, the transparent nature of OPR allows for ongoing feedback and improvement of research papers. Valuable discussions and improvements can arise through open debates and additional insights, enabling a more thorough evaluation of research papers and fostering collaborative efforts to enhance the quality of published work.

In an OPR setting, authors can understand the reasoning behind reviewers’ decisions and suggestions, while researchers who are inexperienced with peer review find valuable learning opportunities, further enhancing their skills and knowledge. Moreover, with open identities, the concept of OPR allows recognition of reviewers’ contributions, enabling them to build their professional reputation and receive credit for their contributions.

However, the benefits of OPR extend beyond the academic community. The wider public can gain insights into the scientific discourse, fostering public trust and engagement with scientific findings. Students, journalists, and policymakers can access review comments and understand the evaluation process, contributing to a more informed and engaged society. OPR also facilitates networking and collaboration among diverse individuals and groups, leading to potential future collaborations and advancements in the field.

Challenges of Open Peer Review

Despite its many benefits, OPR also presents challenges to researchers. One significant challenge is the potential influence of open identities on judgments, which can introduce biases and prejudices. Unlike traditional peer review, where anonymity helps mitigate these issues, addressing them in an open setting can be more difficult, potentially compromising the integrity of the review process. Consequently, reviewers may hesitate to participate in OPR due to concerns about their reputation or fear of backlash. This is particularly relevant for researchers from marginalized communities or early-career researchers, resulting in a smaller pool of available reviewers and a limited diversity of feedback. Furthermore, authors may worry that their work will be prematurely judged or criticized before undergoing revisions or improvements, which can discourage them from submitting their work for OPR. Limited researcher participation also poses a challenge as OPR may require more time and resources from both authors and reviewers. Engaging in discussions and responding to comments promptly can be demanding, especially for researchers with limited time or funding.

Developing a Success Story of Open Peer Review

OPR has the potential to transform the way research is evaluated and disseminated, making it a transformative force in the academic publishing landscape. However, the success of OPR relies heavily on its specific implementation and context:

1. Transparent Guidelines: Clear and comprehensive guidelines should be established to provide reviewers and authors with a framework for conducting OPR. This will help streamline the review process and ensure constructive and respectful feedback.
2. Training and Support: Researchers should be provided with adequate training and support to navigate the challenges of OPR. This can include workshops, resources, and mentorship programs to enhance their understanding and skills.
3. Diverse Reviewer Pool: Efforts should be made to actively recruit reviewers from diverse backgrounds and communities. This will help ensure a broader range of perspectives and minimize biases in the review process.
4. Constructive Feedback Culture: Encouraging a culture of constructive feedback is crucial for the success of OPR. Reviewers and authors should be encouraged to provide feedback that is helpful, respectful, and focused on improving the quality of the research.
5. Oversight: Efforts should be made to actively monitor reviewer feedback to identify and address instances of bias or discrimination. Tracking reviewer feedback, and addressing concerns raised by participants can help maintain fairness.
6. Recognition and Incentives: Recognizing the efforts of reviewers and authors who participate in OPR can help incentivize their engagement. This can be done through acknowledgments, certificates, or other forms of recognition within the scientific community.

It’s important to recognize that there is much more to the success story of this approach. By addressing the specific challenges, OPR can significantly increase its chances of becoming a true success story, driving impactful advancements in academic publishing and contributing significantly to the overall progress of scientific knowledge.

DOI (Digitalausgabe): https://doi.org/10.48664/gyh9-eq29

Open Science Festival 2023