Abstract
Surveys are widely used to measure attitudes, perceptions, experiences and other constructs that cannot be observed directly. However, developing a questionnaire that produces meaningful and interpretable data involves considerably more than writing a set of Likert-scale questions. This session will explore the process of designing and validating quantitative survey instruments, drawing on practical examples from research developing measures of students' attitudes towards astronomy and science, and experiences of inclusion in science engagement.

This webinar will consider how to move from defining a construct to developing appropriate survey items, including the use of existing literature and instruments, expert review, cognitive testing, and piloting. Key quantitative approaches to evaluating instruments will be reviewed, including item analysis, internal consistency and reliability, and exploratory and confirmatory factor analysis. The session will then discuss complimentary qualitative approaches to evaluating the appropriateness and effectiveness of survey instruments.

Rather than treating validation as a single statistical test, the session will emphasise validation as an ongoing process of accumulating evidence about whether an instrument supports the interpretations researchers wish to make from its scores. Participants will leave with a foundation for validating their own surveys, with practical examples, and an overview of common pitfalls and limitations.

Abstract
In this lecture, I will share the background to ROSES and how the project developed from an identified need in science education research (SER) at the time. Drawing on my experience of large-scale quantitative research, I will discuss how the project’s purpose shaped its design and how it was coordinated across participating countries, including decisions concerning questionnaire development, translation and adaptation, sampling, data collection, and the subsequent treatment and analysis of data. I will highlight key findings to illustrate why international comparisons matter and reflect on what worked well and what proved more challenging. With particular relevance to research on astronomy and space education, I will also reflect on the importance of sustained collaboration and active participation in the SER community, both for keeping pace with developments in the field and for contributing to the evolution of research questions, theoretical perspectives and methods.

By the end of this seminar, participants will be able to:
• Critically reflect on how an identified research need can shape the purpose and design of an international comparative study.
• Identify key methodological challenges in coordinating cross-national surveys, from instrument design and translation to sampling, data collection, data analysis and publication.
• Critically discuss how international comparisons reveal differences between countries, and explain how collaboration and participation in the SER community can support the continued development of research on young people’s relationships with S&T.

Glossary terms:
• ROSES
• Science education research
• Research design

Abstract TBD

Abstract
This seminar introduces international large-scale assessment research through two influential studies: the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS). We explore what these assessments measure, why their designs differ, and how researchers use carefully selected samples, questionnaires, and statistical methods to investigate patterns across countries and over time. These rich datasets offer opportunities to study student achievement, engagement with science, learning opportunities, educational inequality, and pathways into further science education.

At the same time, international comparisons require careful interpretation. We consider whose knowledge is represented, whether questions have comparable meanings across cultures, who may be excluded from the data, and why statistical relationships do not necessarily demonstrate causation. No advanced statistical background is required. The seminar introduces the main ideas in accessible terms and considers how large-scale evidence could contribute to astronomy education research.

By the end of this seminar, participants will be able to:
• Explain the main purposes and designs of international large-scale assessments
• Describe opportunities for using large-scale assessment data in education research
• Recognise key methodological challenges and interpret findings from these assessments critically

Glossary terms:
• International large-scale assessment (ILSA)
• Programme for International Student Assessment (PISA)
• Trends in International Mathematics and Science Study (TIMSS)
• Representative sample
• Plausible values
• Cross-national comparability
• Multilevel analysis