Skip to main content
Alliance 2024 Annual Conference Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Conference Presentation

Measurement and Evaluation

Beyond the Numbers: The Value and Benefits of a Mixed-Methods Design in CE (Level 3)

Tuesday, February 6, 2024
2:15 PM - 3:15 PM CST
Location: Oak Alley
Session Description:
As the field of education research continues to evolve, it is important to evaluate the applicability of modern measurement and evaluation approaches in the context of current CME programs and activities. Traditionally, CME program outcome measurement heavily relies on quantitative analysis, employing methods such as pre- and post-assessments and surveys. However, qualitative methods may also have an important role in further understanding participating learners, their social frameworks, values and motivations. Qualitative outcomes research diverges from quantitative research by providing a descriptive account of the complexity and range of occurrences, instead of focusing on frequency, incidence, and magnitude. 

Recognizing the respective strengths of quantitative and qualitative methods, mixed methods approach is increasingly recognized as a valuable outcomes design. By integrating results from both research approaches, mixed methods can corroborate findings and enhance insights. The proposed session aims to provide an in-depth review of quantitative methods, qualitative methods, and the principles and practices specific to mixed methods approaches. The specific topic areas that will be discussed include:

The role of quantitative analysis 

Quantitative analysis enables researchers to evaluate measurable outcomes such as knowledge gains, skill acquisition, and behavioral changes. Statistical models and analyses are essential in quantifying these outcomes and ensuring their reliability and validity. To accommodate various types of data and research questions, both traditional and non-traditional statistical models should be considered. Inferential statistics like t-tests and regression analyses, for example, can determine the statistical significance of differences in outcomes, whereas multilevel modeling can account for nested data structures in CME programs with multiple levels of participants (e.g., individuals within hospitals or healthcare systems). Furthermore, modern statistical approaches, such as machine learning algorithms and network analysis, may reveal complex patterns and associations in CME outcomes.

Qualitative analysis provides depth and richness to outcomes studies by uncovering the stories, experiences, and perspectives of the learners. Data collection methods may include in-depth interviews, focus groups, observation and document review. The constant comparative method is a commonly used analytic approach, where data are systemically reviewed line by line, assigning codes associated with key concepts, and refining code dimensions throughout the process. Qualitative data analysis can yield classifications, themes, theories and sentiments, all of which contribute contextual information on factors influencing educational outcomes. By delving into participants' narratives and lived experiences, qualitative analysis reveals insights that quantitative data alone may miss. The qualitative lens can facilitate a nuanced understanding of the true educational needs and gaps.

Learning Objectives:

  • Understand the benefits of using a mixed-method approach for enhancing outcomes analysis
  • Design CE programs that include both qualitative and quantitative data collection components
  • Utilize qualitative analysis techniques to enhance outcome reporting in CME programs