Construction and validation of an instrument for evaluating the quality of university servicelearning projects using the Delphi method

Construcción y validación de un instrumento para la evaluación de la calidad de proyectos de aprendizaje-servicio universitario a través del método Delphi

Elena López-de-Arana Prado, Universidad Autónoma de Madrid
L. Fernando Martínez Muñoz, Universidad Autónoma de Madrid
María Teresa Calle-Molina, Universidad Autónoma de Madrid
Raquel Aguado Gómez, Universidad Autónoma de Madrid
M.ª Luisa Santor-Pastor, Universidad Autónoma de Madrid

DOI

10.22550/REP81-2-2023-07

Abstract

Service-learning has spread significantly in higher education in recent decades. Its effects in the academic field (students and teachers) and in the community (disadvantaged groups at risk of social exclusion and socio-educational partners) are backed by research. However, few works have considered the evaluation of these projects, and there are few instruments available for guiding their development and for assessing their quality. The aim of this study is to develop criteria to evaluate university service-learning projects. To do so, we used the Delphi method with three rounds of expert consultation. The result is a university service-learning, indicator matrix with 9 dimensions and 43 indicators. We conclude that, as well as evaluating the quality of service-learning projects, this instrument could also be valid for validating social innovation from the educational sphere. The principal limitations to overcome are the still-existing welfare perspective and difficulties with involving the recipients of the service.

Referencias | References

Alonso-Sáez, I., Arandia, M., Martínez, I., Martínez, B., & Gezuraga, M. (2013). El Aprendizaje-Servicio en la innovación universitaria. Una experiencia realizada en la formación de educadoras y educadores sociales [Service-learning in university innovation an experience conducted in the training of social educators]. RIEJS, Revista Internacional de Educación para la Justicia Social, 2 (2), 195-216. http://www.rinace.net/riejs/numeros/vol2-num2/art10.pdf

Álvarez-Castillo, J. L., Martínez-Usarralde, M. J., González González, H., & Buenestado Fernández, M. (2017). El aprendizaje-servicio en la formación del profesorado de las universidades españolas [Service-learning in teacher Çtraining in Spanish universities]. Revista Española de Pedagogía, 75 (267), 199-217. 10.22550/REP75-2-2017-02

Ash, S. L., Clayton, P. H., & Atkinson, M. P. (2005). Integrating reflection and assessment to capture and improve student learning. Michigan Journal of Community Service Learning, 49-60. http://hdl.handle.net/2027/spo.3239521.0011.204

Batlle, R. (2020). Aprendizaje-servicio. Compromiso social en acción [Service-learning. Social commitment in action]. Santillana.

Billig, S. H., & Waterman, A. S. (2014). Studying service-learning: Innovations in education research methodology. Routledge.

Bringle, R. G., & Clayton, P. H. (2021). Civic learning: A sine qua non of service learning. Front. Educ, 6, 606443. http://doi.org/10.3389/feduc.2021.606443

Carrington, S., Mercer, K. L., Iyer, R., & Selva, R. (2015). The impact of transformative learning in a critical service-learning program on teacher development: building a foundation for inclusive teaching. Reflective Practice, 16 (1), 61-72. http://doi.org/10.1080/14623943.2014.969696

Case, L., Schram, B., Jung, J., Leung, W., & Yun, J. (2020). A meta-analysis of the effect of adapted physical activity service-learning programs on college student attitudes toward people with disabilities. Disability and Rehabilitation, 43 (21), 2990-3002. http://doi.org/10.1080/09638288.2020.1727575

Chambers D., & Lavery, S. (2022). International service learning: benefits, challenges and experiences of pre-service teachers. Asia Pacific Journal of Teacher Education, 1-17. http://doi.org/10.1080/1359866X.2022.2050355

Chiva-Bartoll, O., & Fernández-Río, J. (2021). Advocating for service-learning as a pedagogical model in Physical Education: Towards an activist and transformative approach. Physical Education and Sport Pedagogy, 27 (5), 545- 558. http://doi.org/10.1080/17408989.2021.1911981

Choo, J., Kong, T. Y., Ong, F., Shiuan, T. S., Nair, S., Ong, J., & Chan A. (2019). What works in service-learning? Achieving civic outcomes, academic connection, career preparation and personal growth in students at Ngee Ann Polytechnic. Michigan Journal of Community Service Learning, 25 (2), 95-132. http://doi.org/10.3998/mjcsloa.3239521.0025.208

Clark-Taylor, A. (2017). Developing critical consciousness and social justice self-efficacy: Lessons from feminist community engagement student narratives. Journal of Higher Education Outreach and Engagement, 21 (4), 81-116. http://bit.ly/2Xh5Y7h

Conner, J., & Erickson, J. (2017). When does service-learning work? Contact theory and service-learning courses in higher education. Michigan Journal of Community Service Learning, 23, 53-65. http://doi.org/10.3998/mjcsloa.3239521.0023.204

Darby, A., & Willingham, L. (2022). Faculty motivation in service-learning based on expectancy x value theory. Journal of Experiential Education, 45 (3). http://doi.org/10.1177/10538259211058292

De Castro, A., Amaris, M. C., & Guerra, D. (2017). A manera de conclusión: experiencia del aprendizaje-servicio en la Universidad del Norte [By way of conclusion: Service-learning experience at Universidad del Norte]. In A. de Castro, & E. Domingues (Eds.), Transformar para Educar 4. Aprendizaje-Servicio (pp. 241- 264). Universidad del Norte.

Deeley, S. J. (2016). El aprendizaje-servicio en educación superior: Teoría, práctica y perspectiva crítica [Service-learning in higher education: Theory, practice and critical perspective]. Narcea.

Dolgon, C., Mitchell, T. D., & Eatman, T. K. (2017). The Cambridge handbook of service learning and community engagement. Cambridge University Press.

Escofet, A., & Rubio, L. (2017). El valor de la reflexión en el aprendizaje-servicio [The value of reflection in service-learning]. In L. Rubio & A. Escofet (Eds.), Aprendizaje-Servicio (ApS): Claves para su desarrollo en la universidad (pp. 83-96). Octaedro.

European Union (2013). Guide to social innovation. http://doi.org/10.2776/72046

Furco, A., & Norvell, K. (2019). What is service-learning? Making sense of the pedagogy and practice. In P. Aramburuzabala, L. McIlrath, & H. Opazo (Eds.), Embedding service-learning in European higher education. Developing a culture of civic engagement (pp. 13-36). Routledge.

Galván, C., Meaney, K., & Gray, V. (2018). Examining the reciprocal nature of service-learning for underserved students and preservice teachers. Journal of Teaching in Physical Education, 37 (4), 363- 372. http://doi.org/10.1123/jtpe.2018-0051

García-Gutiérrez, J., & Ruiz-Corbella, M. (2022). La idea de universidad desde un enfoque humanista: la contribución del aprendizaje-servicio como filosofía de la educación superior [The idea of university from a humanist approach: The contribution of service-learning as a philosophy of higher education]. Teoría de la Educación. Revista Interuniversitaria, 34 (2), 159- 176. http://doi.org/10.14201/teri.27887

García-Romero, D., & Lalueza, J. L. (2019). Procesos de aprendizaje e identidad en aprendizaje-servicio universitario: una revisión teórica [Learning and identity processes in university service-learning: A theoretical review]. Educación XXI, 22 (2), 45- 68. 10.5944/educxx1.22716

George, C. E., & Trujillo, L. (2018). Aplicación del método Delphi modificado para la validación de un cuestionario de incorporación de las TIC en la práctica docente [Application of the modified Delphi method for the validation of a questionnaire on the incorporation of ICT in teaching practice]. Revista Iberoamericana de Evaluación Educativa, 11 (1), 113-135. http://doi.org/10.15366/riee2018.11.1.007

GREM. (2015). Rúbrica para la autoevaluación y la mejora de los proyectos de aprendizaje-servicio [Rubric for self-assessment and improvement of service-learning projects]. Fundació Jaume Bofill. http://bit.ly/33FhY2F

Liesa-Orus, M., Cored, S., Latorre, C., Vázquez, S., & Revilla, A. (2019). El impacto de la mentoría entre profesores universitarios en programas de aprendizaje-servicio [The impact of mentoring among university professors in service-learning programmes]. Universidad de Zaragoza.

López-de-Arana Prado, E., Aramburuzabala Higuera, P., & Opazo Carvajal, H. (2020). Diseño y validación de un cuestionario para la autoevaluación de experiencias de aprendizaje-servicio universitario [Design and validation of a questionnaire for selfassessment of university service-learning experiences]. Educación XX1, 23 (1), 319-347. 10.5944/educXX1.23834

López-Gómez, E. (2018). El método Delphi en la investigación actual en educación: una revisión teórica y metodológica [The Delphi method in current educational research: A theoretical and methodological review]. Educación XX1, 21 (1), 17-40. http://doi.org/10.5944/educxx1.20169

Lorenzo, M., Ferraces-Otero, M. J., Pérez, C., & Naval, C. (2019). El profesorado universitario ante el aprendizaje-servicio: variables explicativas [The professor in the service-learning: Explanatory variables]. Revista de Educación, 386, 37-61. http://doi.org/10.4438/1988-592X-RE-2019-386-426

Lucas, S. (2021). Aprendizaje-servicio en la universidad. Desarrollo de proyectos emprendedores socialmente responsables con la comunidad [Service-learning at the university. Development of socially responsible entrepreneurial projects with the community]. Dykinson. https://www.dykinson.com/libros/aprendizaje-servicio-en-la-universidad/9788413779133

MacPhail, A., & Sohun, R. (2019). Interrogating the enactment of a service-learning course in a physical education teacher education programme: Less is more? European Physical Education Review, 25 (3), 876-892. http://doi.org/10.1177/1356336X18783922

Maravé-Vivas, M., Gil-Gómez, J., Valverde-Esteve, T., Salvador-García, C., & Chiva-Bartoll, O. (2022). A longitudinal study of the effects of servicelearning on physical education teacher education students. Frontiers in Psychology, 13. http://doi.org/10.3389/fpsyg.2022.787346

Martínez-Usarralde, M. J., Gil-Salom, D., & Macías-Mendoza, D. (2019). Revisión sistemática de responsabilidad social universitaria y aprendizaje servicio. Análisis para su institucionalización [Systematic review of university social responsibility and service learning. Analysis for institutionalization]. Revista Mexicana de Investigación Educativa , 24 (80), 149- 172. http://www.scielo.org.mx/scielo.php?script=sci_arttext&pid=S1405-66662019000100149&lng=es&nrm=iso

McIlrath, L., Aramburuzabala, P., & Opazo, H. (2019). Introduction. In P. Aramburuzabala, L. McIlrath, & H. Opazo (Eds.), Embedding service learning in European higher education: Developing a culture of civic engagement (pp. 1-9). Routledge.

Menon, S., & Suresh, M. (2020). Synergizing education, research, campus operations, and community engagements towards sustainability in higher education: A literature review. International Journal of Sustainability in Higher Education, 21 (5), 1015- 1051. http://doi.org/10.1108/IJSHE-03-2020-0089

Miller, M. (2012). The role of service-learning to promote early childhood physical education while examining its influence upon the vocational call to teach. Physical Education and Sport Pedagogy, 17 (1), 61-77. http://doi.org/10.1080/17408981003712810

Mtawa, N., & Wilson-Strydom, M. (2018). Community service learning: Pedagogy at the interface of poverty, inequality and privilege. Journal of Human Development and Capabilities, 19 (2), 249-265. http://doi.org/10.1080/19452829.2018.1448370

Patrascu, C. (2022). ‘Scaling up’ the social impact of innovation in services - significant findings and best practice examples from specialty research. LUMEN Proceedings, 17, 552-558. http://doi.org/10.18662/wlc2021/56

Pino, E., Rojas, W., Tamayo, M., & Castro, J. (2016). Programa de aprendizaje-servicio en la educación superior [Service-learning program in higher education]. Revista Ciencias Pe-dagógicas e Innovación, 4 (1), 101-111. http://doi.org/10.26423/rcpi.v4i1.122

Puig-Rovira, J. M. (2022). Aprendizaje-servicio, cambio de paradigma y revolución educativa [Service-learning, paradigm shift and educational revolution]. RIDAS, Revista Iberoameri-cana de Aprendizaje-Servicio, 14, 12-35. 10.1344/RIDAS2022.14.2

Redondo-Corcobado, P., & Fuentes, J. L. (2020). La investigación sobre el aprendizaje-servicio en la producción científica española: una revisión sistemática [Research on service-learning in the Spanish scientific production: A systematic review]. Revista Complutense de Educación, 31 (1), 69-82. 10.5209/rced.61836

Ruiz-Corbella, M., & García-Gutiérrez, J. (2019). Aprendizaje-servicio: los retos de la evaluación [Service-learning: the challenges of assessment]. Narcea.

Salam, M., Awang Iskandar, D. N., Ibrahim, D. H. A., & Farooq, M. S. (2019). Service learning in higher education: A systematic literature review. Asia Pacific Education Review, 20, 573- 593. http://doi.org/10.1007/s12564-019-09580-6 Sánchez-Taraza, L., & Ferrández-Berrueco, R. (2022). Aplicación del método Delphi en el diseño de un marco para el aprendizaje por competencias [Application of the Delphi method in the esigno f a competency-based education framework]. Revista de Investigación Educativa, 40 (1), 219-235. http://doi.org/10.6018/rie.463611

Sanders, M. J., Vans, T., & McGeary, S. (2015). Analyzing reflections in service learning to promote personal growth and community self-efficacy. Journal of Experiential Education, 39, 73-88. http://doi.org/10.1177/1053825915608872

Santos-Pastor, M. L., Ruiz-Montero, P. J., Chiva- Bartoll, O., & Martínez-Muñoz, L. F. (2021a). Guía práctica para el diseño e intervención de programas de aprendizaje-servicio en actividad física y deporte [Practical guide for the design and intervention of service-learning programmes in physical activity and sport]. Junta de Andalucía. http://digibug.ugr.es/handle/10481/70964

Santos-Pastor, M. L., Martínez-Muñoz, L. F., Garoz- Puerta, I., & García-Rico, L. (2021b). La reflexión en el aprendizaje-servicio universitario en actividad física y deporte. Claves para el aprendizaje personal, académico y profesional [Reflection in university service-learning in physical activity and sport. Keys to personal, academic and professional Learning]. Contextos Educativos, 27 (1), 9-29. 10.18172/con.4574

Tryon, E., & Stoecker, R. (2008). The unheard voices: Community organizations and service-learning. Journal of Higher Education Outreach and Engagement, 12 (3), 47-60. http://cutt.ly/VTkU4mH

Ward, K., & Wolf-Wendel, L. (2000). Community-centered service learning: Moving from doing for to doing with. American Behavioral Scientist, 43, 767-780. http://doi.org/10.1177/00027640021955586

Author Biography

Elena López-de-Arana Prado. Associate Professor at the Universidad Autónoma de Madrid. PhD in Humanities and Education in the Entities of the Future from the Universidad de Mondragon. Her lines of research focus on improvement processes in educational centres, teacher training (practicum, ICTs, etc.), social justice and service-learning. She has led several projects on university service-learning.

https://orcid.org/0000-0002-6962-5469

Fernando Martínez-Muñoz. Associate Professor at the Department of Physical Education, Sport and Human Motricity of the Universidad Autónoma de Madrid. His field of research is related to formative assessment and active methodologies in Physical Education, with special interest in service-learning and the development of physical-sports activities in the natural environment. He is currently a member of RIADIS, REEFNAT and REFyCE.

https://orcid.org/0000-0001-5209-7527

María Teresa Calle-Molina. Assistant Professor at the Department of Physical Education, Sport and Human Motricity at the Universidad Autónoma de Madrid. Her lines of research focus on the history of sport and Olympism, physical activity and people with intellectual disabilities and service-learning. Member of the National Research Network on Service- Learning in Physical Activity and Sport for Social Inclusion (RIADIS).

https://orcid.org/0000-0001-7877-8283

Raquel Aguado-Gómez. PhD in Physical Activity and Sport Sciences. She works in the Department of Physical Education, Sport and Human Motricity of the Faculty of Teacher Training and Education of the Universidad Autónoma de Madrid. She is a member of the GEDAF Research Group, of the Research Network on Service-Learning in Physical Activity and Sport for Social Inclusion, and of the team of the UNESCO Chair in Education for Social Justice.

https://orcid.org/0000-0002-7943-7744

Mª Luisa Santos-Pastor. Lecturer at the Universidad Autónoma de Madrid. PhD in Physical Education from the Universidad de Valladolid. Her field of research is related to university service-learning, formative assessment and active methodologies, as well as in the field of physical activities in the natural environment. She has supervised six doctoral theses. She coordinates the Research Net-work on Service-Learning in Physical Activity and Sport for Social Inclusion of the Consejo Superior de Deportes (Call 2019, 2020 and 2022).

https://orcid.org/0000-0002-4985-0810

Citación recomendada | Recommended citation

Construcción y validación de un instrumento para la evaluación de la calidad de proyectos de aprendizaje-servicio universitario a través del método Delphi | Construction and validation of an instrument for evaluating the quality of university service-learning projects using the Delphi method. Revista Española de Pedagogía, 81 (285), 381-402. 10.22550/REP81-2-2023-07