Meta-Framework de calidad: MLOps, NLP y analítica del aprendizaje
Barra lateral del artículo
Contenido principal del artículo
Resumen
Se presenta una propuesta teórica de meta-framework integrado de gestión de calidad educativa basado en Procesamiento de Lenguaje Natural (NLP) y Analítica del Aprendizaje (LA), orientado a posibilitar la evaluación escalable, trazable y auditable de las interacciones docente–estudiante en entornos digitales. La arquitectura conceptual articula NLP y LA para transformar el discurso académico y los metadatos conductuales en indicadores clave de desempeño (KPIs), alineados con las dimensiones pedagógica, técnica, social y psicológica de la calidad educativa. El modelo se estructura bajo una arquitectura cíclica de tres fases —Dirección, Operación y Optimización— que integra diseño estratégico, despliegue técnico y calibración continua. Asimismo, incorpora una estrategia híbrida de anotación que combina la escalabilidad de los Grandes Modelos de Lenguaje (LLMs) con validación humana periódica, garantizando control de sesgo, consistencia semántica y supervisión institucional. Su principal contribución reside en la incorporación explícita de un marco de Gobernanza Algorítmica (GA), sustentado en prácticas de MLOps, que operacionaliza la calidad mediante métricas auditables, protocolos de validación y mecanismos de monitoreo de deriva de datos y modelos. De esta forma, la propuesta trasciende enfoques basados en KPIs aislados y establece una arquitectura de segundo orden que integra evaluación pedagógica, control técnico y responsabilidad institucional, promoviendo transparencia, rendición de cuentas y mejora continua en sistemas educativos basados en IA.
Detalles del artículo
Esta obra está bajo una licencia internacional Creative Commons Atribución-NoComercial-SinDerivadas 4.0.
Los autores conservan los derechos de autor y ceden a la revista el derecho de la primera publicación y pueda editarlo, reproducirlo, distribuirlo, exhibirlo y comunicarlo en el país y en el extranjero mediante medios impresos y electrónicos. Asimismo, asumen el compromiso sobre cualquier litigio o reclamación relacionada con derechos de propiedad intelectual, exonerando de responsabilidad a la Editorial Tecnológica de Costa Rica. Además, se establece que los autores pueden realizar otros acuerdos contractuales independientes y adicionales para la distribución no exclusiva de la versión del artículo publicado en esta revista (p. ej., incluirlo en un repositorio institucional o publicarlo en un libro) siempre que indiquen claramente que el trabajo se publicó por primera vez en esta revista.
Citas
[1] K. Alalawi, R. Athauda, y R. Chiong, “An extended learning analytics framework integrating machine learning and pedagogical approaches for student performance prediction and intervention,”International Journal of Artificial Intelligence in Education, vol. 35, no. 3, pp. 1239–1287, 2025.
[2] A. Anwar et al., “Emotions matter: A systematic review and meta analysis of the detection and classification of students’ emotions in STEM during online learning,” Education Sciences, vol. 13, no. 9, p. 914, 2023.
[3] Axelos, ITIL Foundation: ITIL 4 Edition. TSO (The Stationery Office), 2019.
[4] CMMI Institute, CMMI for Development, Version 2.0. CMMI Institute, 2018.
[5] D. Demszky, J. Liu, H. C. Hill, D. Jurafsky, y C. Piech, “Can automated feedback improve teachers’ uptake of student ideas? Evidence from a randomized controlled trial in a large-scale online course,” Educational Evaluation and Policy Analysis, vol. 46, no. 3, pp. 483–505, 2024.
[6] D. Herrero Pascual, Ciclo de Vida de los Datos y MLOps: Herramientas, metodologías, puesta en producción y mantenimiento de proyectos de Data Science, 2022.
[7] ISACA, COBIT 2019 Framework: Introduction and Methodology. ISACA, 2019.
[8] ISO/IEC,Information technology—Governance of IT for the organization, ISO/IEC 38500:2015, 2015.
[9] ISO, Quality management systems — Fundamentals and vocabulary, ISO 9000:2015, 2015.
[10] S. Issar y A. Aneesh, “What is algorithmic governance?,” Sociology Compass, vol. 16, no. 1, e12955, 2022.
[11] K. F. Hew, C. Qiao, y Y. Tang, “Understanding student engagement in large-scale open online courses: A machine learning facilitated analysis of student’s reflections in 18 highly rated MOOCs,” Int. Rev. Res. Open Distrib. Learn., vol. 19, no. 3, 2018.
[12] R. Kaliisa, B. Rienties, A. I. Mørch, y A. Kluge, “Social learning analytics in computer-supported collaborative learning environments:A systematic review of empirical studies,”Computers and Education Open,vol.3, p.100073, 2022.
[13] C. Katzenbach y L. Ulbricht, “Algorithmic governance,” Internet Policy Review, vol. 8, no. 4, pp. 1–18, 2019.
[14] S. Knight y S. Buckingham Shum,”Theory and learning analytics,”Handbook of Learning Analytics, vol. 1, pp. 17–22, 2017.
[15] S. Knight y K. Littleton, “Dialogue as data in learning analytics for productive educational dialogue,” Journal of Learning Analytics, vol. 2, no. 3, pp. 111–143, 2015.
[16] J. O. Krugmann y J. Hartmann, “Sentiment analysis in the age of generative AI,” Customer Needs and Solutions, vol. 11, no. 3, 2024.
[17] A. Kucharavy, O. Plancherel, V. Mulder, A. Mermoud, y V. Lenders, Eds., Large Language Models in Cybersecurity: Threats, Exposure and Mitigation. Springer, 2024.
[18] V. Lukarov, “Scaling up Learning Analytics in Blended Learning Scenarios,” Ph.D. dissertation, RWTH Aachen Univ., 2019.
[19] Q. Lu et al., “Sentiment analysis: Comprehensive reviews, recent advances, and open challenges,” IEEE Trans. Neural Netw. Learn. Syst., 2023.
[20] R. Martinez-Maldonado et al., “Interactive surfaces and learning analytics: Data, orchestration aspects, pedagogical uses and challenges,” in Proc. 6th Int. Conf. on Learning Analytics & Knowledge, 2016, pp. 124–133.
[21] X. Ochoa et al., “Multimodal learning analytics—Rationale, process, examples, and direction,” en The Handbook of Learning Analytics, vol. 2, pp. 54–65, 2022.
[22] D. Owusu-Mensah, P. A. Sarfo, y G. A. Kusi, “Exploring the Impact of Artificial Intelligence on Customer Experience Personalization and Marketing Strategy Optimization in Digital Marketing: An Empirical Analysis,” Journal of Management and Informatics, vol. 4, no. 2, pp. 822-843, 2025.
[23] S. Rüdian, J. Podelo, J. Kužílek, y N. Pinkwart, “Feedback on feedback: Student’s perceptions for feedback from teachers and few-shot LLMs,” in Proc. 15th Int. Learning Analytics and Knowledge Conf., Dublin, Ireland: ACM, Mar. 2025, pp. 82–92.
[24] M. Sokovic, D. Pavletic, y K. K. Pipan,”Quality improvement methodologies–PDCA cycle, RADAR matrix, DMAIC and DFSS,” Journal of achievements in materials and manufacturing engineering,vol. 43,no. 1, pp. 476-483, 2010.
[25] B. T. Wong y K. C. Li, “Learning analytics intervention: A review of case studies,” en 2018 International Symposium on Educational Technology (ISET), IEEE, 2018, pp. 178-182.
[26] B. Yang, “Learning analytics through machine learning and natural language processing,” 2023.
[27] L. Yan, L. Zhao, D. Gašević, y R. Martinez-Maldonado, “Scalability, sustainability, and ethicality of multimodal learning analytics,” in Proc. LAK’22: 12th Int. Learning Analytics and Knowledge Conf., 2022, pp. 13–23.
[28] L. Yan et al., “Practical and ethical challenges of large language models in education: A systematic scoping review,” British Journal of Educational Technology, vol. 55, no. 1, pp. 90–112, 2024.
[29] U. Hanson, C. A. Okonkwo, y C. U. Orakwe, “Implementing AI-enhanced learning analytics to improve educational outcomes using psychological insights,” Journal of Educational Technology Innovations, 2024.
[30] A. Alam y A. Mohanty, “Business models, business strategies, and innovations in EdTech companies: Integration of learning analytics and artificial intelligence in higher education,” in Proc. IEEE 6th Conf. on Information and Communication Technology (CICT), 2022, pp. 1–6.
[31] Dutta, D. (2022). Edtech with Perspective of Business Analysis. perception, 6(03).
[32] B. Williamson, “Meta-edtech,” Learning, Media and Technology, vol. 46, no. 1, pp. 1–5, 2021, doi: 10.1080/17439884.2021.1876089.
[33] M.A.Chatti et al.,“Learning analytics:Challenges and future research directions,”E-Learning and Education, no.10, 2014.
[34] R. Pelánek, “Learning analytics challenges: Trade-offs, methodology, scalability,” in Proc. 10th Int. Conf. on Learning Analytics & Knowledge, 2020, pp. 554–558.
[35] C. Herodotou et al., “The scalable implementation of predictive learning analytics at a distance learning university: Insights from a longitudinal case study,” The Internet and Higher Education, vol. 45, p. 100725, 2020.
[36] A. Wilson, C. Watson, T. L. Thompson, V. Drew, y S. Doyle, “Learning analytics: Challenges and limitations,” Teaching in Higher Education, vol. 22, no. 8, pp. 991–1007, 2017.
[37] S. Dyer, Feedback by Design: Exploring Interactions Between Novice Programmers and Automated Teaching Assistants, Ph.D. dissertation, Univ. of Auckland, 2025.
[38] J. C. Debuse, M. Lawley, y R. Shibl, “Educators’ perceptions of automated feedback systems,” Australasian Journal of Educational Technology, vol. 24, no. 4, 2008.
[39] UNESCO, A Global Conversation on the Ethics of AI for Children, UNESCO, 2024. [Online]. Available: https://media.unesco.org/sites/default/files/webform/ed3002/390204eng.pdf
[40] K. Vayadande, P. A. Bailke, L. S. Khedekar, R. Kumar, y V. R. Dange, “A review on text analysis using NLP,” in How Machine Learning is Innovating Today’s World: A Concise Technical Guide, pp. 13–23, 2024.
[41] R. Ranganath, A. K. Fetterman, A. N. Gesselman, y E. J. Johnson, “Using natural language processing to analyse text data in behavioural science,” Nature Human Behaviour, 2024.
[42] S. Rathje et al., “GPT is an effective tool for multilingual psychological text analysis,” Proc. Natl. Acad. Sci. USA, vol. 121, e2308950121, 2024.
[43] F. Gilardi, M. Alizadeh, y M. Kubli, “ChatGPT outperforms crowd workers for text-annotation tasks,” Proc. Natl. Acad. Sci. USA, vol. 120, e2305016120, 2023.
[44] A. S. Ziabari et al., “Reinforced multiple instance selection for speaker attribute prediction,” in Proc. Conf. North American Chapter of the Assoc. Computational Linguistics: Human Language Technologies, 2024, pp. 3307–3321.
[45] M. Kurday y G. Vladova, “Learning analytics and educational data mining applications: Bibliometric and ChatGPT-based analysis of research publications from 2014 to 2023,” Journal of Computers in Education, pp. 1–36, 2025.
[46] A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008, 2017.
[47] C. Romero y S. Ventura, “Educational data mining: A review of the state of the art,” IEEE Trans. Syst., Man, Cybern., Part C (Applications and Reviews), vol. 40, no. 6, pp. 601–618, 2010.
[48] V. Kagklis, A. Karatrantou, M. Tantoula, C. T. Panagiotakopoulos, y V. S. Verykios, «A learning analytics methodology for detecting sentiment in student fora: A Case Study in Distance Education.», European Journal of Open, Distance and E-learning, vol. 18, n.o 2, pp. 74-94, 2015.
[49] C. Lang, G. Siemens, A. Wise, y D. Gasevic, «Handbook of learning analytics», 2017.
[50] V. Kovanovic, S. Joksimovic, D. Gašević, y M. Hatala, “What is the source of social capital? The association between social network position and social presence in communities of inquiry,” in CEUR Workshop Proc., CEUR-WS, pp. 21–28, 2014.
[51] J. P. Bernius, S. Krusche, y B. Bruegge, “Machine learning based feedback on textual student answers in large courses,” Computers and Education: Artificial Intelligence, vol. 3, p. 100081, 2022.
[52] M. Kumar, L. Khan, y H.-T. Chang, “Evolving techniques in sentiment analysis: A comprehensive review,” PeerJ Comput. Sci., vol. 11, p. e2592, 2025.
[53] Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.
[54] Long, P.,& Siemens, G.(2011).Penetrating the fog:Analytics in learning and education.EDUCAUSE Review,46(5), 31–40.
[55] Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In Learning analytics (pp. 61–75). Springer.
[56] OECD (2019). OECD Principles on Artificial Intelligence.
[57] European Commission (2019). Ethics Guidelines for Trustworthy AI.
[58] NIST (2023). AI Risk Management Framework (AI RMF 1.0).
[59] Sculley, D. et al. (2015). Hidden technical debt in machine learning systems. NeurIPS.
[60] Breck, E. et al. (2017). The ML Test Score: A rubric for ML production readiness. Google Research.
[61] Amershi, S. et al. (2019). Software engineering for machine learning: A case study. ICSE.
[62] IBM (2020). AI Governance Framework.