Towards an Outcome-based Discovery and Filtering of MOOCs using moocrank Israel Gutiérrez-Rojas, Derick Leony, Carlos Alario-Hoyos, Mar Pérez-Sanagustín and Carlos Delgado-Kloos needs information from users who previously completed the courses and indicated the achieved learning outcomes. Until a critical mass of courses is annotated, the ranked list is not very accurate. But, precisely due to the fact that the filtering is not too accurate for the first users, the system has not been used as widely as expected. That problem is known as the bootstrapping (or cold start) problem. In order to solve that situation, we are developing an automatic annotation tool. This tool annotates the courses with their intended learning outcomes based on existing information about MOOCs, like description, outline, background information, etc. The automatic annotation system makes use of a natural language processing algorithm in order to identify what outcomes are most likely provided by the course. Still, the automatic annotation tool can lead to mistakes. For that reason, learners will be able to amend the automatic annotations, and therefore the relationships between MOOCs and learning outcomes will evolve, driven by the moocrank users. In the case that users provide divergent annotations for the LOs of a course, the automatic annotations will serve as the judgment to decide the final LOs to be assigned. Once we get enough activity in the platform, we will be able to perform a better evaluation of its performance using typical metrics of information retrieval systems, such as precision and recall. On this line, one of the next steps is to thoroughly explore the literature in the recommenders’ educational systems domain so as to incorporate a recommender module based on similarity measures already tested and evaluated in this field. The next steps in the short term include leveraging moocrank with more social features to enable users to rank the courses, provide comments about their experience to help future participants, provide feedback about the teachers, the contents, the learning pace, etc. With that social information, learners will have more data to make more informed decisions about what course to take next. Another future line is to improve the usability of the moocrank site, since we detected that the selection of learning outcomes by learners is a somewhat cumbersome task, due to large list of learning outcomes provided by CSC2013. Furthermore, we plan to update other components in the user interface like the recommendation screen, so that moocrank presents the courses ordered by relevance, although this information is not explicitly reported to the user. Further plans also include collecting the dependencies between courses (what courses are pre-requisite for others) from the users themselves and other sources of information. With that information, we would be able to offer learning paths for the users to follow, aligned to the intended learning outcomes, and moocrank will not recommend advanced courses to novice learners. Finally, the last future line we are going to explore is the application of moocrank for learning in the workplace. We think that the CSC2013 taxonomy could be used by employers to indicate the learning outcomes that a company wants for its employees. Following this idea, employees will be recommended which online courses to follow; courses that are also aligned with the training expectations of their employer. This could be useful to complement the training used in the workplace, by recommending courses aligned to company objectives. Furthermore, the application would aggregate the learning outcomes achieved during an employee’s career, offering information on how the worker has updated their knowledge and skills to adapt to new company needs. Acknowledgements This work has been funded by the Spanish Ministry of Economy and Competitiveness Project TIN2011- 28308-C03-01, the Regional Government of Madrid project S2009/TIC-1650, and the postdoctoral fellowship Alianza 4 Universidades. Prof. Carlos Delgado-Kloos wishes to acknowledge support from Fundación Caja Madrid to visit Harvard University and MIT in the academic year 2012/13. He also thanks numerous people from MIT, Harvard and edX for fruitful conversations carried out during this research stay. References Boyatt, R. & Sinclair, J. (2012). Navigating the Educational Cloud. Workshop on Learning Technology for Education in Cloud (LTEC’12). Advances in Intelligent Systems and Computing, 173, 179–191. Springer Brusilovsky, P. & Millán, E. (2007). User models for adaptive hypermedia and adaptive educational systems. In The adaptive web, 3-53. Springer-Verlag. Cooper, S. and Sahami, M. (2013). Reflections on Stanford’s MOOCs, Communications of the ACM, 56, 2, 28-30. Crespo, R.M., Najjar, J., Derntl, M., Leony, D., Neumann, S., Oberhuemer, P., Totschnig, M., Simon, B., Gutiérrez, I. & Delgado Kloos, C. (2010). Aligning assessment with learning outcomes in outcome-based education. Proceedings of the IEEE Engineering Education Conference (EDUCON 2010), 1239-1246. IEEE. Draschler, H., Hummel, H.G. & Koper, R. (2008). Personal recommender systems for learners in lifelong learning networks: the requirements, techniques and model. International Journal of Learning Technology, 3, 4, 404-423. Highsmith, J. & Cockburn, A. (2001). Agile software development: the business innovation. IEEE Computer Society, 34, 9, 120-127. Kalz, M., Specht, M., Nadolski, R., Bastiaens, Y., Leirs, N. & Pawlowski, J. (2010). OpenScout: Competence based management education with Research Track | 56
Towards an Outcome-based Discovery and Filtering of MOOCs using moocrank Israel Gutiérrez-Rojas, Derick Leony, Carlos Alario-Hoyos, Mar Pérez-Sanagustín and Carlos Delgado-Kloos community-improved open educational resources. Proceedings of the 17th EDINEB Conference: Crossing Borders in Education and workbased learning, 137-146. FEBA ERD Press. Manouselis, N., Draschler, H., Vuorikari, R., Hummel, H. & Koper, R. (2011). Recommender Systems in Technology Enhanced Learning. In Recommender Systems Handbook, 387–415. Springer. Retrieved September 2013 from http://www.springerlink.com/index/10.1007/978-0-387-85820-3. Najjar, J., Grant, S., Simon, B., Derntl, M., Klobučar, T., Crespo-García, R.M., Delgado-Kloos, C., Vu, A., Nguyen-Ngoc, A.V., Pawlowski, J., Hoel, T. & Oberhuemer, P. (2009). ISURE: Model for describing learning needs and learning opportunities taking context ontology modelling into account. Deliverable D2.2 from ICOPER. Retrieved September 2013 from http://www.icoper.org/results/deliverables/D2-2. Pappano, L. (2012). The year of the MOOC. The New York Times, Retrieved September 2013 from: http://www.nytimes.com/2012/11/04/ education/edlife/massive-open-online-courses-are-multiplying-at-arapid-pace.html. Paquette, G. (2007). An ontology and a software framework for competency modeling and management. Educational Technology & Society, 10, 3, 1-21. Sahami M., Roach, S., Danyluk, A., Fincher, S., Fisher, K., Grossman, D., Hawthorne, B., Katz, R., LeBlanc, R., Reed, D., Cuadros-Vargas, E., Dodge, R., France, R., Kumar, A., Robinson, B., Seker, R. & Thompson, A. (2013). Computer Science Curricula 2013 Ironman Draft (Version 1.0). The Joint Task Force on Computing Curricula Association for Computing Machinery IEEE-Computer Society, Retrieved September 2013 from: http://ai.stanford.edu/users/sahami/CS2013/ironman-draft/cs2013-ironman-v1.0.pdf. Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M., Draschler, H., Bosnic, I. & Duval, E. (2012). Context-Aware Recommender Systems for Learning: A Survey and Future Challenges. IEEE Transactions on Learning Technologies, 5, 4, 318–335. Wang, F. & Hannafin, M.J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53, 4, 5-23. Research Track | 57
Change language