We demonstrate the feasibility of Finite State Machine (FSM) logic to design adaptively scaffolded activities, presenting early work integrating adaptive learning into a learning tool in widespread use globally. We describe how integrating FSM logic with existing assessment architecture enables us to extend from direct measurement to scaffolding and feedback interventions on a spectrum of timescales from 1-second to several hours. Four prototypes are shared, demonstrating how this FSM logic affords design across differing learning contexts. Implications for design of efficiency and empowerment at scale, potential for content co-creation, and transformation of learning are discussed.
{"title":"Adaptive Learning using Finite State Machine Logic","authors":"M. Waterman, D. C. Frezzo, Michael X. Wang","doi":"10.1145/3386527.3406720","DOIUrl":"https://doi.org/10.1145/3386527.3406720","url":null,"abstract":"We demonstrate the feasibility of Finite State Machine (FSM) logic to design adaptively scaffolded activities, presenting early work integrating adaptive learning into a learning tool in widespread use globally. We describe how integrating FSM logic with existing assessment architecture enables us to extend from direct measurement to scaffolding and feedback interventions on a spectrum of timescales from 1-second to several hours. Four prototypes are shared, demonstrating how this FSM logic affords design across differing learning contexts. Implications for design of efficiency and empowerment at scale, potential for content co-creation, and transformation of learning are discussed.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82928200","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Retrieval practice (also known as testing effect or test-enhanced learning) is a well-studied and established technique for improving the retention of knowledge. Many previous works have confirmed the benefits of retrieval practice in laboratory experiments involving the memorization of words or facts. In this study, we build on these works and analyze retrieval practice in an intelligent tutoring system. Using a large data set composed of the actions of almost 4 million students studying math and chemistry, we look at the possible benefits of retrieval practice in the ALEKS adaptive learning and assessment system. We compare two different types of retrieval practice---one involving the assessment of learned material, and another involving the learning of closely related content that builds on the learned material---leveraging the scale of the available data to control for several confounding variables. Finally, we look at the timing of retrieval practice within the system and the possible effect it has on forgetting. The results indicate that a delay in retrieval practice is associated with better retention and that, while being assessed on learned material is beneficial, the learning of closely related content is associated with an even higher rate of retention.
{"title":"Studying Retrieval Practice in an Intelligent Tutoring System","authors":"Jeffrey Matayoshi, Hasan Uzun, Eric Cosyn","doi":"10.1145/3386527.3405927","DOIUrl":"https://doi.org/10.1145/3386527.3405927","url":null,"abstract":"Retrieval practice (also known as testing effect or test-enhanced learning) is a well-studied and established technique for improving the retention of knowledge. Many previous works have confirmed the benefits of retrieval practice in laboratory experiments involving the memorization of words or facts. In this study, we build on these works and analyze retrieval practice in an intelligent tutoring system. Using a large data set composed of the actions of almost 4 million students studying math and chemistry, we look at the possible benefits of retrieval practice in the ALEKS adaptive learning and assessment system. We compare two different types of retrieval practice---one involving the assessment of learned material, and another involving the learning of closely related content that builds on the learned material---leveraging the scale of the available data to control for several confounding variables. Finally, we look at the timing of retrieval practice within the system and the possible effect it has on forgetting. The results indicate that a delay in retrieval practice is associated with better retention and that, while being assessed on learned material is beneficial, the learning of closely related content is associated with an even higher rate of retention.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"16 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88241454","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mixed reality creates exciting new opportunities for computer-aided language learning. By combining markerless tracking technology with a user's geolocation, software systems can dynamically locate and generate personalized interactive language practice exercises. The Locabulary mobile app uses a combination of markerless tracking and metadata from the user's location information to construct that utilize the learner's physical surroundings to provide unique and relevant content. Additionally, Locabulary employs context-aware spaced repetition to help language learners develop mastery over the material in the exercises it creates.
{"title":"Locabulary","authors":"Zachary Phillips-Gary","doi":"10.1145/3386527.3406762","DOIUrl":"https://doi.org/10.1145/3386527.3406762","url":null,"abstract":"Mixed reality creates exciting new opportunities for computer-aided language learning. By combining markerless tracking technology with a user's geolocation, software systems can dynamically locate and generate personalized interactive language practice exercises. The Locabulary mobile app uses a combination of markerless tracking and metadata from the user's location information to construct that utilize the learner's physical surroundings to provide unique and relevant content. Additionally, Locabulary employs context-aware spaced repetition to help language learners develop mastery over the material in the exercises it creates.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"40 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"82208477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
David S. Park, Robert W. Schmidt, Charankumar Akiri, Stephanie Kwak, David A. Joyner
Following the initial proliferation of Massive Open Online Courses (MOOCs), a more recent trend has emerged toward offering "Affordable Degrees at Scale" or "Large, Internet-Mediated Asynchronous Degrees". In this research, we set out to understand this space: the range in tuition costs for these programs, the variety of admissions standards, and the types of assessments used to evaluate these non-traditional students. In the process, however, we found that in many ways, these programs may not be as new as we initially perceived: similarly-priced online programs have existed from traditional universities for years. In this research, we explore these two questions: what are these new degrees at scale, and how do they actually differ from traditional programs? To explore this, we collected materials for 35 MOOC-based graduate degrees and numerous non-MOOC-based comparable degrees. We then explored the patterns in tuition, admissions requirements, and syllabus information. In this paper, we report the trends we identified in MOOC-based degrees, and attempt to answer the question: what makes these programs different from non-MOOC-based online programs of the past? Ultimately, we find that this new era of programs is similar in many observable ways.
{"title":"Affordable Degrees at Scale: New Phenomenon or New Hype?","authors":"David S. Park, Robert W. Schmidt, Charankumar Akiri, Stephanie Kwak, David A. Joyner","doi":"10.1145/3386527.3405923","DOIUrl":"https://doi.org/10.1145/3386527.3405923","url":null,"abstract":"Following the initial proliferation of Massive Open Online Courses (MOOCs), a more recent trend has emerged toward offering \"Affordable Degrees at Scale\" or \"Large, Internet-Mediated Asynchronous Degrees\". In this research, we set out to understand this space: the range in tuition costs for these programs, the variety of admissions standards, and the types of assessments used to evaluate these non-traditional students. In the process, however, we found that in many ways, these programs may not be as new as we initially perceived: similarly-priced online programs have existed from traditional universities for years. In this research, we explore these two questions: what are these new degrees at scale, and how do they actually differ from traditional programs? To explore this, we collected materials for 35 MOOC-based graduate degrees and numerous non-MOOC-based comparable degrees. We then explored the patterns in tuition, admissions requirements, and syllabus information. In this paper, we report the trends we identified in MOOC-based degrees, and attempt to answer the question: what makes these programs different from non-MOOC-based online programs of the past? Ultimately, we find that this new era of programs is similar in many observable ways.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"88893147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Concepts are basic elements in any learning module and are thus very useful for modeling, summarizing, and previewing the content of any module. Automatic extraction of the major concepts from online education materials enables many useful applications. In this paper, we propose to leverage textbooks and their back-of-the-book indexes as training data to train a supervised machine learning algorithm for automatic extraction of concepts from text data in the education domain. We evaluate this idea by training neural networks on three textbooks and applying the trained neural networks to extract concepts from the lecture transcripts of two MOOCs. Our results suggest great promise for further exploration of this direction.
{"title":"Leveraging Book Indexes for Automatic Extraction of Concepts in MOOCs","authors":"Assma Boughoula, Aidan San, Chengxiang Zhai","doi":"10.1145/3386527.3406749","DOIUrl":"https://doi.org/10.1145/3386527.3406749","url":null,"abstract":"Concepts are basic elements in any learning module and are thus very useful for modeling, summarizing, and previewing the content of any module. Automatic extraction of the major concepts from online education materials enables many useful applications. In this paper, we propose to leverage textbooks and their back-of-the-book indexes as training data to train a supervised machine learning algorithm for automatic extraction of concepts from text data in the education domain. We evaluate this idea by training neural networks on three textbooks and applying the trained neural networks to extract concepts from the lecture transcripts of two MOOCs. Our results suggest great promise for further exploration of this direction.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"5 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72592446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Meng Xia, Yuya Asano, J. Williams, Huamin Qu, Xiaojuan Ma
"Gaming the system" is the phenomenon where students attempt to perform well by systematically exploiting properties of the learning system, rather than learning the material. Frequent gaming tends to cause bad learning outcomes. Though existing studies tackle the problem by redesigning the system workflow to change students' behaviors automatically, gaming students discover new ways to game. We instead propose a novel way, reflective nudge, to reflectively influence students' attitudes by conveying reasons not to game via information visualizations. Particularly, we identify three common gaming contexts and involve students and instructors in co-designing three context-specific persuasive visualizations. We deploy our information visualizations in a real online learning platform. Through embedded surveys and in-person interviews, we find some evidence that the designs can promote students' reflection on gaming, and suggestive data that two of them can reduce gaming compared with control groups. Furthermore, we present insights into reflective nudge designs and practical issues concerning deployment.
{"title":"Using Information Visualization to Promote Students' Reflection on \"Gaming the System\" in Online Learning","authors":"Meng Xia, Yuya Asano, J. Williams, Huamin Qu, Xiaojuan Ma","doi":"10.1145/3386527.3405924","DOIUrl":"https://doi.org/10.1145/3386527.3405924","url":null,"abstract":"\"Gaming the system\" is the phenomenon where students attempt to perform well by systematically exploiting properties of the learning system, rather than learning the material. Frequent gaming tends to cause bad learning outcomes. Though existing studies tackle the problem by redesigning the system workflow to change students' behaviors automatically, gaming students discover new ways to game. We instead propose a novel way, reflective nudge, to reflectively influence students' attitudes by conveying reasons not to game via information visualizations. Particularly, we identify three common gaming contexts and involve students and instructors in co-designing three context-specific persuasive visualizations. We deploy our information visualizations in a real online learning platform. Through embedded surveys and in-person interviews, we find some evidence that the designs can promote students' reflection on gaming, and suggestive data that two of them can reduce gaming compared with control groups. Furthermore, we present insights into reflective nudge designs and practical issues concerning deployment.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"22 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"78510953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
It is hard for experts to create good instructional resources due to a phenomenon known as the expert blind spot: They forget what it was like to be a novice, so they cannot pinpoint exactly where novices commonly struggle and how to best phrase their explanations. To help overcome these expert blind spots for computer programming topics, we created a learnersourcing system that elicits explanations of misconceptions directly from learners while they are coding. We have deployed this system for the past three years to the widely-used Python Tutor coding website (pythontutor.com) and collected 16,791 learner-written explanations. To our knowledge, this is the largest dataset of explanations for programming misconceptions. By inspecting this dataset, we found surprising insights that we did not originally think of due to our own expert blind spots as programming instructors. We are now using these insights to improve compiler and run-time error messages to explain common novice misconceptions.
{"title":"Learnersourcing at Scale to Overcome Expert Blind Spots for Introductory Programming: A Three-Year Deployment Study on the Python Tutor Website","authors":"Philip J. Guo, Julia M. Markel, Xiong Zhang","doi":"10.1145/3386527.3406733","DOIUrl":"https://doi.org/10.1145/3386527.3406733","url":null,"abstract":"It is hard for experts to create good instructional resources due to a phenomenon known as the expert blind spot: They forget what it was like to be a novice, so they cannot pinpoint exactly where novices commonly struggle and how to best phrase their explanations. To help overcome these expert blind spots for computer programming topics, we created a learnersourcing system that elicits explanations of misconceptions directly from learners while they are coding. We have deployed this system for the past three years to the widely-used Python Tutor coding website (pythontutor.com) and collected 16,791 learner-written explanations. To our knowledge, this is the largest dataset of explanations for programming misconceptions. By inspecting this dataset, we found surprising insights that we did not originally think of due to our own expert blind spots as programming instructors. We are now using these insights to improve compiler and run-time error messages to explain common novice misconceptions.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"8 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80658333","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hao Shen, Leming Liang, N. Law, Erik Hemberg, Una-May O’Reilly
A goal of learning analytics is to inform and improve learning design. Previous studies have attempted to interpret learners' clickstream data based on learning science theories. Many of these interpretations are made without reference to the specific learning designs of the courses being analyzed. Here, we report on a learning design informed analytics exploration of an introductory MOOC on Computer Science and Python programming. The learning resources (videos) and practice resources (short exercises and problem sets) are analyzed according to the knowledge types and cognitive process levels respectively, both based on a revised Bloom's Taxonomy. A heat map visualization of the access intensity on a learner resource access transition matrix and social network analysis are used to analyze learners' behavior with respect to the different resource categories. The results show distinctively different patterns of access between groups of students with different course performance and different academic backgrounds.
{"title":"Understanding Learner Behavior Through Learning Design Informed Learning Analytics","authors":"Hao Shen, Leming Liang, N. Law, Erik Hemberg, Una-May O’Reilly","doi":"10.1145/3386527.3405919","DOIUrl":"https://doi.org/10.1145/3386527.3405919","url":null,"abstract":"A goal of learning analytics is to inform and improve learning design. Previous studies have attempted to interpret learners' clickstream data based on learning science theories. Many of these interpretations are made without reference to the specific learning designs of the courses being analyzed. Here, we report on a learning design informed analytics exploration of an introductory MOOC on Computer Science and Python programming. The learning resources (videos) and practice resources (short exercises and problem sets) are analyzed according to the knowledge types and cognitive process levels respectively, both based on a revised Bloom's Taxonomy. A heat map visualization of the access intensity on a learner resource access transition matrix and social network analysis are used to analyze learners' behavior with respect to the different resource categories. The results show distinctively different patterns of access between groups of students with different course performance and different academic backgrounds.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79907876","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Tina Papathoma, Rebecca Ferguson, Francisco Iniesto, Irina Rets, D. Vogiatzis, Victoria Murphy
While learning at scale has the potential to widen access to education, the accessibility of courses offered on Massive Open Online Course (MOOC) platforms has not been researched in depth. This paper begins to fill that gap. Data was gathered using the participatory 'Evidence Café' method. Thematic analysis identified characteristics of accessible courses on these platforms. These characteristics include elements of both technology and pedagogy. Capturing and analysing expert insights enables this paper to provide guidance on how online courses can be made more accessible. The findings suggest that course production teams need to work collaboratively with providers to address issues of accessibility and involve learners in design, testing and evaluation. Well-designed tutor-supported activities that follow web accessibility and usability guidelines are needed, as well as educator training on accessibility.
{"title":"Guidance on How Learning at Scale Can be Made More Accessible","authors":"Tina Papathoma, Rebecca Ferguson, Francisco Iniesto, Irina Rets, D. Vogiatzis, Victoria Murphy","doi":"10.1145/3386527.3406730","DOIUrl":"https://doi.org/10.1145/3386527.3406730","url":null,"abstract":"While learning at scale has the potential to widen access to education, the accessibility of courses offered on Massive Open Online Course (MOOC) platforms has not been researched in depth. This paper begins to fill that gap. Data was gathered using the participatory 'Evidence Café' method. Thematic analysis identified characteristics of accessible courses on these platforms. These characteristics include elements of both technology and pedagogy. Capturing and analysing expert insights enables this paper to provide guidance on how online courses can be made more accessible. The findings suggest that course production teams need to work collaboratively with providers to address issues of accessibility and involve learners in design, testing and evaluation. Well-designed tutor-supported activities that follow web accessibility and usability guidelines are needed, as well as educator training on accessibility.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"323 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76298736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
C. Piech, Lisa Yan, Lisa Einstein, Ana Saavedra, B. Bozkurt, Eliška Šestáková, Ondřej Guth, N. McKeown
Programming is fast becoming a required skill set for students in every country. We present CS Bridge, a model for cross-border co-teaching of CS1, along with a corresponding open-source course-in-a-box curriculum made for easy localization. In the CS Bridge model, instructors and student-teachers from different countries come together to teach a short, stand-alone CS1 course to hundreds of local high school students. The corresponding open-source curriculum has been specifically designed to be easily adapted to a wide variety of local teaching practices, languages, and cultures. Over the past six years, the curriculum has been used to teach CS1 material to over 1,000 high school students in Colombia, the Czech Republic, Turkey, and Guinea. A large majority of our students continue on to study CS or CS-related fields in university. More importantly, many of our undergraduate student-teachers stay involved with teaching beyond the program. Joint teaching creates a positive, high-quality learning experience for students around the world and a powerful, high-impact professional development experience for the teaching team---instructors and student-teachers alike.
{"title":"Co-Teaching Computer Science Across Borders: Human-Centric Learning at Scale","authors":"C. Piech, Lisa Yan, Lisa Einstein, Ana Saavedra, B. Bozkurt, Eliška Šestáková, Ondřej Guth, N. McKeown","doi":"10.1145/3386527.3405915","DOIUrl":"https://doi.org/10.1145/3386527.3405915","url":null,"abstract":"Programming is fast becoming a required skill set for students in every country. We present CS Bridge, a model for cross-border co-teaching of CS1, along with a corresponding open-source course-in-a-box curriculum made for easy localization. In the CS Bridge model, instructors and student-teachers from different countries come together to teach a short, stand-alone CS1 course to hundreds of local high school students. The corresponding open-source curriculum has been specifically designed to be easily adapted to a wide variety of local teaching practices, languages, and cultures. Over the past six years, the curriculum has been used to teach CS1 material to over 1,000 high school students in Colombia, the Czech Republic, Turkey, and Guinea. A large majority of our students continue on to study CS or CS-related fields in university. More importantly, many of our undergraduate student-teachers stay involved with teaching beyond the program. Joint teaching creates a positive, high-quality learning experience for students around the world and a powerful, high-impact professional development experience for the teaching team---instructors and student-teachers alike.","PeriodicalId":20608,"journal":{"name":"Proceedings of the Seventh ACM Conference on Learning @ Scale","volume":"704 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2020-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81736963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}