Johannes F. von Hoyer, Georg Pardi, Yvonne Kammerer, Peter Holtz
Metacognitive self-assessments of one's learning performance (calibration) are important elements of Searching as Learning (SAL) tasks. In this SAL study, N = 115 participants were asked to learn for up to 30 minutes about the formation of thunderstorms and lightning by using any suitable internet resources (including multimedia resources). Participants rated their performance in comparison to other participants (placement), estimated the percentage of correct answers (estimation), and indicated their confidence in the correctness of their answers (confidence) in a multiple-choice knowledge test that was filled in one week before (T1) and directly after (T2) the learning phase. Participants furthermore rated the 'familiarity' of terms that do or do not exist in the context of meteorology (overclaiming). Learners tended to underestimate their performance at T1 and there were indicators of a potential Dunning-Kruger effect. Overall, placement and estimation ratings tended to be more accurate at T2. Surprisingly, confidence ratings increased approximately equally for correct as well as incorrect answers. A propensity for overclaiming was positively correlated with most confidence measures and the amount of time learners spent on YouTube was correlated to lower confidence scores. Implications for the design of SAL tasks and SAL studies are discussed.
{"title":"Metacognitive Judgments in Searching as Learning (SAL) Tasks: Insights on (Mis-) Calibration, Multimedia Usage, and Confidence","authors":"Johannes F. von Hoyer, Georg Pardi, Yvonne Kammerer, Peter Holtz","doi":"10.1145/3347451.3356730","DOIUrl":"https://doi.org/10.1145/3347451.3356730","url":null,"abstract":"Metacognitive self-assessments of one's learning performance (calibration) are important elements of Searching as Learning (SAL) tasks. In this SAL study, N = 115 participants were asked to learn for up to 30 minutes about the formation of thunderstorms and lightning by using any suitable internet resources (including multimedia resources). Participants rated their performance in comparison to other participants (placement), estimated the percentage of correct answers (estimation), and indicated their confidence in the correctness of their answers (confidence) in a multiple-choice knowledge test that was filled in one week before (T1) and directly after (T2) the learning phase. Participants furthermore rated the 'familiarity' of terms that do or do not exist in the context of meteorology (overclaiming). Learners tended to underestimate their performance at T1 and there were indicators of a potential Dunning-Kruger effect. Overall, placement and estimation ratings tended to be more accurate at T2. Surprisingly, confidence ratings increased approximately equally for correct as well as incorrect answers. A propensity for overclaiming was positively correlated with most confidence measures and the amount of time learners spent on YouTube was correlated to lower confidence scores. Implications for the design of SAL tasks and SAL studies are discussed.","PeriodicalId":347114,"journal":{"name":"Proceedings of the 1st International Workshop on Search as Learning with Multimedia Information","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124698745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jianwei Shi, Christian Otto, Anett Hoppe, Peter Holtz, R. Ewerth
Ranking and recommendation of multimedia content such as videos is usually realized with respect to the relevance to a user query. However, for lecture videos and MOOCs (Massive Open Online Courses) it is not only required to retrieve relevant videos, but particularly to find lecture videos of high quality that facilitate learning, for instance, independent of the video's or speaker's popularity. Thus, metadata about a lecture video's quality are crucial features for learning contexts, e.g., lecture video recommendation in search as learning scenarios. In this paper, we investigate whether automatically extracted features are correlated to quality aspects of a video. A set of scholarly videos from a Mass Open Online Course (MOOC) is analyzed regarding audio, linguistic, and visual features. Furthermore, a set of cross-modal features is proposed which are derived by combining transcripts, audio, video, and slide content. A user study is conducted to investigate the correlations between the automatically collected features and human ratings of quality aspects of a lecture video. Finally, the impact of our features on the knowledge gain of the participants is discussed.
{"title":"Investigating Correlations of Automatically Extracted Multimodal Features and Lecture Video Quality","authors":"Jianwei Shi, Christian Otto, Anett Hoppe, Peter Holtz, R. Ewerth","doi":"10.1145/3347451.3356731","DOIUrl":"https://doi.org/10.1145/3347451.3356731","url":null,"abstract":"Ranking and recommendation of multimedia content such as videos is usually realized with respect to the relevance to a user query. However, for lecture videos and MOOCs (Massive Open Online Courses) it is not only required to retrieve relevant videos, but particularly to find lecture videos of high quality that facilitate learning, for instance, independent of the video's or speaker's popularity. Thus, metadata about a lecture video's quality are crucial features for learning contexts, e.g., lecture video recommendation in search as learning scenarios. In this paper, we investigate whether automatically extracted features are correlated to quality aspects of a video. A set of scholarly videos from a Mass Open Online Course (MOOC) is analyzed regarding audio, linguistic, and visual features. Furthermore, a set of cross-modal features is proposed which are derived by combining transcripts, audio, video, and slide content. A user study is conducted to investigate the correlations between the automatically collected features and human ratings of quality aspects of a lecture video. Finally, the impact of our features on the knowledge gain of the participants is discussed.","PeriodicalId":347114,"journal":{"name":"Proceedings of the 1st International Workshop on Search as Learning with Multimedia Information","volume":"127 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115177492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Search engine results pages (SERPs) are a frequent gateway to Internet content. Prior research has extensively documented strong effects of SERPs (e.g. rank order or the spatial distribution of the results) on users' attention to and selection of particular Web pages [1,2]. In the context of Web search, a common user behavior is the 'top link' or 'Google trust' heuristic, that is, the inspection and selection of only the first few search results presented by the search engine, without evaluating all other search results available. This heuristic behavior allows users to find information in an efficient way, as search engines tend to provide relevant documents on top of the list, especially when it comes to simple facts. But just relying on the top results of the SERP to access information may not be as efficient when users search for learning purposes about controversial topics, such as climate change, for at least two reasons. First, users can be easily mislead by, for example, commercially biased Web Pages located on top of the SERP [3,4]. Second, by looking at just few hits users miss the opportunity to use SERP information to reflect on the relationships between available web pages, an essential step when learning about controversial topics [5]. An emerging research line is currently exploring the role of SERPs in supporting users from different student populations to search the Internet for learning purposes. Based on a synthesis of existing research, in this talk I will argue that SERPs' design can influence users' perceptions and learning of controversial topics, to the extent that design can counteract the effects of 'top link' heuristic [5,6,7]. In addition, I will show that SERP effects are moderated by users (e.g. prior knowledge) [4,8] and task's characteristics (e.g. task complexity) [3]. Finally, I will present results from a recent short intervention study designed to support users' systematic exploration of SERPs when learning about a controversial topic.
{"title":"Search Interfaces and Learning about Controversial Topics","authors":"L. Salmerón","doi":"10.1145/3347451.3356729","DOIUrl":"https://doi.org/10.1145/3347451.3356729","url":null,"abstract":"Search engine results pages (SERPs) are a frequent gateway to Internet content. Prior research has extensively documented strong effects of SERPs (e.g. rank order or the spatial distribution of the results) on users' attention to and selection of particular Web pages [1,2]. In the context of Web search, a common user behavior is the 'top link' or 'Google trust' heuristic, that is, the inspection and selection of only the first few search results presented by the search engine, without evaluating all other search results available. This heuristic behavior allows users to find information in an efficient way, as search engines tend to provide relevant documents on top of the list, especially when it comes to simple facts. But just relying on the top results of the SERP to access information may not be as efficient when users search for learning purposes about controversial topics, such as climate change, for at least two reasons. First, users can be easily mislead by, for example, commercially biased Web Pages located on top of the SERP [3,4]. Second, by looking at just few hits users miss the opportunity to use SERP information to reflect on the relationships between available web pages, an essential step when learning about controversial topics [5]. An emerging research line is currently exploring the role of SERPs in supporting users from different student populations to search the Internet for learning purposes. Based on a synthesis of existing research, in this talk I will argue that SERPs' design can influence users' perceptions and learning of controversial topics, to the extent that design can counteract the effects of 'top link' heuristic [5,6,7]. In addition, I will show that SERP effects are moderated by users (e.g. prior knowledge) [4,8] and task's characteristics (e.g. task complexity) [3]. Finally, I will present results from a recent short intervention study designed to support users' systematic exploration of SERPs when learning about a controversial topic.","PeriodicalId":347114,"journal":{"name":"Proceedings of the 1st International Workshop on Search as Learning with Multimedia Information","volume":"40 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132970260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Proceedings of the 1st International Workshop on Search as Learning with Multimedia Information","authors":"","doi":"10.1145/3347451","DOIUrl":"https://doi.org/10.1145/3347451","url":null,"abstract":"","PeriodicalId":347114,"journal":{"name":"Proceedings of the 1st International Workshop on Search as Learning with Multimedia Information","volume":"480 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115876953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：