Learning online is essential for an agent to perform well in an ever-changing world. An agent has to learn online not only out of necessity — a non-stationary world might render past learning useless — but also because continual tracking in a temporally coherent world can result in better performance than a fixed solution. Despite the necessity of online learning, we have made little progress towards building robust online learning methods. More specifically, a scalable online representation learning method for neural network function approximators has remained elusive. In this thesis, I investigate the reasons behind this lack of progress. I propose the idea of online-aware representations – data representations explicitly optimized for online learning – and argue that existing representation learning methods do not learn such representations. I investigate if neural networks are capable of learning these representations. My results suggest that neural networks can indeed learn representations that are highly effective for online learning, but learning these representations online using gradient-based methods is challenging. More specifically, long-term credit assignment using back-propagation through time (BPTT) does not scale with the size of the problem. To address this, I propose Learning with Backtracking for slowly and continually improving representations online. The primary idea behind LwB is that while it is not possible to compute an accurate estimate of the representation update online, it is possible to verify if an update is useful online.
{"title":"The Problem","authors":"H. Mounce","doi":"10.4324/9781315203225-3","DOIUrl":"https://doi.org/10.4324/9781315203225-3","url":null,"abstract":"Learning online is essential for an agent to perform well in an ever-changing world. An agent has to learn online not only out of necessity — a non-stationary world might render past learning useless — but also because continual tracking in a temporally coherent world can result in better performance than a fixed solution. Despite the necessity of online learning, we have made little progress towards building robust online learning methods. More specifically, a scalable online representation learning method for neural network function approximators has remained elusive. In this thesis, I investigate the reasons behind this lack of progress. I propose the idea of online-aware representations – data representations explicitly optimized for online learning – and argue that existing representation learning methods do not learn such representations. I investigate if neural networks are capable of learning these representations. My results suggest that neural networks can indeed learn representations that are highly effective for online learning, but learning these representations online using gradient-based methods is challenging. More specifically, long-term credit assignment using back-propagation through time (BPTT) does not scale with the size of the problem. To address this, I propose Learning with Backtracking for slowly and continually improving representations online. The primary idea behind LwB is that while it is not possible to compute an accurate estimate of the representation update online, it is possible to verify if an update is useful online.","PeriodicalId":389393,"journal":{"name":"Tolstoy on Aesthetics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129790565","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Art and Morality","authors":"H. Mounce","doi":"10.4324/9781315203225-5","DOIUrl":"https://doi.org/10.4324/9781315203225-5","url":null,"abstract":"","PeriodicalId":389393,"journal":{"name":"Tolstoy on Aesthetics","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133757024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Art and Society","authors":"H. Mounce","doi":"10.4324/9781315203225-2","DOIUrl":"https://doi.org/10.4324/9781315203225-2","url":null,"abstract":"","PeriodicalId":389393,"journal":{"name":"Tolstoy on Aesthetics","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125267904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Central Features","authors":"H. Mounce","doi":"10.4324/9781315203225-4","DOIUrl":"https://doi.org/10.4324/9781315203225-4","url":null,"abstract":"","PeriodicalId":389393,"journal":{"name":"Tolstoy on Aesthetics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134038577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Summary and Elaboration","authors":"H. Mounce","doi":"10.4324/9781315203225-9","DOIUrl":"https://doi.org/10.4324/9781315203225-9","url":null,"abstract":"","PeriodicalId":389393,"journal":{"name":"Tolstoy on Aesthetics","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129117257","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The program is designed to foster a lifelong interest in and sensitivity to the arts as a fundamental form of human expression and communication, equipping students with strong skills in a world in which visual literacy is vitally important. Based at one of the nation's top art schools, Tyler's art history major offers access to dynamic faculty members with a diverse range of interests; the resources of a large research university; study abroad programs in Rome and Tokyo; and opportunities for field trips, research, internships and enrichment in Philadelphia, a vibrant center of art, culture and history located near both New York and Washington, D.C.
{"title":"Art in History","authors":"H. Mounce","doi":"10.4324/9781315203225-6","DOIUrl":"https://doi.org/10.4324/9781315203225-6","url":null,"abstract":"The program is designed to foster a lifelong interest in and sensitivity to the arts as a fundamental form of human expression and communication, equipping students with strong skills in a world in which visual literacy is vitally important. Based at one of the nation's top art schools, Tyler's art history major offers access to dynamic faculty members with a diverse range of interests; the resources of a large research university; study abroad programs in Rome and Tokyo; and opportunities for field trips, research, internships and enrichment in Philadelphia, a vibrant center of art, culture and history located near both New York and Washington, D.C.","PeriodicalId":389393,"journal":{"name":"Tolstoy on Aesthetics","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122370439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：