Fisher information, stochastic complexity, and universal modeling

J. Rissanen
{"title":"Fisher information, stochastic complexity, and universal modeling","authors":"J. Rissanen","doi":"10.1109/WITS.1994.513849","DOIUrl":null,"url":null,"abstract":"The main objective in universal modeling is to construct a process for a class of model processes which for long strings, generated by any of the models in the class, behaves like the data generating one. Hence, such a universal process may be taken as a representation of the entire model class to be used for statistical inference. If f(x/sup n/) denotes the probability or density it assigns to the data string x/sup n/=x/sub 1/,..,x/sub n/, then the negative logarithm - log f(x/sup n/), which may be viewed as the shortest ideal code length for the data obtainable with the model class, is called the stochastic complexity of the string, relative to the considered model class. Unlike in related universal modeling, where the mean code length is sufficient, we also need an explicit asymptotic formula for the stochastic complexity. This is because it permits a comparison of different model classes by their stochastic complexity in accordance with the MDL (minimum description length) principle.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"30 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"2","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of 1994 Workshop on Information Theory and Statistics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/WITS.1994.513849","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 2

Abstract

The main objective in universal modeling is to construct a process for a class of model processes which for long strings, generated by any of the models in the class, behaves like the data generating one. Hence, such a universal process may be taken as a representation of the entire model class to be used for statistical inference. If f(x/sup n/) denotes the probability or density it assigns to the data string x/sup n/=x/sub 1/,..,x/sub n/, then the negative logarithm - log f(x/sup n/), which may be viewed as the shortest ideal code length for the data obtainable with the model class, is called the stochastic complexity of the string, relative to the considered model class. Unlike in related universal modeling, where the mean code length is sufficient, we also need an explicit asymptotic formula for the stochastic complexity. This is because it permits a comparison of different model classes by their stochastic complexity in accordance with the MDL (minimum description length) principle.
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
费雪信息、随机复杂性和通用建模
通用建模的主要目标是为一类模型过程构造一个过程,对于由类中的任何模型生成的长字符串,该过程的行为与数据生成过程类似。因此,这种通用过程可以作为整个模型类的表示,用于统计推断。如果f(x/sup n/)表示它分配给数据字符串x/sup n/=x/sub 1/的概率或密度,..,x/下标n/,然后是负对数- log f(x/下标n/),它可以被视为模型类可获得的数据的最短理想代码长度,称为字符串的随机复杂度,相对于所考虑的模型类。不像在相关的通用模型中,平均码长是足够的,我们还需要一个明确的渐近公式的随机复杂性。这是因为它允许根据MDL(最小描述长度)原则对不同的模型类进行随机复杂度的比较。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
自引率
0.00%
发文量
0
期刊最新文献
Large deviations and consistent estimates for Gibbs random fields Markov chains for modeling and analyzing digital data signals Maximized mutual information using macrocanonical probability distributions Coding for noisy feasible channels Identification via compressed data
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1