Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513888
G. Cheang
Approximation and estimation bounds were obtained by Barron (see Proc. of the 7th Yale workshop on adaptive and learning systems, 1992, IEEE Transactions on Information Theory, vol.39, pp.930-944, 1993 and Machine Learning, vol.14, p.113-143, 1994) for function estimation by single hidden-layer neural nets. This paper highlights the extension of his results to the two hidden-layer case. The bounds derived for the two hidden-layer case depend on the number of nodes T/sub 1/ and T/sub 2/ in each hidden-layer, and also on the sample size N. It is seen from our bounds that in some cases, an exponentially large number of nodes, and hence parameters, is not required.
Barron(参见第7届耶鲁自适应和学习系统研讨会，1992,IEEE Transactions on Information Theory, vol.39, pp.930-944, 1993和Machine learning, vol.14, p.113-143, 1994)获得了单隐藏层神经网络函数估计的近似和估计边界。本文着重将其结果推广到两隐层情况。两个隐藏层情况的边界取决于每个隐藏层的节点数量T/sub 1/和T/sub 2/，也取决于样本大小n。从我们的边界可以看出，在某些情况下，不需要指数级的节点数量，因此不需要参数。
{"title":"Neural network approximation and estimation of functions","authors":"G. Cheang","doi":"10.1109/WITS.1994.513888","DOIUrl":"https://doi.org/10.1109/WITS.1994.513888","url":null,"abstract":"Approximation and estimation bounds were obtained by Barron (see Proc. of the 7th Yale workshop on adaptive and learning systems, 1992, IEEE Transactions on Information Theory, vol.39, pp.930-944, 1993 and Machine Learning, vol.14, p.113-143, 1994) for function estimation by single hidden-layer neural nets. This paper highlights the extension of his results to the two hidden-layer case. The bounds derived for the two hidden-layer case depend on the number of nodes T/sub 1/ and T/sub 2/ in each hidden-layer, and also on the sample size N. It is seen from our bounds that in some cases, an exponentially large number of nodes, and hence parameters, is not required.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129352434","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513863
M. D. Riley
Several applications of statistical tree-based modelling are described to problems in speech and language, including prediction of possible phonetic realizations, segment duration modelling in speech synthesis and end of sentence detection in text analysis.
{"title":"Tree-based models for speech and language","authors":"M. D. Riley","doi":"10.1109/WITS.1994.513863","DOIUrl":"https://doi.org/10.1109/WITS.1994.513863","url":null,"abstract":"Several applications of statistical tree-based modelling are described to problems in speech and language, including prediction of possible phonetic realizations, segment duration modelling in speech synthesis and end of sentence detection in text analysis.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123356625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513876
F. Comets
Large deviations estimates yield a convenient tool to study asymptotics of Gibbs fields. Applications to parametric estimation and detection of phase transition are given. Gibbs random fields provide pertinent statistical models for spacial data, where important features of the dependence structure can be captured in a very natural way.
{"title":"Large deviations and consistent estimates for Gibbs random fields","authors":"F. Comets","doi":"10.1109/WITS.1994.513876","DOIUrl":"https://doi.org/10.1109/WITS.1994.513876","url":null,"abstract":"Large deviations estimates yield a convenient tool to study asymptotics of Gibbs fields. Applications to parametric estimation and detection of phase transition are given. Gibbs random fields provide pertinent statistical models for spacial data, where important features of the dependence structure can be captured in a very natural way.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"54 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114030217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513846
T. Cover
The main theorems in information theory and statistics are put in context, the differences are discussed, and some of the open research problems are mentioned. The author demonstrates some of the points of intersection of information theory and statistics, and discuss some problems in physics and computer science that require a rigorous probabilistic treatment.
{"title":"Information theory and statistics","authors":"T. Cover","doi":"10.1109/WITS.1994.513846","DOIUrl":"https://doi.org/10.1109/WITS.1994.513846","url":null,"abstract":"The main theorems in information theory and statistics are put in context, the differences are discussed, and some of the open research problems are mentioned. The author demonstrates some of the points of intersection of information theory and statistics, and discuss some problems in physics and computer science that require a rigorous probabilistic treatment.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115348909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513903
J. Solka, C. Priebe, G. Rogers, W. Poston, D. Marchette
This paper examines the application of Akaike (1974) information criterion (AIC) based pruning to the refinement of nonparametric density estimates obtained via the adaptive mixtures (AM) procedure of Priebe (see JASA, vol.89, no.427, p.796-806, 1994) and Marchette. The paper details a new technique that uses these two methods in conjunction with one another to predict the appropriate number of terms in the mixture model of an unknown density. Results that detail the procedure's performance when applied to different distributional classes are presented. Results are presented on artificially generated data, well known data sets, and some features for mammographic screening.
{"title":"The application of Akaike information criterion based pruning to nonparametric density estimates","authors":"J. Solka, C. Priebe, G. Rogers, W. Poston, D. Marchette","doi":"10.1109/WITS.1994.513903","DOIUrl":"https://doi.org/10.1109/WITS.1994.513903","url":null,"abstract":"This paper examines the application of Akaike (1974) information criterion (AIC) based pruning to the refinement of nonparametric density estimates obtained via the adaptive mixtures (AM) procedure of Priebe (see JASA, vol.89, no.427, p.796-806, 1994) and Marchette. The paper details a new technique that uses these two methods in conjunction with one another to predict the appropriate number of terms in the mixture model of an unknown density. Results that detail the procedure's performance when applied to different distributional classes are presented. Results are presented on artificially generated data, well known data sets, and some features for mammographic screening.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123985638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513862
Q. Xie, R. Ward, C. Laszlo
VQ-based method is developed as an effective data reduction technique for nonparametric classifier design. This new technique, while insisting on competitive classification accuracy, is found to overcome the usual disadvantage of traditional nonparametric classifiers of being computationally complex and of requiring large amounts of computer storage.
{"title":"Nonparametric classifier design using vector quantization","authors":"Q. Xie, R. Ward, C. Laszlo","doi":"10.1109/WITS.1994.513862","DOIUrl":"https://doi.org/10.1109/WITS.1994.513862","url":null,"abstract":"VQ-based method is developed as an effective data reduction technique for nonparametric classifier design. This new technique, while insisting on competitive classification accuracy, is found to overcome the usual disadvantage of traditional nonparametric classifiers of being computationally complex and of requiring large amounts of computer storage.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126559803","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513857
Bin Yu
This paper focuses on lower bound results on expected redundancy for universal compression of i.i.d. data from parametric and nonparametric families. Two types of lower bounds are reviewed. One is Rissanen's almost pointwise lower bound and its extension to the nonparametric case. The other is minimax lower bounds, for which a new proof is given in the nonparametric case.
{"title":"Lower bounds on expected redundancy","authors":"Bin Yu","doi":"10.1109/WITS.1994.513857","DOIUrl":"https://doi.org/10.1109/WITS.1994.513857","url":null,"abstract":"This paper focuses on lower bound results on expected redundancy for universal compression of i.i.d. data from parametric and nonparametric families. Two types of lower bounds are reviewed. One is Rissanen's almost pointwise lower bound and its extension to the nonparametric case. The other is minimax lower bounds, for which a new proof is given in the nonparametric case.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129590019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513902
W. Poston, J. Solka
In many situations it is desirable to operate on a subset of the data only. These can arise in the areas of experimental design, robust estimation of multivariate location, and density estimation. The paper describes a method of subset selection that optimizes the determinant of the Fisher information matrix (FIM) which is called the effective independence distribution (EID) method. It provides some motivation that justifies the use of the EID, and the problem of finding the subset of points to use in the estimation of the minimum volume ellipsoid (MVE) is examined as an application of interest.
{"title":"Choosing data sets that optimize the determinant of the Fisher information matrix","authors":"W. Poston, J. Solka","doi":"10.1109/WITS.1994.513902","DOIUrl":"https://doi.org/10.1109/WITS.1994.513902","url":null,"abstract":"In many situations it is desirable to operate on a subset of the data only. These can arise in the areas of experimental design, robust estimation of multivariate location, and density estimation. The paper describes a method of subset selection that optimizes the determinant of the Fisher information matrix (FIM) which is called the effective independence distribution (EID) method. It provides some motivation that justifies the use of the EID, and the problem of finding the subset of points to use in the estimation of the minimum volume ellipsoid (MVE) is examined as an application of interest.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129600136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513864
Xiaolin Wu
Image compression is often approached from an angle of statistical image classification. For instance, VQ-based image coding methods compress image data by classifying image blocks into representative two-dimensional patterns (codewords) that statistically approximate the original data. Another image compression approach that naturally relates to image classification is segmentation-based image coding (SIC). In SIC, we classify pixels into segments of certain uniformity or similarity, and then encode the segmentation geometry and the attributes of the segments. Image segmentation in SIC has to meet some more stringent requirements than in other applications such as computer vision and pattern recognition. An efficient SIC coder has to strike a good balance between accurate semantics and succinct syntax of the segmentation. From a pure classification point of view, free form segmentation by relaxation, region-growing, or split-and-merge techniques offers an accurate boundary representation. But the resulting segmentation geometry is often too complex to have a compact description, defeating the purpose of image compression. Instead, we adopt a bintree-structured segmentation scheme. The bintree is a binary tree created by recursive rectilinear bipartition of an image.
{"title":"Image coding via bintree segmentation and texture VQ","authors":"Xiaolin Wu","doi":"10.1109/WITS.1994.513864","DOIUrl":"https://doi.org/10.1109/WITS.1994.513864","url":null,"abstract":"Image compression is often approached from an angle of statistical image classification. For instance, VQ-based image coding methods compress image data by classifying image blocks into representative two-dimensional patterns (codewords) that statistically approximate the original data. Another image compression approach that naturally relates to image classification is segmentation-based image coding (SIC). In SIC, we classify pixels into segments of certain uniformity or similarity, and then encode the segmentation geometry and the attributes of the segments. Image segmentation in SIC has to meet some more stringent requirements than in other applications such as computer vision and pattern recognition. An efficient SIC coder has to strike a good balance between accurate semantics and succinct syntax of the segmentation. From a pure classification point of view, free form segmentation by relaxation, region-growing, or split-and-merge techniques offers an accurate boundary representation. But the resulting segmentation geometry is often too complex to have a compact description, defeating the purpose of image compression. Instead, we adopt a bintree-structured segmentation scheme. The bintree is a binary tree created by recursive rectilinear bipartition of an image.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133700788","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513924
T. Robert, J. Tourneret
Bayes decision theory is based on the assumption that the decision problem is posed in probabilistic terms, and that all of the relevant probability values are known. The aim of this paper is to show how blind sliding window AR modeling is corrupted by an abrupt model change and to derive a statistical study of these parameters.
{"title":"Continuously evolving classification of signals corrupted by an abrupt change","authors":"T. Robert, J. Tourneret","doi":"10.1109/WITS.1994.513924","DOIUrl":"https://doi.org/10.1109/WITS.1994.513924","url":null,"abstract":"Bayes decision theory is based on the assumption that the decision problem is posed in probabilistic terms, and that all of the relevant probability values are known. The aim of this paper is to show how blind sliding window AR modeling is corrupted by an abrupt model change and to derive a statistical study of these parameters.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133674996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：