Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513888
G. Cheang
Approximation and estimation bounds were obtained by Barron (see Proc. of the 7th Yale workshop on adaptive and learning systems, 1992, IEEE Transactions on Information Theory, vol.39, pp.930-944, 1993 and Machine Learning, vol.14, p.113-143, 1994) for function estimation by single hidden-layer neural nets. This paper highlights the extension of his results to the two hidden-layer case. The bounds derived for the two hidden-layer case depend on the number of nodes T/sub 1/ and T/sub 2/ in each hidden-layer, and also on the sample size N. It is seen from our bounds that in some cases, an exponentially large number of nodes, and hence parameters, is not required.
Barron(参见第7届耶鲁自适应和学习系统研讨会，1992,IEEE Transactions on Information Theory, vol.39, pp.930-944, 1993和Machine learning, vol.14, p.113-143, 1994)获得了单隐藏层神经网络函数估计的近似和估计边界。本文着重将其结果推广到两隐层情况。两个隐藏层情况的边界取决于每个隐藏层的节点数量T/sub 1/和T/sub 2/，也取决于样本大小n。从我们的边界可以看出，在某些情况下，不需要指数级的节点数量，因此不需要参数。
{"title":"Neural network approximation and estimation of functions","authors":"G. Cheang","doi":"10.1109/WITS.1994.513888","DOIUrl":"https://doi.org/10.1109/WITS.1994.513888","url":null,"abstract":"Approximation and estimation bounds were obtained by Barron (see Proc. of the 7th Yale workshop on adaptive and learning systems, 1992, IEEE Transactions on Information Theory, vol.39, pp.930-944, 1993 and Machine Learning, vol.14, p.113-143, 1994) for function estimation by single hidden-layer neural nets. This paper highlights the extension of his results to the two hidden-layer case. The bounds derived for the two hidden-layer case depend on the number of nodes T/sub 1/ and T/sub 2/ in each hidden-layer, and also on the sample size N. It is seen from our bounds that in some cases, an exponentially large number of nodes, and hence parameters, is not required.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129352434","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513876
F. Comets
Large deviations estimates yield a convenient tool to study asymptotics of Gibbs fields. Applications to parametric estimation and detection of phase transition are given. Gibbs random fields provide pertinent statistical models for spacial data, where important features of the dependence structure can be captured in a very natural way.
{"title":"Large deviations and consistent estimates for Gibbs random fields","authors":"F. Comets","doi":"10.1109/WITS.1994.513876","DOIUrl":"https://doi.org/10.1109/WITS.1994.513876","url":null,"abstract":"Large deviations estimates yield a convenient tool to study asymptotics of Gibbs fields. Applications to parametric estimation and detection of phase transition are given. Gibbs random fields provide pertinent statistical models for spacial data, where important features of the dependence structure can be captured in a very natural way.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"54 3","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114030217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513875
P. Hall, S. Lahiri, Y. Truong
The authors discuss the optimal bandwidth choice and optimal convergence rates for density estimation with dependent data, as the amount of information in the sample is altered by adjusting the range of dependence. They assume that data are observed from a stationary stochastic process that may be taken to be an unknown function of a Gaussian process.
{"title":"Bandwidth choice and convergence rates in density estimation with long-range dependent data","authors":"P. Hall, S. Lahiri, Y. Truong","doi":"10.1109/WITS.1994.513875","DOIUrl":"https://doi.org/10.1109/WITS.1994.513875","url":null,"abstract":"The authors discuss the optimal bandwidth choice and optimal convergence rates for density estimation with dependent data, as the amount of information in the sample is altered by adjusting the range of dependence. They assume that data are observed from a stationary stochastic process that may be taken to be an unknown function of a Gaussian process.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126648795","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513846
T. Cover
The main theorems in information theory and statistics are put in context, the differences are discussed, and some of the open research problems are mentioned. The author demonstrates some of the points of intersection of information theory and statistics, and discuss some problems in physics and computer science that require a rigorous probabilistic treatment.
{"title":"Information theory and statistics","authors":"T. Cover","doi":"10.1109/WITS.1994.513846","DOIUrl":"https://doi.org/10.1109/WITS.1994.513846","url":null,"abstract":"The main theorems in information theory and statistics are put in context, the differences are discussed, and some of the open research problems are mentioned. The author demonstrates some of the points of intersection of information theory and statistics, and discuss some problems in physics and computer science that require a rigorous probabilistic treatment.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"15 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115348909","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513855
J. Ziv
An important class of universal encoders is the one where the encoder is fed by two inputs: a) The incoming string of data to be compressed. b) An N-bit description of the source statistics (i.e. a "training sequence"). We consider fixed-to-variable universal encoders that noiselessly compress blocks of length l.
{"title":"Bounds on universal coding: the next generation","authors":"J. Ziv","doi":"10.1109/WITS.1994.513855","DOIUrl":"https://doi.org/10.1109/WITS.1994.513855","url":null,"abstract":"An important class of universal encoders is the one where the encoder is fed by two inputs: a) The incoming string of data to be compressed. b) An N-bit description of the source statistics (i.e. a \"training sequence\"). We consider fixed-to-variable universal encoders that noiselessly compress blocks of length l.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"38 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129856443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513857
Bin Yu
This paper focuses on lower bound results on expected redundancy for universal compression of i.i.d. data from parametric and nonparametric families. Two types of lower bounds are reviewed. One is Rissanen's almost pointwise lower bound and its extension to the nonparametric case. The other is minimax lower bounds, for which a new proof is given in the nonparametric case.
{"title":"Lower bounds on expected redundancy","authors":"Bin Yu","doi":"10.1109/WITS.1994.513857","DOIUrl":"https://doi.org/10.1109/WITS.1994.513857","url":null,"abstract":"This paper focuses on lower bound results on expected redundancy for universal compression of i.i.d. data from parametric and nonparametric families. Two types of lower bounds are reviewed. One is Rissanen's almost pointwise lower bound and its extension to the nonparametric case. The other is minimax lower bounds, for which a new proof is given in the nonparametric case.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129590019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513902
W. Poston, J. Solka
In many situations it is desirable to operate on a subset of the data only. These can arise in the areas of experimental design, robust estimation of multivariate location, and density estimation. The paper describes a method of subset selection that optimizes the determinant of the Fisher information matrix (FIM) which is called the effective independence distribution (EID) method. It provides some motivation that justifies the use of the EID, and the problem of finding the subset of points to use in the estimation of the minimum volume ellipsoid (MVE) is examined as an application of interest.
{"title":"Choosing data sets that optimize the determinant of the Fisher information matrix","authors":"W. Poston, J. Solka","doi":"10.1109/WITS.1994.513902","DOIUrl":"https://doi.org/10.1109/WITS.1994.513902","url":null,"abstract":"In many situations it is desirable to operate on a subset of the data only. These can arise in the areas of experimental design, robust estimation of multivariate location, and density estimation. The paper describes a method of subset selection that optimizes the determinant of the Fisher information matrix (FIM) which is called the effective independence distribution (EID) method. It provides some motivation that justifies the use of the EID, and the problem of finding the subset of points to use in the estimation of the minimum volume ellipsoid (MVE) is examined as an application of interest.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129600136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513903
J. Solka, C. Priebe, G. Rogers, W. Poston, D. Marchette
This paper examines the application of Akaike (1974) information criterion (AIC) based pruning to the refinement of nonparametric density estimates obtained via the adaptive mixtures (AM) procedure of Priebe (see JASA, vol.89, no.427, p.796-806, 1994) and Marchette. The paper details a new technique that uses these two methods in conjunction with one another to predict the appropriate number of terms in the mixture model of an unknown density. Results that detail the procedure's performance when applied to different distributional classes are presented. Results are presented on artificially generated data, well known data sets, and some features for mammographic screening.
{"title":"The application of Akaike information criterion based pruning to nonparametric density estimates","authors":"J. Solka, C. Priebe, G. Rogers, W. Poston, D. Marchette","doi":"10.1109/WITS.1994.513903","DOIUrl":"https://doi.org/10.1109/WITS.1994.513903","url":null,"abstract":"This paper examines the application of Akaike (1974) information criterion (AIC) based pruning to the refinement of nonparametric density estimates obtained via the adaptive mixtures (AM) procedure of Priebe (see JASA, vol.89, no.427, p.796-806, 1994) and Marchette. The paper details a new technique that uses these two methods in conjunction with one another to predict the appropriate number of terms in the mixture model of an unknown density. Results that detail the procedure's performance when applied to different distributional classes are presented. Results are presented on artificially generated data, well known data sets, and some features for mammographic screening.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123985638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513863
M. D. Riley
Several applications of statistical tree-based modelling are described to problems in speech and language, including prediction of possible phonetic realizations, segment duration modelling in speech synthesis and end of sentence detection in text analysis.
{"title":"Tree-based models for speech and language","authors":"M. D. Riley","doi":"10.1109/WITS.1994.513863","DOIUrl":"https://doi.org/10.1109/WITS.1994.513863","url":null,"abstract":"Several applications of statistical tree-based modelling are described to problems in speech and language, including prediction of possible phonetic realizations, segment duration modelling in speech synthesis and end of sentence detection in text analysis.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123356625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 1994-10-27DOI: 10.1109/WITS.1994.513862
Q. Xie, R. Ward, C. Laszlo
VQ-based method is developed as an effective data reduction technique for nonparametric classifier design. This new technique, while insisting on competitive classification accuracy, is found to overcome the usual disadvantage of traditional nonparametric classifiers of being computationally complex and of requiring large amounts of computer storage.
{"title":"Nonparametric classifier design using vector quantization","authors":"Q. Xie, R. Ward, C. Laszlo","doi":"10.1109/WITS.1994.513862","DOIUrl":"https://doi.org/10.1109/WITS.1994.513862","url":null,"abstract":"VQ-based method is developed as an effective data reduction technique for nonparametric classifier design. This new technique, while insisting on competitive classification accuracy, is found to overcome the usual disadvantage of traditional nonparametric classifiers of being computationally complex and of requiring large amounts of computer storage.","PeriodicalId":423518,"journal":{"name":"Proceedings of 1994 Workshop on Information Theory and Statistics","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1994-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126559803","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：