基于近红外光谱GADF和RGB图像的木材树种识别深度学习多模态融合框架

IF 2.2 3区 农林科学 Q2 FORESTRY Holzforschung Pub Date : 2023-11-09 DOI:10.1515/hf-2023-0062
Xi Pan, Zhiming Yu, Zhong Yang
{"title":"基于近红外光谱GADF和RGB图像的木材树种识别深度学习多模态融合框架","authors":"Xi Pan, Zhiming Yu, Zhong Yang","doi":"10.1515/hf-2023-0062","DOIUrl":null,"url":null,"abstract":"Abstract Accurate and rapid wood species identification is vital for wood utilization and trade. This goal is achievable with the fast development of deep learning (DL). Several studies have been published related to this topic; however, they were limited by their generalization performance in practical applications. Therefore, this study proposed a DL multimodal fusion framework to bridge this gap. The study utilized a state-of-the-art convolutional neural network (CNN) to simultaneously extract both short-wavelength near-infrared (NIR) spectra and RGB image feature, fully leveraging the advantages of both data types. Using portable devices for collecting spectra and image data enhances the feasibility of onsite rapid identification. In particular, a two-branch CNN framework was developed to extract spectra and image features. For NIR spectra feature extraction, 1 dimensional NIR (1D NIR) spectra were innovatively encoded as 2 dimensional (2D) images using the Gramian angular difference field (GADF) method. This representation enhances better data alignment with CNN operations, facilitating more robust discriminative feature extraction. Moreover, wood’s spectral and image features were fused at the full connection layer for species identification. In the experimental phase conducted on 16 difficult-to-distinguish wood samples from the Lauraceae family, all achieved identification metrics results exceed 99 %. The findings illustrate that the proposed multimodal fusion framework effectively extracts and fully integrates the wood’s features, thereby, improving wood species identification.","PeriodicalId":13083,"journal":{"name":"Holzforschung","volume":" 38","pages":"0"},"PeriodicalIF":2.2000,"publicationDate":"2023-11-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A deep learning multimodal fusion framework for wood species identification using near-infrared spectroscopy GADF and RGB image\",\"authors\":\"Xi Pan, Zhiming Yu, Zhong Yang\",\"doi\":\"10.1515/hf-2023-0062\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract Accurate and rapid wood species identification is vital for wood utilization and trade. This goal is achievable with the fast development of deep learning (DL). Several studies have been published related to this topic; however, they were limited by their generalization performance in practical applications. Therefore, this study proposed a DL multimodal fusion framework to bridge this gap. The study utilized a state-of-the-art convolutional neural network (CNN) to simultaneously extract both short-wavelength near-infrared (NIR) spectra and RGB image feature, fully leveraging the advantages of both data types. Using portable devices for collecting spectra and image data enhances the feasibility of onsite rapid identification. In particular, a two-branch CNN framework was developed to extract spectra and image features. For NIR spectra feature extraction, 1 dimensional NIR (1D NIR) spectra were innovatively encoded as 2 dimensional (2D) images using the Gramian angular difference field (GADF) method. This representation enhances better data alignment with CNN operations, facilitating more robust discriminative feature extraction. Moreover, wood’s spectral and image features were fused at the full connection layer for species identification. In the experimental phase conducted on 16 difficult-to-distinguish wood samples from the Lauraceae family, all achieved identification metrics results exceed 99 %. The findings illustrate that the proposed multimodal fusion framework effectively extracts and fully integrates the wood’s features, thereby, improving wood species identification.\",\"PeriodicalId\":13083,\"journal\":{\"name\":\"Holzforschung\",\"volume\":\" 38\",\"pages\":\"0\"},\"PeriodicalIF\":2.2000,\"publicationDate\":\"2023-11-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Holzforschung\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1515/hf-2023-0062\",\"RegionNum\":3,\"RegionCategory\":\"农林科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"FORESTRY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Holzforschung","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1515/hf-2023-0062","RegionNum":3,"RegionCategory":"农林科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"FORESTRY","Score":null,"Total":0}
引用次数: 0

摘要

摘要准确、快速的树种鉴定对木材的利用和贸易至关重要。随着深度学习(DL)的快速发展,这个目标是可以实现的。已经发表了几项与此主题相关的研究;然而,在实际应用中,它们的泛化性能受到了限制。因此,本研究提出了一种DL多模态融合框架来弥补这一差距。该研究利用最先进的卷积神经网络(CNN)同时提取短波近红外(NIR)光谱和RGB图像特征,充分利用了两种数据类型的优势。使用便携式设备采集光谱和图像数据,提高了现场快速识别的可行性。特别是,开发了一种双分支CNN框架来提取光谱和图像特征。在近红外光谱特征提取方面,创新性地采用格拉曼角差场(GADF)方法将一维近红外光谱编码为二维(2D)图像。这种表示增强了与CNN操作的更好的数据一致性,促进了更鲁棒的判别特征提取。在全连接层融合木材的光谱和图像特征,进行物种识别。在实验阶段对16个樟科难辨木材样品进行了鉴定,鉴定指标均达到99%以上。结果表明,所提出的多模态融合框架能够有效地提取和充分融合木材的特征,从而提高木材的种类识别。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
A deep learning multimodal fusion framework for wood species identification using near-infrared spectroscopy GADF and RGB image
Abstract Accurate and rapid wood species identification is vital for wood utilization and trade. This goal is achievable with the fast development of deep learning (DL). Several studies have been published related to this topic; however, they were limited by their generalization performance in practical applications. Therefore, this study proposed a DL multimodal fusion framework to bridge this gap. The study utilized a state-of-the-art convolutional neural network (CNN) to simultaneously extract both short-wavelength near-infrared (NIR) spectra and RGB image feature, fully leveraging the advantages of both data types. Using portable devices for collecting spectra and image data enhances the feasibility of onsite rapid identification. In particular, a two-branch CNN framework was developed to extract spectra and image features. For NIR spectra feature extraction, 1 dimensional NIR (1D NIR) spectra were innovatively encoded as 2 dimensional (2D) images using the Gramian angular difference field (GADF) method. This representation enhances better data alignment with CNN operations, facilitating more robust discriminative feature extraction. Moreover, wood’s spectral and image features were fused at the full connection layer for species identification. In the experimental phase conducted on 16 difficult-to-distinguish wood samples from the Lauraceae family, all achieved identification metrics results exceed 99 %. The findings illustrate that the proposed multimodal fusion framework effectively extracts and fully integrates the wood’s features, thereby, improving wood species identification.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Holzforschung
Holzforschung 工程技术-材料科学:纸与木材
CiteScore
4.60
自引率
4.20%
发文量
83
审稿时长
3.3 months
期刊介绍: Holzforschung is an international scholarly journal that publishes cutting-edge research on the biology, chemistry, physics and technology of wood and wood components. High quality papers about biotechnology and tree genetics are also welcome. Rated year after year as one of the top scientific journals in the category of Pulp and Paper (ISI Journal Citation Index), Holzforschung represents innovative, high quality basic and applied research. The German title reflects the journal''s origins in a long scientific tradition, but all articles are published in English to stimulate and promote cooperation between experts all over the world. Ahead-of-print publishing ensures fastest possible knowledge transfer.
期刊最新文献
Degradation of Pinus sylvestris and Populus tremula by laccate Ganoderma species Wood density and chemical composition variation of Eucalyptus urophylla clone in different environments Wood discrimination of six commonly traded Phoebe and Machilus species using high-resolution plastid and nuclear DNA barcodes Physical, vibro-mechanical and optical properties of pernambuco in relation to bow-making qualitative evaluation and wood diversity Comparative wood and charcoal anatomy of Manilkara sp.: contribution for market inspections
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1