Pub Date : 2017-04-01Epub Date: 2017-05-11DOI: 10.1109/IC2E.2017.23
Shunxing Bao, Andrew J Plassard, Bennett A Landman, Aniruddha Gokhale
Traditional in-house, laboratory-based medical imaging studies use hierarchical data structures (e.g., NFS file stores) or databases (e.g., COINS, XNAT) for storage and retrieval. The resulting performance from these approaches is, however, impeded by standard network switches since they can saturate network bandwidth during transfer from storage to processing nodes for even moderate-sized studies. To that end, a cloud-based "medical image processing-as-a-service" offers promise in utilizing the ecosystem of Apache Hadoop, which is a flexible framework providing distributed, scalable, fault tolerant storage and parallel computational modules, and HBase, which is a NoSQL database built atop Hadoop's distributed file system. Despite this promise, HBase's load distribution strategy of region split and merge is detrimental to the hierarchical organization of imaging data (e.g., project, subject, session, scan, slice). This paper makes two contributions to address these concerns by describing key cloud engineering principles and technology enhancements we made to the Apache Hadoop ecosystem for medical imaging applications. First, we propose a row-key design for HBase, which is a necessary step that is driven by the hierarchical organization of imaging data. Second, we propose a novel data allocation policy within HBase to strongly enforce collocation of hierarchically related imaging data. The proposed enhancements accelerate data processing by minimizing network usage and localizing processing to machines where the data already exist. Moreover, our approach is amenable to the traditional scan, subject, and project-level analysis procedures, and is compatible with standard command line/scriptable image processing software. Experimental results for an illustrative sample of imaging data reveals that our new HBase policy results in a three-fold time improvement in conversion of classic DICOM to NiFTI file formats when compared with the default HBase region split policy, and nearly a six-fold improvement over a commonly available network file system (NFS) approach even for relatively small file sets. Moreover, file access latency is lower than network attached storage.
{"title":"Cloud Engineering Principles and Technology Enablers for Medical Image Processing-as-a-Service.","authors":"Shunxing Bao, Andrew J Plassard, Bennett A Landman, Aniruddha Gokhale","doi":"10.1109/IC2E.2017.23","DOIUrl":"10.1109/IC2E.2017.23","url":null,"abstract":"<p><p>Traditional in-house, laboratory-based medical imaging studies use hierarchical data structures (e.g., NFS file stores) or databases (e.g., COINS, XNAT) for storage and retrieval. The resulting performance from these approaches is, however, impeded by standard network switches since they can saturate network bandwidth during transfer from storage to processing nodes for even moderate-sized studies. To that end, a cloud-based \"medical image processing-as-a-service\" offers promise in utilizing the ecosystem of Apache Hadoop, which is a flexible framework providing distributed, scalable, fault tolerant storage and parallel computational modules, and HBase, which is a NoSQL database built atop Hadoop's distributed file system. Despite this promise, HBase's load distribution strategy of region split and merge is detrimental to the hierarchical organization of imaging data (e.g., project, subject, session, scan, slice). This paper makes two contributions to address these concerns by describing key cloud engineering principles and technology enhancements we made to the Apache Hadoop ecosystem for medical imaging applications. First, we propose a row-key design for HBase, which is a necessary step that is driven by the hierarchical organization of imaging data. Second, we propose a novel data allocation policy within HBase to strongly enforce collocation of hierarchically related imaging data. The proposed enhancements accelerate data processing by minimizing network usage and localizing processing to machines where the data already exist. Moreover, our approach is amenable to the traditional scan, subject, and project-level analysis procedures, and is compatible with standard command line/scriptable image processing software. Experimental results for an illustrative sample of imaging data reveals that our new HBase policy results in a three-fold time improvement in conversion of classic DICOM to NiFTI file formats when compared with the default HBase region split policy, and nearly a six-fold improvement over a commonly available network file system (NFS) approach even for relatively small file sets. Moreover, file access latency is lower than network attached storage.</p>","PeriodicalId":92127,"journal":{"name":"Proceedings of the IEEE International Conference on Cloud Engineering. IEEE International Conference on Cloud Engineering","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2017-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5584067/pdf/nihms843798.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"35335885","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
There are many more things with moving parts in the world than computers. These are the objects that are being connected, initially artefacts, but also the natural world. They are connected both by being sensed, and via actuators. For a true Internet of things to emerge with all its potential value for innovation in efficiencies, the sensors and actuators must actually be reachable from anywhere, anytime, just like computers on today's internet. And they must be locally and remotely programmable. Of course, there must be mechanisms to implement policies about access and use. However, these policies are complex, since they don't merely reflect informational rules, but also rules about the physical world - a car may be restricted to certain speeds in certain areas, but also to different speeds and areas at different times, due to the driver. Unfortunately, in the rush to instrument and control the world of things, the complexity of the world seems to have been forgotten. Worse, the typical system software being deployed in many places does not reflect the last few decades evolution of safety and security work that has gone in to the implementation of operating systems and protocols. All too often, we here another system uses an embedded OS with no isolation or a protocol stack with known vulnerabilities, or is shipped with default access control credentials to millions of customers. This is not good enough. In this talk, I will cover some of the work we've been doing in the Microsoft sponsored project in Cambridge and QMUL, on the technical and legal challenges that are now facing our community.
{"title":"What Could Possibly go Wrong?","authors":"J. Crowcroft","doi":"10.1109/IC2E.2016.51","DOIUrl":"https://doi.org/10.1109/IC2E.2016.51","url":null,"abstract":"There are many more things with moving parts in the world than computers. These are the objects that are being connected, initially artefacts, but also the natural world. They are connected both by being sensed, and via actuators. For a true Internet of things to emerge with all its potential value for innovation in efficiencies, the sensors and actuators must actually be reachable from anywhere, anytime, just like computers on today's internet. And they must be locally and remotely programmable. Of course, there must be mechanisms to implement policies about access and use. However, these policies are complex, since they don't merely reflect informational rules, but also rules about the physical world - a car may be restricted to certain speeds in certain areas, but also to different speeds and areas at different times, due to the driver. Unfortunately, in the rush to instrument and control the world of things, the complexity of the world seems to have been forgotten. Worse, the typical system software being deployed in many places does not reflect the last few decades evolution of safety and security work that has gone in to the implementation of operating systems and protocols. All too often, we here another system uses an embedded OS with no isolation or a protocol stack with known vulnerabilities, or is shipped with default access control credentials to millions of customers. This is not good enough. In this talk, I will cover some of the work we've been doing in the Microsoft sponsored project in Cambridge and QMUL, on the technical and legal challenges that are now facing our community.","PeriodicalId":92127,"journal":{"name":"Proceedings of the IEEE International Conference on Cloud Engineering. IEEE International Conference on Cloud Engineering","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77541425","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Summary form only given. Cloud computing represents a fundamental change in the business model behind IT: a shift from manufacturing of software and hardware products towards packaging infrastructure, processing, and storage as services. Cloud data centers, given their intended use for general purpose computing, would seem to push towards homogeneity in architectures and platforms. Modern applications and use cases, from scientific computing to big data, push in exactly the opposite direction: an increase in specialization as a way to efficiently meet demanding requirements. In this talk I will illustrate both trends and argue that, contradictory as they seem to be, there are many opportunities in combining them. Doing so requires to work in two areas. One is to find better ways to extend the performance and efficiency advantages of specialization to general purpose settings. The other is to develop the necessary software and hardware layers to allow generalized use of specialized systems. Taking together, these efforts outline an exciting research and development landscape that I will outline as a conclusion of the talk.
{"title":"Generalization versus Specialization in Cloud Computing Infrastructures","authors":"G. Alonso","doi":"10.1109/IC2E.2016.50","DOIUrl":"https://doi.org/10.1109/IC2E.2016.50","url":null,"abstract":"Summary form only given. Cloud computing represents a fundamental change in the business model behind IT: a shift from manufacturing of software and hardware products towards packaging infrastructure, processing, and storage as services. Cloud data centers, given their intended use for general purpose computing, would seem to push towards homogeneity in architectures and platforms. Modern applications and use cases, from scientific computing to big data, push in exactly the opposite direction: an increase in specialization as a way to efficiently meet demanding requirements. In this talk I will illustrate both trends and argue that, contradictory as they seem to be, there are many opportunities in combining them. Doing so requires to work in two areas. One is to find better ways to extend the performance and efficiency advantages of specialization to general purpose settings. The other is to develop the necessary software and hardware layers to allow generalized use of specialized systems. Taking together, these efforts outline an exciting research and development landscape that I will outline as a conclusion of the talk.","PeriodicalId":92127,"journal":{"name":"Proceedings of the IEEE International Conference on Cloud Engineering. IEEE International Conference on Cloud Engineering","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2016-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77673351","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
扫码关注我们
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号