Summary form only given: Today a huge amount of geospatial data is being created, collected and used more than ever before. The ever increasing observations and measurements of geo-sensor networks, satellite imageries, point clouds from laser scanning, geospatial data of Location Based Services (LBS) and location-based social networks has become a serious challenge for data management and analysis systems. Traditionally, Relational Database Management Systems (RDBMS) were used to manage and to some extent analyze the geospatial data. Nowadays these systems can be used in many scenarios but there are some situations when using these systems may not provide the required efficiency and effectiveness. More specifically when the geospatial data has high volume, high frequency of change (in both data content and data structure) and variety of structures, the conventional data storage systems cannot provide needed efficiency in online systems in terms of performance and scalability. In these situations, NoSQL solutions can provide the efficiency necessary for applications using geospatial data. This paper provides an overview of the characteristics of geospatial big data, possible solutions for managing and processing them. Then the paper provides an overview of the major types of NoSQL solutions, their advantages and disadvantages and the challenges they present in managing geospatial big data. Then the paper elaborates on serving geospatial data using standard geospatial web services with a NoSQL XML database as a backend.
{"title":"Efficient Online Sharing of Geospatial Big Data Using NoSQL XML Databases","authors":"P. Amirian, A. Bassiri, A. Winstanley","doi":"10.1109/COMGEO.2013.34","DOIUrl":"https://doi.org/10.1109/COMGEO.2013.34","url":null,"abstract":"Summary form only given: Today a huge amount of geospatial data is being created, collected and used more than ever before. The ever increasing observations and measurements of geo-sensor networks, satellite imageries, point clouds from laser scanning, geospatial data of Location Based Services (LBS) and location-based social networks has become a serious challenge for data management and analysis systems. Traditionally, Relational Database Management Systems (RDBMS) were used to manage and to some extent analyze the geospatial data. Nowadays these systems can be used in many scenarios but there are some situations when using these systems may not provide the required efficiency and effectiveness. More specifically when the geospatial data has high volume, high frequency of change (in both data content and data structure) and variety of structures, the conventional data storage systems cannot provide needed efficiency in online systems in terms of performance and scalability. In these situations, NoSQL solutions can provide the efficiency necessary for applications using geospatial data. This paper provides an overview of the characteristics of geospatial big data, possible solutions for managing and processing them. Then the paper provides an overview of the major types of NoSQL solutions, their advantages and disadvantages and the challenges they present in managing geospatial big data. Then the paper elaborates on serving geospatial data using standard geospatial web services with a NoSQL XML database as a backend.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117202583","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Motivated by lacking the capability of supporting geospatial workflow composition and execution in a Web environment from leading GIS (such as ESRI ArcGIS), we have developed a prototype system by integrating mature open source and commercial software packages in an innovative way. Our prototype system includes a client module for visual and interactive workflow editing based on Ptolemy II (a modeling and design system), a geospatial actor library representing 500+ ArcGIS geoprocessing tools for drag-and-drop-based workflow composition, a middleware as a workflow engine to schedule and execute ArcGIS Geoprocessing tools based on composed geospatial workflows, and, a Web-GIS to visualize original and derived data along a workflow processing pipeline. By reusing the mature software packages, we are able to complete the prototype development within weeks instead of months or years. A site selection problem that involves multiple geospatial operations are used to demonstrate the functionality and features of the prototype system.
{"title":"A practical approach to developing a web-based geospatial workflow composition and execution system","authors":"Jianting Zhang","doi":"10.1145/2345316.2345341","DOIUrl":"https://doi.org/10.1145/2345316.2345341","url":null,"abstract":"Motivated by lacking the capability of supporting geospatial workflow composition and execution in a Web environment from leading GIS (such as ESRI ArcGIS), we have developed a prototype system by integrating mature open source and commercial software packages in an innovative way. Our prototype system includes a client module for visual and interactive workflow editing based on Ptolemy II (a modeling and design system), a geospatial actor library representing 500+ ArcGIS geoprocessing tools for drag-and-drop-based workflow composition, a middleware as a workflow engine to schedule and execute ArcGIS Geoprocessing tools based on composed geospatial workflows, and, a Web-GIS to visualize original and derived data along a workflow processing pipeline. By reusing the mature software packages, we are able to complete the prototype development within weeks instead of months or years. A site selection problem that involves multiple geospatial operations are used to demonstrate the functionality and features of the prototype system.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"256 2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123286560","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The augmentation of objects by textual annotations provides a powerful means for visual data exploration. Especially in interactive scenarios, where the view on the objects and, thus, the preferred placement of annotations changes continually, efficient labeling procedures are required. As identified by a preliminary study for this paper, these procedures have to consider a number of requirements for achieving an optimal readability, e.g. cartographic principles, visual association and temporal coherence. In this paper, we present a force-based labeling algorithm for 2D and 3D scenes, which can compute the placements of annotations at very high speed and fulfills the identified requirements. The efficient labeling of several hundred annotations is achieved by computing their layout in parallel on the GPU. This allows for a real-time and collision-free arrangement of both dynamically changing and static information. We demonstrate that our method supports a large variety of applications, e.g. geographical information systems, automotive navigation systems, and scientific or information visualization systems. We conclude the paper with an expert study which confirms the enhancements brought by our algorithm with respect to visual association and readability.
{"title":"Temporally coherent real-time labeling of dynamic scenes","authors":"M. Vaaraniemi, M. Treib, R. Westermann","doi":"10.1145/2345316.2345337","DOIUrl":"https://doi.org/10.1145/2345316.2345337","url":null,"abstract":"The augmentation of objects by textual annotations provides a powerful means for visual data exploration. Especially in interactive scenarios, where the view on the objects and, thus, the preferred placement of annotations changes continually, efficient labeling procedures are required. As identified by a preliminary study for this paper, these procedures have to consider a number of requirements for achieving an optimal readability, e.g. cartographic principles, visual association and temporal coherence. In this paper, we present a force-based labeling algorithm for 2D and 3D scenes, which can compute the placements of annotations at very high speed and fulfills the identified requirements. The efficient labeling of several hundred annotations is achieved by computing their layout in parallel on the GPU. This allows for a real-time and collision-free arrangement of both dynamically changing and static information. We demonstrate that our method supports a large variety of applications, e.g. geographical information systems, automotive navigation systems, and scientific or information visualization systems. We conclude the paper with an expert study which confirms the enhancements brought by our algorithm with respect to visual association and readability.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"65 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124682555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Today's business and government organizations are challenged when trying to manage and analyze information from enterprise databases, streaming servers, social media and open source. This is compounded by the complexity of integrating diverse data types (relational, text, spatial, images, spreadsheets) and their representations (customers, products, suppliers, events, and locations) - all of which need to be understood and re-purposed in different contexts. Identifying meaningful patterns across these different information sources is non-trivial. Moreover, conventional IT tools, such as conventional data warehousing and business intelligence alone, are insufficient at handling the volumes, velocity and variety of content at hand. A new framework and associated tools are needed. Dr. Lopez outlines how data scientists and analysts are applying Spatial and Semantic Web concepts to make sense of this Big Data stream. He will describe new approaches oriented toward search, discovery, linking, and analyzing information on the Web, and throughout the enterprise. The role of Map Reduce is described, as is importance of engineered systems to simplify the creation and configuration of Big Data environments. The key take away is use of spatial and linked open data concepts to enhance content alignment, interoperability, discovery and analytics in the Big Data stream.
{"title":"Big data and advanced spatial analytics","authors":"Xavier Lopez","doi":"10.1145/2345316.2345322","DOIUrl":"https://doi.org/10.1145/2345316.2345322","url":null,"abstract":"Today's business and government organizations are challenged when trying to manage and analyze information from enterprise databases, streaming servers, social media and open source. This is compounded by the complexity of integrating diverse data types (relational, text, spatial, images, spreadsheets) and their representations (customers, products, suppliers, events, and locations) - all of which need to be understood and re-purposed in different contexts. Identifying meaningful patterns across these different information sources is non-trivial. Moreover, conventional IT tools, such as conventional data warehousing and business intelligence alone, are insufficient at handling the volumes, velocity and variety of content at hand. A new framework and associated tools are needed. Dr. Lopez outlines how data scientists and analysts are applying Spatial and Semantic Web concepts to make sense of this Big Data stream. He will describe new approaches oriented toward search, discovery, linking, and analyzing information on the Web, and throughout the enterprise. The role of Map Reduce is described, as is importance of engineered systems to simplify the creation and configuration of Big Data environments. The key take away is use of spatial and linked open data concepts to enhance content alignment, interoperability, discovery and analytics in the Big Data stream.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121553891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Data mining is the process of discovering new patterns and relationships in large datasets. However, several studies have shown that general data mining techniques often fail to extract meaningful patterns and relationships from the spatial data owing to the violation of fundamental geospatial principles. In this tutorial, we introduce basic principles behind explicit modeling of spatial and semantic concepts in data mining. In particular, we focus on modeling these concepts in the widely used classification, clustering, and prediction algorithms. Classification is the process of learning a structure or model (from user given inputs) and applying the known model to the new data. Clustering is the process of discovering groups and structures in the data that are "similar," without applying any known structures in the data. Prediction is the process of finding a function that models (explains) the data with least error. One common assumption among all these methods is that the data is independent and identically distributed. Such assumptions do not hold well in spatial data, where spatial dependency and spatial heterogeneity are a norm. In addition, spatial semantics are often ignored by the data mining algorithms. In this tutorial we cover recent advances in explicitly modeling of spatial dependencies and semantic concepts in data mining.
{"title":"Modeling spatial dependencies and semantic concepts in data mining","authors":"Ranga Raju Vatsavai","doi":"10.1145/2345316.2345369","DOIUrl":"https://doi.org/10.1145/2345316.2345369","url":null,"abstract":"Data mining is the process of discovering new patterns and relationships in large datasets. However, several studies have shown that general data mining techniques often fail to extract meaningful patterns and relationships from the spatial data owing to the violation of fundamental geospatial principles. In this tutorial, we introduce basic principles behind explicit modeling of spatial and semantic concepts in data mining. In particular, we focus on modeling these concepts in the widely used classification, clustering, and prediction algorithms. Classification is the process of learning a structure or model (from user given inputs) and applying the known model to the new data. Clustering is the process of discovering groups and structures in the data that are \"similar,\" without applying any known structures in the data. Prediction is the process of finding a function that models (explains) the data with least error. One common assumption among all these methods is that the data is independent and identically distributed. Such assumptions do not hold well in spatial data, where spatial dependency and spatial heterogeneity are a norm. In addition, spatial semantics are often ignored by the data mining algorithms. In this tutorial we cover recent advances in explicitly modeling of spatial dependencies and semantic concepts in data mining.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114830412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Attacks like fault data injection are not easy to prevent in resource-limited sensor networks. Especially in environments with urgent decision making trustworthy sensor networks are mandatory. Redundancy can be used to detect and isolate malicious behaving nodes and thus to secure the network. The presented approach implements trust based on off-the-shelf wireless sensor nodes and is more power efficient than one-single trusted node implementations with TPM technology.
{"title":"Evidence theory for reputation-based trust in wireless sensor networks","authors":"A. Matheus, Björn Stelte","doi":"10.1145/2345316.2345360","DOIUrl":"https://doi.org/10.1145/2345316.2345360","url":null,"abstract":"Attacks like fault data injection are not easy to prevent in resource-limited sensor networks. Especially in environments with urgent decision making trustworthy sensor networks are mandatory. Redundancy can be used to detect and isolate malicious behaving nodes and thus to secure the network. The presented approach implements trust based on off-the-shelf wireless sensor nodes and is more power efficient than one-single trusted node implementations with TPM technology.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"25 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126214820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents a new approach to improve the accuracy and time needed to assess the structural damage based on imaging and sensor fusion technologies. The major structural properties (i.e., global properties, temperature, and deformation) are employed, which can be obtained through different kinds of sensors. Enhancements of visual images including thermal imaging and historical data are important methods to determinate both visible and invisible structural stability. Crack detection is given to further enhance the assessment. The latest GPGPU (General-Purpose Graphics Processing Unit) technology to help improve computation performance is introduced in briefly. An expert system is created to assist final sensor fusion and analysis for structural stability determination.
{"title":"Image-based structural damage assessment with sensor fusion","authors":"P. Chang, Duoduo Liao","doi":"10.1145/2345316.2345351","DOIUrl":"https://doi.org/10.1145/2345316.2345351","url":null,"abstract":"This paper presents a new approach to improve the accuracy and time needed to assess the structural damage based on imaging and sensor fusion technologies. The major structural properties (i.e., global properties, temperature, and deformation) are employed, which can be obtained through different kinds of sensors. Enhancements of visual images including thermal imaging and historical data are important methods to determinate both visible and invisible structural stability. Crack detection is given to further enhance the assessment. The latest GPGPU (General-Purpose Graphics Processing Unit) technology to help improve computation performance is introduced in briefly. An expert system is created to assist final sensor fusion and analysis for structural stability determination.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"81 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132003028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The cross section and profile of waterways are constantly subjected to alteration due to changes in flow volume and velocity. Banks and beds of waterways could experience excessive scouring especially at locations with sharp bents. At these locations, the soil constituting the cross section of the waterway could be subjected to considerable degradation, which could significantly alter the flow pattern at these locations. With severe degradation, the volume of sediment transport increases and may exceed the water carrying capacity resulting in the phenomenon known as aggradation. Contrary to degradation, aggradation results from the deposition of carried aggregate transport, which can hinder water flow in the waterway and obstruct its path. As degradation and aggradation take hold at some location along the waterway, their effect gets compounded over time which exacerbates the problem and make it difficult for the waterway to recover. Geographic Information Systems (GIS) is used to study the effect of degradation and aggradation along the Schoharie Creek, which is one of the major tributaries of the Mohawk River in upstate New York. The change in the selected locations will be examined over time to show the gradual alteration that a given section experiences and its effect on flow pattern and waterway profile. The analysis will also include a Digital Elevation Model (DEM) study of bank slopes based on the creation of a contour map. The August and September 2011 Tropical Storms Irene and Lee, respectively, left in their wake tremendous change to the waterway due to excessive degradation and aggradation. This effect was sensed due to the severe brunt it brought on the area landscape and its infrastructure. This study will identify the areas most in need for buffering and most susceptible to the impact of these natural phenomena. This will help implement proper protection methods, and in case of a damage, it will help plan for effective restoration systems.
{"title":"Variation of flow pattern in waterways due to degradation and aggradation","authors":"A. Ghaly","doi":"10.1145/2345316.2345356","DOIUrl":"https://doi.org/10.1145/2345316.2345356","url":null,"abstract":"The cross section and profile of waterways are constantly subjected to alteration due to changes in flow volume and velocity. Banks and beds of waterways could experience excessive scouring especially at locations with sharp bents. At these locations, the soil constituting the cross section of the waterway could be subjected to considerable degradation, which could significantly alter the flow pattern at these locations. With severe degradation, the volume of sediment transport increases and may exceed the water carrying capacity resulting in the phenomenon known as aggradation. Contrary to degradation, aggradation results from the deposition of carried aggregate transport, which can hinder water flow in the waterway and obstruct its path. As degradation and aggradation take hold at some location along the waterway, their effect gets compounded over time which exacerbates the problem and make it difficult for the waterway to recover. Geographic Information Systems (GIS) is used to study the effect of degradation and aggradation along the Schoharie Creek, which is one of the major tributaries of the Mohawk River in upstate New York. The change in the selected locations will be examined over time to show the gradual alteration that a given section experiences and its effect on flow pattern and waterway profile. The analysis will also include a Digital Elevation Model (DEM) study of bank slopes based on the creation of a contour map. The August and September 2011 Tropical Storms Irene and Lee, respectively, left in their wake tremendous change to the waterway due to excessive degradation and aggradation. This effect was sensed due to the severe brunt it brought on the area landscape and its infrastructure. This study will identify the areas most in need for buffering and most susceptible to the impact of these natural phenomena. This will help implement proper protection methods, and in case of a damage, it will help plan for effective restoration systems.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"69 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114813423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The amount of data each organization deals with today has been rapidly growing. However, analyzing large datasets commonly referred to as "big data" has been a huge challenge due to lack of suitable tools and adequate computing resources. Why are organizations, both in public sector and private sector, so keen on unlocking business insights from all structured and unstructured data? What is the current state of big data solutions and service providers? How effective are some of the solutions that have been put into real world practices? What is the current state of cloud computing technologies? What impacts have cloud computing technologies available in public clouds and private clouds had on the way organizations addressing big data challenges? How to secure big data in the clouds? What are the future roadmaps for cloud-based big data solutions, especially for geospatial related applications? This panel discussion will include a short presentation or discussion related to big data and cloud computing by each panelist, followed by questions and questions from the audience and the panel.
{"title":"Cloud computing & big data computing","authors":"Zhiming Xue","doi":"10.1145/2345316.2345328","DOIUrl":"https://doi.org/10.1145/2345316.2345328","url":null,"abstract":"The amount of data each organization deals with today has been rapidly growing. However, analyzing large datasets commonly referred to as \"big data\" has been a huge challenge due to lack of suitable tools and adequate computing resources. Why are organizations, both in public sector and private sector, so keen on unlocking business insights from all structured and unstructured data? What is the current state of big data solutions and service providers? How effective are some of the solutions that have been put into real world practices? What is the current state of cloud computing technologies? What impacts have cloud computing technologies available in public clouds and private clouds had on the way organizations addressing big data challenges? How to secure big data in the clouds? What are the future roadmaps for cloud-based big data solutions, especially for geospatial related applications?\u0000 This panel discussion will include a short presentation or discussion related to big data and cloud computing by each panelist, followed by questions and questions from the audience and the panel.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116986694","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mobile technology, especially mobile phone, is very popular nowadays. Increasing number of mobile users and availability of GPS-embedded mobile phones generate large amount of GPS trajectories that can be used in various research areas such as people mobility and transportation planning. However, how to handle such a large-scale dataset is a significant issue particularly in spatial analysis domain. In this paper, we aimed to explore a suitable way for extracting geo-location of GPS coordinate that achieve large-scale support, fast processing, and easily scalable both in storage and calculation speed. Geo-locations are cities, zones, or any interesting points. Our dataset is GPS trajectories of 1.5 million individual mobile phone users in Japan accumulated for one year. The total number was approximately 9.2 billion records. Therefore, we conducted performance comparisons of various methods for processing spatial data, particularly for a huge dataset. In this work, we first processed data on PostgreSQL with PostGIS that is a traditional way for spatial data processing. Second, we used java application with spatial library called Java Topology suite (JTS). Third, we tried on Hadoop Cloud Computing Platform focusing on using Hive on top of Hadoop to allow SQL-like support. However, Hadoop/Hive did not support spatial query at the moment. Hence, we proposed a solution to enable spatial support on Hive. As the results, Hadoop/hive with spatial support performed best result in large-scale processing among evaluated methods and in addition, we recommended techniques in Hadoop/Hive for processing different types of spatial data.
{"title":"Performance comparisons of spatial data processing techniques for a large scale mobile phone dataset","authors":"Apichon Witayangkurn, T. Horanont, R. Shibasaki","doi":"10.1145/2345316.2345346","DOIUrl":"https://doi.org/10.1145/2345316.2345346","url":null,"abstract":"Mobile technology, especially mobile phone, is very popular nowadays. Increasing number of mobile users and availability of GPS-embedded mobile phones generate large amount of GPS trajectories that can be used in various research areas such as people mobility and transportation planning. However, how to handle such a large-scale dataset is a significant issue particularly in spatial analysis domain. In this paper, we aimed to explore a suitable way for extracting geo-location of GPS coordinate that achieve large-scale support, fast processing, and easily scalable both in storage and calculation speed. Geo-locations are cities, zones, or any interesting points. Our dataset is GPS trajectories of 1.5 million individual mobile phone users in Japan accumulated for one year. The total number was approximately 9.2 billion records. Therefore, we conducted performance comparisons of various methods for processing spatial data, particularly for a huge dataset. In this work, we first processed data on PostgreSQL with PostGIS that is a traditional way for spatial data processing. Second, we used java application with spatial library called Java Topology suite (JTS). Third, we tried on Hadoop Cloud Computing Platform focusing on using Hive on top of Hadoop to allow SQL-like support. However, Hadoop/Hive did not support spatial query at the moment. Hence, we proposed a solution to enable spatial support on Hive. As the results, Hadoop/hive with spatial support performed best result in large-scale processing among evaluated methods and in addition, we recommended techniques in Hadoop/Hive for processing different types of spatial data.","PeriodicalId":400763,"journal":{"name":"International Conference and Exhibition on Computing for Geospatial Research & Application","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2012-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114640358","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：