Data mining ocean model output at the Naval Oceanographic Office Shared Resource Center

Abstract

One of the computational technology areas supported by the High Performance Computing Modernization Program is Climate, Weather, and Ocean (CWO) modeling. To this end, state-of-art computing architectures are leveraged against the extremely difficult problem of mathematically modeling and predicting the behavior of a variety of ocean climatological parameters. The problem at hand is the technology to store, retrieve, manipulate, and display these data has not kept pace with the computational technology. During the last five years, we have seen significant cost reductions associated with applying the status quo in visualization techniques to scientific data sets. This is due in large part to the computer gaming industry, driven by the huge profit margins associated with that market. The scientific community has benefited by these advances in low-cost architectures, but only as a by-product of its original intent, which is entertainment. Even so, these low-cost architectures are not designed to handle the scale of data sizes presented by the scientific community and serve only to make inadequate techniques cheaper to field and use. The Naval Oceanographic Office Major Shared Resource Center (NAVO MSRC) Visualization Center is challenged with providing its users state-of-the-art analysis environments for the interrogation of their increasingly large data sets. This paper deals with the data generated by the CWO community, all of whom work with large domains and high resolutions (either vertically, horizontally, or both) that all vary over time. This leads to very large data sets (rows columns layers attribute per cell) for each time step and can challenge even the most powerful architectures when trying to extract or "mine" information from the raw data. As in most visualization applications, the model output deals with physical parameters that are invisible to the naked eye. This means effective methods of display are required for ocean circulation or currents, sea surface height, temperature, salinity, and so on. One analogy, which no doubt started the concept of "data mining", is that the raw data represent a huge block of ore from which gold nuggets of valuable information (features) must be extracted or mined. This paper concerns the technical solutions that were built to solve the challenges described above, including algorithms, data descriptions, and formats.