Big data create values for business and research, but pose signiﬁcant challenges in terms of networking, storage, management, analytics and ethics. Multidisciplinary collaborations from engineers, computer scientists, statisticians and social scientists are needed to tackle, discover and understand big data. This survey presents an overview of big data initiatives, technologies and research in industries and academia, and discusses challenges and potential solutions.
BIG DATA IN COMPUTER SCIENCE AND ENGINEERING
This section discusses the big data research and applications from the work of computer scientists and engineers in academia and industries. It is primarily based on publications from the Association for Computer Machinery (ACM), IEEE Xplore Digital Library and Google Scholar using keywords such as big data, large-scale or high-dimensional data.
BIG DATA IN STATISTICS
In the big data era, a greater use of analytics is required to uncover hidden patterns and relationships among big data. New tools are needed for big data exploration and visualization to support fast or even real-time decision making, and ﬁnd information that were unable to be discovered in the past. Big data need computational statisticians in collaboration with networking engineers and computer scientists. For this section, materials were obtained from publications including Journal of American Statistical Association (JASA), Journal of the Royal Statistical Society, Pattern Recognition, Biostatistics, Biometrics, Biometrika, Statistica Sinica, Bioinformatics, and Statistics in Medicine.
Big data has the advantages of dramatic cost reductions, substantial improvement in the time to perform a computing task, and in offering new services. With the advance of big data, we could answer questions that are beyond research in the past, extract knowledge and insight from data, and can even improve the productivity of business and create substantial values for the world economy. However, it should be noted that the primary values of big data come not from its raw form, but from its processing and analysis. The sweeping changes in big data technologies and management will result in the multidisciplinary collaborations to support decision making and service innovation.
Two trends are making big data increasingly attractive: the ubiquity of mobile phones and advances in physical instrumentation. While big data can yield extremely useful information and value, it presents different kinds of challenges. For example, we need to understand how much data to store, how much it costs, whether the data will be secured, and how long it must be maintained. McKinsey Global Institute identiﬁed big data challenges in ﬁve domains: healthcare in the United States, the public sector in Europe, retail in the United States, manufacturing, and personal-location data globally.
Source: University of Massachusetts Medical School
Authors: Hua (Julia) Fang | Zhaoyang Zhang | Chanpaul Jin Wang