Working with Big Data
By Sara Royster
The purpose of this article is to inform the reader about what big data is, who uses it, how it is used, what challenges are present, and the overall requirements needed to work with big data.
What is Big Data
Big data is the collection of large datasets that cannot be analyzed with normal statistical methods. The data can be anything from numbers, words, pictures, videos, maps, etc. There are two different types of big data.
Structured data is where numbers and words can be easily categorized and analyzed.
Unstructured data is where the data is too complex and cannot be easily put into categories and or analyzed. Usually deals with what people are saying.
Who Uses Big Data
Workers who deal with big data are called data scientists. Data scientists run computer programs and algorithms to detect patterns or for usable information. Some occupations that use big data:
∑ Managers- The chief data officers that decide on how the data will be used/supervise.
∑ Postsecondary Teachers- Instruct students in statistical analysis and computer science.
∑ Software Developers- Write programs that aggregate, process, analyze, and visualize the data.
How is Big Data Used
How the big data is used is determined by the field. Here are several fields that use big data:
∑ Business- Uses big data to help with the businessís efficiency and overall improvements.
∑ E-commerce- Uses commercial websites to help with costumer service.
∑ Finance- Uses big financial data to look for fraud and security breaches.
∑ Government- Uses big data to help their constituents and improve policy decisions.
∑ Healthcare- Uses big data to help patients, track diseases, and drug development.
∑ Social Networking- Uses big data to help with advertising and costumer service.
∑ Telecommunications- Uses big data to track users and help with costumer preferences.
Challenges with Big Data
With big data being a new phenomenon, the funding for it is very limited. Since there can be thousands and even millions of data points, the overall storage of the points can be a challenge. Most have to be stored on multiple servers. Given the large amount of data, finding the usable data can be quite difficult. With the usable data being so hard to find and organize, the accuracy of what is being found can be in jeopardy.
Requirements to Use Big Data
In order to work with big data one usually needs knowledge of statistical analysis, computer systems and experience in the relevant field. Most workers in big data not only have a bachelorís degree but usually a masters or even higher. A background in math, statistics, and computer programming are a must for data scientists. Engineers are the most common data scientists because most already have that sort of background. Some important skills needed for working with big data are communication skills, teamwork and curiosity.