Executive Summary: Working with big data


In recent year, the amount of data that is needed and stored throughout society as become very large. “90 percent of data that is used today was created in the last two years”. The amounts of data are continuously changing and growing. Many times, this vast amount of data cannot be analyzed by conventional statistical methods. The name for such data is “big data”. This data is not just numbers, but can include media, words, maps, phrases, exc. The big data can be separated into to categorize; structured and unstructured. Structured big data are large amounts of numbers or words that can easily be categorized and analyzed. Unstructured is just the opposite. Primarily, unstructured big data is compiled with more complex information; such as customer reviews and profiles.

Working with big data

In many cases, workers are needed to collect, process, and analyze big data. These workers are given the name of “data scientists”. This field is growing steadily over the past decade and will continue to show growth for years to come. Many types of career fields are in need of data scientists. Some of these fields include Business, E-commerce, Finance, Government, Healthcare, Science, Social Networking, and Telecommunications. Employment for data scientists in these fields is on a steady increase. The wage for data scientists in 2012 were around $75,000. This wage is more than double the median wage for all workers, at roughly $35,000. As the amounts of data needed to be processed grows, so too will the demand for data scientists and careers alike.

Challenges presented by big data

Although so much insight can be gain from big data, it does have its limitations. First, gathering the necessary funds for improved computing and software for big data analysis has become arduous. Due to the recent economic recession, the government has been targeting these programs for budget cuts. Continuing, in order to process and analyze the copious amounts of data, there needs to be adequate storage. Having the amount of servers needed to retain all this information has proved to be difficult. Lastly, protecting big data is a challenge as well. Sensitive information can be found in big data, and keeping such information from the wrong hands is crucial. Financial medical records, location data, and telecommunications data are among those deemed highly sensitive.

Preparing to work with big data

When attempting to be a possible data scientist, having the specified academic curriculum and degree go a long way. Taking mathematical courses as well as statistic courses teaches the students to think in a problem solving way. When attempting to enter the software side of this field, it is a necessity to have experience in computer-programming. Taking this type of course load teaches students more than just what can be learn through on the job experience.


Problem solving is one of many skills that are sought after when entering a field involving big data. Having the ability to communicate information clearly to co-workers is a positive. Team work is equally as important. Being able to work well with others in an efficient manner is imperative. And most important, showing genuine intellectual curiosity is a must. Technology is continuously changing, and being able to adapt and learn to use these new technologies is vital.