Executive Summary

Big Data Strategy Issues Paper

This paper provides an opportunity to consider the wide variety of options for agencies when using big data, and the tools that would help them understand what this data means. While considering these options, it is also important to consider the potential concerns that might come with it, like maintaining privacy policies.


Where are we now and why should one consider a big data strategy?

Now a day's data is produced at an increasing rate. This is being driven by:
  1. Individuals increased use of media
  2. Organizations
  3. Switching from analogue to digital technologies
  4. The increase in internet connecting devices and systems
  5. An increase in machine-generated and unstructured data which now hold 80% or more of all data holdings

The most commonly accepted definition of big data is high-volume, high-velocity, and/or high-variety information assets that need cost-effective, innovative forms of information processing for enhanced insight, decision making, and optimization.
The purpose for the development of a big data strategy is the need for a strategy to enhance cross-agency data analytic capability for improved policy and service delivery. Big data has a more evidence-based policy design and service implementation which allows citizens to interact with their government in a personalized and seamless way.

Opportunities big data presents for Australian Government agencies

  1. Data management - There are potential savings in time and money if agencies implement smarter data management practices that are conscious of the needs of big data analysis.
  2. Personalization of services- Can reveal a clear picture of the customer so it is easier to provide personalized services tailored to the individual and delivered by the government
  3. Problem solving and predictive analytics - The ability to unify multiple datasets from separate sources while using analytic techniques will advance problem solving capabilities which can help support decision making
  4. Productivity and efficiency - analysis of big data sources can be used to identify cost savings and opportunities to increase efficiency

What the future looks like

  1. The better delivery of services- Allows government agencies to deliver more personalized services that are tailored to meet the citizen's needs and preferences
  2. Improved efficiency of government operations- Allows governement agencies to better assess risk and feasibility and to detect fraud and error when using predictive analysis
  3. Open engagement Allows groups to better interact with industry, academia, non-government organizations and other interested parties locally and internationally


Privacy, security and trust

When government agencies are collecting or managing citizens' data, they need to follow legislative controls, and must comply with a number of acts and regulations. These legislative controls are designed to maintain public confidence in the government that it has an effective and secure way to store citizen information. One must understand and carefully manage certain threats that could allow opportunity for an unfriendly state and non-state actors to glean sensitive information.

Data management and sharing

Accessible information is the lifeblood of a robust democracy and a productive economy. Government agencies realize that for data to have any value it needs to be discoverable, accessible and usable, and the significance of these requirements only increases as the discussion turns towards big data. Government agencies must achieve these requirements whilst still adhering to privacy laws.

Technology and analytical systems

If big data analytics is to be adopted by agencies, a large amount of stress may be placed upon current ICT systems and solutions which presently carry the burden of processing, analysing and archiving data. Government agencies will need to manage these new requirements efficiently in order to deliver net benefits through the adoption of new technologies.


Due to its relative youth and complexity, big data will require agencies to attract employees with diverse new skill sets including: science, technological, research, statistical, analytical and interpretive skills, business acumen and creativity as well as an understanding of the underlying nature of the business process or policy intent. It is unlikely to find in one person with this skill set so this means that collaborative teams will be needed in order for agencies to meet these needs.