Back

Big Data: An Introduction and Application in the Social Sector

Big data, as the name indicates, signifies a large set in the range of at least one terabyte of unstructured and structured data. The large set of data makes it difficult to handle with traditional methods. Instead, a unique set of techniques and algorithms are needed to explore trends and patterns for decision-making – called ‘Big Data Analytics’.

Big data is characterized by 3Vs, i.e., Volume, Velocity, and Variety, as described in the section below:

Volume

Volume is central to the concept of Big data. In today’s time, social media generates a huge volume of data ranging in billions of tweets, YouTube and Instagram views every day. Additionally, huge government records or data generated through new-age technology or big corporation user data must be handled using big data analytics for real-time decision-making.

Velocity

Velocity signifies the speed with which data is being generated. For instance, in the world of social media, every second and minute data is posted. Big Data architecture and analytics play a crucial role in storing the inflow of the data, making sense of it, and building on the process to adapt to the velocity of the data for decision-making.

Variety

The variety of the big data available alongside its volume and velocity makes the whole concept exciting and compelling. It includes both structured and unstructured data generated either through social media or by large corporations and includes texts, tweets, pictures & videos, voicemails, audio recordings, etc. Looking at the variety of the data available, it is quintessential to use a range of techniques to help classify data into various categories and explore it for patterns to enabling companies in decision-making.

Application of Big Data in the Social Sector

The social sector has, in recent years, started to use big data analytics to improving efficiency in operations for better targeting. For example, Big Data Analytics can be used effectively in the education sector to develop content for students based on their early learning experiences. Technology-enabled devices can be used to diagnose basic health screenings at the primary health care facility and predictive analytics can be used to predict health coverage indicators and footfalls across health facilities for assessment.

Multi-lateral and bilateral institutions have also supported global collaboration for social good. One such example is the UN Global Pulse initiative to collaborate on big data and AI for social good. The platform lists several projects ranging from operational response simulation tools for epidemics to how disaggregated data can help in providing more inclusive transport.

References

  • B. W. Wirtz, J. C. Weyerer, and C. Geyer. (2019). Artificial Intelligence and the Public Sector—Applications and Challenges(link is external). International Journal of Public Administration. 4 (:7). pp. 596–615
  • J. Berryhill, et al. 2019. Hello, World: Artificial Intelligence and Its Use in the Public Sector(link is external). OECD Working Papers on Public Governance. No. 36. Paris: OECD Publishing.
  • Yu, K.-H., Beam, A. & Kohane, I. S. (2018). Artificial intelligence in healthcare. Nat. Biomed. Eng.2, 719–731.
  • January 22, 2019, Center for Inclusive Growth. Mastercard and The Rockefeller Foundation Announce ‘Data Science for Social Impact’ with Initial $50 Million Commitment. Retrieved from: Data Science Project

For more, check out:

Data Science for Social good

UN Global Pulse Labs

AI for Social Good

Microsoft AI for Humanity

Kultar Singh – Chief Executive Officer, Sambodhi

Kultar Singh