What is BIG Data?

Big Data is a relatively new term in the business world. This post aims to shed some light on what we mean when we say Big Data.

As we can infer from its translation, Big Data is simply a set of technologies and tools that make it possible to work with massive datasets. There are countless formal definitions of Big Data, but we are going to mention one of the earliest: the one formulated by Gartner around 2001, which is still a reference today: “Big Data is data that contains greater variety and is presented in increasing volumes and at higher velocity”.

Thus, Big Data emerged to respond to a reality that did not previously exist: how to store and process excessive, growing volumes of data that are generated at enormous speeds.

The 5 Vs of Big Data

Gartner’s definition refers to the intrinsic characteristics that differentiate Big Data environments: Volume, Velocity, and Variety. These are the so-called 3 Vs of Big Data.

• Volume: in Big Data, we always refer to large datasets. Big Data literally means “massive data”, so we can say that volume is the characteristic most associated with Big Data.
• Velocity: this data is generated at breakneck speed, and systems must record it quickly to avoid bottlenecks. Likewise, it is sometimes necessary to process it in real time for effective decision-making.
• Variety: the data you work with may be structured data, such as that stored in database systems; or unstructured data, which includes text documents, emails, audio, video, images, sensor data, etc. Most of the data generated today is unstructured, which adds significant difficulty when it comes to managing it and extracting information from it.

Although Gartner’s definition refers to the three fundamental characteristics of any Big Data ecosystem, in recent years it has become somewhat insufficient, and two important characteristics have been added to consider: veracity and value.

• Veracity: With enormous volumes of data coming from multiple and varied sources and received at very high speeds, we have more than enough reason to question the veracity of the data received. This poses a major challenge for organizations, as it is essential to ensure that the information received is accurate and trustworthy. It is common to receive incorrect or incomplete data which, depending on the data in question, can cause harm when carrying out the relevant analyses. However, thorough data cleansing can be extremely costly, so it is vital to correctly assess which data to clean.
• Value: In our view, this is one of the most important aspects of what we understand by a Big Data system. If the data, once processed and converted into information, does not provide value to the organization, we will not have achieved anything. In other words, it is essential that Big Data systems are built with the objective of delivering value to the organization, generating information that helps streamline and guide decision-making, improve internal management, define new optimized strategies, and ultimately give organizations the opportunity to make the most of it.

21st-century Big Data

It was from around 2005 that Big Data systems truly took off, driven mainly by two factors: first, the enormous amount of data generated by users of social networks and online services that needed to be processed; and second, the development of Hadoop technologies (and from 2014, Apache Spark), which reduced the cost of using and storing data.

Since then, the volume of data generated by users has skyrocketed, growing exponentially. In addition, it is no longer only humans who generate data: electronic devices connected to the Internet (the Internet of Things, or IoT) and machine learning are generating more and more data.

Today, Big Data techniques are applied to solve problems as diverse as the creation and development of new products, customer experience analysis, organizational operational efficiency, or predictive maintenance of production lines.

Share on Facebook
Share on Google
Share on Twitter
Share on LinkedIn
Share on Pinterest
Share on Print
Share via Email