Sven Löffler
25. February 2015 0

Evolution of Big Data

In the past and more than ever today Big Data is one of the topics for modern companies as they can gain a huge benefit from analyzing their data. So, everybody is talking about Big Data, but how many actually know: “Where are the origins of Big Data and how it has developed in the past 90 years?“.

The beginning of Big Data

The recording and analysis of large amount of Data ,for that time, already started before computer even were developed. During the 1920´s criminal and arrest records could be barely kept up to date. As population growth continued to problems became worse. More and more labor had to be employed to deal with the information and the storage-systems in libraries, the original systems of storing and sorting data, had to be improved. This massive increase in information was referred to as the “Information Explosion”. Therefore, there had to be a other, more sufficient way to deal with large amount of data, other than by just employing more people.

First ideas to deal with Big Data

To be able to cope with the immense amount of data, first machines were developed to make the work more efficient and faster. One of the first was the 405 Electric Accounting Machine, which was able to process 150 80-character punch cards per minute. The machines helped to increase the speed of analysis and storing, in addition requiring less labor. However, there was still the problem of storage. As time went by machines became more and more developed and in the 1960´s first centralized computing systems were used to automate data analysis.
Revolutionary ideas were driving the development process. The concept of virtual memory, that would treat finite storage as infinite, was proposed by Fritz-Rudolf. In 1970 Edgar F. Codd introduced the idea of relational database, which showed how data could be accessed from a database without knowing how the information was structured or where exactly it was stored in the database.

Business Intelligence

The usage of just storing data was limited, therefore firms wanted to use their data to improve their decision making. Therefore concepts and methods using fact-based support-systems were used to improve business decisions. This was defined as business intelligence. Business intelligence started to boom, as a reaction to the success of the world wide net in the 90´s.The influx of information was a new challenge for data management and increased the cost for storing data. To offer more functionality, digital storage quickly became more and more cost-effective for storing data and the first BI platforms began to form.

The 21th Century

In the 21th Century Big Data was strongly promoted and had a rapid growth. Big Data stood for the 4 V´s: volume, velocity, variety and value. In addition to this, Web Services were focused to allow customers to easily create new applications. The increasing data flow due to the success of the world wide web was shown by studies in 2008, stating:“Americans consumed information for about 1.3 trillion hours, an average of almost 12 hours per day”. Therefore, Hadoop was created, which aimed at storing and processing data. The internet continued to become more and more important and first cloud solution were offered. Also other new in-memory technologies such as SAP HANA® are used until today to analyze and optimize mass quantities of data and using the analysis for competitive advantage in this very rapidly changing market in the today´s world.

So we can ask: ”What will the future bring for Big Data?”. Experts estimate a 4300% increase in annual data generation by 2020. So we can be curious to see, how we will deal with this amount of data, which is surrounding all of us.

Big Data regards
Sven Löffler

Leave a Reply

Your email address will not be published. Required fields are marked *

By sending this comment you accept our comment policy.

a) Blog visitors are always invited to comment.

b) Comments are supposed to increase the value of this weblog.

c) Comments will be activated only after validation.

d) Comments which do not relate to the topic, obviously violate copyrights, have offensive content or contain personal attacks will be deleted.

e) Links can be inserted to the comment but should refer to the topic of the blog post. Links to other websites or blogs which do not refer to the posting will be considered as spam and will be deleted.



tsystemsCom @tsystemsCom
T-Systems  @tsystemsCom
Let's take a quiz: What do you remember about #WannaCry? @BackofenD about the need of a comprehensive immunization… 
T-Systems  @tsystemsCom
#Digitisation also holds great promise in Public #Healthcare but fundamental challenges be overcome to create the r… 
T-Systems  @tsystemsCom
#DigitalTransformation is front of mind for many senior executives, but too often #security is left behind, knows… 
T-Systems  @tsystemsCom
Making the most from your #cloud: 10 best practices recommended by cloud computing experts and IT managers who are… 
T-Systems  @tsystemsCom
The Internet of Things needs your #Mobile: #IoT can’t connect the next billion until we reinvent mobile @SAP:… 
T-Systems  @tsystemsCom
Seven techniques that will help create natural project checkpoints, stakeholder feedback loops, and system adjustme… 
T-Systems  @tsystemsCom
Big savings with the #cloud: Just being in the cloud costs about the same as for 500 workloads as it does for 2,000… 
T-Systems  @tsystemsCom
Digital twins are the next innovation in manufacturing and by 2020 at least 50% of big manufacturers will have a… 
T-Systems  @tsystemsCom
#Blockchain and your enterprise: There are four areas to consider when deciding if your business is ready for block…