Just about every player in the Big Data and analytics game was in New York last week at the Strata + Hadoop World conference, to showcase their latest technologies.
Over 7,000 people attended the event where keynote speakers, including White House chief data scientist DJ Patil, laid out their visions for where machine learning, analytics, the Internet of Things, autonomous vehicles and smart cities will be taking us in the near future.
Here are a few of my highlights from the event and some of the major announcements from key players.
DJ Patil spoke about how during his time so far as the U.S.’s first chief data scientist, his mission has been to “responsibly unleash the power of data to benefit all Americans.”
He spoke about how Big Data and analytics is helping to reduce damage by opioid abuse, and the importance of openness.
“When the president first started in office there was about 10 [open] data sets put out there, now there are about 2,000”, he said. Whatever you think of Barack Obama’s presidency that is an impressive achievement, as it means that anyone from major corporations to armchair data scientists can now use data to develop new strategies and technologies to harness it.
Martin Hall, chief data scientist for Big Data solutions at Intel, told his audience that the explosion of interest and activity in Big Data means that “we now have the data, the analytics and the compute power to deliver more than insights – we can enable intelligence.”
The arrival of personalized medicine, autonomous cars and smart connected devices mean that we are now entering the age of AI, and rather than a simple Internet of Things, we are heading towards an Internet of Intelligent Things, with ever increasing levels of automation. This vision undoubtedly has huge implications on just about everything concerning how we live our lives, jobs, and how we interact and communicate with the world and each other.
This year’s conference served as confirmation that streaming real time analytics has moved firmly into the mainstream of data science, and rather than simply being a pipe dream or end goal, is fast becoming a reality. Apache’s open source Kafka engine is seen as the driving force which is enabling this shift, and big players were keen to show their support for this particular piece of technology.
Cloudera – one of the biggest distributors of open source platforms – announced the upcoming release of version 5.9 of its own Hadoop distribution which will ship with Spark 2.0 for the first time, as well as the latest release of Apache Kudu, which is specifically tooled toward real time analytics. It also announced that its distribution will run on Microsoft’s Azure cloud infrastructure for the first time (as well as Amazon Web Services and Google’s Cloud Platform). It also announced a new pay-as-you-go pricing model which it will offer alongside its existing annual subscription model.
IBM’s rock, paper, scissors-playing robot Marvin was let loose to entertain the crowds. Taking on all comers and appearing to show, by the increasing win rate, that computers are increasing in their ability to predict our behavior. Marvin is powered by Apache Spark and you can see a brief video of him in action here. IBM also announced a new initiative known as Project Dataworks which aims to use Spark to make more data available for processing through its Watson cognitive computing engine.
SAP spoke about how their recently announced acquisition of Californian Big Data startup Altiscale will boost its HANA cloud service offerings, such as offering access to its cloud-based Spark services. It also showcased its Vora query engine which deploys machine learning to enhance contextual awareness of AI-driven Big Data operations.
These are just a few of the highlights from this year’s event – expect more from me soon, in the meantime if you were at Strata + Hadoop World, why not let me know what you were most excited by?
Bernard Marr is a best-selling author & keynote speaker. His new book: 'Big Data in Practice: How 45 Successful Companies Used Big Data Analytics to Deliver Extraordinary Results'
Original source of this article is forbes
