What do foreign currency exchanges, online gaming and the internet of things have in common? Most obviously, they throw off huge data volumes which need to be analysed and understood. More significantly, the value of that data for organisations is realised in the moment it is generated, rather than when looked at subsequently (even if that means minutes, or just micro-seconds after an event).
For ITRS, realising the commonality of this problem across sectors was the trigger for the creation of its Insights solution, a big data analytics platform that ingests streaming data and applies machine learning algorithms to generate insight into business operations. It builds on the vendor’s Geneos platform which was created for application performance analytics and has found adoption across financial services to support business-critical systems management.
As Justo Ruiz Ferrer, chief technology officer at ITRS, explained to DataIQ: “Why streaming data? It is a problem for everybody and a growth industry. There is a time value of information that becomes a problem.”
“From the operational side, we have been focusing on data centres and visualisation of the business. It gives the client a depth view of whether servers are working, networks are behaving properly, is the business going to cope with the load - questions that it is important to understand,” said Ferrer.
ITRS has been operating in the financial services sector for the last 20 years as a niche player with around 200 clients, including nine out of ten of the largest investment banks. Ferrer’s own background is in senior front office and risk practices at hedge funds and investment banks, where he worked as a computer scientist in foreign exchange, rates and credit functions, before joining his current company four years ago. “One thing I have done well in the past is advocacy of performing computation on streaming data, focusing on what is happening now, not one minute or five minutes ago. That is very valuable to clients,” he noted.
“For financial services, they are trying to put value on storing data. They have a lot of systems that store information on nano or micro-second events, such as high-frequency trading, that can be very complicated. But there is only so much competitiveness in one milisecond,” he pointed out.
Operations such as trading desks and market makers on the buy and sell sides are becoming more complicated with the growth of electronic marketplaces and increasingly complexity in the products on offer. Said Ferrer: “You need to be able to look at the trading patterns, how they impact on prices. Traders need streams of information, not just on the marketplace and trading data, but the whole operation of the business itself. Having an holistic view is extremely important because you can see variables that are really significant, but you also need to understand the whole company and where you are at.”
Since the creation of Geneos and its adoption by financial services, a new category of streaming data has emerged via the internet of things. With the introduction of smart meters, the embedding of sensors into a growing number of appliances and the new strategy among manufacturers of operating their products as a service, a new market opportunity has emerged. He said: “For us, from a traditional, mature space and with extensive understanding of fhnancial systems and their performance, to providing a platform capable of taking company data and combining that with technical data to give the client a unique view into those behaviours is fantastic.”
Said Ferrer: “Our platform is able to access data at scale, integrate it and run analyses against historic patterns. We have seen interest from other sectors, such as agriculture, energy, gaming. We have even had discussions with CERN because of the volume of data they generate. Our platform makes it easy to collect data on a massive scale and analyse it.”
Based on its proprietary developer ptalform Valo, Insights runs in genuine real-time, not micro-batches and can be run on commodity hardware ingesting a wide range of data types, from numerical to free-form text. The complex machine learning algorithms sit behind an intuitive user interface that can drive visualisations and dashboards.
In one example of how the IoT is creating a new demand for this type of solution, Ferrer pointed to three Canadian provinces which have deployed hundreds of expensive devices to monitor the conditon of roads. These devices are generating streaming information on the traffic load, temperature, humidity, solar radiation and so on.
“They are starting to explore the value in that data and immediately their investment has become more positive because if they provide information in real-time, their relationship with partners, such as snow clearing companies, becomes more effective. They can tell them where to go, check if they are meeting their service level agreements. As a public function, they are responsible for ensuring the safety of their citizens. But they can go even further and use it as a platform to provide services - why not sell that data to navigation devices or other agencies, such as weather?” asked Ferrer.
Platforms to support this shift in data types, complexity and timeliness will be vital to support the development of IoT. Analytics and insight are where the value is generated that justifies the investment, allowing organisations to build their infrastructure knowing they are likely to see a payback.
ITRS is currently discussing whether to develop Insights into an as-a-service or hosted proposition. The company had 12 proofs of concept running at the time of the interview. Ferrer explained: “It is a brand new technoology and it takes time to provide the confidence that clients want to see in ciritical infrastuucture, but our expectations are high.”