Nuno Godinho, Director of Cloud Services, Europe for Aditi Technologies discusses how cloud technologies can enable business analytics and big data.
Ever heard of data scientists? Well the Harvard Business Review named it the sexiest job of the 21st Century and it is has quickly risen to prominence in a number of industries including retail, oil and gas, telecommunications and financial services. So what has this got to do with cloud? Should we all be packing up our RESTful APIs and retraining? Not at all. Before looking at what the cloud has to offer the data scientist and IT departments that work with them, lets have a quick dash through history and give an explanation of the role.
Analysing data with computers has gone through a number of significant changes over the years, from looking at raw numbers by hand, to spreadsheets, Business Intelligence and, more recently, visualisations and complex real-time predictive analytics. One thing that has largely been consistent throughout that period has been the data and analysis taking place on-premise. For a long time this was down to technical reasons, but now that has been overcome with many vendors offering PaaS and SaaS versions of their tools and platforms, which means most of the barrier can be put down to the commercial sensitivity of data and it going outside the firewall. Will the service be reliable, safe from hackers, and adequately protected with encryption etc?
In terms of the data scientists themselves, in truth this role has been around for a long time. Essentially it boils down to extracting deep meaning from data. However, the key to doing it is understanding the data itself – not just the results – the raw data, its sources, formats and relationships and then combining this with strategies to analyse the data, from descriptive and prescriptive to predictive perspectives. The data scientist has to have a mind that combines analytic, business and IT skills. Previously representatives from these departments would (read might) have worked to generate the required analysis normally leveraging some kind of data warehouse tool.
Traditional business intelligence, reviews and presents historic data, whether that is two-seconds or two-years old. For years analysts have been taking those same datasets and using them to built prescriptive models that describe the relationships between the data elements and how the numbers interact together. But the latest frontier is real-time predictive analytics: using those models to predict (for example) an action, value or preference. Usually an event, such as an attempted credit card transaction, triggers a model to be run against the transaction details to determine the chances of whether it could be fraudulent, or to make recommendations for other products an online shopper might like. In financial services trading floor systems make thousands of these predictions a second to assess stock movements.
So where can the cloud fit into all this, and should it? The answer is, the cloud can be of benefit throughout the process of collecting and getting data to the point that it can be used in predictive analytics. Firstly, it can be an aggregation point. If your data sources are sensors distributed across an oil field, or even mobile such as truck geo-location data, the cloud can be the point where all those resources are brought together into a single data source for further processing.
Raw compute power from the cloud can also be used to process the big data associated with predictive analytics. Creating models can be an intensive task depending on the size of the data sets, if you don’t actually need to do this often, why make the capital investment when you can just buy the machine time?
The cloud can also be used to enrich your data sets, by providing additional data sources for your models. There are hundreds if not thousands of sources that can enhance your data, whether you need traffic data, government information, or simply temperature data. These sources are validated, reliable and can substantially improve the quality of your models, whilst reducing the costs.
The cloud can of course be responsible for the predictive analysis itself and it is at this point more than any other that you have to consider how quickly you need the results and whether speed and reliability demand you have the infrastructure on-site. For example if you are making thousands of transactions a second that rely on predictive analytics, and the internet connection to your cloud provider is lost – what happens? You may be able to switch to a back-up line, but how long does it take and what is the impact?
Data Scientists and IT departments alike should not ignore the role that the cloud can play in any analytics scenario. That is not to say that it right for all of them, but as we have explained above, there are a number of ways that cloud computing can play a role, it doesn’t have to be all or nothing. It can enhance models, lower costs and give smaller companies access to intelligence that they would otherwise not be able to afford. Basically in order to do these kinds of activities and analysis we require huge amounts of compute power and storage space. This is why the cloud is the perfect partner for big data.
So, to say ‘No’ outright is to deny yourself the possibility of improving or simplifying the way in which analytics is executed in your company.
He has been an MVP for the last six years, first as an MVP in ASP.NET and the last three years as a Windows Azure MVP. His is also a speaker at some of Microsoft's key events such as TechEd North America, TechEd Europe, Tech Days Worldwide Online, TechDays Netherlands and at other community events such as GASP - Grupo de Arquitectura de Software Português, Windows Azure UK User Group, Azure BE UG and so on. He is a prolific blogger and community creator.
David Grimes, Chief Technology Officer at NaviSite explains what the next generation of cloud services will be and how they will differ from the current infrastructure-as-a-service solutions The cloud computing market has matured significantly in the past two to three years, becoming almost synonymous with infrastructure-as-a-service (IaaS). But if the rapid adoption of IaaS was Cloud 1.0, what does the future hold for cloud beyond IaaS and what services will we see being enabled by cloud in the future? Primarily cloud adoption was driven by cost. Not always in the sense that switching to cloud would reduce spend immediately, but the benefits of moving from a CAPEX to OPEX model was, and continues to be, a compelling proposition to businesses. Other factors have also helped to shape today’s cloud industry. These include the maturation of virtualisation technologies, the ever increasing capabilities of the underlying hardware platforms, and the expertise which service providers have developed in delivering robust, secure solutions using a shared infrastructure model. It is this attraction to ‘as-a-service’ delivery that will continue to fuel the next evolution in cloud services, or Cloud 2.0. Over the coming years we will see many existing offerings adapting to an as-a-service model enabling greater levels of flexibility and operational efficiency.
One emerging opportunity in the as-a-service suit is Desktop-as-a-Service (DaaS). Previously a model that had limited success in the market, it is now experiencing increasing demand. This trend can be explained by businesses’ move towards ‘bring your own device’ (BYOD) policies, which in itself is a trend that has resulted from a combination of different elements. Firstly, the current generation of employees has expectations of personal choice when it comes to devices in the office. A company supplied mobile phone simply doesn’t cut it for many workers. They want freedom to choose their own — often more advanced — smartphones or tablets. Also, the trend toward a more global, distributed workforce means people need to work where they want, how they want. The flexible nature of desktop-as-a-service helps address many of these issues. For many companies their intellectual property (IP) is the lifeblood of their business, so protecting access to it is critical. DaaS brings with it a host of information security benefits and many companies now use it to help mitigate IP control concerns. Some of the businesses that most benefit from DaaS use large numbers of overseas contractors to work on specific projects for limited periods of time. The business wants to ensure that contractors have access to the source code but not allow them to copy it for use outside of the organisation, onto a USB stick or printer for example. By using a hosted DaaS model for their contracted developers they are able to provide the tools they need and at the same time retain more control, restricting and remotely cutting off access as soon as needed. Other drivers of DaaS are the advances in protocols and networks that have made the hosted desktop experience more acceptable, and in some cases superior, to the traditional desktop. For enterprises, DaaS represents an opportunity to satisfy their employees BYOD desires, replace CAPEX oriented desktop refresh cycles with a more predictable OPEX model, reduce overall IT support costs, and more readily address security and compliance needs.
Another area that will form a significant part of the Cloud 2.0 as-a-service future is storage. Exponential growth in storage needs will be driven by regulatory and compliance requirements, an increase in mobile devices, and the unprecedented growth of unstructured data. It would be difficult to have a discussion about storage today without using the phrase ‘Big Data’. Almost everything we do creates data, and as a result storage requirements are growing rapidly. This rate of growth cannot be met using traditional procurement methods. Storage-as-a-service (STaaS), where servers can be provisioned via the cloud, enables companies to add storage at the click of a mouse rather than the week and months it previously took to get new servers up and running. The clear business advantages in terms of speed of delivery and flexibility that STaaS provides, means that this area has the potential to grow significantly over the coming years.
Aside from DaaS and STaaS there is huge opportunity to transform other traditional methods of service delivery into an as-a-service model. All of these services will embody the essential characteristics of cloud and will likely be delivered from the common platform which defined the original Cloud 1.0. For example, many applications will benefit from both a cost model and operational efficiency perspective through database-as-a service offerings. The new generation of NoSQL databases are much better suited to cloud applications as they are designed to deal with multiple small requests and work in a distributed, horizontally scalable model – several databases spread across different geographies in the cloud. We could also soon see increased adoption of platform-as-a-service (Paas), but that will be dependent on an evolution in applications, which is clearly underway but likely to be a bit slower growing in the immediate future. Disaster recovery is yet another area where the move away from traditional hardware-based solutions will enable the infrastructure to provide additional capabilities. While any complete disaster recovery plan will include considerations at the application layer, the infrastructure itself can do more today than ever before. In this context of Cloud 2.0 it is best to think of cloud as a set of guiding principles as opposed to a term that defines a specific offering or capability. Our vision for Cloud 2.0 will see the evolutionary move from a pure IaaS to a comprehensive suite of as-a-service offerings. To that end, the specific offerings identified here should be considered representative, not exhaustive. The market is continuing to accelerate and we will likely see additional opportunities to apply the as-a-service mindset to legacy and new offerings.
David Grimes is the CTO of NaviSite and has responsibility for the overall technology vision and direction of the company and leads the Research and Development. Grimes moved to NaviSite in 2002 through the acquisition of Clear Blue where he was responsible for overseeing all internal and operational support systems. Prior to that, he was the lead software engineer at AppliedTheory.