The Smile-IT Blog » Blog Archives

Tag Archives: Cloud Orchestration

Automation and Orchestration – a Conclusion

This post is part of the "Automation-Orchestration" architecture series. Posts of this series together comprise a whitepaper on Automation and Orchestration for Innovative IT-aaS Architectures.

 

Automation and Orchestration are core capabilities in any IT landscape.

Traditionally, there’d be classical on-premise IT, comprised of multiple enterprise applications, (partly) based on old-style architecture patterns like file exchange, asynchronous time-boxed export/import scenarios and historic file formats.

At the same time, the era of the Cloud hype has come to an end in a way that Cloud is ubiquitous; it is as present as the Internet as such has been for years, and the descendants of Cloud – mobile, social, IoT – are forming the nexus for the new era of Digital Business.

For enterprises, this means an ever-increasing pace of innovation and a constant advance of business models and business processes. As this paper has outlined, automation and orchestration solutions form the core for IT landscapes to efficiently support businesses in their striving for constant innovation.

Let’s once again repeat the key findings of this paper:

  • Traditional “old style” integration capabilities – such as: file transfer, object orientation or audit readiness – remain key criteria even for a cloud-ready automation platform.
  • In an era where cloud has become a commodity, just like the internet as such, service centered IT landscapes demand for a maximum of scalability and adaptability as well as multi-tenancy in order to be able to create a service-oriented ecosystem for the advancement of the businesses using it.
  • Security, maximum availability, and centralized management and control are fundamental necessities for transforming an IT environment into an integrated service center supporting business expansion, transformation, and growth.
  • Service orchestration might be the ultimate goal to achieve for an IT landscape, but system orchestration is a first step towards creating an abstraction layer between basic IT systems and business-oriented IT-services.

Therefore, for IT leaders, choosing the right automation and orchestration solution to support the business efficiently might be the majorly crucial decision to either become a differentiator and true innovation leader or (just) remain the head of a solid – yet: commodity – enterprise IT.

The CIO of the future is a Chief Innovation (rather than “Information”) Officer – and Automation and Orchestration both build the core basis for innovation. What to look at in getting to the right make-or-buy decision was the main requirement for this paper.

 

Published by:

DevOps style performance monitoring for .NET

 

{{ this article has originally been published in DevOps.com }}

 

Recently I began looking for an application performance management solution for .NET. My requirements are code level visibility, end to end request tracing, and infrastructure monitoring in a DevOps production setup.

DotTrace is clearly the most well-known tool for code level visibility in development setups, but it can’t be used in a 24×7 production setup. DotTrace also doesn’t do typical Ops monitoring.

Unfortunately a Google search didn’t return much in terms of a tool comparison for .NET production monitoring. So I decided to do some research on my own. Following is a short list of well-known tools in the APM space that support .NET. My focus is on finding an end-to-end solution and profiler-like visibility into transactions.

New Relic was the first to do APM SaaS, focused squarely on production with a complete offering. New Relic offers web request monitoring for .NET, Java, and more. It automatically shows a component-based breakdown of the most important requests. The breakdown is fairly intuitive to use and goes down to the SQL level. Code level visibility, at least for .NET, is achieved by manually starting and stopping sampling. This is fine for analyzing currently running applications, but makes analysis of past problems a challenge. New Relic’s main advantage is its ease of us, intuitive UI, and a feature set that can help you quickly identify simple issues. Depth is the main weakness of NewRelic. As soon as you try to dig deeper into the data, you’re stuck. This might be a minor point, but if you’re used to working with a profiler, you’ll miss CPU breakdown as New Relic only shows response times.

net-1-newrelic

Dynatrace is the vendor that started the APM revolution and is definitely the strongest horse in this race. Its feature set in terms of .NET is the most complete, offering code level monitoring (including CPU and wait times), end to end tracing, and user experience monitoring. As far as I can determine, it’s the only tool with a memory profiler for .NET and it also features IIS web request insight. It supports the entire application life cycle from development environments, to load testing, to production. As such it’s nearly perfect for DevOps. Due to its pricing structure and architecture it’s targeted more at the mid to enterprise markets. In terms of ease of use it’s catching up to competition with a new Web UI. It’s rather light on infrastructure monitoring on its own, but shows additional strength with optional Dynatrace synthetic and network monitoring components.

net-2-dynatrace

Ruxit is a new SaaS solution built by Dynatrace. It’s unique in that it unites application performance management and real user monitoring with infrastructure, cloud, and network monitoring into a single product. It is by far the easiest to install, literally takes 2 minutes. It features full end to end tracing, code level visibility down to the method level, SQL visibility, and RUM for .NET, Java, and other languages, with insight into IIS and Apache. Apart from this it has an analytics engine that delivers both technical and user experience insights. Its main advantages are its ease of use, web UI, fully automated root cause analysis, and frankly, amazing breadth. Its flexible consumption based pricing scales from startups, cloud natives, and mid markets up to large web scale deployments of ten-thousands of servers.

net-3-ruxit

AppNetta‘s TraceView takes a different approach to application performance management. It does support tracing across most major languages including database statements and of course .NET. It visualizes things in charts and scatter plots. Even traces across multiple layers and applications are visualized in graphs. This has its advantages but takes some time getting used to it. Unfortunately while TraceView does support .NET it does not yet have code level visibility for it. This makes sense for AppNetta, which as a whole is more focused on large scale monitoring and has more of a network centric background. For DevOps in .NET environments however, it’s a bit lacking.

net-4-TraceView

Foglight, originally owned by Quest and now owned by Dell, is a well-known application performance management solution. It is clearly meant for operations monitoring and tracks all web requests. It integrates infrastructure and application monitoring, end to end tracing, and code level visibility on .NET, among other things. It has the required depth, but it’s rather complex to set up and obviously generates alert storms as far as I could experience. It takes a while to configure and get the data you need. Once properly set up though, you get a lot of insight into your .NET application. In a fast moving DevOps scenario though it might take too long to manually adapt to infrastructure changes.

net-5-foglight

AppDynamics is well known in the APM space. Its offering is quite complete and it features .NET monitoring, quite nice transaction flow tracing, user experience, and code level profiling capabilities. It is production capable, though code level visibility may be limited here to reduce overhead. Apart from these features though, AppDynamics has some weaknesses, mainly the lack of IIS request visibility and the fact that it only features walk clock time with no CPU breakdown. Its flash-based web UI and rather cumbersome agent configuration can also be counted as negatives. Compared to others it’s also lacking in terms of infrastructure monitoring. Its pricing structure definitely targets the mid market.

net-6-AppDynamics

Manage Engine has traditionally focused on IT monitoring, but in recent years they added end user and application performance monitoring to their portfolio called APM Insight. Manage Engine does give you metric level insight into .NET applications and transaction trace snap shots which give you code level stack traces and database interactions. However it’s apparent that Manage Engine is a monitoring tool and APM insight doesn’t provide the level of depth one might be accustomed to from other APM tools and profilers.

net-7-ME

JenniferSoft is a monitoring solution that provides nice real-time dashboarding and gives an overview of the topology of your environment. It enables users to see deviations in the speed of transactions with real time scatter charts and analysis of transactions. It provides “profiling” for IIS/.NET transactions, but only on single tiers and has no transaction tracing. Their strong suit is clearly cool dashboarding but not necessarily analytics. For example, they are the only vendor that features 3D animated dashboards.

net-8-JenniferSoft

Conclusion: There’s more buzz around on the APM space than a Google search would reveal on first sight and I did actually discover some cool vendors to target my needs; however, the field clears up pretty much when you dig for end-2-end visibility from code down to infrastructure, including RUM, any web service requests and deep SQL insights. And if you want to pair that with a nice, fluent, ease-of-use web UI and efficient analytics, there’s actually not many left …

Published by:

(It’s) The End of the Cloud (as we know it)

“It’s the end of the world as we know it”, they say, “and I feel fine”. I do indeed. Things change. And change is always moving us and all around us. And in the best case it’s moving us forward.

So, let me start with a very moving (and maybe touching) statement:

 

“The Cloud has ended!
You can now stop talking about it.”

After about 5 years (if you think of the Middle-of-Europe geography – maybe some more in the US), the Cloud has finally come to its end. That’s great! Because eventually we can luckily move on to the really important things in our day-2-day IT life!

So – why is that? Why do I have the strong believe, that this hype is now finally over?

Some 5 years ago I participated in a panel discussion. And the main area it circulated on was whether Cloud can ever be secure or not – multi tenant or not – reliable or not – compliant or not. The panel discussion as such went quite well and eventually attendees of some real industry giants, like e.g. Siemens, for the first time felt assured that this was

 

The next Big Thing to Come …

And it was! Indeed! It remained to be perceived as the next big thing for the next – well – let’s say: 2 to 3 years.

That occasion back then was only one example – however – of how conversations went going forward. People lost themselves in arguments of security, compliance, controlability and segregatability of a cloudy, foggy and fuzzy monster which none could really grasp for a long time. Discussing these was just more easy than to understand the big advantages to come …

Still, it remained not only a hype but took its turn into our everyday life! Even more than that – Cloud Computing became the basis for what we today call the third IT revolution. And many talks today aim at spanning everybody’s perception regarding this revolution – one within our everyday lifes with IT being planted into any kind of small or big device – one within our industry lifes with production processes being shifted by 90° to create a complete new way of delivering applications – and eventually also a revolution for the big players as many of them might lose traction when remaining one foot on the platform, the other on the train for too long …

Soon after that panel discussion back in 2009, me and my then team entered the Windows Azure “Technology Adoption Program”. We created a first Cloud-based software distribution platform; you could compare it to those Systems Management architectures which have been very common for long in enterprise IT. By using Cloud patterns, we were able to address a broader reach and distribute more scalable and more flexible. Azure – back then – had its really major leaks and intensively improved down the path – together with fellow teams like ours who recommended changes and enhancements to the platform without end. Amazon – by that time – had already its place in the market. But what were they doing? Nothing more and nothing less than providing servers – more or less. Infrastructure-as-a-Service, it was called (question is: would you claim this to be disruptive today?).

Everybody talked about

 

The Big 3

Amazon, Microsoft and of course — ? — Google. Question: What’s one common thing of all 3 of them??

Exactly: All are US-based, spreading their capabilities over the world pretty quickly – backed by major investments – but still US-based. And every enterprise also back then knew: This was not going to work in compliancy to their respective country’s law or their anti trust regulatory.

So what remained the major discussion over years? Still? Security. Compliance. Controlability. Segregatability – still the same. Resentiments were omnipresent and the Big 3 kept having a hard time generating adoption of their brilliant new technologies (whereas admittedly Microsoft had the hardest time, because they leaked a $-rollin ecosystem such as brilliant search or brilliant book-sells).

Still, that Cloud-hype was not killable. Despite all concerns of private and public endavours, of small businesses and large enterprises, Cloud Computing remained and even strengthened its position as the basis for more to come: Technology ecosystems that evolved solely and purely because cloud took its turn into our everyday life and everyday enterprise IT.

And how couldn’t it?

How many of us are using eMail, some kind of file sharing facility, a social network of any kind? Anyone not using any remotely hosted collaboration product, authoring tools, design helpers – like the Adobe family – etc. etc. …

Who’s been using that within one’s work environment daily? For how long? 2 years, 3 years, …

We have all long ago started to enhance our day-2-day work experience by adding usefull little helpers into our IT-wise behaviour – sometimes without even noticing, into how far away premises we’re providing our data to. Haven’t we?

And many times we’ve all done that long before our enterprise IT provider offered us the same convenience, the same workforce enhancements from within our company’s premises. Either privately or even on our company PCs. Enterprise IT departments have been facing the worst time of their existence with sleepless nights for CIOs asking themselves the very same question ever and ever again: How the hell can I stop that Cloud thingy to happen to my enetrprise IT when it is so unsecure and uncompliant and at the same time, all my CxO’s are using it on their iPads? How can I adopt it without losing control completely.

Well – Ladies and Gentlemen – let me assure you: The nightmare has an end as it’s

 

The End of the Cloud as we know it!

(BTW: Ever had a look to a recent Gartner hypecycle and its positioning of “Cloud Computing”: They kept having it on the declinign edge for the last 2 years or so …) So I think we can rest assured: Our struggle is over.

But why? Why can a hype that big, that it managed to become the basis for all new and disruptive power in IT – social, mobile apps, data management and analytics – finally end?

Because it finally became utterly boring! Because it simply isn’t Cloud anymore, we’re talking about! Stop talking Cloud. Stop talking SaaS. It’s not retaining the importance it used to have. It has ceased to exist as a self-contained technology arguing its relevance by itself. It has ceased to exist on its own.

Hence – and here comes another good news – we’re now finally able to adopt this great new technology precisely for what we really neeed it: To accelerate and grow the businesses, which we – the IT guys amongst us – are bound to support. To integrate it into the devices we’re creating and stay as connected as needed. Or to accelerate our delivery into continuity. Or to simply consume the Services, we need to consume. Right now. Right as much and rapid as we need’em. And ultimately: To bridge the great IT systems we’ve all been managing for decades with the elasticity and scalability and accuracy of what was formerly called Cloud.

Let me talk you through a few examples:

  • Salesforce started in 2001. Claiming the decline of Software. They were facing the same concerns, the same weak adoption (at least in the beginning), the same hurdles for their entirely subscription based model. Today, Salesforce offers an ecosystem of products – even a platform to create your own: force.com – which are virtually friction-freely integrateable with your on-prem IT; mostly by just a few configurational mouseclicks. And a huge many enterprises trust Salesforce just enough to provide them one of their most critical business assets: their customer and opportunity data.
  • Or take user management: Many of today’s enterprises still struggle with providing their businesses a seamless login and authorization experience. At the same time, many roll out an identity provider which bridges on-prem IT systems with SaaS-provided systems and thereby offer a totally new single-sign-on experience not only within their on-prem IT. It’s a way of shifting – or even vanishing – the borders.
  • Or think of the most obvious of all examples: Infrastructure and Application Provisioning. Who has not yet introduced Virtualization in its IT? Every larger enterprise has, and the smaller ones leverage it from some out-of-prem providers. Finally, the technology that was formerly called “Cloud” has evolved into a level of maturity that it lives up to the promises of some earlier days: There’s means for you to control and manage your on-premise IT infrastructure seamlessly in a joint way with some Infrastructure provided – well – somewhere (I’m not stressing the word again).

However,

 

One Piece is Missing

One extensively important technology item that has the ability and purpose of glueing it all together: Automation. Or to be precise: Automation and Orchestration.

We are entering the era of Orchestration. We’re leaving the self-fullfilling technology-focussed island of pure “Cloud” behind us and can finally commence to create the really interesting stuff: By bridging our solidly designed and implemented IT systems and architectures with some even more solid, secure and reliable, scalable, elastic architectures, we are accelerating the business value of technology. We start to shift our thinking towards “Orchestration” and “Service”:

So, trust me: It’s the End of the Cloud as we know it! And we indeed can feel entirely fine about it. Security concerns have been addressed all the past years. Compliancy has been factored into vendor’s platforms – as has multi tenancy. And we are able to control where our data resides. Hence, the change can finally move on. We can stop talking “Cloud” as it became so boring as a standalone technology.

 

The Hype is Over

We’re entering the era of Orchestration – of Service Orchestration and entirely Service-based delivery. And Automation is the glue to make it happen. Automation is the glue between those valuable, well-defined, secure IT architectures and whole-new ecosystems of platforms – for the benefit of Service Orchestration.

So: Let us stop the “Cloud” talk. And start to

Automate to Orchestrate

 

 

Published by:
%d bloggers like this: