Home > Blogs > We practice Agile and Devops, but how do we measure and monitor Devops?
We practice Agile and Devops, but how do we measure and monitor Devops?
There are innumerable DevOps and Agile success stories around. Whether it is a start-up, digital natives or traditional IT Service organization, many have gained experience at least with Agile, but there are instances where DevOps is an uncharted giant. However, an organization had overcome this challenge in last few years and built on the capability and experience to implement Agile for their clients. Many organizations developed DevOps and Agile in a Box, which would allow customers to test, deploy and implement applications in a controlled environment before stepping onto on a large scale transformation. DevOps and Agile in a Box are ready to solve a real-world business problem. But does anyone know that right DevOps practices are being followed, or DevOps actually brings in the benefit, do the organization measure and monitor DevOps?
While there are many different tools and approaches to DevOps, it is an accepted fact that it is extremely important to gain experience than to get the tools right the first time around. As a result, many organizations have developed a full set of processes, tools and underlying architecture to get started and rapidly progress to actually delivering value.
With so many organization implementing Agile and DevOps, experts across the industry have conducted 100+ workshops on DevOps to understand the common challenge areas – and the top three challenges are No DevOps metrics, No DevOps Centre of Excellence and Planning in a DevOps environment. Though there are multiple success stories establishing the fact of DevOps plus Agile, there is multitude of instances where the majority of the organization’s struggle with the latest transformation journey. This may be due to a number of reasons:
- As an organization, one is not sure where to start from and what is most suitable – what tools should one use, what DevOps practices to follow
- There are too many unknowns, difficult to develop a business case
- Too many discussions, planning sessions and technical directions, no one actually focuses on the real-time business problem
- Leadership is afraid of the investments required, no clarity on the areas of investment
- Lack of DevOps training and capability
- Last, not the least, difficult to change people culture, very difficult to shift from existing working behavior to the DevOps style of delivery. Everyone thinks that they are not ready.
Though Agile methodology has its own metrics but measuring DevOps or its success is still a dark cloud. But what should we measure – is it the User Experience, Productivity, Efficiency or the Software/Product Quality. The organization should have the goals defined linking together software quality, the speed of delivery, customer experience, business success and definitely client satisfaction. The below table details some:
Depending on the industry, domain, software being implemented, one may not focus on all the areas but can measure specific metrics. For e.g. a retail online shopping website checks on application performance, but an online railway ticket booking system will focus more on the speed or application response time, an airline booking and luggage check-in app will stress more on the user experience, etc.
Related Blogs All Blogs
Balancing Risk, Effectiveness, and Customer experience in debt collection
April 09, 2019
Pension-Tension-Freedom!
February 28, 2018
Leave a Reply
146 thoughts on “We practice Agile and Devops, but how do we measure and monitor Devops?”
-
Like!! Great article post.Really thank you! Really Cool.
-
From this article I got some new informtion.The topics are useful to learn.The Blog is more informative.
-
Very good article on DevOps and Agile. Well written too. Thanks Rupak, for sharing the knowledge !!
-
Good article. Was a smooth read.
-
Good insight about Agile methodologies.
-
Good article. DevOps being the buzz word across all organizations these days, this articles stresses the importance of measuring the DevOps implementation and the parameters to consider. I think this is the next step in this area to close the loop in the automation world.
-
Really a great article, Rupak
-
Excellent. It’s very nice article
-
Very good article on DevOps and Agile. Well written too. Thanks Rupak, for sharing the knowledge !! Keep up the good work.
-
Very good article on DevOps and Agile. Well written too. Thanks Rupak, for sharing the knowledge !!
-
Useful blog. Well written.
-
I think there are 2 aspects of measurementconstraint . One is maximum risk apetite of organization and other is investment success criteria. For each different approach either agile or devops or something else we need to define what makes the deliverable acceptable from risk apetite perspective as well as investment success perspective.
-
That#8217;s a wonderful article on DevOps and Agile.
-
Rupak- Really good article!!!
-
Good info on Agile and DevOps
-
Rupak, that is one well written article.
-
Well written Rupak
-
Thanks for sharing information on Agile and DevOps!
-
Very good article Rupak!
-
Good Article!!!
-
Good article with in sight on Agile
-
A nice article with insightful thoughts
-
We hear so much of dev ops on a daily basis but the details provided in the blog is what we don#8217;t get to hear. Thanks Rupak for shedding some light on it!!
-
Good and well articulated and need to measurement of DevOps
-
Quite informative
-
Worth reading, expecting more blogs.
-
I am new to this technology, but i got some clear ideas after reading your thoughts#8230;
Thanks for the share of knowledge, continue the same ) -
Good share Rupak
-
Very nice blog and well articulates the challenges faced in the transformation journey into DevOps .
-
A nice article with insightful thoughts
-
nice article Rupak, thanks for sharing your insight on how to measure devops amp; agile when they are bundling together. Its very important to measure, so we know exactly how to take full benefit out of it. it#8217;s good to know the detailed break down so we can pickup the goals aligning to our business.
-
Good one.
-
Yes, it is key to have metrics to measure the effectiveness of an organization#8217;s DevOps implementation and to gain insights into various aspects for its continuous improvement. How quickly we could harness the data generated by the DevOps tool chain plays an important role here.
-
Very interesting article. Focuses on a key industry aspect thats often left unattended.
-
Very informative and useful. Thanks for sharing this.
-
Nice on Rupak.. Please share any metrics that you at least attempted to measure DevOps. Like for example Deployment Time before and after DevOps, Escaped Defects in Regression bed before and after DevOps, Defects noticed before an incident after deployment before and after DevOps etc. are a good starting point in my view. etc.
-
Good one Rupak.
-
Very informative and well summarized!
-
Excellent Blog Rupak#8230;Undoubtedly DevOps is the combination of cultural philosophies, practices, and tools that increases an organization#8217;s ability to deliver applications and services at high velocity. This is reality which most of the organizations are facing, though we follow Agile methodology and DevOps there are many hiccups as your have cleared in your blog. Once project is into Agile using DevOps during the phase of development and automation most of them are unaware of the underlying facts as such developments are beings stopped without any conclusion facts. Really this blog is meant to all the development teams who use DevOps, I believe one should have clear understanding of end-to-end functionality of how and where to use.
-
Very well written
-
Very well written!
-
Very useful and well explained the problem areas.
-
Great insight of devops and definitely strike a chord with the current area that our team is focusing on. You have raised the right questions i believe.
-
Very nice article and very well written. Its MAD (Modern Application Development) world
-
Well explained amp; combined. Really very useful.
-
We are also trying ti implement Agile and Devops. You have almost nailed what we discuss as drawbacks or improvements area . Good one
-
Hi Rupak,
Now a days devops and agile both are hot topic.You have written such a nice artical on top of it.Keep doing.You expacting lot more interesting topic from you in near future. Thanks -
Good article!..while the frameworks help it is the culture of Agile that makes it successful#8230;.has to be believed in spirit by both the service provider and the receiving organization.
-
Well written by foresight amp; considering the actual facts what#8217;s happening on the ground.
-
This article nicely explain the role of DevOps in Agile methodology and current implementation challenges of DevOps in industry.
-
Nice article Rupak!! Really great idea to measure Devops practicemetrics as per Domain expertise of the client.
-
Good read
-
Great Article
-
Good one Rupak!
-
It#8217;s clear and crisp
-
Very thoughtful Rupak. )
-
Very Insightful. well done.
-
Excellent blog on Agile and highlighting DevOps implementation challenges faced by Transformation teams.
-
Good Insights . Well written sir
-
A great article! Based on this approach and industry experience, we know that improvement doesn#8217;t happen overnight, but instead is an iterative, continuous process. We want to be able to measure our improvement by applying a scientific step sequence of
Hypothesis-gt;test-gt;analysis-gt;proceed on the outcome-gt;Refinement #8230; -
Good points to ponder
-
The value of DevOps in a large organisations and growing organisations is intangible. And the rewards far outweigh the effort required to transition in the long run. This was an interesting read. Looking forward to a follow up article, highlighting the strategies used to tackle the challenges mentioned here.
-
I totally agree that DevOps is not a separate concept but a mere extension of Agile. Excellent topic to talk about in today#8217;s IT world.
-
Excellent insights on the agile methodology of product development amp; Devops platforms. You have put it in a very good manner, explaining the intricate details in a very simple way.
-
Good insights!
Maintainability index is something that can come up in code quality -
Excellent point of view Rupak! While everyone is full steam on the implementation, quantitative measurement will be the key to success of any project. It will be interesting as to how the various tools also respond to the need to measure and provide the data views.
-
Well explained
-
Excellent blog Rupak!! Measuring DevOps success is important and this article provides direction. Thanks!
-
Excellent article Rupak ! It reflects your years of experience in Agile and DevOps processes.
I my opinion about #8220;measuring DevOps or its success is still a dark cloud#8221;, I believe the improvement in time to market and product quality (defect metrics) would be one of the biggest indicators to success of DevOps implementation. -
nice blog
-
Good insight to measure DevOps.
Helps in agile and DevOps in a box. -
Very good information. Thank you Rupak
-
You have nailed it, shot and crisp on how to measure DevOps and the common challenges in implementing DevOps, It is Amazing concept to measure and defend KPI for DevOps..
-
I would recommend similar in my organization.
-
-
Very good analysis of devops. Worth the read
-
This is great news.
-
Rupak has excellent and very vast experience on delivery management as well as he is very technical leader i have met in my professional life.
He is true leader. -
Excellent Article ! We would like have more articles on the DevOps Strategy amp; how an Account can get matured on DevOps space ( Step by Step Approach )
-
quite thought provoking blog on DevOps
-
Informative content Rupak..!! Look forward for more blogs like this..
-
Thanks Rupak, Good one .. This covers one of the important aspect of measurement and monitoring at high level, which every one want to see at the end of the day !
-
As rightly mentioned here, there should be proven processmatrix to measure or quantify the degree of DevOps effort. Looks like, going forward there will be an additional process in-parallelin-congestion with above mentioned processes.
-
Good write-up guide to refer for the teams looking towards the Devops functioning.
-
It truly captures real world problem
-
Liked the valuable subject and contents of this article. I think speed (number of code releases) should be one of the best measure for devops.
-
Very useful information. Nicely described about the importance of DevOps.
-
Great write up Rupak. This will help drive the thought process for current organizations and agile projects leveraging Devops.
-
Great article Rupak in agile world we do need to tailor what would be measured and how based on the project requirements.
-
Great article Rupak.
-
Was really good, informative and helpful.
-
Very aptly written.
As management thinker Peter Drucker said #8220;you can#8217;t manage, what you can#8217;t measure#8221; and of course it can not be improved either. Ultimate objective should be value addition to end user in #8220;measureable#8221; term. -
Really, It is very nice blog Sir#8230;!!!!
-
Lovely article Rupak, also very much relevant to the current way of working
-
Good info. Its a great article. I will follow this article for next comments in any blog. I think it will help me a lot.
-
Good article#8230;Would like see more blogs#8230;
-
Very well tabled points, highlighting the gaps in the current DevOps practices in most organizations. Loved the fact, that the blog points to possible solutions as well and gives the reader some food for thought.
-
Hello Rupak- You are a wonderful blogger.
Thanks for writing and sharing this article. It is a good piece of information! -
Good read and very well written.
-
Nice article Rupak. A different perspective to measure Devops efficiency would certainly help the industry.
-
Very well articulated Rupak! Great read! Keep it up!
-
Awesome read, gives a very clear and concise insight into DevOps importance in Agile.
-
This is indeed a very pertinent perspective to actually understand what are the best DevOps practices, and what benefits it brings. Companies may have a wide range of infrastructure setup, but it all boils down to the use case as you have rightly mentioned, and making sure you have all bases covered.
-
DevOps is the new hot topic in the market. Well articulated.
-
A very good article Rupak, and stating what exactly are the problems which are faced in real time . The blog flow is also very good
-
Very good article Rupak. As we are getting more maturity in agile we feel the need to address the challenges.
Waiting for more such blogs.. -
Demistyfies Devops.An excellent blog.
-
Very informative article #8230;.
-
Never thought of devops#8230; good information#8230;
-
Never thought of devops#8230; good information#8230;
-
Nice article capturing various goals and metrics in different areas. Although all may not be able to be achieved but this serve as a good guide for the scrum masterproject manager to focus.
-
Excellent article covering agile and DevOps and the suggested metrics to measure success!
-
Well articulated content and perfectly explained in the context of using methodologies to achieve business goals. Will be of great value to businesses if implemented in their respective scenarios.
-
Very much relevant to our work environment. Definitely a value add.
-
A very nice and informative blog. A few thoughts -
1) The most imp metric should be customer experience. Ultimately if Devops doesn#8217;t lead to increase in the Cust satisfactionexperience its of no use..
2)The 2nd most imp will be the Business value it adds. It should add both to the Top Line as well as Bottom Line
3)As long as we can put these things in Numbers and not just mention Qualitatively i think we can measure Devops -
This is really a great insight on Agile and DevOps . By simply including these additional metrics, it will become really very very helpful for an agile team to understand their progress in respect to their milestonebenchmark and improvement areas. The team can identify the area of improvements which will bring significant difference in improving end user experience.
-
very nice article covering challenges faced by the companies following devops model. well written !!
-
Good insights there rupak , contextual based simplified story helps the stakeholders internalise this and thereby to get the best out of devops + agile when it comes to realising the same.
-
Good read Rupak.
-
Good article kindling thoughts given DevOps is nascent in our industry and is emerging.
-
Well written and throws some good light in a concise and crisp manner, on measuring DevOps which is quite a grey area.
-
Very well written and nicely highlighted about DevOps
-
A very nice read. I would take away that good metric for measurement would be a combination of all the goals listed in the table.
-
Really useful blog#8230;. )
-
nice )
-
Interesting Read
-
Good information.
-
Good article
-
Very insightful and useful.
-
Very well written
-
Good article. I agree with the point that the Devops metrics may vary from organization to organization and will have to aligned with the overall business goals.
-
Good one Rupak. I like the metrics defined to assist managers in taking a decision on use of Devops in their projects.
-
a very insightful write-up )
-
Good one Rupak. Due to absence of a standardized metrics, in our projects we have created our own #8211; which very much is in line with the proposed.
-
There is no one measure that fits all, it has to be customised, depending on the focus.
-
Hi Rupak,
Good to see your blog on DevOps and Agile methodology. Today, it is of pivotal significance that companies (Established or Start-ups) understand the essence on DevOps and Agile; embracing the transformation and their disciplined practice leads to effective results.
Thanks for sharing the blog. Kindly, keep blogging. Let us together spread the success stories on DevOps and Agile; right approach and implementation is the key.
Regards,
Suraj -
Good insights, from my view #8211; DevOps measurement is about implmentability #8211; how quick release can be done and how many can be done for how many changes, etc
-
Great thought provoking article Rupak, in my opinion a true measure for DevOps would be the number of releases happening at what interval. From Container Orchestration to release automation the entire journey should one key parameter for the success of true DevOps Model.
-
This was really great and very helpful. Thank you !!
-
Excellent read. Very informative.
-
Good one
-
This Blog is very relevant in today#8217;s Agile world .
Would love to read more about Devops because mostly this blog leave us with many questions in our mind )
Good Work Rupak#8230;.will look forward for more articles from you. -
Nice perspective on agile and continuously changing business landscape. Nicely written Rupak.
-
Good insights and inputs on using devops and agile practices and creating achievable and measurable devops targets and kpis.
-
Very nicely constructed#8230;.
-
Very nicely articulated the challenges faced on ground. These DevOps measures if focused would surely result in a successful DevOps and Agile in a Box.
-
excellent article
Load more comments...
Rupak Das
Principal, Business Excellence
"Let's enhance success by removing impediments" Rupak, a product excellence buff, has rich cross-functional expertise in Engineering & Delivery Management. He is passionate about enhancing client experience.
More blogs from Rupak Das >