There are innumerable DevOps and Agile success stories around. Whether it is a start-up, digital natives or traditional IT Service organization, many have gained experience at least with Agile, but there are instances where DevOps is an uncharted giant. However, an organization had overcome this challenge in last few years and built on the capability and experience to implement Agile for their clients. Many organizations developed DevOps and Agile in a Box, which would allow customers to test, deploy and implement applications in a controlled environment before stepping onto on a large scale transformation. DevOps and Agile in a Box are ready to solve a real-world business problem. But does anyone know that right DevOps practices are being followed, or DevOps actually brings in the benefit, do the organization measure and monitor DevOps?

While there are many different tools and approaches to DevOps, it is an accepted fact that it is extremely important to gain experience than to get the tools right the first time around. As a result, many organizations have developed a full set of processes, tools and underlying architecture to get started and rapidly progress to actually delivering value.

With so many organization implementing Agile and DevOps, experts across the industry have conducted 100+ workshops on DevOps to understand the common challenge areas – and the top three challenges are No DevOps metrics, No DevOps Centre of Excellence and Planning in a DevOps environment. Though there are multiple success stories establishing the fact of DevOps plus Agile, there is multitude of instances where the majority of the organization’s struggle with the latest transformation journey. This may be due to a number of reasons:

  • As an organization, one is not sure where to start from and what is most suitable – what tools should one use, what DevOps practices to follow
  • There are too many unknowns, difficult to develop a business case
  • Too many discussions, planning sessions and technical directions, no one actually focuses on the real-time business problem
  • Leadership is afraid of the investments required, no clarity on the areas of investment
  • Lack of DevOps training and capability
  • Last, not the least, difficult to change people culture, very difficult to shift from existing working behavior to the DevOps style of delivery. Everyone thinks that they are not ready.

Though Agile methodology has its own metrics but measuring DevOps or its success is still a dark cloud. But what should we measure – is it the User Experience, Productivity, Efficiency or the Software/Product Quality. The organization should have the goals defined linking together software quality, the speed of delivery, customer experience, business success and definitely client satisfaction. The below table details some:

Depending on the industry, domain, software being implemented, one may not focus on all the areas but can measure specific metrics. For e.g. a retail online shopping website checks on application performance, but an online railway ticket booking system will focus more on the speed or application response time, an airline booking and luggage check-in app will stress more on the user experience, etc.

Tags: ,

Rupak Das

Principal, Business Excellence

"Let's enhance success by removing impediments" Rupak, a product excellence buff, has rich cross-functional expertise in Engineering & Delivery Management. He is passionate about enhancing client experience.

More blogs from Rupak Das

134 thoughts on “We practice Agile and Devops, but how do we measure and monitor Devops?

  • excellent article

  • Very nicely articulated the challenges faced on ground. These DevOps measures if focused would surely result in a successful DevOps and Agile in a Box.

  • Very nicely constructed….

  • Good insights and inputs on using devops and agile practices and creating achievable and measurable devops targets and kpis.

  • Nice perspective on agile and continuously changing business landscape. Nicely written Rupak.

  • This Blog is very relevant in today’s Agile world .
    Would love to read more about Devops because mostly this blog leave us with many questions in our mind 🙂
    Good Work Rupak….will look forward for more articles from you.

  • Good one

  • Excellent read. Very informative.

  • This was really great and very helpful. Thank you !!

  • Great thought provoking article Rupak, in my opinion a true measure for DevOps would be the number of releases happening at what interval. From Container Orchestration to release automation the entire journey should one key parameter for the success of true DevOps Model.

  • Good insights, from my view – DevOps measurement is about implmentability – how quick release can be done and how many can be done for how many changes, etc

  • Hi Rupak,
    Good to see your blog on DevOps and Agile methodology. Today, it is of pivotal significance that companies (Established or Start-ups) understand the essence on DevOps and Agile; embracing the transformation and their disciplined practice leads to effective results.

    Thanks for sharing the blog. Kindly, keep blogging. Let us together spread the success stories on DevOps and Agile; right approach and implementation is the key.

    Regards,
    Suraj

  • There is no one measure that fits all, it has to be customised, depending on the focus.

  • Good one Rupak. Due to absence of a standardized metrics, in our projects we have created our own – which very much is in line with the proposed.

  • a very insightful write-up 🙂

  • Good one Rupak. I like the metrics defined to assist managers in taking a decision on use of Devops in their projects.

  • Good article. I agree with the point that the Devops metrics may vary from organization to organization and will have to aligned with the overall business goals.

  • Very well written

  • Very insightful and useful.

  • Good article

  • Good information.

  • Interesting Read

  • nice 🙂

  • Really useful blog…. 🙂

  • A very nice read. I would take away that good metric for measurement would be a combination of all the goals listed in the table.

  • Very well written and nicely highlighted about DevOps

  • Well written and throws some good light in a concise and crisp manner, on measuring DevOps which is quite a grey area.

  • Good article kindling thoughts given DevOps is nascent in our industry and is emerging.

  • Good read Rupak.

  • Good insights there rupak , contextual based simplified story helps the stakeholders internalise this and thereby to get the best out of devops + agile when it comes to realising the same.

  • very nice article covering challenges faced by the companies following devops model. well written !!

  • This is really a great insight on Agile and DevOps . By simply including these additional metrics, it will become really very very helpful for an agile team to understand their progress in respect to their milestone/benchmark and improvement areas. The team can identify the area of improvements which will bring significant difference in improving end user experience.

  • A very nice and informative blog. A few thoughts :-
    1) The most imp metric should be customer experience. Ultimately if Devops doesn’t lead to increase in the Cust satisfaction/experience its of no use..
    2)The 2nd most imp will be the Business value it adds. It should add both to the Top Line as well as Bottom Line
    3)As long as we can put these things in Numbers and not just mention Qualitatively i think we can measure Devops

  • Very much relevant to our work environment. Definitely a value add.

  • Well articulated content and perfectly explained in the context of using methodologies to achieve business goals. Will be of great value to businesses if implemented in their respective scenarios.

  • Excellent article covering agile and DevOps and the suggested metrics to measure success!

  • Nice article capturing various goals and metrics in different areas. Although all may not be able to be achieved but this serve as a good guide for the scrum master/project manager to focus.

  • Never thought of devops… good information…

  • Never thought of devops… good information…

  • Very informative article ….

  • Demistyfies Devops.An excellent blog.

  • Very good article Rupak. As we are getting more maturity in agile we feel the need to address the challenges.
    Waiting for more such blogs..

  • A very good article Rupak, and stating what exactly are the problems which are faced in real time . The blog flow is also very good

  • DevOps is the new hot topic in the market. Well articulated.

  • This is indeed a very pertinent perspective to actually understand what are the best DevOps practices, and what benefits it brings. Companies may have a wide range of infrastructure setup, but it all boils down to the use case as you have rightly mentioned, and making sure you have all bases covered.

  • Awesome read, gives a very clear and concise insight into DevOps importance in Agile.

  • Very well articulated Rupak! Great read! Keep it up!

  • Nice article Rupak. A different perspective to measure Devops efficiency would certainly help the industry.

  • Good read and very well written.

  • Hello Rupak- You are a wonderful blogger.
    Thanks for writing and sharing this article. It is a good piece of information!

  • Very well tabled points, highlighting the gaps in the current DevOps practices in most organizations. Loved the fact, that the blog points to possible solutions as well and gives the reader some food for thought.

  • Good article…Would like see more blogs…

  • Good info. Its a great article. I will follow this article for next comments in any blog. I think it will help me a lot.

  • Lovely article Rupak, also very much relevant to the current way of working

  • Really, It is very nice blog Sir…!!!!

  • Very aptly written.
    As management thinker Peter Drucker said “you can’t manage, what you can’t measure” and of course it can not be improved either. Ultimate objective should be value addition to end user in “measureable” term.

  • Was really good, informative and helpful.

  • Great article Rupak.

  • Great article Rupak in agile world we do need to tailor what would be measured and how based on the project requirements.

  • Great write up Rupak. This will help drive the thought process for current organizations and agile projects leveraging Devops.

  • Very useful information. Nicely described about the importance of DevOps.

  • Liked the valuable subject and contents of this article. I think speed (number of code releases) should be one of the best measure for devops.

  • It truly captures real world problem

  • Good write-up guide to refer for the teams looking towards the Devops functioning.

  • As rightly mentioned here, there should be proven process/matrix to measure or quantify the degree of DevOps effort. Looks like, going forward there will be an additional process in-parallel/in-congestion with above mentioned processes.

  • Thanks Rupak, Good one .. This covers one of the important aspect of measurement and monitoring at high level, which every one want to see at the end of the day !

  • Informative content Rupak..!! Look forward for more blogs like this..

  • quite thought provoking blog on DevOps

  • Excellent Article ! We would like have more articles on the DevOps Strategy & how an Account can get matured on DevOps space ( Step by Step Approach )

  • Rupak has excellent and very vast experience on delivery management as well as he is very technical leader i have met in my professional life.
    He is true leader.

  • This is great news.

  • Very good analysis of devops. Worth the read

  • You have nailed it, shot and crisp on how to measure DevOps and the common challenges in implementing DevOps, It is Amazing concept to measure and defend KPI for DevOps..

    • I would recommend similar in my organization.

  • Very good information. Thank you Rupak

  • Good insight to measure DevOps.
    Helps in agile and DevOps in a box.

  • nice blog

  • Excellent article Rupak ! It reflects your years of experience in Agile and DevOps processes.
    I my opinion about “measuring DevOps or its success is still a dark cloud”, I believe the improvement in time to market and product quality (defect metrics) would be one of the biggest indicators to success of DevOps implementation.

  • Excellent blog Rupak!! Measuring DevOps success is important and this article provides direction. Thanks!

  • Well explained

  • Excellent point of view Rupak! While everyone is full steam on the implementation, quantitative measurement will be the key to success of any project. It will be interesting as to how the various tools also respond to the need to measure and provide the data views.

  • Good insights!
    Maintainability index is something that can come up in code quality

  • Excellent insights on the agile methodology of product development & Devops platforms. You have put it in a very good manner, explaining the intricate details in a very simple way.

  • I totally agree that DevOps is not a separate concept but a mere extension of Agile. Excellent topic to talk about in today’s IT world.

  • The value of DevOps in a large organisations and growing organisations is intangible. And the rewards far outweigh the effort required to transition in the long run. This was an interesting read. Looking forward to a follow up article, highlighting the strategies used to tackle the challenges mentioned here.

  • Good points to ponder

  • A great article! Based on this approach and industry experience, we know that improvement doesn’t happen overnight, but instead is an iterative, continuous process. We want to be able to measure our improvement by applying a scientific step sequence of
    Hypothesis->test->analysis->proceed on the outcome->Refinement …

  • Good Insights . Well written sir

  • Excellent blog on Agile and highlighting DevOps implementation challenges faced by Transformation teams.

  • Very Insightful. well done.

  • Very thoughtful Rupak. 🙂

  • It’s clear and crisp

  • Good one Rupak!

  • Great Article

  • Good read

  • Nice article Rupak!! Really great idea to measure Devops practice/metrics as per Domain expertise of the client.

  • This article nicely explain the role of DevOps in Agile methodology and current implementation challenges of DevOps in industry.

  • Well written by foresight & considering the actual facts what’s happening on the ground.

  • Good article!..while the frameworks help it is the culture of Agile that makes it successful….has to be believed in spirit by both the service provider and the receiving organization.

  • Hi Rupak,
    Now a days devops and agile both are hot topic.You have written such a nice artical on top of it.Keep doing.You expacting lot more interesting topic from you in near future. Thanks

  • We are also trying ti implement Agile and Devops. You have almost nailed what we discuss as drawbacks or improvements area . Good one

  • Well explained & combined. Really very useful.

  • Very nice article and very well written. Its MAD (Modern Application Development) world

  • Great insight of devops and definitely strike a chord with the current area that our team is focusing on. You have raised the right questions i believe.

  • Very useful and well explained the problem areas.

  • Very well written!

  • Very well written

  • Excellent Blog Rupak…Undoubtedly DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity. This is reality which most of the organizations are facing, though we follow Agile methodology and DevOps there are many hiccups as your have cleared in your blog. Once project is into Agile using DevOps during the phase of development and automation most of them are unaware of the underlying facts as such developments are beings stopped without any conclusion facts. Really this blog is meant to all the development teams who use DevOps, I believe one should have clear understanding of end-to-end functionality of how and where to use.

  • Very informative and well summarized!

  • Good one Rupak.

  • Nice on Rupak.. Please share any metrics that you at least attempted to measure DevOps. Like for example Deployment Time before and after DevOps, Escaped Defects in Regression bed before and after DevOps, Defects noticed before an incident after deployment before and after DevOps etc. are a good starting point in my view. etc.

  • Very informative and useful. Thanks for sharing this.

  • Very interesting article. Focuses on a key industry aspect thats often left unattended.

  • Yes, it is key to have metrics to measure the effectiveness of an organization’s DevOps implementation and to gain insights into various aspects for its continuous improvement. How quickly we could harness the data generated by the DevOps tool chain plays an important role here.

  • Good one.

  • nice article Rupak, thanks for sharing your insight on how to measure devops & agile when they are bundling together. Its very important to measure, so we know exactly how to take full benefit out of it. it’s good to know the detailed break down so we can pickup the goals aligning to our business.

  • A nice article with insightful thoughts

  • Very nice blog and well articulates the challenges faced in the transformation journey into DevOps .

  • Good share Rupak

  • I am new to this technology, but i got some clear ideas after reading your thoughts…
    Thanks for the share of knowledge, continue the same 🙂

  • Worth reading, expecting more blogs.

  • Quite informative

  • Good and well articulated and need to measurement of DevOps

  • We hear so much of dev ops on a daily basis but the details provided in the blog is what we don’t get to hear. Thanks Rupak for shedding some light on it!!

  • A nice article with insightful thoughts

  • Good article with in sight on Agile

  • Good Article!!!

  • Very good article Rupak!

  • Thanks for sharing information on Agile and DevOps!

  • Well written Rupak

  • Rupak, that is one well written article.

  • Good info on Agile and DevOps

  • Rupak- Really good article!!!

  • That’s a wonderful article on DevOps and Agile.

Leave a Reply

Your email address will not be published. Required fields are marked *

Gartner Magic Quadrant for Global Retail Core Banking 2017

Infosys Finacle positioned a leader for the tenth time in a row
EdgeVerve Systems (for Infosys Finacle Core Banking solution) has been positioned as a Leader in Gartner’s Magic Quadrant for Global Retail Core Banking (GRCB).