Digital Transformation for Modern Enterprises Through DevOps — A Complete Guide17th May 2019
Digital Transformation for Modern Enterprises Through DevOps — A Complete Guide
Five commandments for modern enterprises for a successful journey.
1. Mindset matters: Setting the ‘Culture First’ approach
Fixed Mindset — “It’s always been done this way.”
Growth Mindset — “Let’s find a way to do it better.”
Solid engineering culture is the need today. Proposing tools without knowing the business and/or platform context doesn’t help anybody. It is not just about moving fast and breaking things; moving with confidence is what matters! It is not about automating stuff; it is about learning and knowledge sharing across the organization. That is how culture is built. It’s a team sport and hence the need for DevOps.
Change is the only constant, or, as Charles Darwin put it, “It Is Not the Strongest of the Species that Survives But the Most Adaptable.”
In the journey of agility, you might fail sometimes, but you need to remember what Aristotle said, “We are what we repeatedly do. Excellence, then, is not an act, but a habit.”
‘Do not automate a bad process, you end up with an automated bad process’ — DOES 2018
You can’t scale a bad process or culture. If DevOps puts a bandaid on a gunshot wound you know that eventually, you will bleed from the inside. DevOps/Agile is a culture in reality, so organizational buy-in is critical.
2. Ownership and responsibility
In practical scenarios, the core assumption is that deployments work better if the people who wrote the code are also the administrators of that code. Giving ownership of a product motivates the individuals to take fuller responsibility for their work.
At Amazon, the development and deployment processes are so mature that it handles a new software deployment every second.
Shared responsibility is the core principle of good DevOps teams. No buck-passing, No gimmicks. I love this piece of advice by Mike Miller, Co-founder, and Chief Scientist at Cloudant.
3. It is NEVER about the tools: The solution first and tools next approach
It is never about the tools. It is a mindset. Tools will help you get the work done only once you know what you want to achieve.
We should constantly be asking ourselves: why do we need this and why are we doing this? It’s not about a ‘have to’ use mindset. Still, we see most of the folks thinking about ‘tools first’ approach when it comes to DevOps.
Breaking the silos and making things work with clarity is important for any company that wants to embrace digital transformation through DevOps. Bridging the gap between Dev and Ops is the essence of DevOps with culture in mind.
At the core, DevOps should always aim at establishing an environment where building, testing and releasing software can happen rapidly, frequently, and more reliably. It should be a team effort and shouldn’t be assigned to an individual.
4. Security is a priority
While DevOps ensures value and availability, DevSecOps ensures safety, trust, and confidence. While many companies are transforming themselves to become DevOps companies, security has become a concern & has become the priority.
With ever-increasing technological advancement in recent years, companies are expected to move fast, deliver fast and at the same time maintain security as the top priority. DevSecOps is the new game.
Integrating DevOps + Security = DevSecOps
Security should be a focus from the beginning of the development lifecycle. From source control to managing your dependencies, environments, and binaries to packaging them and the whole process-flow across the organization. I personally didn’t know that artifacts repository management systems are so critical. It seems like they add the required real shield in the software development process.
Neglecting security comes with a huge cost because your security flaws directly impact your business, brand, and authenticity. That’s why DevSecOps is becoming so important. Developers these days are required to think from the customers perspective and embed security right from the beginning.
‘Security’ is one of the facts your customers are mostly inclined towards, and you need to make sure they are in the good, safe hands. Don’t just say it, prove it. That is where DevSecOps principles come in, making certain ‘security’ is of the utmost priority and embedding security from the beginning and at each stage of the software development cycle. I always believe that engineering, product, marketing, and sales teams should be united for any organization to succeed, so you are unitedly aware and passing the critical information to each other. It’s a connected world. Make sure your security is more powerful than your threats.
5. Automation is key
For many years, organizations have struggled to adopt Agile. That’s where DevOps comes to the rescue.
A methodical approach to the road of DevOps is with a mindset of zero-touch automation. Yes. Just like Norman Vincent’s famous quote, “Shoot for the moon. Even if you miss, you’ll land among the stars”. But hold on, that doesn’t mean you have to deploy straight to the production. Just that if you want to do it, you should be able to.
You should be able to attain that stage where even an operation guy can do it with just a push of a button and not going SSH’ing bunch of servers.
The CI/CD pipeline is one of the best practices for DevOps teams to implement, for delivering code changes more frequently and reliably.
A CI pipeline is a set of build and test processes that produce a release candidate of your product. A CD pipeline takes that release candidate, subjects it to more rigorous system testing in a testing environment, and if those tests pass (including now also DevSecOps testing such as automated pen testing) deploys said candidate into production.
Here are some cool CI/CD tutorials you can try.
- Continuous Delivery with Docker, Kubernetes, and GoCD:
2. CI/CD Pipeline using JFrog Artifactory, which also serves as an artifact repository:
3. CI:CD for NODE.JS and KUBERNETES:
4. OpenShift 4 CI/CD Pipelines:
5. CI-CD for Azure Kubernetes Service AKS using Azure DevOps:
I have a special attachment with Kubernetes these days, and I believe that such tools revolutionize the software development ecosystem. With a vibrant community and backed by giants, the popularity of this tool is going to explode in the coming years. So, I would love to share some case studies that have a significant impact on software powered organizations.
Recently, many companies have gone through the digital transformation with the involvement of tools like ‘Kubernetes’, let’s talk about some examples and real cases.
Famous Kubernetes digital transformation case studies
Tinder’s move to Kubernetes
Due to high traffic volume, Tinder’s engineering team faced challenges of scale and stability. What did they do?
Yes, the answer is Kubernetes.
Tinder’s engineering team solved interesting challenges to migrate 200 services and run a Kubernetes cluster at scale totaling 1,000 nodes, 15,000 pods, and 48,000 running containers.
Was it easy? No way. But they had to do it for smoother business operations going forward. One of their Engineering leaders said, “As we onboard more and more services to Kubernetes, we found ourselves running a DNS service that was answering 250,000 requests per second.” Fantastic culture, Tinder’s entire engineering organization now has knowledge and experience on how to containerize and deploy their applications on Kubernetes.
Read this fascinating case study below,
Tinder’s move to Kubernetes
Reddit’s Kubernetes story:
Reddit is one of the top busiest sites in the world. Kubernetes forms the core of Reddit’s internal Infrastructure.
From many years, the Reddit infrastructure team followed traditional ways of provisioning and configuring. However, this didn’t go long until they saw some huge drawbacks and failures happening while doing the things the old way. They moved to ‘Kubernetes.’
See this amazing video where their infrastructure release engineering manager describes the Kubernetes story at Reddit.
The New York Times Journey to Kubernetes
Today the majority of their customer-facing applications are running on Kubernetes. What an amazing story! The biggest impact has been to speed up deployment and productivity. Legacy deployments that took up to 45 minutes are now pushed in just a few. It’s also given developers more freedom and fewer bottlenecks. The New York Times has gone from a ticket-based system for requesting resources and weekly deploy schedules to allowing developers to push updates independently.
Check out the evolution & the fascinating story of The New York Times tech stack
The Evolution of The New York Times Tech Stack | StackShare
Airbnb’s Kubernetes story
Airbnb’s transition from a monolithic to a microservices architecture is pretty amazing. They needed to scale continuous delivery horizontally, and the goal was to make continuous delivery available to the company’s 1000 or so engineers so they could add new services. Airbnb adopted to support over 1000 engineers concurrently configuring and deploying over 250 critical services to Kubernetes (at a frequency of about 500 deploys per day on average). I want you to see this excellent presentation from Melanie Cebula, the infrastructure engineer at Airbnb.
Develop Hundreds of Kubernetes Services at Scale with Airbnb
Pinterest’s Kubernetes story
With over 250 million monthly active users and serving over 10 billion recommendations every single day, that is huge. (The numbers might have changed now) As they knew these numbers are going to grow day by day, they began to realize the pain of scalability and performance issues.
Their initial strategy was to move their workload from EC2 instances to Docker containers; hence they first moved their services to Docker to free up engineering time spent on Puppet and to have an immutable infrastructure.
And then the next strategy was to move to Kubernetes. Now they can take ideas from ideation to production in a matter of minutes whereas previously, they would take hours or even days. They have cut down so much of overhead cost by utilizing Kubernetes and have removed a lot of manual work without making engineers worry about the underlying infrastructure.
Read their impressive story on Kubernetes website ‘Pinterest Case Study’
Pokemon Go’s Kubernetes story
How was ‘Pokemon Go’ able to scale so efficiently & became so successful? The answer is Kubernetes. Pokemon Go was developed and published by Niantic Inc. 500+ million downloads and 20+ million daily active users.
Pokemon Go engineers never thought their user base would increase exponentially surpassing the expectations within a short time, they were not ready for it, and even the servers couldn’t handle this much traffic.
The horizontal scaling on one side but Pokemon Go also faced a severe challenge when it came to vertical scaling because of the real-time activity by millions of users worldwide. Niantic was not prepared for this.
The magic of containers. The application logic for the game ran on Google Container Engine (GKE) powered by the open source Kubernetes project. Niantic chose GKE for its ability to orchestrate their container cluster at planetary-scale, freeing its team to focus on deploying live changes for their players. In this way, Niantic used Google Cloud to turn Pokémon GO into a service for millions of players, continuously adapting and improving. This got them more time to concentrate on building the game’s application logic and new features rather than worrying about the scaling part.
Impressive, isn’t it? Read the complete case study shared on Google Cloud.
Talking about Microservices, I have a famous Netflix example
Do you know how Netflix works so flawlessly? Hundreds of microservices, one giant service — Netflix.
What Netflix has gone through is a modern technology revolution and an example of how to scale suitably. Just hit the play button and watch your favorite shows, so simple, right? Nope.
There goes a lot of work behind the scenes to make sure the end user is happy. Netflix was one such company that understood the importance of microservices very early. It changed its monolithic architecture to microservices architecture where different microservices talked to each other through APIs. One microservice stores all the shows you’ve watched, one deducts the monthly fee from your credit card, one provides your device with the correct video files that it can play, one takes a look at your watching history and uses algorithms to guess a list of movies that you will like, etc.
This approach has transformed the Netflix’s business and has made them the world’s trusted brand. Again, why microservices? Microservices architecture is simple to develop, test, deploy, and scale.
Read more about how Netflix works with microservices architecture.
Though there were over 100 announcements made at the Cloud Next conference, Anthos stands out. Google CEO Sundar Pichai announced the idea behind Anthos it is to “write once and run anywhere.”
I know, it can be confusing to understand everything. But, I have a suggestion to make if you want to know more — Janakiram MSV recently shared an amazing article on Forbes, ‘Everything You Want To Know About Anthos — Google’s Hybrid And Multi-Cloud Platform’
Many companies are software companies now, and each of them could become a DevOps company soon. IoT has a lot to do with technology advancement & in software development space.
Maslow 2.0: DevOps Will Dictate All Software in the World
IoT and DevOps is such a fantastic combination. If IoT products & processes have to be successful, they need DevOps. I was going through a video where Google CEO Sundar Pichai was talking about the IoT devices & the advancements. I was impressed to see how a traditional watch can be modified into an IoT device by adding computing power, connectivity and a layer of intelligence to it. This is what the end users want, an upgraded version of the technology and enhanced experience.
For IoT things to be connected always and communicate with each other, you need to make your software as liquid as possible. This is where DevOps plays a vital role, Development and deployments are moving away from fixed, versioned releases, toward streams of verified software components. Liquifying the software development process produces huge benefits in faster and more frequent releases at a lower cost. (I love JFrog’s vision of liquid software)
What is the scope for IoT and DevOps?
For 2020, the installed base of the Internet of Things devices is forecast to grow to almost 31 billion worldwide. This number is enough to see a huge potential ahead for both DevOps & IoT.
Talking about the tools that enable digital transformation — Today’s tools will get replaced by the upcoming and more modern solutions, we have seen this happening over the years now.
BTW, did you read my recent article on DevOps Trends? If not, here is the link below,
8 DevOps Trends to Be Aware of in 2019
It is our quick response to rapidly changing technology trends that generate the excitement and drive the brand value, DevOps helps in growth hacking your brand value exponentially by delivering trending things (features) faster to the market in a more organized way.
Anything you would like to let me know? Let’s connect on Twitter.
Digital Transformation for Modern Enterprises Through DevOps — A Complete Guide was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.