DevOps is a set of practices that combines software development (Dev) and IT operations (Ops). It aims to shorten the systems development life cycle and provide continuous delivery with high software quality.DevOps is complementary with Agile software development; several DevOps aspects came from the Agile methodology.
There is no How-to-guide when it comes to DevOps. To be a well rounded DevOps'er you need to have had hands on experience with both Development and Operation.
Once a DevOps has this experience. That person will never say that he does not need to worry about performance or writing code that contain empty try/catch blocks.
His Ops experience will tell him what major headaches this can be in a production environment.
At Sharks4IT we believe the world of Kubernetes is here to stay and they make the world of IT more manageable for Developers and Operations as well.
We have worked a lot with kubernetes in different varieties. Managed Kubernetes such as Openshift. On premise vanilla installation and Kubernetes in the Cloud. In all we have worked with kubernetes for 6+ years
We have experience running a Kubernetes cluster from an administrative perspective. And have hands on with CI/CD and automated deployment into a Kubernetes Cluster using a variaty of scripting languages such as Ansible, terraform and other shell scripts.
Writing container images using Docker and creating Deployment , ReplicaSet and much more using tools such as Helm or just running simple Kubernetes admin commands.
We have worked with Cluster Setup where High Availability is required and a need for fast recovery time on different scales and approces.
If you have a Kubernetes related tasks then Please Contact Us
At Sharks4IT we see ansible as the glue that holds infrastructure as code together. And we have been using ansible for 5+ years
Ansible is very good at managing infrastructure as code and can also be used to provide self-service solutions and provisioning of kubernetes infrastructure using ansible and k8s modules.
We have experience with using ansible for infrastructure testing such as VM-Server sizing validation, network configuration verification and not only for on premise we also used ansible to do infrastructure testing of Confluent Cloud. The solution was using ansible and kubernetes resources to set up the initial testing mechanisms.
If you have a Ansible related tasks then Please Contact Us
We have worked with Openshift 3.11 for 4 years as administrators managing the platform. Openshift patching using redHat ansible playbooks.
Capacity management , capacity calculations and cost analysis. To do this we made sure that application teams were forced to consider their resource needs. This was done by setting the default values for cpu and memory so low. That the solutions would not be able to start. The reason for this is that kubernetes can better manage the cluster when quota & limits have been specified. Otherwise you can risk application pods starting up on already overloaded nodes.
The setup required a high availability and security. We configured 3 masters and 4 Infra nodes for the higher ENV settings. To make sure we would have as little outages as possible. The custers ended up having 30-40 worker nodes.
The implementation of the 3 pod policy was also introduced. Stating that if an application running on Openshift wanted guaranteed HA. They would as a minimum have 3 pods running. We also made sure that we had affinity and anti-affinity rules making sure that not all pods would be running on the same nodes.This way if Operations were doing patching and Openshift would be moving pods around there would still be at least 1 pod running.
The setup also required that we exported logs and metrics to an external log and metric system. In this case ELK was chosen. The solutions were many but we installed file and metricbeat as daemonsets making sure we would collect logs and metrics from each node in the cluster. We also used the metricbeat prometheus module to scrape and enrich metric data so we could present cost information in ELK based on a project's resource usage and consumption.
Migrating from 3.11 to 4.x was also part of the task. Instead of having openshift on prem. we would move to a managed solution. And for this we analyzed different approaches to how we could do the upgrade. Because going from 3.11 to 4 is not a standard upgrade but a lift and shift operation.
If you have a Openshift related tasks then Please Contact Us
Doing devops work in kubernetes sometimes requires debug tools. However container images do not always come with the desired tools that are needed. And to debug connection issues, certificate issues etc. it’s a good idea to be able to do this from within a container.
That is why we wrote a devopshelper docker image that we use to debug certain situations that arise when working with kubernetes.
This image can be deployed into any namespace alongside the containers that are having the issues. This way our devopshelper can test a variety of situations using the tools in the image.
Beside the devopshelper we have also extended existing images that mostly are used on the operations side of things.
We have however also assisted in writing different java related docker images
If you have a Docker related tasks then Please Contact Us
Sharks4IT have been working with Automated Deployment for 18+ years.
What we have learned is that people are not really built to do manually changes at 2 am to a production environment following a guide. This approach is just destined to fail and result in employee burn-out faster than anything.
We have a saying “If you have to do something once that’s fine. If you have to do it, twitch Automate it”.
“but I can do it way faster than if I have to automate it”. Yes that is true but you can’t guarantee that the results are the same if you have to do something over and over again. And when something goes wrong it’s 10 times as difficult to find the error.
So making the decision to invest the time in automation will be worth it in the long run.
We have done a lot of automation written in different languages such as Ant, python/jython, ansible, Terraform and a variety of shell and bash scripts.
On this page you will find a more detailed explanation about what we have done and what we have worked with.
If you have Automated Deployment related tasks then Please Contact Us
Sharks4IT have been working with python for 8+ years.
We helped develop a python deployengine in a time where tools like ansible and terraform did not exist. The deployengine could deploy a variety of middleware application platforms such as websphere, websphere cluster, weblogic, tomcat , jboss and many more.
The engine worked on the principle that everything should be created from scratch to make sure nothing was missing or out of place. Was in version control was the only truth.
The Deployengine was also before the time of Azure or AWS so everything would be deployed on WM’s running windows or linux.
The deployengine also where using QA/QC to guarantee quality. Each middleware server would be deployed every 3 hours to verify that everything worked.
The idea for the engine arose from using a XML based script language called ANT. But because of deployment time ANT was replaced by our own python deploy engine.
The complexity and maintenance responsibility grow overtime but the deployengine was able to withstand the test of time
If you have Python related tasks then Please Contact Us
Sharks4IT have been introduced to terraform and have helped setup self service and infrastructure deployment for the Confluent Cloud Platform. We have worked with terraform for about 2 year
We have used providers from Confluent Cloud and created a reusable child module configuration. That will make it possible to define ready to go kafka clusters and environments according to a department's need.
When working in the cloud. terraform is a great tool to maintain infrastructure as code. And you can quickly be up and running because a lot of the work has already been done in the community modules.
The trick however is to configure your terraform repository to fit your need’s and define the reuse of code that is desired.
If you have terraform related tasks then Please Contact Us
When working with Kubernetes and containers. You need a way to handle and maintain the kubernetes configuration files.
When you're doing simple tests, running kubectl commands is fine. But for larger scale deployments you need to have more control and a way to manage the state of your kubernetes deployments
This is where Helm comes in. Helm is a tool designed to manage kubernetes objects using variables and templates.
but also a very powerful feature is that it keeps track of the state of your deployments and you can use helm to install, upgrade roll-back and delete whatever you create in a kubernetes cluster.
At Sharks4IT we highly recommend using Helm when working with kubernetes. We have been using Helm for 3+ years
If you have helm related tasks then Please Contact Us
When it comes to CI/CD the tool of choice has been mostly Jenkins. And we spend time working on jenkins pipelines to be able to run ansible, python and kubernetes related tasks.
Since we have been working with Kubernetes we have also been looking into alternatives such as argoCD.
The main benefit/difference between jenkins and argoCD is that argoCD can monitor and manage the state of anything that is deployed into a kubernetes cluster.
It also has the ability to make sure that what is in version control is the truth. So if something has been changed manually. ArgoCD can detect this and redeploy what is defined in version control.
From a DevOps point of view this tool can make it much easier to manage once kubernetes infrastructure and container images.
If you have CI/CD related tasks then Please Contact Us
When it comes to DevOps one discipline that is sometimes not prioritized is Observability. However the information provided is key to understanding why something goes wrong. You will also have the ability to be alerted to potential problems that can arise in your production environment.
A platform like ELK is a very well rounded platform to manage observability. At Sharks4IT we have been involved in implementing various beats to solve different observability requirements.
One task was to use Metricbeat to extract metrics from prometheus running in Openshift. The idea was to extract capacity usage information and then design dashboards in ELK. So each team would be able to see the total cost for a project to run on Openshift.
We have also installed metricbet & filebeat on kubernetes clusters to collect performance and log data and send it to ELK.
Another task was to implement a Rest Proxy solution for Confluent Cloud and from here we also needed to collect metrics in relation to Confluent Kafka instances running both on prem and in cloud. For this solution Logstash was used to collect and enrich metrics. And again the data was used to design project specific dashboards to show performance data and possible bottlenecks in the message management.
If you have ELK related tasks then Please Contact Us
When Sharks4IT works for a customer we make sure that the work we do is well documented and handed over to the department team.
We excel at documentation and the writing of how-to guides. We believe that when a team member joins a department there should be how-to’s in place that quickly help the employee get started with his assignments.
Instead of relying on other team members to make the time to introduce procedures and processes to the new employee’s. An approach using how-to / run-books that explain the procedure/process step-by-step is a way to go.
Another benefit that can be used is recorded sessions of knowledge sharing done in the teams.
These approaches are a very good way to secure knowledge in a department. And Sharks4IT will do what we can to make sure this is in place
Copyright © Alle rettigheder forbeholdes
