In computing, latency = elapsed time
It is an expression of how much time it takes for a data packet to travel from one device to another.
- A low latency indicates a good performing network communication.
- A high latency indicates a poor performing network communication.
High latency may occur for many reasons such as
- CPU utilization
- Disk Paging errors
- Application stuck threads
- High volume log writing nature of applications
- Encryption/decryption utilities
- OS tuning parameters and
- Legacy system integration.
It may also happen in highly available environment due to poor session tracking algorithms, singleton classes etc.
The IT Engineers have a good control to diagnose and troubleshoot latency issues in the conventional IT however the diagnostic tools are very expensive. Further it has become more expensive with siloed approach, resources and other licensing requirements. Coordinating a troubleshooting issue on a production environment is a nightmare. The term ‘It is not my fault’ from different team members is a common theme.
In the Public Cloud environment, diagnostic tools are readily available. There is no siloed approach, no need for unique licensing requirements etc. A single pane of view for most of the diagnostic and monitoring information exists. On the side note, the Cloud consumer has no control on underlying IT resources. MultiCloud changes the monitoring landscape of Cloud resources. The on-call engineer at MultiCloud is no longer an expert to troubleshoot an integrated issue across AWS, Azure, GCP and On-Premise.
Well, you don’t get the best of both worlds.
MultiCloud pose many challenges to organizations and one of them is performance.
- Who is responsible for what?
- Where is the origination of high latency point?
- How do we gather metrics and find the root cause?
There are many unanswered questions.
Cloud provider offerings such as unlimited storage, computing power, availability zones and other boundless promises creates an unaccountable atmosphere in the model leading to high latency.
To take full advantage of MultiCloud for high performance, you need to break down your requirements, individual Cloud provider capabilities and understand that you distribute your work load across multiple Clouds.
Below listed are a few pointers that may help with your MultiCloud journey.
Network Latency Primer
Key points:
- Moving application and data closer to users.
- Ensuring nodes within the application are close together.
Bringing application closer to user is an expensive process in Conventional IT. It was suitable for Fortune 100 companies who have enough money and resources to establish their data centers across the globe. It created massive ‘server sprawling’ and unmanaged data centers for them.
Cloud Computing made this primer easy. With 54 regions and 140 availability zones across the globe, Microsoft Azure brings your applications and data closer to the end user in a specific geographical location.
It is not advisable to host your applications from Cloud A and database from Cloud B. Both must reside in Cloud A to deliver high performance but you can host your right fax server, batch processing and other supporting systems in Cloud B. Also note that, integration with On-Premise systems such as Mainframe, ERP etc. has an impact on your overall application performance.
Do not spread your MVC Architecture across multiple Clouds.
Proactively Identify bottlenecks
Key points:
- Instrument application components.
- Conduct periodical performance test.
Traditional approach of instrumenting applications is applicable for the Cloud environment too. More than ever before, the data center components at the Cloud environment became visible. Every Cloud provider has their system management tools to instrument, monitor and manage the applications. Using those tools proactively mitigate the high latency issues for your applications.
AWS provides a set of tools such as AWS CloudWatch, AWS Systems Manager, AWS CloudTrail that all work together to control, operate and monitor all parts of your Cloud Infrastructure and applications. Microsoft Azure and Google Cloud also provides the similar set of tools to support their customers.
There is no unified view (Single pane) in the MultiCloud environment to view the above tools metrics and unfortunately, they should be viewed separately to troubleshoot. Prometheus comes handy for many customers however the speed of light innovations at the Public Cloud requires continuous objects update at your in-house monitoring solution. Therefore, it is important to practice the Cloud management tools to identify bottlenecks in the application pipeline and consolidate them.
Periodical performance tests in Conventional IT is expensive. The constrains are limited environment resources, unique performance scripts for each application and iterations, coordination with monitoring and networking teams, interpreting the results and charts.
The fast provisioning of Cloud resources and robust configuration management tools makes periodical performance test iterations automated and a test engineer can apply them as templates on various environments. It is advisable to turn off elasticity of the virtual machines and execute the test on an expected resource utilization metrics. The performance environment can be archived when not in use to save the cloud fee.
The right distribution of your application architecture across MultiCloud is crucial to utilize the Cloud provider management tools in order to get the right latency metrics.
Hybrid MultiCloud
With a hybrid architecture, you can host some of your latency sensitive applications On-premise and migrate other Cloud native applications to the Cloud in order to use the best of both worlds. Though it may seem to be an expensive solution, hybrid MultiCloud may also fulfill some government regulations, data protection and security requirements.
Google GKE (Google Kubernetes Engine) allows you to extend your Kubernetes Cluster to On-premise to manage the workloads efficiently in the DevOps world.
Cloud native applications
Lift and Shift (Rehost) of legacy applications doesn’t perform well in the Cloud. These monolithic applications are not designed to utilize the Cloud resources such as elasticity, autoscaling, software load balancers, REST API’s etc.
A Cloud Native application is loosely coupled, service oriented, containerized, declarative and independent Microservices. Refer my article to know more about Cloud native applications here.
It is inevitable to rewrite your applications as Cloud native in the coming years in order to fulfill the SLA/SLO requirements from MultiCloud hosting. Containerizing your applications may also enable them to spread across MultiCloud without rework.
Microservices are inherently scalable that has high velocity to deploy new features rapidly. The rapid deployment of Microservices may create bad deployments and instable code. A thorough code review and approval process should be in place to make the applications robust and high performing.
Latency based routing
Public Cloud providers have several routing policies to choose. One of them is latency-based routing. At AWS, you can create latency records and store them to reference with Route 53. When Route 53 receives a DNS query for your domain, it determines which AWS Regions you’ve created latency records for, determines which region gives the user the lowest latency, and then selects a latency record for that region to divert the traffic.
Azure and Google also provide similar latency-based routing with their own resources mechanism.
Conclusion
It may not be possible to control the latency at MultiCloud environment fully. Monitoring, conducting periodical performance tests, utilize cloud provider network capabilities, convert your applications to Cloud native and a fine application distribution architecture should become your priorities to fulfill the 99.9% up time objective.
References
Cloud Architecture Patterns, by Bill Wilder – O’Reilly Media
AWS, Azure and GCP documents
Want to be a cloud computing expert? Check out these training courses
