The OVF package is invalid and cannot be deployed – In the trenches with the AWS Discovery Connector

I was working with a customer recently who had trouble deploying the AWS Discovery Connector to their VMware environment. AWS offer this appliance as an OVA file. For those who aren’t aware, OVA (Open Virtualisation Archive) is an open standard used to describe virtual infrastructure to be deployed on a hypervisor of your choice. Typically speaking, these files are hashed with an algorithm to ensure that the contents of the files are not changed or modified in transit (prior to being deployed within your own environment.)

At the time of writing, AWS currently offer this appliance hashed in two flavours… MD5 or SHA256. All sounds quite reasonable right?

  • Download the OVA with a hash of your choice
  • Deploy to VMware.
  • Profit???

Wrong! I was surprised to receive an email from my customer stating that their deployment had failed (see below.)

There’s a small clue here…

The Solution

My immediate response was to fire up google and do some reading. Surely someone had blogged about this before? After all…. I am no VMware expert. I finally arrived at the VMware knowledge base, where I began sifting through supported ciphers for ESX/ESXi and vCenter. The findings were quite interesting, you can find them summarised below:

  • If your VMware cluster consists of hosts which run ESX/ESXi 4.1 or less (hopefully no one) – MD5 is supported
  • If your VMware cluster consists of hosts which run ESX/ESXi 5.x or 6.0 – SHA1 is supported
  • If your VMware cluster consists of hosts which run ESX/ESXi 6.5 or greater – SHA256 is supported

In the particular environment I was working in, the customer had multiple environments with a mix of 5.5 and 6.0 physical hosts. As I was short on time, I had no real way of telling if the MD5 hashed image would deploy on a newer environment. I also don’t have a VMware development environment to test this approach on (by design.)

After a few more minutes of googling, I was rewarded with another VMware knowledge base article. VMware provide a small utility called “OVFTool.” This applications sole purpose in life is to convert OVA files (you guessed it) ensuring that they are hashed with supported cipher of your choice. In my scenario, the file was re-written using the supported SHA1 cipher. All of this was triggered from a windows command line by executing:

ovftool.exe –shaAlgorithm=SHA1 <source image.ova> <destination image.ova>

After this I was able to successfully deploy the AWS Discovery Connector OVA as expected using my freshly minted image.

You can grab a copy of the tool – here

You can read more about VMware supported ciphers – here

Finally, I should call out that this solution is not specific to deploying the AWS Discovery Connector. Consider this approach if you are experiencing similar symptoms deploying another OVA based appliance in your VMware environment.


Planning a move to the cloud with the AWS Application Discovery Service

Here at cloudstep, we love to help our customers achieve their goals. We believe that the cloud is a tool in the toolbox and we can use that multi-facet tool to help our customers realise success. Planning for success starts with goals, and goals come in many different shapes and sizes.

For any given solution, a customers goal may be focused on achieving financial or competitive advantage. Alternatively, they may be looking to realise operational efficiency by improving a day-to-day process using automation and orchestration. No matter your goal, you need a solid plan to ensure success. More often than not, that starts with validating that you have a sound understanding of the current state environment which will enable you to move forward towards achieving your goals.

Today I want to talk about a capability provided as part of the Migration Hub offering in AWS, the Application Discovery Service. This is a tool that we regularly use and encounter when meeting with customers. The core idea behind this capability (aptly named) is to help you discover critical details about your environment. This includes performance metrics and resource utilisation data which can be used for cost modelling, in our case… cloudstep.io. The tooling can also gather detailed network metrics to help you better understand the integrations and interfaces between applications in your environment. All of this data is at your disposal once you have decided upon which deployment model you would like to utilise.

AWS offer both an Agentless Discovery service and an Agent Based discovery service. Ordinarily, we typically use the Agentless discovery service. This is a great approach for organisations that operate entirely virtualised VMware infrastructure. Using this approach allows you to quickly inventory each of your VM’s that reside within your vCenter without the requirement of installing an agent on each guest VM. Choosing this path means that the agentless discovery service will query the VMware vCenter for performance metrics (irrespective of which OS the guest is running.) It can’t actually reach inside the virtual machine, therefore it is dependent on having a compatible version of the “VMware Tools” running inside each VM.

If you have a mixture of Physical and Virtual servers in your fleet, or you run another Hypervisor (such as Hyper-V) you may need to consider the Agent based deployment model. This approach is generally considered more labour intensive to get up and running due to the requirement to get hands on with each server. There are also some constraints around which OS’s it can fetch data from. So be mindful of this. You may even find that the best approach is to run a mix of the two deployment models. The outcome of both approaches is a series of performance data metrics which is shipped outbound using HTTPS to an S3 bucket. This bucket can then be queried by the AWS Migration Hub service. Alternatively you can export the data and analyse it using tooling of your choice.

For the remainder of the article, I will focus on our experience with the Agentless discovery approach. As I mentioned earlier, this is our preferred approach because it takes about an hour to get up and running and it generally produces more than enough quality data. In our experience, this provides an excellent baseline for commencing our cloudstep.io cost modelling engagement.

The AWS Agentless discovery connector operates as a VMware appliance within your vCenter environment. AWS provide a pre-canned OVA file which is around 2GB in size. You simply deploy this, the same way you would with any other open virtualisation archive. If you run multiple vCenters for different physical locations, you will need to deploy multiple instances of the appliance to service each stack.

If you experience issues deploying the OVA image within VMware, review my other blog – here

Deploying these appliances in enterprise environments often presents unique challenges. In our experience, this is where customers tend to have issues. Sometimes they deploy the appliances to management networks which don’t provide DHCP so they need to manually bind IP addresses, or there may be firewall rules which prevent connections from an access layer switch to perform the configuration process. The appliance does offer a terminal console (sudo setup.rb) where you can configure foundation services such as IP configs and DNS servers.

Another consideration you should make is “How will my appliance get outbound access to the internet?” After all, its sole purpose is to ship data outbound using HTTPS to an AWS S3 bucket via the Migration Hub. From a firewalling perspective, this is usually quite nice as outbound TCP443 generally doesn’t warrant a discussion with your security team. However, should your security team raise concern about corporate data being shipped off to the internet, AWS provide a detailed article on exactly what information is collected – here.

A final consideration you should make is proxy servers. If you utilise upstream proxy servers to police internet access, consider any rules you may need to define here. Typically speaking, the appliance will run headless in a “SYSTEM” context so you may need to allow it unauthenticated outbound internet access. Take a moment to think through any pitfalls you may encounter and also consider how you intend on interfacing with the appliance.

Once you have deployed your shiny new VM, you can fire up a web browser and configure it using the native web interface ( http://127.0.0.1 ) There are two things you will need:

  1. Read-only credentials to the vCenter you will inventory.
  2. AWS IAM Credentials to authenticate to the Migration Hub service.

Once you have completed the wizard, you will be greeted with a summary screen that presents instance specific configuration such as the appliances AWS connector ID.

The final step in the process is to to start the data collection process. You can action this by making API calls using the AWS CLI

aws discovery start-data-collection-by-agent-ids –agent-ids <connector ID>

Alternatively, you can also navigate to the Migration Hub console and manually approve the data collection process. If you have more than one appliance, you will have multiple connector ID’s registered here. You can validate that these line up by browsing to the appliance web interface where it will list its respective connector ID. The service polls the vCenter environment every 60 minutes, therefore it is reasonable to expect that you should be able to query your data within the AWS migration hub within an hour or two assuming everything is functioning as expected. Alternatively you can export the collected data to a CSV to commence your migration analysis.

In this blog I have explored the Application Discovery Service which is a capability provided by AWS’ Migration Hub. We have talked through common pitfalls that customers often experience when working with the agentless discovery service in effort to simply the deployment process. The data collected provides powerful insights into your environment which is crucial to success when planning a cloud migration. Should you need further assistance, do not hesitate to reach out to the team at cloudstep.io. We’d love to hear from you, and to help you on the road to success

To the cloud!


AWS obtain PROTECTED level certification for Australian Region

Earlier this week Amazon Web Services made a statement, indicating that the battle of tier-one public cloud providers is still heating up. Yesterday Matthew Graham (AWS Head of Security Assurance for Australia and New Zealand) announced that The Australian Cyber Security Centre (ACSC) had awarded PROTECTED certification to AWS for 42 of their cloud services. 

In what appears to be a tactical move that has been executed hot off the trail of Microsoft announcing their PROTECTED accredited Azure Central Regions in the back half of last year. This clearly demonstrates that AWS aren’t prepared to reduce the boil to a gentle simmer any time soon.

Graham announced “You will find AWS on the ACSC’s Certified Cloud Services List (CCSL) at PROTECTED for AWS services, including Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3), AWS Lambda, AWS Key Management Service (AWS KMS), and Amazon GuardDuty.”

He continued to state “We worked with the ACSC to develop a solution that meets Australian government security requirements while also offering a breadth of services so you can run highly sensitive workloads on AWS at scale. These certified AWS services are available within our existing AWS Asia-Pacific (Sydney) Region and cover service categories such as compute, storage, network, database, security, analytics, application integration, management and governance. “

Finally, delivering a seemingly well orchestrated jab “Importantly, all certified services are available at current public prices, which ensures that you are able to use them without paying a premium for security.”

It is no secret that the blue team currently charges a premium for entry into their PROTECTED level facility (upon completion of a lengthy eligibility assessment process) due to a finite amount of capacity available.

Both vendors state that consumers must configure services in line with the guidance in the respective ACSC certification report and consumer guidelines. This highlights that additional security controls must be implemented to ensure workloads are secured head to toe whilst storing protected level data. Ergo, certification is not implicit by nature of consuming accredited services.

AWS have released the IRAP assessment reports under NDA within their Artefact repository. For more information, review the official press release here.


Using the AWS CLI for Process Automation

Amazon Web Services is a well established cloud provider. In this blog, I am going to explore how we can interface with the orange cloud titan programmatically. First of all, lets explore why we may want to do this. You might be thinking “But hey, the folks at AWS have built a slick web interface which offers all the capability I could ever need.”Whilst this is true, repetitive tasks quickly become onerous. Additionally, manual repetition introduces the opportunity to introduce human error. That sounds like something we should avoid, right? After all, many of the core tenets of the DevOps movement is built on these principles (“To increase the speed, efficiency and quality of software delivery”– amongst others.)

From a technology perspective, we achieve this by establishing automated services. This presents a significant speed advantage as automated processes are much faster than their manual counterparts. The quality of the entire release process improves because steps in the pipeline become standardised, thus creating predictable outcomes.

Here at cloudstep, this is one of our core beliefs when operating a cloud infrastructure platform. Simply put, the portal is a great place to look around and check reporting metrics. However, any services should be provisioned as code. Once again, to realise efficiency and improve overall quality.

How do we go about this and what are some example use cases?”

AWS provide an open source CLI bundle which enables you to interface directly with their public API’s. Typically speaking, this is done using a terminal of your choice (Linux shells, Windows Command Line, PowerShell, Puty, Remotely.. You name it, its there.) Additionally, they also offer SDK’s which provide a great starting point for developing applications on-top of their services in many different languages (PowerShell, Java, .NET, JavaScript, Ruby, Python, PHP and GO.)   

So lets get into it… The first thing you’ll want to do is walk through the process of aligning your operating environment with any mandatory prerequisites, then you can get install the AWS CLI tools in a flavour of your choice. The process is well documented, so I wont cover it off here.

Link – https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html

Once you have the tools installed, you will need to provide the CLI tools with a base level of configuration which is stored in a profile of your choice. Running “AWS Configure” from a terminal of your choice is the fastest way to do this. Here you will provide IAM credentials to interface with your tenant, a default region and an output format. For the purpose of this example I’ve set my region to “ap-southeast-2” and my output format to “JSON.”

aws configure example

From here I could run “aws ec2 describe-instances” to validate that my profile had been defined correctly within the AWS CLI tools. The expected return would be a list of EC2 instances hosted within my AWS subscription as shown below.

aws ec2 describe-instances example

This shouldn’t take more than 5 minutes to get you up and running. However, don’t stop here. The AWS CLI supports almost all of the capability which can be found within the management portal. Therefore, if you’re in an operations role and your company is investing in AWS in 2019. You should be spending some time to learn about how to interface with services such as DynamoDB, EC2, S3/Glacier, IAM, SNS and SWF using the AWS CLI.

Lets have a look at a more practical example whereby automating a simple task can potentially save you hours of time each year. As a Mac user (you’ve probably already picked up on that) I often need to fire up a windows PC for Visual Studio or Visio. AWS is a great use case for this. I simply fire up my machine when I need it and shut it down when I’m done. I pay them a couple of bucks a month for some storage costs and some compute hours and I’m a happy camper. Simple right?

Lets unpack it further. I am not only a happy camper. I’m also a lazy camper. Firing up my VM to do my day job means:

  • Opening my browser and navigating to the AWS management console
  • Authenticating to the console
  • Navigating to the EC2 service
  • Scrolling through a long list of instances looking for my jumpbox
  • Starting my VM
  • Waiting for the network interface to refresh so I can get the public IP for RDP purposes.

This is all getting too hard right? All of this has to happen before I can even do my job and sometimes I have to do this a few times each day. Maybe its time to practice what I preach? I could automate all of this using the AWS tools for PowerShell, which would allow me to automate this process by running a script which saves me hours each year (employers love that.) Whilst this example wont necessarily increase the overall quality of my work, it does provide me with a predictable outcome every single time.

For a measly 20 lines of PowerShell I was able to define an executable script which authenticates to the AWS EC2 service, checks the power state of my VM in question. If the VM is already running it will return the connectivity details for my RDP client. If the VMis not running, it will fire up my instance, wait for the NIC to refresh and then return the connectivity details for my RDP client. I then have a script based on the same logic to shutdown my VM to save money when I’m not using the service. All of this takes less than 5 seconds to execute.

PowerShell Automation Example

The AWS CLI tools provide an interface to interact with the cloud provider programmatically. In this simple example we looked at automating a manual process which has the potential to save hours of time each year whilst also ensuring a predictable outcome for each execution. Each of the serious public cloud players offer similar capability. If you are looking to increase your overall efficiency, improve the quality of your work whilst automating monotonous tasks, consider investing some effort into learning a how to interface with your cloud provider of choice programmatically. You will be surprised how many repetitive tasks you can bowl over when you maximise the usage of the tools you have available to you.