Azure Firewall vs Network Security Group (NSG)

An important Security measure when running workloads in Azure or any Cloud service is to control the type of traffic that flows in and out of resources.  The resources can be virtual machines running a SQL database, web applications or domain services.  In Azure, there are two security features that can be used to manage both inbound and outbound traffic to resources:  Azure Firewall and Network Security Groups (NSGs).  In this article, I’m going to show how the two compare to each other and can be used together to protect to traffic resources in Azure.

Azure Firewall and NSG Overview
Lets start with Network Security Groups.  An NSG filters traffic at the network layer and consists of security rules that allows or denies traffic based on 5-tuple information:
1. Protocol – such as TCP, UDP, ICMP
2. Source – IP address,
3. Source port
4. Destination
5. Destination port

You can associate an NSG with a subnet or the network interface of an Azure VM.  Fun fact – in your mother’s Azure (the old classic model), it was possible to link an NSG to a VM as well as subnet.  In accordance with Best practices, it’s recommended to scope NSGs at the subnet level or network interface, not both. This can make it complicated when having to troubleshoot network issues.  Also, the same NSG can be applied to multiple subnets.

You can probably imagine how NSG rules can become difficult to manage in large environments that contain multiple subnets and virtual machines. Who wants to manually input rules allowing traffic to individual IP addresses?  This is where Application Security Groups (ASGs) come to the rescue.  An ASG is a logical grouping of virtual machines that allows you to apply security rules at scale.  For example, if you have a group of VM’s serving a web application, the VM’s can be placed in an ASG called “webappvms”.  The webappvms group can then be added to a rule within an NSG allowing HTTP (TCP) traffic over port 80.  This alleviates the need to add individual IP addresses to the security rule.

Azure Firewall is a highly available, managed firewall service that filters network and application level traffic.  It has the ability to process traffic across subscriptions and VNets that are deployed in a hub-spoke model.  Azure Firewall is priced in two ways: 1) $1.25/hour of deployment, regardless of scale and 2) $0.016/GB of data processed.

Azure Firewall and NSG Comparison
An NSG is a firewall, albeit a very basic one.  It’s a software defined solution that filters traffic at the Network layer.  However, Azure Firewall is more robust.  It’s a managed firewall service that can filter and analyze L3-L4 traffic, as well as L7 application traffic.  Azure Firewall provides the same capabilities as an NSG, plus more. The following chart offers a comparative illustration of each solution:

Here are some definitions if you’re not familiar with all of the features listed in the above chart:

  • Service Tags – these are labels that represent a range of IP addresses for particular services such as Azure Key Vault, Data Lake, Container Registry, etc.  These are managed by Microsoft and cannot be customized.  You can learn more about them here. As an example, here’s a Service tag I have configured for Event Hub in an outbound NSG rule.  This same rule can also be created in Azure Firewall.
  • FQDN Tags – represent a group of fully qualified domain names of Microsoft services such as Windows Update or Azure Backup.  Like Service tags, they are maintained by Microsoft and cannot be customized.  The are a significantly fewer number of  FQDN tags than Service tags.  Go here to see the list of FQDN tags. Here’s an example of FQDN tag I have for Windows Update in my Azure Firewall application rule.  This allows you to avoid creating multiple application rules for each of the numerous Windows Update endpoints.  One tag to rule them all!
  • SNAT – Source Network Address Translation is a feature of Azure Firewall. It’s possible to configure Azure Firewall with a Public IP address (PIP) that can be used to masked the IP address of Azure Resources that are sending out via the Firewall.
  • DNAT – Source Destination Address Translation is used to translate incoming traffic to the firewall’s Public IP to the Private IP addresses of the VNet.

Azure Firewall and NSG in Conjuction
NSGs and Azure Firewall work very well together and are not mutually exclusive or redundant.  You typically want to use NSGs when you are protecting network traffic in or out of a subnet.  An example would be a subnet that contains VMs that require RDP access (TCP over 3389) from a Jumpbox. Azure Firewall is the solution for filtering traffic to a VNet from the outside.  For this reason, it should be deployed in it’s own VNet and isolated from other resources.  Azure Firewall is a highly available solution that automatically scales based on its workload.  Therefore, it should be in a /26 size subnet to ensure there’s space for additional VMs that are created when it’s scaled out.

A scenario to use both would be a Hub-spoke VNet environment with incoming traffic from the outside.  Consider the following diagram:

The above model has Azure Firewall in the Hub VNet which has peered connections to two Spoke VNets.  The Spoke Vnets are not directly connected, but their subnets contain a User Defined Route (UDR) that points to the Azure Firewall, which serves as a gateway device.  Also, Azure Firewall is public facing and is responsible for protecting inbound and outbound traffic to the VNet.  This is where features like Application rules, SNAT and DNaT come in handy.  If you have a simple environment, then NSGs should be sufficient for network protection.

The following links are to Microsoft docs that provide detailed information about Azure Firewall and Network Security Groups and were used as source material for this article:
https://docs.microsoft.com/en-us/azure/virtual-network/security-overview
https://docs.microsoft.com/en-us/azure/firewall/overview
https://docs.microsoft.com/en-us/azure/firewall/integrate-lb

 

 

 

Standard

Advice for First-time or Aspiring Speakers

Back in April of this year, I had the opportunity to present at the Global Azure Bootcamp.  It was a great experience and my first time ever doing a Technical presentation.  To be honest, there was a time I could not see myself standing before an audience and delivering a presentation. I’m sure many of you feel the same way.  If you are considering being a speaker or preparing to deliver your first talk, perhaps these tips can be of some benefit to you.

Your Topic
It’s important to choose a topic that you care about, instead of something you think other people care about.  How you feel about your topic will have an affect on your preparation and performance. Full disclosure: I initially chose a topic because I thought it was cool or “sexy”. I knew enough about the subject to present on it, but I found myself lacking motivation to prepare for it. So I decided to speak about something that resonated with me. Once I did that, I became motivated and putting together my presentation became effortless. Additionally, talking about something you care about enhances your performance. If you love your topic, you’ll feel better while speaking. And if you feel better, you perform better.

To Err is Human
You will make mistakes; I guarantee it.  No matter how much preparation you undergo, you will either overlook something you intended to mention, have technical issues, stumble over an explanation or experience some type of screw up. Murphy’s Law ( what can go wrong, will go wrong) will be in full affect. But prepare yourself in advance by accepting the fact that you will mess up somehow and it’s OK. Also, there’s no one in the audience waiting to blast you with tomatoes or commenting on Social media about how stupid you are (the latter may be true, but who cares). But seriously, what I discovered is that no one cared about my mistakes more than I did. And some of my mistakes were only noticed by me and my inner Imposter. So remember, just relax and keep it moving when you commit an error.

Anyone Can Do It
I used to believe that speaking is for experts only…WRONG! You don’t have to be among the most knowledgeable people on a particular subject in order to be a speaker.  No matter your skill level or degree of knowledge, you can give a talk. Speaking is simply an exercise in sharing what you know based on your experience. This also holds true for blogging or any other medium of knowledge transfer.  So to use a popular phrase, “Just Do It”.

Don’t Shortchange Yourself
Too often we minimize the value of our experience and knowledge, particularly if you’re in the beginning stages of learning a technology. Don’t short change yourself. There’s always someone out there who knows less than you and can benefit from what you’ve learned. Even people who are more informed than you can get some value from what you know, what you’ve done and how you did it.  Each person’s experience is unique. They way you went about configuring something or resolving a problem, can be very insightful.  Also, how you explain something is special to you and can be a factor in someone understanding a subject that they previously struggled to understand.

Here’s the bottom-line:  if I can do it, so can you. Don’t let the fear of failure or the belief that you aren’t good enough, prevent you from taking a step forward in a direction that can have a monumental impact on your career or personal life.  More importantly, you have the potential to make an impact on the lives or careers of others.

 

 

 

 

Standard

Azure Monitor Logs and Kusto Query Language (KQL)

The Azure platform consists of a variety of resources that generate large volumes of activity and diagnostic log data.  The source of this data can be subscription level events such as deallocating a virtual machine, deleting a resource group or creating a load balancer – essentially any create, update or delete operation on a resource.  It can also include resource level activities such as a VM Windows event logs, VM performance data, web app response times – logs related to resource utilitization. In this article, I will provide an overview of how Azure Monitor organizes all of this data, along with some examples of how Kusto Query Language (KQL) can be leveraged to parse this information.

AZURE MONITOR LOGS OVERVIEW
Azure Monitor Logs is responsible for collecting all log and telemetry data and organizing it in a structured format.  The data is stored in a Log Analytics Workspace, which organizes it into categorical units.  Within each unit or solution are tables that contain columns for various types of data.  The data types can be string, numerical or date/time.  The graphic below shows the Schema pane within Azure Monitor logs, which gives a hierarchical view of this structure.  By default, it is scoped to a Workspace that I created which has three categories called ChangeTracking, LogManagement and SecurityCenterfree, as you can see here:

By the way,  you have the ability to change the scope to either another Workspace, Resource group or subscription.  To do this, click here and the following window will appear:

But let’s go back to the originally scoped Workspace and see what it contains.  By expanding the LogManagement unit, you can see its tables.  Some of the tables are AuditLogs, AzureActivity, AppCenterError, etc.

As I mentioned, each table has columns for various types of data.  If we look inside the AppCenterError table above, there are icons next to each column that represents the type of data it contains.  For example, the next to the Createdat column means it has date/time data; a  next to the Errorline column is indicative of numerical data; and the indicates a column has textual data.

KUSTO QUERY LANGUAGE (KQL)
Now that we’ve gone over the Azure Monitor Logs data platform, let’s take a look some ways to analyze all of the data it holds using Kusto Query Language.  All queries performed by KQL have at least one table as its search base.  However, the scope of a query can expand across all tables, multiple tables or only one table – you have the right to choose.  Also, when executing queries in the Query editor, the default time frame is 24 hours, which can be customized. Here are some examples.

The following query searches for the keyword “compliant”  in the AzureDiagnostics table.  It retrieves all of the records in the AzureDiagnostics table and pipes it to the “search” operator.

 

Here’s another way to perform the previous query.  Instead of retrieving all records in the table and then filtering, the search operation is executed directly on the AzureDiagnostics table. Although, there are times when the former query is more suitable.

 

You can further narrow the scope of the previous query to a certain column in the AzureDiagnostics table.  This command takes all data from the AzureDiagnostics table and filters out any records in the DSCResourceStatus_s column with the keyword Compliant.

 

How about performing queries across multiple tables? You can perform a query across an array of tables with the following example:

If you want to reduce the result size of data set, use the Limit operator.  The Take operator can be used as well, as the two operators are synonymous. Also, they will retrieve a random subset of records.

 

By default, the maximum number of records that can be produced by the Kusto Data engine is 500,000 or or no more than 64 MB in size.  Therefore, if you specify a limit of say 550,000, the query will fail.  If you find this to be unacceptable, you can summon the Dark Side of the Force and suppress the default query limit with the notruncation option and retrieve any number of records (not recommended, however):

In conclusion, Azure Monitor logs is a valuable tool for performing data gathering and analysis.  The Cloud has elevated the importance of data and being able to parse it for insight into your environment.  With Azure Monitor logs comes the ability to consume log data from a variety resources and perform Kusto queries across a multitude of data sets.

 

 

 

 

 

 

 

 

 

 

 

 

 

Standard

Azure Tip: Create a Subscription Budget to Manage Costs

Are you Cloud computing on a budget?  This article explains how to manage your costs by configuring a budget for your Azure subscription and receive alerts for when you are nearing the budget limit.  This a great feature to leverage to avoid any costly surprises for your dev/test or pay-as-you-go account.  Let’s face it, there are many cool features to explore in Azure and it’s very easy to deploy a resource or service and forget about it. I’ve done that and quite possibly, so have you.

Here are the quick steps to setup a budget and associated alerts to avoid any financial pain.

In the Azure Portal go to the Cost Management + Billing menu blade.

Next select “Cost Management”

Then go to “Budgets” under Cost Management.

To create your budget, click on the “Add” button, which will bring up the following screen.

Compete the following fields and click Create:

  • Name – give your budget a name
  • Amount – the budget spending limit (e.g., $200)
  • Resets – the budget time frame (monthly, quarterly or annually)
  • Expiration date – when the budget ends
  • % of Budget – the budget limit percentage that will trigger an alert (e.g., 75% of limit amount)
  • Alert Recipients (email) – where to send a budget alert

Once your Budget has been created, you can view it under “Budgets

Finally, you can click on it to a view a summary of your Budget

In conclusion, it’s very easy to setup a budget for your Azure subscription as way to manage costs and receive alerts when your are close to exceeding your spending limit.  Budgeting is not the most glorious task but it’s necessary to avoid spending more than you can handle.

 

Standard

Configuration Management of Non-Azure VM’s with Azure Automation – Part 3

We’ve now reached the final article in this three part series covering Configuration Management in Azure automation.  In Part 1, I discussed the Inventory tool and how to onboard an AWS EC2 virtual machine to Azure.  Part 2 covered Change tracking and how to monitor changes to various resources on the AWS instance.  In this article, Part 3, I will cover Azure State configuration (DSC) and how to register an AWS VM as a DSC node to apply a desired state.

An Overview of State configurtion (DSC) Components
Azure State configuration builds upon Powershell’s Desired State Configuration (DSC), which was introduced in version 4.0 of Powershell.  These are the main components of Azure DSC, along with a brief description of each:

  • Configuration files – Powershell scripts written in a declarative syntax that defines how a target should be configured.  These contain building blocks that define the nodes to be configured and the resources to be applied.
  • Resources – These are Powershell modules that contains the code that determines the state of a machine.
  • Local Configuration Manager (LCM) – This runs on the target nodes.  It serves as the engine that consumes and applies DSC configurations to the target machine.  It also has settings that determine how often a node checks for new configs from the Pull server in Azure automation and what action to take when a node drifts from a desired state.
  • DSC Metaconfigurations – These files define the settings for the Local Configuration Manager (LCM).
  • Pull Server – located in Azure automation.  It functions as a repository for the configuration files that are applied to DSC target nodes.

Why Azure State configuration (DSC)
By leveraging DSC as a service in Azure, this eliminates the need to deploy and maintain an On-Premises Pull server, and all of it’s necessary components such as SSL certificates.  Also, State configuration (DSC) data can be forwarded to Azure Monitor, which provides searching capabilities with Log Analytics and alerting on compliance failures.  Furthermore, if you’re doing the DevOps thing by implementing agile practices, State configuration fits in seamlessly with continuous deployments in Azure DevOps.

Prerequisites
To complete the tasks in this example, the following will be needed:
1. An Azure automation account
2. An AWS EC2 VM
3. WMF 5.1 – on the AWS VM
4. WinRM – on the AWS VM

Steps to Onboard the AWS EC2 VM to Azure
Even though DSC is a part of Azure’s Configuration Management, any machine that it manages has to be onboarded separately from Inventory and Change Tracking.  There are a few ways to onboard an AWS instance to State configuration (DSC):

  1. The AWS DSC Toolkit created by the Powershell Team – this method includes installing the toolkit using PSGet, then running commands to login to your Azure subscription and register the AWS EC2 instance.  Go here to get more details.
  2. A Powershell DSC script – involves running a Powershell DSC configuration script locally on the AWS EC2 instance.  The script will generate a Metaconfig file that contains settings for the Local Configuration Manager (LCM). This Microsoft doc explains more.
  3. Azure Automation Cmdlets – a quick method to produce the DSC Metaconfig file using AzureRm cmdlets in Windows Powershell on the AWS VM.

Options 1 and 3 will allow you to register the node in Azure without having to write a Powershell DSC configuration script to generate the Metaconfiguration.  This is fine as long as you’re comfortable with the default LCM settings. In this case, I will use Option 3 since it’s simple and this is only for demonstration purposes.  Here are the steps to be executed on the AWS VM:

  1. In a Powershell console, install the AzureRm module from the Powershell Gallery using  install-module AzureRM
  2. Login to Azure using login-azurermaccount
  3. Download the Powershell DSC Metaconrigurations from Azure automation with the following Powershell command :
    $Params = @{
    ResourceGroupName = 'myautogroup'; 
    AutomationAccountName = 'myautoacct1'; 
    ComputerName = @('IP address or computer name'); #this will be the Private IP not Public 
    OutputFolder = "$env:UserProfile\Desktop\";
    }
    Get-AzureRmAutomationDscOnboardingMetaconfig @Params
    
  4. Run Set-DscLocalConfigurationManager -path “$env:UserProfile\Desktop\”

To verify that the AWS VM has been successfully onboarded, in the Azure portal, go to the Automation account.  Under Configuration Management, click on State configuration (DSC) to go to the main where it shows the EC2 instance and it’s configuration status.

The next step is to assign a configuration to the node.

Add and Assign State Configurations
Now that the AWS VM is registered as a DSC node in Azure, configuration files can be assigned to it.   This can be done by composing your own configuration files and uploading them to Azure; or using the ones available in the Gallery.

If you select “Configurations“, there’s an option to upload a configuration file or compose one in the Portal.  I will click “Add” and upload a previously created config file called “WindowsFeatureSet.ps1 that will ensure the IIS and Web server features are enabled.

Next browse to the location of configuration script file and hit “Ok” to import it.

I’m not going to walk through the steps of writing a DSC script, but only provide an overview of assigning it to a node.  Once the import is complete, the config file is now available under “Configurations“.

Before it’s assigned to a node, the uploaded file will then need to be compiled into a MOF document by the LCM on the Pull server.  To begin this process, select the imported config file and click on the “Compile” button.

Once complete, the file will show as compiled and will be in the format of filename.nodename.  At this point, the configuration can be assigned to the node.

To assign the configuration, select the DSC node from the Nodes screen and click on the “Assign node configuration” button.

Confirm the assignment by clicking “Ok

The following figure shows that the configuration file has been assigned to the DSC node.

Here I remoted into the AWS VM and verified that IIS has been installed.

Powershell commands to manage State configuration (DSC)
The following Powershell script and commands can be used to complete some of the above tasks that were done in the Portal.

Script to upload config file and compile it:

Import-AzureRmAutomationDscConfiguration
-ResourceGroupName myautogroup –AutomationAccountName myautoacct1
-SourcePath $env:userprofile\Desktop\AzureAutomationDsc\WindowsFeaturSet.ps1
-Published –Force

$jobData = Start-AzureRmAutomationDscCompilationJob
-ResourceGroupName myautogroup –AutomationAccountName myautoacct1
-ConfigurationName Featureset

$compilationJobId = $jobData.Id

Get-AzureRmAutomationDscCompilationJob
-ResourceGroupName myautogroup –AutomationAccountName myautoacct1
-Id $compilationJobId

Command to view DSC Nodes

Command to view the status of a DSC compilation job

Command to view the a DSC configuration

Troubleshooting – Notes from the Field
During the process of onboarding the AWS VM to Azure, I ran into a couple of errors when running the Set-DscLocalConfigurationManager command.

Problem 1 – Here’s a screenshot of the first error:

Fix 1 – I had to log into the AWS console and re-configure the Security group that manages traffic to the virtual machine.  The inbound traffic rule needed an allowance for WinRm-HTTPS (Powershell remoting) on Port 5986.

Problem 2 – when I ran the Set-DscLocalConfigurationManager command again after allowing Powershell remoting in the EC2 Security group, I got this error:

Fix 2 – I had to open the Local Group Policy editor on the AWS VM, enable trusted hosts for the WinRM client and add the source as a Trusted host.  Open gpedit.msc and go to Computer Configuration > Administrative Templates > Windows Components > Windows Remote Management (WinRM) > WinRM Client.  From there, enable Trusted Hosts and add the source server.

Summary
This concludes the third article in the series on Configuration management of a non-Azure VM.  Azure State configuration is a service  that can be used to manage Azure, On-premises and other non-Azure VMs to configure and maintain a desired state.

 

 

 

 

 

Standard

Configuration Management of Non-Azure VM’s with Azure Automation – Part 2

In part 1 of this series, I discussed the Inventory tool that is a part of Azure Automation’s config management and how to on-board an AWS VM for management.  In this article, I will cover Change Tracking.  With Inventory, you get a report on the Windows files, registry and services, as well installed software for the machines being monitored.  However, Change Tracking takes it a step further and provides a notification whenever there is a change to anything that’s being tracked on the machine.  It also provides the capability to perform queries against the change logs.  Let’s take a look and see how it works.

Since I previously enabled Inventory, there’s nothing that needs to be done to enable Change Tracking.  If you go to your Automation account in the Azure Portal, look under the Configuration Management section and click on “Change Tracking“.  On the main screen, there’s a graphical layout of changes made to Windows services, registry, files, Linux daemons and software being tracked on the AWS VM that was previously on-boarded.  In this case, there was 1 software change and a large number of changes to Windows services.

Below the graph, there’s a tabular layout indicating the resource name, resource type, source machine and time of change.

You can click on one of the resources to see more details about how it was modified.  If I select one of the Windows Update changes, more details are provided about exactly what changed.  Below it shows that the service state changed from “Running” to “Stopped”.

Configure Change Tracking
Change Tracking is customizable by allowing you to configure which Registry keys, services or files are tracked.  To configure Change Tracking, click on the “Edit Settings” button at the top of the main screen.  This will bring you to the Workspace Configuration page for Change Tracking.

Here you will see sections for each type of resource that is able to be tracked.  Under “Windows Registry” in the below screenshot, you will see a list of recommended keys to track.  They’re disabled by default.  To enable a key, just click on it and set enabled to “true”  You can also add a registry key by clicking on the “Add” button.

Under “Windows Files” you can view the files being tracked.  These have to be added manually.  For example, I have added the “c:\windows\system32\drivers\etc” folder path.  If a change is made to the Hosts file or any other file in this location, it will be recorded in Change Tracking.  Also, the process to enable tracking for Linux files is the same.

Another thing you can do under Workspace Configuration is enable the content of modified files to be saved in a Storage account.  To get started, click on the “File Content” tab. Enabling this feature will generate a Share Access Signature (SAS) Uri that can be used to access the stored data.  I won’t go into the details on how this works, but it’s very helpful if you would like to provide others access to the change data.

Lastly, let’s see what can be done under “Windows Services”.  The tracking for Windows services is enabled by default, but you can adjust the frequency that changes to them are reported.  The default setting is every 30 minutes, but you can adjust to anywhere between that and 10 seconds.  Here, I set the collection frequency to 10 minutes.

Querying Change Tracking Logs
Now I am going to discuss my favorite feature in Change Tracking – Logs search and query.  Click on “Log Analytics” to view the page to execute queries against the logged changes.

In the left side pane, there’s a schema that is built on the Workspace called “Trackchangesspace” that Log Analytics uses for Change tracking.  In here there are two databases:  ChangeTracking and LogManagement, which holds records of the data collected by the Tracking tool.  with the Change Tracking database expanded to show its tables ConfigurationChange and ConfigurationData.  In the main window is where queries are executed and the results are shown.  When you first open the Log Analytics page, all the changes collected in the last 24 hours are displayed by default in a tabular view.

By clicking on the “>” symbol next to an item, you can get more details.  For example, below are details about one of the software modifications.  It shows the SoftwareName, Computer, ChangeCategory, Previous and Current states.  In this case, an update for Windows Defender Antivirus was added.

What if you want to run a query of all Windows services that are stopped?  In the top pane, you can write a KQL query that searches in the ConfigurationChange table for all services whose SvcState is equal to stopped in the following format:
Configuration | where SvcState == “Stopped”.  I was pleasantly surprised to discover that the query editor has built-in tab completion and intellisense.  This is very helpful for lousy typists like me, or you are used to IDE’s like VScode or Powershell ISE.  Do note this: the query is case sensitive. If you type “Stopped” with a lower case “s”, the query will not yield any results.  If you are a Powersheller, your experience is that comparison values are not case-sensitive.

Here’s another time saver for building queries:  In the below figure, I went back to the results pane and expanded one of the Windows services to get the detailed information.  By placing the cursor next to “SvcState”, two buttons will appear:  a plus and minus.  Clicking on the plus button will add the search logic for all Windows services where the SvcState is stopped.  Then just click “Run” to get the results.  Nice!

Change Alerting
Lastly, Change Tracking provides the ability to configure alert rules if you would like to receive notifications on certain changes.  In the Logs window, click on “New alert rule” to set this up.  In the screen blow, set the resource as the Workspace used by Change tracking, add the appropriate logic for the condition and define how to receive notifications under Action groups.  You can have the alerts emailed or sent via text messages.

Supplemental resources
The following links are to articles that discuss topics that are related to material covered in this article:
1. https://docs.microsoft.com/en-us/azure/kusto/query/ – an overview of Kusto Query Language (KQL)
2. https://docs.microsoft.com/en-us/azure/automation/automation-change-tracking – an overview of Chang tracking
3. https://docs.microsoft.com/en-us/azure/automation/change-tracking-file-contents –  how to view contents of tracked files

This concludes Part 2 of 3 on Configuration management using Azure automation.  In summary, this article explained how to utilize Change tracking to monitor particular resources on an AWS virtual machine, how to perform queries against logged changes and setup alerts for those changes. The final article, part 3, will cover Azure State configuration (DSC).

 

 

 

 

 

 

 

 

 

 

Standard

Configuration Management of Non-Azure VM’s with Azure Automation – Part 1

One of the first questions asked whenever a system or application goes down is “what changed recently”.  Ill-planned or unplanned changes are often the underlying cause of failures.  And if you live on the Operations side of the IT fence like me, a large portion of your existence is dedicated to mitigating the negative impact of these age accelerating events.

To help in this ongoing battle, you can leverage Azure Configuration Management.  This feature is comprised of three tools:  Inventory, Change tracking and State configuration (DSC).  Inventory gives you a view of installed software, Windows services, registry keys, files and Linux Daemons that are present on your machine.  Change Tracking monitors and reports changes to these items.  And DSC allows you to compile configuration files to maintain a desired state for these monitored sources.  Configuration Management can be used for machines that are in Azure of course, but also an On-Premises data center and other cloud services.  For non-Azure machines, the data is gathered by an installed agent and reported back to Azure, where it is stored in a Log Analytics Workspace.

This article will be the first in a three part series that covers the components of Azure Configuration Management and how they can be used with a virtual machine running in AWS EC2. In part 1, I will show you how to get started using Inventory and provide an overview of it’s capabilities.  Part 2 will cover Change Tracking and Part 3 will outline State configuration (DSC).

Prerequisites
Here’s what is needed:

  • An Automation account in Azure
  • A Log Analytics Workspace
  • A non-Azure VM (AWS)

I’m not going to outline the steps of creating the above resources.  Here, I will just focus on the process of enabling and using Inventory.

How to Enable Inventory
In the Azure portal, go to your Automation account and select “Inventory” under Configuration Management.  On the popup screen that appears, select your subscription and the Log Analytics Workspace in the drop-down menus, and click on the enable button. This step will also enable Change Tracking.

The next step is to configure the virtual machine to communicate with Log Analytics.

Configure Non-Azure Machine
This Microsoft doc explains the following in more detail. For this task, I will be using an AWS t2.micro instance of Windows 2016.  The first thing that needs to be done is to add registry subkeys for TLS 1.2 and .Net Framework to use strong cryptography.  The Microsoft Monitoring Agent (MMA) that will be installed, uses TLS 1.2 to report changes to the Log Analytics service in Azure.  Here are the Registry Keys for TLS 1.2 and .NET 4.6 that need to be added:

  1. HKLM\System\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client – you will need to create the TLS 1.2 and Client subkeys.  And in the Client subkey, create the following DWORD value:
    • Enabled [Value = 1]
    • DisabledByDefault [Value = 0]
  2. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319 – no subkeys need to be created; create the following DWORD value: SchUseStrongCrypto with a value of 1.
  3. HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\.NETFramework\v4.0.30319 – no subkeys need to be created; created the DWORD value SchUseStrongCrypto with a value of 1.

Now that the Registry edits have been completed, the MMA can be installed.  There are three options for installing the agent: the installation wizard, command line or Powershell DSC.  In this case, I will use the command line.  Also, there are two agent options:  a 64-bit and 32-bit version.  Once the appropriate agent has been downloaded, open an elevated command prompt, navigate to the location of the file and run the following command to extract the MSI file to a folder path.  Finally, install the agent with the extracted MSI file.

Once the agent is installed you can verify the installation a couple of ways.  In Control Panel, go to MMA Agent to check that it’s intstalled as seen here.

Also, in the Azure Portal, you can run a search query to check that the agent has a heartbeat and is communicating with Log Analytics.  Go to the Logs tab on your Workspace to run the query. The following figure shows there’s a heartbeat detected from the AWS EC2 instance.

Now that the VM is communicating with Azure, we can complete the onboarding of the AWS VM.  Go back to Inventory and you will see a new screen with various options:  A button to Add Azure VMs; a doc on how to Add non-Azure machine; another button to Manage Machines.  Click on Manage Machines.

Now you can see the AWS EC2 instance as an available machine.  You have three options to choose on how to enable Change Tracking and Inventory.  I select the default to “Enable on all available machines and click enable.

Inventory
Now that the Inventory has been enabled, the agent has been installed and the machine is reporting to Azure, let’s see what we get out of the box.  First, if you go to the Inventory tab of your Automation account, it will show you the machine(s) reporting, along with details about the machine (O/S, version, platform, etc).  Additionally, there are tabs to view the software installed on the node, along with Windows services, Windows Registry keys and Linux daemons if applicable.  You will need to configure Files and Registry keys to monitor under Edit settings.  I will demonstrate that in the next article on Change Tracking.

To see the installed software, click on the “Software” tab.  It will show the software name, version, publisher, last refreshed time and number of machines it’s installed on.

Also, go to the “Windows Services” tab to see details such as the service name, status, startup type, last refreshed time and number of machines.

Lastly, the Inventory tools gives you the ability to add machines to a Machine Group.  A default group is created when a machine is added to Inventory.

A Machine group is a group of machines defined by a Log Analytics KQL query.  For instance, if you go to Machine groups and select the “MicrosoftDefaultComputerGroup“, you can see the query that was used to generate the group as well as member machines.

You can create your own Machine group as well.  Click on “Create a Machine Group” on the Inventory screen and fill in the fields on the popup window below.  Here, I’m creating a group of machines that contain “EC2” in the name.  Once you enter a query in the definition box, validate the query and then create it.  A more practical use case would be a group of machines that are missing updates.

This completes part 1 of Azure Configuration Management.  Here, I covered how to enable Inventory, install the monitoring agent on an AWS VM and use Inventory to view a machines configuration.  In Part 2 of this series I will cover Change Tracking.

 

 

Standard