Skip to main content

Access Azure Batch with Azure's REST API using PowerShell and AZ CLI

To start off, what is a REST API? In short, a REST API is how you programmatically interact with an application via HTTP requests. You can do things like get data (GET API call), write data (POST API CALL), and even delete data (DELETE API call). Most of the libraries or modules you work with in PowerShell are API calls. Let's see how to set up a REST API call with PowerShell and AZ CLI. Every service in Azure can be called via REST. Today as an example, we'll use Batch accounts.

What is Batch? From the Microsoft documentation: "Azure Batch runs large-scale applications efficiently in the cloud. Schedule compute-intensive tasks and dynamically adjust resources for your solution without managing infrastructure." - https://docs.microsoft.com/en-us/azure/batch/

Let's get started.

Pre-requisites:

1. A Batch account. To create one:  https://docs.microsoft.com/en-us/azure/batch/batch-account-create-portal
2. Access to an Azure portal
3. VSCode
4. AZ CLI


The first way we'll call the Batch REST API is by creating a signed string to list the current jobs. A signed string is made up of your Batch account primary key, and the current date. The signed string is then encoded via UTF8, converted to Base64, and hashed with SHA256. This is a lot of information, so let's look at the code.

The first part is your parameters.

param(
$Key = "your_batch_account_primary_key",
$region = "your_batch_account_region",
$BatchAccount = "your_batch_account_name",
$BatchAccountURL = "Https://$BatchAccount.$region.batch.azure.com"

)

Your parameters consist of your batch account primary key (go to Batch account > Keys > Primary access key), the region that you Batch account sits in, the name of your Batch account, and the Batch account URL (go to Batch account > Keys > URL).


The second part is converting the key from Base64 and retrieving the current date in UTC.

$sharedKey = [System.Convert]::FromBase64String($Key)
$date = [System.DateTime]::UtcNow.ToString("R")


The third part is the signed string. This signed string can typically be found in the examples of the REST API documentation. The format is typically the same except it tells you what API you're calling and what version.

$stringToSign = "GET`n`n`n`n`n`n`n`n`n`n`n`nocp-date:$date`n/$BatchAccount/jobs`napi-version:2019-08-01.10.0"

The fourth part is encoding the signed string in UTF8, creating a new PSObject for SHA256, converting our key from Base64, and converting to Base64 but with SHA256 encryption.

[byte[]]$dataBytes = ([System.Text.Encoding]::UTF8).GetBytes($stringToSign)
$hmacsha256 = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha256.Key = [Convert]::FromBase64String($key)
$sig = [Convert]::ToBase64String($hmacsha256.ComputeHash($dataBytes))


 The fifth part is setting up a variable for our Authorization header and the header itself. A Header is simply telling the endpoint that you're trying to communicate with "hey, this is me and I have access. Please let me in". You will run across APIs that have different types of authorization. Common ones and Bearer, Basic, and SharedKey authorization. Bearer, for example, is the type of token you use when you authenticate via Azure Active Directory oAuth.

$authhdr = "SharedKey $BatchAccount`:$sig"
$headers = @{
    "ocp-date" = $date;
    "Authorization" = "$authhdr"
;
}

As you can see, the Batch API header requires Authorization and a date. The date is what we set up in the second part.

Finally, we're ready to make an API call.

Invoke-RestMethod -Headers $headers -Uri "$BatchAccountURL/jobs?api-version=2019-08-01.10.0"

What the above does is call your Headers that you created for authorization and the URI of the API. All URIs are typically different. In our case, we're using the jobs URI to list jobs.

The entire script together looks like the below.

param(
$Key = "your_batch_account_primary_key",
$region = "your_batch_account_region",
$BatchAccount = "your_batch_account_name",
$BatchAccountURL = "Https://$BatchAccount.$region.batch.azure.com"
)

$sharedKey = [System.Convert]::FromBase64String($Key)
$date = [System.DateTime]::UtcNow.ToString("R")

$stringToSign = "GET`n`n`n`n`n`n`n`n`n`n`n`nocp-date:$date`n/$BatchAccount/jobs`napi-version:2019-08-01.10.0"

[byte[]]$dataBytes = ([System.Text.Encoding]::UTF8).GetBytes($stringToSign)
$hmacsha256 = New-Object System.Security.Cryptography.HMACSHA256
$hmacsha256.Key = [Convert]::FromBase64String($key)
$sig = [Convert]::ToBase64String($hmacsha256.ComputeHash($dataBytes))

$authhdr = "SharedKey $BatchAccount`:$sig"
$headers = @{
    "ocp-date" = $date;
    "Authorization" = "$authhdr";
}

Invoke-restmethod -Headers $headers -Uri "$BatchAccountURL/jobs?api-version=2019-08-01.10.0"


Once we run it, we can see our listed jobs.



Now that we've seen how to access an API with a signed string, let's see how to do it with oAuth. For this, you'll need to be logged into the AZ PowerShell module by running Set-AzContext -Subscription your_sub_id

Once you've done the above, head over to my Github to generate an oAuth token based on your AZ CLI creds.

https://github.com/AdminTurnedDevOps/AZ-203-Code/blob/master/AzureAD/Get-oAuthToken.ps1

The above script can be ran to get an access token. After running the entire script, simply run Get-BearerToken for your token to be printed out on screen.

Once you have your Bearer token, we can move onto the next API call which is listing existing batch accounts. As you can see from the below, it's much more straight forward.

param(
$subscriptionID,
$oAuthToken
)

$headers = @{
    "Authorization" = "Bearer $oAuthToken"
}

Invoke-restmethod -Headers $headers -Uri "https://management.azure.com/subscriptions/$subscriptionID/providers/Microsoft.Batch/batchAccounts?api-version=2019-08-01"


Simply add in your subscription ID and your oAuth token in the params.


The third and final way we'll talk about today is using the az rest command. az rest was introduced in July of 2019. This allows a much simpler way to make a REST API call with your AZ creds.

Please Note: To use this method, you must have at least version 2.0.67 of AZ CLI

param(
    [string]$method = 'GET',
    [string]$subscriptionID = "",
    [string]$restCall = ""
)

az rest --method $method --uri $restCall


As you can see from the above, this is much simpler than the other two ways of creating REST API calls. You simply add in what method you want, your subscription ID, and what REST call you want to make. For example, my screenshots below show the REST URL for getting a list of Batch accounts (same as the second demonstration of doing a REST CALL).




There you have it folks! I admit, some of these ways of making REST API calls is cumbersome and a bit confusing. However, once you understand it, your DevOps tool belt will be much more filled. While practicing DevOps, a lot of our job is to automate. Automation becomes much easier once you understand REST API calls.

All code for this can be found at: https://github.com/AdminTurnedDevOps/AZ-203-Code/tree/master/Batch

Thanks for reading!

Comments

Popular posts from this blog

DevOps tooling in the Microsoft realm

When I really started to dive into automation and practicing DevOps with specific tooling, there were a few key players. At the time Microsoft was not one of them. They were just starting to embrace the open source world, including the art and practice of DevOps. Since then Microsoft has went all in and the tech giant has made some incredible tooling. Recently I switched to a Microsoft-heavy environment and I love it. I went from AWS/Python/Ansible/Jenkins to Azure/PowerShell/ARM/Azure DevOps. My first programming language was PowerShell so being back in the saddle allowed me to do a full circle between all of the different types of tooling in both worlds. Today I want to share some of that tooling with you.

The first thing I want to talk about is ARM. What is ARM? ARM is a configuration management tool that allows you to perform software-defined-infrastructure. Much like Ansible and Terraform, ARM allows you to define what you want your environment to look like at scale. With ARM, yo…

Monitoring your containers in an AKS cluster with Prometheus

Monitoring and alerting is arguably one of the most important thing in Cloud Engineering and DevOps. It's the difference between your clients stack being up and a client being down. Most of us have SLA's to abide by (for good reason). Today we're going to learn how to spin up Prometheus in an AKS cluster to monitor our applications.

Pre-reqs;
1. Intermediate knowledge of Kubernetes
2. An AKS cluster spun up in Azure

Recently AKS supports Prometheus via Helm, so we'll use that for an automated solution to spin this up. This installs kube-prometheus, which is a containerized version of the application. With raw Prometheus, there are a few things that are needed for the operator;

1. Prometheus: Defines a desired deployment.
2. ServiceMonitor: Specifies how groups of services should be monitored
3. Alertmanager: Defines the operator to ensure services and deployments are running by matching the resource

With kube-prometheus, it is all packaged for you. This means configuri…

Run PowerShell code with Ansible on a Windows Host

Ansible is one of the Configuration Manager kings in the game. With it's easy-to-understand syntax and even easier to use modules, Ansible is certainly a go-to when you're picking what Configuration Management you want to use for your organization. Your question may be "but Ansible is typically on Linux and what happens when I'm in a Windows environment?". Luckily I'm here to tell you that Ansible will still work! I was pleasantly surprised with how easy it is to use Ansible on Windows with a little WinRM magic. Let's get started.

Pre-requisites for this post:
1) WinRM set up to connect to your Windows host from Ansible
2) Ansible set up for Windows Remote Management
3) SSH access to the Ansible host
4) Proper firewall rules to allow WinRM (port 5985) access from your Ansible host to your Windows host
5) Hosts file set up in Ansible that has your IP or hostname of your Windows Server.
6) At least one Linux host running Ansible and one Windows Server host …