Azure Storage Deep Dive
Azure Storage Deep Dive
Azure Storage Deep Dive
DEEP DIVE
The Azure Foundation
Sergio Navarro Pino @snavarropino
Sergio León González @panicoenlaxbox
http://panicoenlaxbox.blogspot.com.es/
http://www.serginet.com
What is Azure Storage?
An storage Microsoft managed service that is highly available, secure, durable, scalable,
and redundant.
Services:
• Disk: Storage for your VM’s
• Files: Simple, distributed, cross-platform file system
• Blob: Massively-scalable object storage for unstructured data
• Queue: Durable queues for large-volume cloud services
• Table? Flexible Key-Value NoSQL database
Azure Storage Highlights
Security:
● The storage account can be secured using Role-Based Access Control and Azure Active
Directory.
● Transport level encryption (HTTPS) may be forced
● Client side encyption: https://blogs.msdn.microsoft.com/windowsazurestorage/2015/04/28/client-side-
encryption-for-microsoft-azure-storage-preview/
● Delegated access to the data objects in Azure Storage can be granted using Shared Access
Signatures.
● The authentication method used by someone when they access storage can be tracked using
Storage analytics
● Files and disk can be encrypted, in a transparent way
https://azure.microsoft.com/en-us/support/legal/sla/storage/v1_0/
When your application reaches the upper limits, Azure Storage begins to return error code 503
(Server Busy) or error code 500 (Operation Timeout) responses. If these errors are occurring, then
your application should use an exponential backoff policy for retries that may allow the load to
decrease.
If the needs of your application exceed the scalability targets of a single storage account, you can
build your application to use multiple storage accounts.
Azure Storage limits
https://docs.microsoft.com/en-us/azure/storage/common/storage-scalability-targets
Geographies & Regions
Azure regions are organized into geographies.
An Azure geography ensures that data residency, sovereignty, compliance,
and resiliency requirements are honored within geographical boundaries.
Regions & Availability zones
A region is a set of datacenters deployed
within a latency-defined perimeter and
connected through a dedicated regional
low-latency network.
Hot / cold must be set on account creation, in order to set default blob tier
In any case we can set the desired tier for each blob
Archive (the coolest one) can only be set at blob level.
- Expensive storage
- Cheap transactions
Account type Performance Tier can be set?
• However in order to read or modify the blob must be rehydrated (ask for a
change to tier hot or cold)
• Long-term data retention
Reference: https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-storage-tiers
Azure Storage Replication
The data in your Microsoft Azure storage account is always replicated to ensure durability and
high availability. Replication copies your data so that it is protected from transient hardware
failures, preserving your application up-time.
• Region
• Amount of stored data (Gb’s)
• Acces tier in caso of blob (hot/cold/archive)
• Performance tier (standard/premium) in case of page blobs
• Replication option
• Transactions (operations)
• Data transfer
https://azure.microsoft.com/en-us/pricing/details/storage/
https://azureprice.net/
Demo, crear cuenta geo-replication
Tooling
• PowerShell
• CLI
• https://azure.microsoft.com/es-es/downloads/
• Storage Explorer
• https://azure.microsoft.com/en-us/features/storage-explorer/
• Storage Emulator
• https://go.microsoft.com/fwlink/?linkid=717179&clcid=0x409
• Storage Tools
• https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy
• REST API
• https://docs.microsoft.com/en-us/rest/api/
• Visual Studio
• Cloud Explorer
• Server Explorer
• Templates
PowerShell
Get-Module PowerShellGet -ListAvailable
Install-Module -Name AzureRM -AllowClobber
Get-Command -Module AzureRM.*
PowerShell
https://www.microsoft.com/web/downloads/platform.aspx
PowerShell
Login-AzureRmAccount
New-AzureRmResourceGroup -Location "West Europe" -Name
<resource_group_name>
New-AzureRmStorageAccount -Location "West Europe" -Name
<storage_account_name> -ResourceGroupName <resource_group_name> -SkuName
Standard_RAGRS -Kind Storage
Get-AzureRMStorageAccount
Remove-AzureRmResourceGroup -Name <storage_account_name> -Force
Remove-AzureRmAccount
PowerShell non-interactive login
Login-AzureRmAccount
# en otra ventana
# 98486e3b-1508-4f20-b0ef-34382c4c32b0@sergioleonanalyticalways.onmicrosoft.com
# ApplicationId@<your_tenant_name>
Get-AzureRMStorageAccount
Remove-AzureRmAccount
CLI
1.0 2.0
ASM ARM
Node.js Python
azure az
CLI - Docker
https://docs.microsoft.com/en-us/cli/azure/run-azure-cli-docker?view=azure-cli-latest
az
az login
CLI non-interactive login
http(s)://<storage account>.file.core.windows.net/share
Create a file share
Azure portal
Or
CLI for example:
current_env_conn_string = $(az storage account show-connection-string -n <storage-
account> -g <resource-group> --query 'connectionString' -o tsv)
\\<storage account>.file.core.windows.net\share
Or
Navigate to share in azure portal, click on connect
and follow instructions
Some disadvantages
Three types of blobs, block blobs, append blobs, and page blobs.
You specify the blob type when you create the blob.
https://myaccount.blob.core.windows.net
https://myaccount.blob.core.windows.net/mycontainer
https://myaccount.blob.core.windows.net/mycontainer/myblob
https://myaccount.blob.core.windows.net/myblob
https://myaccount.blob.core.windows.net/$root/myblob
Container ACL
http(s)://<storage account>.queue.core.windows.net/<queue>
● Queue
● Get
● Delete
● Peek
● Update
● Storage queues provide support for updating message content. You can use this
functionality for persisting state information and incremental progress updates into the
message so that it can be processed from the last known checkpoint, instead of starting
from scratch. In services bus this is more complicated
● Clients have the ability to only peek at the messages from the queue, without removing
or locking them.
● Logging capabilities: Users have the ability to activate the loggings mechanism and
track all the actions that are happening on the queue. Tracking information like client IP
are tracked and stored as an out of the box solution.
General purpose Premium Blob, just supporting Page Blobs in private containers (For old VM disks)
Blob storage Only standard is allowed Blob, just supporting Block and Append blobs
General purpose v2 Premium Blob, just supporting Page Blobs in private containers (For old VM disks)
Recommendation: Use “General purpose v2” but taking in account that there are pricing differences: v2
has lower storage prices but higher transaction prices