Author: Nithin Mohan TK

Azure Cosmos DB – 429 Too Many Requests

October 6, 2018 .NET, Azure, CosmosDB, Document DB, Microsoft, Performance, Reliability, Resilliancy, Scalability, Visual Studio 2017, VisualStudio, VS2017 No comments

Recently while I was doing Performance Testing in one of the APIs interacting with Cosmos DB, I encountered a problem as Azure Cosmos DB API’s started returning Http Code 429.  Http Status Code 429 indicates that too many request been received or request rate is very large. This error would happen when we have concurrent users trying to write or read from same cosmos db collection.

Following diagram covers the architecture of the performance test I am performing:

image

Based on analysis it found out to be the Throttling happening from Azure Cosmos DB, as we make requests that may use more than provisioned Request Units(RU) per second. We were using default Cosmos DB configuration for a fixed collection of 1000 RU’s per second which is sufficient enough for a 500 reads and 100 writes for a 1 kb file. You can refer more about Request Units from Azure Docs.

image

 

 

 

Solution(s):

1. Now first logical step we can do is to get rid off this error by increasing the Throughput for the collection.  I am going to increase to 10000 RU/s maximum allocatable for a Storage Capacity: Fixed.   This should ideally improve the Throughput for 250 or more virtual users hitting.

image

2. Second logical step is to improve the code: Improve the connection parameters in the Document DB SDK –> DocumentDbClient. For this I referred to the Microsoft Docs: Performance tips for Azure Cosmos DB and .NET

Providing optimum values to the following Properties in RetryOption class   to be passed as parameter to Connection Policy.

image

 

In my case I provided a value of 30 to give ultimate results:

new RetryOptions() { MaxRetryAttemptsOnThrottledRequests = 30, MaxRetryWaitTimeInSeconds = 30  }

That should resolve most of the 429 issues when dealing with Cosmos DB SDK

Azure Database for MariaDB: Public Preview

October 4, 2018 Azure Database for MariaDB, Managed Services, MariaDB, OpenSource No comments

During Ignite 2018, Microsoft has announced the availability of Maria DB support in Azure Database services. Today it has been opened for Public Preview for all Azure customers.

mariadbhero

What is MariaDB?

MariaDB is a community-developed fork of the MySQL relational database management system intended to remain free under the GNU GPL.Development is led by some of the original developers of MySQL, who forked it due to concerns over its acquisition by Oracle Corporation.Wikipedia

Azure Database for MariaDB: Public Preview Availability

The Azure Database for MariaDB service is now available in preview. It offers an enterprise-ready, fully managed database service that based on the Community Edition of MariaDB.

The service features open-source compatibility, built-in high availability, dynamic scaling, and flexible pricing. Customers can lift and shift to the cloud and use languages and frameworks of their choice, leveraging the power of MariaDB running on Azure.

To learn more about the service, view the service page, pricing, and documentation.

You can create a MariaDB server by using the Azure portal or Azure CLI.

More References:

Azure Cosmos DB–Setting Up New Database using Azure CLI–Sample

October 1, 2018 Azure, Azure Cosmos DB, Codes, CosmosDB, KnowledgeBase, Microsoft, PowerShell, Windows Azure Development No comments

Purpose of this article is to help you with few steps of commands to provision a new Azure Cosmos DB database instance through Azure CLI or Azure Cloud Shell.

Here is the snippet:

 <# 
   This Bash script should help you create a Azure Cosmos DB instance using Azure CLI with bare minimal configuration 
#>

export ACCOUNT_NAME="thingx-retail-store-db"
export DB_RESOURCE_GROUP="thingx-dev"
export DB_LOCATION="southcentralus"
export DB_NAME="Products" 
export DB_THROUGHPUT=1000  ## bare minimal for 500 read and 100 write configuration for a 1KB document.
export DB_COLLECTION_NAME="Groceries"

##Optional: If resource group does not exist create a new one 
az group create --name $DB_RESOURCE_GROUP --location $DB_LOCATION

##1.0 Create the Azure Cosmos DB Account 
az cosmosdb create --name $ACCOUNT_NAME --kind GlobalDocumentDB --resource-group $DB_RESOURCE_GROUP

##2.0 Create Products database in the account  
az cosmosdb database create --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP

##3.0 Create Groceries collection in Products database
az cosmosdb collection create --collection-name $DB_COLLECTION_NAME --partition-key-path "/productId" --throughput $DB_THROUGHPUT --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP


 

For the ease of this article I used Azure Cloud Shell, that you can launch from your azure portal by clicking on the Shell icon on the top portal menu.

image

image

Now you are ready to execute all the commands listed above in the sample bash script

image

Create New Azure Cosmos DB Account

 <span data-mce-type="bookmark" style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" class="mce_SELRES_start"></span>
az cosmosdb create --name $ACCOUNT_NAME --kind GlobalDocumentDB --resource-group $DB_RESOURCE_GROUP

image

Create Products Database in the account

az cosmosdb database create --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP

image

Create Groceries collection in Products database

az cosmosdb collection create --collection-name $DB_COLLECTION_NAME --partition-key-path "/productId" --throughput $DB_THROUGHPUT --name $ACCOUNT_NAME --db-name $DB_NAME --resource-group $DB_RESOURCE_GROUP

image

Now if you browse Azure Portal you can see resources created in “thingx-dev” resource group.

image

Upon browsing with Data Explorer you can see the Groceries collection inside Products DB.

image

So that’s the few silly easy steps to create Cosmos DB database from Azure CLI or Azure Cloud Shell. Hope that makes it easy for you.

Getting Started with Azure CLI 2.0

September 30, 2018 Azure, Azure CLI, Azure Cloud Shell, AzureRM.PowerShell, PowerShell No comments

Older days we used to manage azure resources through AzureRM PowerShell modules . This was very much flexible for any Azure Administrator or Developers to run Automated Deployments to Azure Resource Manager resources.

Azure CLI  is the next improved version with simplified cmdlets to make life easier and it is cross-platform.

You can use Azure CLI in two ways:

  1. Azure Portal – Through Azure Cloud Shell
  2. PowerShell module

Installation Steps:

  • Download Azure CLI designed for Linux/Windows/MacOS based on your OS.
  • Install image and follow the steps.

 

image

image

  • Verify the Installation by executing cmdlet [  az –version  ]
az –-version

image

Running the Azure CLI from PowerShell has some advantages over running the Azure CLI from the Windows command prompt, provides additional tab completion features.

Now let us try logging in to Azure using Azure CLI. There are various ways of logging in, for this article I would try simple web login using az login command.

Execute the following cmdlet to login to Azure:

az login

The Azure CLI will  launch your default browser to open the Azure sign-in page. After a successful sign in, you’ll be connected to your Azure subscription.  If it fails, follow the command-line instructions and enter an authorization code at https://aka.ms/devicelogin.

Create a azure resource group and verify:

az group create –name "thingx-dev" –location "southcentralus" 
az group list --output table

 

Hope that is helpful for you to get started with Azure CLI. To learn more about Azure CLI cmdlets : https://github.com/Azure/azure-cli

NDepend–VSTS/Azure DevOps Integration–Part 01

September 30, 2018 .NET, .NET Core, .NET Framework, Azure DevOps, Best Practices, Code Analysis, Code Quality, Dynamic Analysis, Emerging Technologies, Microsoft, Static Analysis, Tools No comments

In my previous article I wrote an introductory about NDepend and how it will be useful for Agile Team to ensure code quality.

In that article we found how we can use NDepend in a developer machine. Now with this article we will familiarize ourselves in using NDepend in your build automation pipeline in your VSTS/Azure DevOps Build Agent.

There are two types of integration possible for NDepend:

  1. Directly using NDepend Package Extension from VSTS Marketplace
  2. Manual Integration using NDepend Command Line Tool. (This would provide you more control over licensing by setting up the license in your own OnPrem VSTS Build Agent.

For the interest of this article I will cover the use of VSTS Package Extension and using NDepend Build Task in VSTS Build Pipeline.

Installation of NDepend Extension for VSTS/Azure DevOps :

1.) Got to Azure DevOps Market Place:  https://marketplace.visualstudio.com/items?itemName=ndepend.ndependextension

image

2.) Click on Get to Install this extension in to your AzureDevOps account and follow the steps. For the demo purpose I am starting with 30 day free trial, otherwise you can go ahead and buy the full license.

image

image

image

3.) Now when you get back to Azure DevOps project, you can see the NDepend side menu enabled, this is where you would see the report summary of your project.

image

Integration NDepend into Azure DevOps Pipeline :

1.) Select “NDepend Task” and add in to Pipeline

image

image

Note:

  • You can choose to stop the build when at least one quality gate fails.
  • You also need to specify the NDepend project file customized for your project, otherwise NDepend will use their default project file configuration.  Having your own NDepend project file will provide you more control over the policies for the scan.

Queue a new Build and wait for Build to complete. Now you can see the BuildArtifacts includes all NDepend report file.

image

Now you go back to NDepend menu from Left side menu item in Summary Tab. This will provide you detailed view of Technical Debt in your project.

image

image

image

image

image

In the next article I will cover the manual integration steps.

New Microsoft Azure Certifications

September 16, 2018 Azure, Azure SDK, Azure Tools, Certification, Emerging Technologies, MCP, Microsoft, Microsoft Learning, Windows Azure Development No comments

Microsoft has recently announced new certification exam tracks for Azure Administrators, Developers and Architects. Here are the line ups that should help you move your career with right certifications. 

The three new Microsoft Azure Certifications are:

  • Microsoft Certified Azure Developer
  • Microsoft Certified Azure Administrator
  • Microsoft Certified Azure Architect

These certifications would essentially split the previous MCSA/MCSE: Cloud Platform and Infrastructure track and introduce new exams for individual certification track. 

So far I only have limited information available about all the exam numbers for each individual track, as recently Microsoft has made BETA exams available for Microsoft Certified Azure Administrator track. 

These exams are still in BETA, would commence general availability in coming months.  Will keep you posted about newer exams for other tracks as we get to know more. 

References: https://www.microsoft.com/en-us/learning/exam-list.aspx