AWS Command Line Interface (CLI) tool.

What is AWS CLI ?

The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

Why do you use AWS CLI ?

We have been using AWS services for a long time. We are communicating those services through API build on different languages. AWS also provides a command line tool to use those services.
Many of us know that and using it some extent but many of us might not know about this tool. It’s very excellent tool.
You don’t need to write a program to access those services. They provide huge command references; you will get them all from here.

Install and Configure AWS CLI

You can download the AWS Command Line Interface from here.
You can also installed using pip command pip install awscli

Once the instalation is done you need to configure it. You will get the details of configuration from here

Usefulness of AWS CLI with an example

Let me give an example how it very much useful. Suppose you are using an Employee DynamoDb table and you want to know the date and time of the last provisioned throughput increase or decrease for this table or the number of provisioned throughput decreases for the table during this UTC calendar day. How can you get that information? There are basically two ways to get it:

• Write a small program using AWS API to get the desired information.
• Login to the AWS console and go to the Metrics tab of that DynamoDB table and find out your desired data. Though sometimes it’s very hard to find some specific data from the console.

But you can get that information by using a simple command in your command prompt without writing a small application!!!!
aws dynamodb describe-table --table-name Employee

Monitoring DynamoDB using Amazon CloudWatch

You can monitor dynamo using CloudWatch service. If you wish to build up your own auto-scaling or only scale down application then you have to use this service to get different Metrics data. For example, you can capture ConsumedReadCapacityUnits, ConsumedWriteCapacityUnits, ReadThrottleEvents etc using Amazon CloudWatch service.

Here I'm going to show you how to get those data using boto3 python API. Coding structure is almost same for all other API. You can try with any other language.

    import boto3
    import datetime

    cloudwatch = boto3.client('cloudwatch')

    endTime = datetime.datetime.utcnow()
    startTime = endTime - datetime.timedelta(minutes=5)
    consumedWriteCapacityData = cloudwatch.get_metric_statistics(Period=60, 
                        StartTime = startTime, EndTime = endTime,
                        MetricName = 'ConsumedWriteCapacityUnits', Namespace = 'AWS/DynamoDB',
                        Statistics = ['Sum'], Dimensions = [{'Name': 'TableName', 'Value': 'Employee' }])
    for item in consumedWriteCapacityData['Datapoints']:
        print('Time: ', item['Timestamp'], '\t ', item['Sum'])
                

As I am using 60 secods period. So, you have to divide each data point value by 60 to get the actual ConsumedWriteCapacityUnits.

Similarly you can also get WriteThrottleEvents data.

    import boto3
    import datetime

    cloudwatch = boto3.client('cloudwatch')

    endTime = datetime.datetime.utcnow()
    startTime = endTime - datetime.timedelta(minutes=5)

    writeThrottleEventsData = cloudwatch.get_metric_statistics(Period=60, 
                        StartTime = startTime, EndTime = endTime,
                        MetricName = 'WriteThrottleEvents', Namespace = 'AWS/DynamoDB',
                        Statistics = ['Sum'], Dimensions = [{'Name': 'TableName', 'Value': 'Employee' }])
    for item in writeThrottleEventsData['Datapoints']:
        print('Time: ', item['Timestamp'], '\t ', item['Sum'])
                

References:

• http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/dynamo-metricscollected.html
• http://boto3.readthedocs.io/en/latest/reference/services/cloudwatch.html#CloudWatch.Client.get_metric_statistics

The AWS SQS message delete request get success but message does not get deleted from the queue.

Backgraund

We are using AWS SQS service to communicate between web application and background job processor for long running task and when the task completes, we delete the SQS message from the queue. We have a background job processor console application written in C# and a job scheduler which monitor the SQS message and when a new message receive, it start processing with the message. When the same message read by the job scheduler that time it skips the message because the message already in processing. When the message successfully processed, it deletes the message from the SQS queue.

Problem

Recently, we have found that some messages do not get deleted even though the delete request successfully executed. As a result same message is processing multiple times.

Reason of The Problem

If you receive a message more than once, each time you receive it, you get a different receipt handle string value with the message object. You must provide the most recently received receipt handle when you request to delete the message (otherwise, the message might not be deleted).
As some of our message processor taking much time to process the message, in the meantime our job scheduler read the message multiple times and skip the message. So when message processor completes its operation, it requests to delete the message with ReceiptHandle value of the message object. The delete request get success but message does not get deleted because it tried to delete the message with old ReceiptHandle value.

Solution

When the job scheduler read the same message then we capture the ReceiptHandle value only and associate it with the message and skip the current message.

Sample Code to Generate The Issue

Let's you have a FileProcessingQueue queue configured with Default Visibility Timeout to 1 minutes and there is a message in the queue. Now if you run the test method of SqsQueueRunner class then you will see message does not get deleted.

    public class SqsQueueRunner
    {
        private int count;
        private string firstReceiptHandle = string.Empty;

        public void Test()
        {
            var sqsClient =new AmazonSQSClient(RegionEndpoint.USEast1);
            ProcessMessage(sqsClient);
        }

        private void ProcessMessage(IAmazonSQS sqsClient)
        {
            while (true)
            {
                var request = new GetQueueUrlRequest {QueueName = "FileProcessingQueue"};

                var response = sqsClient.GetQueueUrl(request);

                var url = response.QueueUrl;

                var receiveRequest = new ReceiveMessageRequest
                {
                    QueueUrl = url, MaxNumberOfMessages = 1
                };

                var receivedResponse = sqsClient.ReceiveMessage(receiveRequest);
                if (!receivedResponse.Messages.Any())
                    continue;

                var message = receivedResponse.Messages[0];
                if (firstReceiptHandle == string.Empty)
                    firstReceiptHandle = message.ReceiptHandle;

                if (count < 5)
                {
                    Thread.Sleep(61*1000);
                    Console.WriteLine(count);
                    count++;
                    continue;
                }
                sqsClient.DeleteMessage(new DeleteMessageRequest(url, firstReceiptHandle));
                break;
            }
        }
    }
  

If you delete the message with the most recent ReceiptHandle value, the message will be deleted from the queue.

Abstract Factory Pattern

What is Abstract Factory Pattern?

Abstract factory pattern is a creational pattern and provide an interface for creating families of related or dependent objects without specifying their concrete classes.

Abstract Factory patterns work around a super-factory which creates other factories. This factory is also called as factory of factories.

The Abstract Factory defines a Factory Method per product.

When to use it?

1. Create a set of related objects, or dependent objects which must be used together.

2. System should be configured to work with multiple families of products.

3. The creation of objects should be independent from the utilizing system.

4. Concrete classes should be decoupled from clients.

The classes and objects participating in this pattern are:

• AbstractFactory  (IWaltonFactory)
  • This is an interface which is used to create abstract product.
• ConcreteFactory  (BangladeshFactory, BhutanFactory)
  • This is a class which implements the AbstractFactory interface to create concrete products.
• AbstractProduct  (IMobile, IMotorcycle)
  • This is an interface which declares a type of product.
• Product  (PrimoNx, PrimoZx, Fusion, Xplore)
  • This is a class which implements the AbstractProduct interface to create product.
• Client  (WaltonClient)
  • This is a class which use AbstractFactory and AbstractProduct interfaces to create a family of related objects.

Real world example:

Let's Walton company has a number of factories throughout the world.

They are building number of products like mobile, motorcycle, television, freez etc.

Client can create any product from any factory. Here I'm showing how we can create a product using Abstract Factory pattern.

  public enum MobileModelType
    {
        PrimoNx,
        PrimoZx
    }

    public enum MotorcycleModelType
    {
        Fusion,
        Xplore
    }

    public interface IWaltonFactory
    {
        IMobile GetMobile(MobileModelType modelType);

        IMotorcycle GetMotorcycle(MotorcycleModelType modelType);
    }
    
    public class BangladeshFactory : IWaltonFactory
    {
        public IMobile GetMobile(MobileModelType modelType)
        {
            IMobile mobile;
            switch (modelType)
            {
                case MobileModelType.PrimoNx:
                    mobile = new PrimoNx();
                    break;
                case MobileModelType.PrimoZx:
                    mobile = new PrimoZx();
                    break;
                default:
                    throw new ApplicationException("Unknown model type");
            }
            mobile.FactoryName = "Bangladesh";
            return mobile;
        }

        public IMotorcycle GetMotorcycle(MotorcycleModelType modelType)
        {
            IMotorcycle motorcycle;
            switch (modelType)
            {
                case MotorcycleModelType.Fusion:
                    motorcycle = new Fusion();
                    break;
                case MotorcycleModelType.Xplore:
                    motorcycle = new Xplore();
                    break;
                default:
                    throw new ApplicationException("Unknown model type");
            }
            motorcycle.FactoryName = "Bangladesh";
            return motorcycle;
        }
    }
    
    public class BhutanFactory : IWaltonFactory
    {
        public IMobile GetMobile(MobileModelType modelType)
        {
            IMobile mobile;
            switch (modelType)
            {
                case MobileModelType.PrimoNx:
                    mobile = new PrimoNx();
                    break;
                case MobileModelType.PrimoZx:
                    mobile = new PrimoZx();
                    break;
                default:
                    throw new ApplicationException("Unknown model type");
            }
            mobile.FactoryName = "Bhutan";
            return mobile;
        }

        public IMotorcycle GetMotorcycle(MotorcycleModelType modelType)
        {
            IMotorcycle motorcycle;
            switch (modelType)
            {
                case MotorcycleModelType.Fusion:
                    motorcycle = new Fusion();
                    break;
                case MotorcycleModelType.Xplore:
                    motorcycle = new Xplore();
                    break;
                default:
                    throw new ApplicationException("Unknown model type");
            }
            motorcycle.FactoryName = "Bhutan";
            return motorcycle;
        }
    }

   
    public interface IMobile
    {
        string FactoryName { get; set; }
        void Create();
    }

   public interface IMotorcycle
    {
        string FactoryName { get; set; }
        void Create();
    }

    public class PrimoNx : IMobile
    {
        public string FactoryName { get; set; }

        public void Create()
        {
            Console.WriteLine($"Primo Nx model mobile has been created from {this.FactoryName}.");
        }
    }

  public class PrimoZx : IMobile
    {
        public string FactoryName { get; set; }

        public void Create()
        {
            Console.WriteLine($"Primo Zx model mobile has been created from {this.FactoryName}.");
        }
    }

   public class Fusion : IMotorcycle
    {
        public string FactoryName { get; set; }

        public void Create()
        {
            Console.WriteLine($"Fusion model motorcycle  has been created from {this.FactoryName}.");
        }
    }

   
    public class Xplore : IMotorcycle
    {
        public string FactoryName { get; set; }

        public void Create()
        {
            Console.WriteLine($"Xplore model motorcycle  has been created from {this.FactoryName}.");
        }
    }

    public class WaltonClient
    {
        private readonly IWaltonFactory factory;

        public WaltonClient(IWaltonFactory factory)
        {
            this.factory = factory;
        }

        public void CreateMobile(MobileModelType modelType)
        {
            var mobile = this.factory.GetMobile(modelType);
            mobile.Create();
        }

        public void CreateMotorcycle(MotorcycleModelType modelType)
        {
            var motorcycle = this.factory.GetMotorcycle(modelType);
            motorcycle.Create();
        }
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            IWaltonFactory waltonBangladeshFactory = new BangladeshFactory();
            WaltonClient waltonClient = new WaltonClient(waltonBangladeshFactory);
            waltonClient.CreateMobile(MobileModelType.PrimoNx);
            waltonClient.CreateMotorcycle(MotorcycleModelType.Fusion);

            IWaltonFactory waltonBhutanFactory = new BhutanFactory();
             waltonClient = new WaltonClient(waltonBhutanFactory);
            waltonClient.CreateMobile(MobileModelType.PrimoZx);
            waltonClient.CreateMotorcycle(MotorcycleModelType.Xplore);

            Console.ReadKey();

        }
    }
  

Here is the output result:

*********************** Bangladesh Factory ********************************
Primo Nx model mobile has been created from Bangladesh.
Fusion model motorcycle  has been created from Bangladesh.



*********************** Bhutan Factory ********************************
Primo Zx model mobile has been created from Bhutan.
Xplore model motorcycle  has been created from Bhutan.

Note

1. Internally, Abstract Factory use Factory design pattern for creating objects. But it can also use Builder design pattern and prototype design pattern for creating objects. It completely depends upon your implementation for creating objects.

2. Abstract Factory can be used as an alternative to Facade to hide platform-specific classes.

3. When Abstract Factory, Builder, and Prototype define a factory for creating the objects, we should consider the following points :

   1. Abstract Factory use the factory for creating objects of several classes.

   2. Builder use the factory for creating a complex object by using simple objects and a step by step approach.

   3. Prototype use the factory for building a object by copying an existing object.

Getting Started with Angular 2 using TypeScript

In this article I'm trying to show how you can develop a start-up project in Angular 2.

What you will learn ?

After completing this post, you will be able to do the following things.

• You will be able to setup your environment to develop Angular js 2 with typescript application.
• You will be able to create an application in angular js 2
• Basic knowledge on Type Scripts and its transpiler.
• Basic knowledge Node Package Manager tool.
• Real-time application development. Changes automatically reflect the UI without refreshing the browser.

Create a new project folder:

Open the command prompt and run these two commands.

mkdir hello-world-angular-2
cd hello-world-angular-2

Node Js:

If you don't have node on your machine download from Node JS and install it on your machine.
It's pretty easy and straightforward to install.
With node we get a tool called npm or node package manager which we use for managing dependencies of our application.

TypeScript:

So once you install node, open up command prompt on windows and run the following command

npm install -g typescript

which will install the typescript globally in your computer.

Code Editor:

You need a code editor which support typescript, In this tutorial I'm using visual studio code editor which is beutiful, lightweight and cross-platform editor from Microsoft. It's free and open source awesome editor. You can also download it from here. You can use any other code editor you like.

Add the libraries what we need in our project:

Add a package.json file to the project folder and copy/paste the following:

package.json

 {
  "name": "hello-world-angular2",
  "version": "1.0.0",
  "scripts": {
    "start": "concurrent \"npm run tsc:w\" \"npm run lite\" ",    
    "tsc": "tsc",
    "tsc:w": "tsc -w",
    "lite": "lite-server",
    "typings": "typings",
    "postinstall": "typings install" 
  },  
  "license": "ISC",
  "dependencies": {
    "angular2": "2.0.0-beta.7",
    "systemjs": "0.19.22",
    "es6-promise": "^3.0.2",
    "es6-shim": "^0.33.3",
    "reflect-metadata": "0.1.2",
    "rxjs": "5.0.0-beta.2",
    "zone.js": "0.5.15"
  },
  "devDependencies": {
    "concurrently": "^2.0.0",
    "lite-server": "^2.1.0",
    "typescript": "^1.7.5"
  }
}
           

package.json is a standard node package configuration file.
Here we declared all dependencies of our application
You will get details about package.json from this link package.json

In this file we add a script line: "start": "concurrent \"npm run tsc:w\" \"npm run lite\" "
Here we run the two command concurrently.
"npm run tsc:w" => Run the typescript compiler in watch mode.
"npm run lite" => Run the light web server.

With typescript compiler running in the watch mode we can modify and save a type script file and then the typescript compiler will automatically generate a javascript and a javascript map file in the same folder.
The javascript file actually be loaded in browser.

Now run the following command from the command prompt:

npm install

npm will look at our package.json file and download all dependencies, once all dependencies are installed you will see a new folder name node_modules in your project folder.
Scary error messages in red may appear during install. Ignore them.

Add a tsconfig.json file to the project folder and copy/paste the following:

tsconfig.json

 {
  "compilerOptions": {
    "target": "es5",
    "module": "system",
    "moduleResolution": "node",
    "sourceMap": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "removeComments": false,
    "noImplicitAny": false
  },
  "exclude": [
    "node_modules",
    "typings/main",
    "typings/main.d.ts"
  ]
}
           

tsconfig.json is the configuration file for typescript compiler, So it uses this configuration file to determine how to compile or more accurately transpile typescript files into javascript files.
If you want to learn more details about this configuration file please go to ts configuration documentation files:
1. tsconfig-json
2. tsconfig in angualr

Add a typings.json file to the project folder and copy/paste the following:

typings.json

 {
  "ambientDependencies": {
    "es6-shim": "github:DefinitelyTyped/DefinitelyTyped/es6-shim/es6-shim.d.ts#6697d6f7dadbf5773cb40ecda35a76027e0783b2"
  }
}
           

typings.json another configuration file. when we use external javascript files in typescript we need to import what we called a typescript definition file this type definition file gives us static type checking and intelligence for that javascript library.
If you want to learn more details about this configuration file please go to ts configuration documentation files:
1. typings-json
2. typings-config in angualr

Create an application source sub-folder:

We like to keep our application code in a sub-folder off the root called app/. Execute the following command in the console window.
mkdir app
cd app

Add the component file:

Now add a file named app.component.ts and paste the following lines:

app/app.component.ts

import {Component} from 'angular2/core';

@Component({
    selector: 'my-app',
    template: '<h1>Hi, {{name}} <br /> Wellcome to the angular 2 application development world.'
})
export class AppComponent {
    name: string = "Emon Khan";
}
           

app.component.ts is the root component in our application.

Add the boot file:

Now add a file named boot.ts and paste the following lines:

app/boot.ts

import {bootstrap}    from 'angular2/platform/browser'
import {AppComponent} from './app.component'

bootstrap(AppComponent);
           

boot.ts is our main or starting module of our application.

Add the index.html file:

Now add a file named index.html and paste the following lines:

index.html

<html>
<head>
  <title>Hello World Angular 2</title>

  <!-- 1. Load libraries -->
  <!-- IE required polyfills, in this exact order -->
  <script src="node_modules/es6-shim/es6-shim.min.js"></script>
  <script src="node_modules/systemjs/dist/system-polyfills.js"></script>

  <script src="node_modules/angular2/bundles/angular2-polyfills.js"></script>
  <script src="node_modules/systemjs/dist/system.src.js"></script>
  <script src="node_modules/rxjs/bundles/Rx.js"></script>
  <script src="node_modules/angular2/bundles/angular2.dev.js"></script>

  <!-- 2. Configure SystemJS -->
  <script>
      System.config({
        packages: {        
          app: {
            format: 'register',
            defaultExtension: 'js'
          }
        }
      });
      System.import('app/boot')
            .then(null, console.error.bind(console));
    </script>

</head>

<!-- 3. Display the application -->

<body>
    <my-app></my-app>
</body>

</html>
           

Now your file stucture should be like this:

Run the application:

now run the following command from the project folder localtion:

npm start

Now our typescript compiler running in watch mode and our light web server started. This also launches default web browser and you will
"Hi, Emon Khan
Wellcome to the angular 2 application development world." text in the browser.

Now open the app.component.ts and make some changes in template and you will see the changes in the browser without loading the browser. That's cool!!!

Refactoring if else-if and switch statement code block

I will show you how we can refactor an if else-if or switch case statement code block in more batter way.

Though I'm using C# language here but you can use this approach in other programming languages too.

Before Refactoring

Let's we have a switch statement code block in our application which returns the Excel file name depending on ExportType enum.

         public string GetReportFileName(ExportType exportType)
          {
              string fileName;
              switch (exportType)
              {
                  case ExportType.MemberDataVersioning:
                      fileName = "MemberList.xls";
                      break;
                  case ExportType.MemberProfileInformation:
                      fileName = "MemberProfile.xls";
                      break;
                  case ExportType.MemberOtherDataVersioningDetails:
                      fileName = "MemberRecords.xls";
                      break;
                  case ExportType.ProviderDataVersioning:
                      fileName = "ProviderList.xls";
                      break;
                  case ExportType.ProviderProfileInformation:
                      fileName = "ProviderProfile.xls";
                      break;
                  case ExportType.ProviderOtherDataVersioningDetails:
                      fileName = "ProviderRecords.xls";
                      break;
                  case ExportType.ClaimsDataVersioning:
                      fileName = "ClaimList.xls";
                      break;
                  case ExportType.ClaimsDataChangeLog:
                      fileName = "ClaimDataChangeLog.xls";
                      break;
                  case ExportType.ElementWiseValidationReport:
                      fileName = "ElementDetails.xls";
                      break;
                  case ExportType.ReconciliationReport:
                      fileName = "RecociliationReport.xls";
                      break;

                  default:
                      fileName = exportType.ToString() + ".xls";
                      break;
              }

              return fileName;
          }
                    

When a new export type will come then we have to add another switch case statement and will continue this process for every new export type.

After Refactoring

There is nice refactoring way which we can apply to improve the code quality and simplicity.
We can use a dictionary instead of the switch statement. Let's we declare a dictionary of report file types in constructor:

        private readonly Dictionary<ExportType, string> reportFileNames;
    
        public ReportFileHelper()
          {
              this.reportFileNames = new Dictionary<ExportType, string>
                                    {
                                        { ExportType.MemberDataVersioning, "MemberList.xls" },
                                        { ExportType.MemberProfileInformation, "MemberProfile.xls" },
                                        { ExportType.MemberOtherDataVersioningDetails, "MemberRecords.xls" },
                                        { ExportType.ProviderDataVersioning, "ProviderList.xls" },
                                        { ExportType.ProviderProfileInformation, "ProviderProfile.xls" },
                                        { ExportType.ProviderOtherDataVersioningDetails, "ProviderRecords.xls" },
                                        { ExportType.ClaimsDataVersioning, "ClaimList.xls" },
                                        { ExportType.ClaimsDataChangeLog, "ClaimChangeLog.xls" },
                                        { ExportType.ElementWiseValidationReport, "ElementDetails.xls" },
                                        { ExportType.ReconciliationReport, "RecociliationReport.xls" }
                                    };
          }
                    

Now we can easily refactore the GetReportFileName method with this simple lines of code:

         public string GetReportFileName(ExportType exportType)
          {
            string fileName; 
            return reportFileNames.TryGetValue(exportType, out fileName) ? fileName : (exportType.ToString() + ".xls");
          }
                    

When a new report type comes, we will just add an item in reportFileNames dictionary.

Similarly, we can also use this refactoring approach in the if else-if statement.

Before Refactoring

Let's we have a function which return a customer object based on customer type.

Here is the code block:

    public ICustomer GetCustomer(CustomerType customerType)
        {
            ICustomer customer = null;
            if (customerType == CustomerType.Gold)
            {
                customer = new GoldCustomer();
            }
            else if (customerType == CustomerType.Platinum)
            {
                customer = new PlatinumCustomer();
            }
            else if (customerType == CustomerType.Silver)
            {
                customer = new SilverCustomer();
            }
            else if (customerType == CustomerType.Normal)
            {
                customer = new NormalCustomer();
            }

            return customer;
        }
                    

After Refactoring

We will declare a dictionary in the constructor. Here I have used Func of the customer.

Func is a very nice feature in c#. you can get details from here.

        private readonly Dictionary<CustomerType, Func<ICustomer>> customerTypes;

        public CustomerFactory()
        {
            this.customerTypes = new Dictionary<CustomerType, Func<ICustomer>>
                      {
                          { CustomerType.Gold, () => new GoldCustomer() },
                          { CustomerType.Platinum, () => new PlatinumCustomer() },
                          { CustomerType.Silver, () => new SilverCustomer() },
                          { CustomerType.Normal, () => new NormalCustomer() },
                      };
        }
                    

Now we can replace the GetCustomer method in this way.

    public ICustomer GetCustomer(CustomerType customerType)
        {
            Func<ICustomer> customer;
            return this.customerTypes.TryGetValue(customerType, out customer) ? customer() : null;
        }
                    

Any feedbacks are most welcomed. If you know any better approach, please share it here.