Monday, 19 September 2016

Directive to Parse and Format Number from and to Currency

This post will show an example of an AngularJS directive that will automatically format a plain number inserted into an input field into a currency format. As the number is typed into the input field, the value will be formatted into its local currency representation. The directive also converts a number value from model to its currency format in the view.
$formatters, $parsers and $filter are used as part of the codes.
myApp.directive('formatCurrency', ['$filter', '$locale', function ($filter, $locale) {
    return {
        require: '?ngModel',
        link: function (scope, elem, attrs, ctrl) {
            if (!ctrl) return;

            // $formatters is used to process value from code to view
            ctrl.$formatters.unshift(function (modelValue) {
                var formattedValue;
                if (modelValue) {
                    formattedValue = $filter('currency', null, 2)(modelValue);  // use $filter to do some formatting
                } else {
                    formattedValue = '';
                }
                return formattedValue;
            });

            // $parsers is used to process value from view to code
            ctrl.$parsers.unshift(function (viewValue) {
                var plainNumber;
                var formattedValue;
                
                var decimalSeparatorIndex = viewValue.lastIndexOf($locale.NUMBER_FORMATS.DECIMAL_SEP);  // $locale.NUMBER_FORMATS.DECIMAL_SEP variable is the decimal separator for the current culture
                if (decimalSeparatorIndex > 0) {
                    // if input has decimal part
                    var wholeNumberPart = viewValue.substring(0, decimalSeparatorIndex);
                    var decimalPart = viewValue.substr(decimalSeparatorIndex + 1, 2);
                    plainNumber = parseFloat(wholeNumberPart.replace(/[^\d]/g, '') + '.' + decimalPart).toFixed(2); // remove any non number characters and round to two decimal places

                    formattedValue = $filter('currency', null, 2)(plainNumber);
                    formattedValue = formattedValue.substring(0, formattedValue.lastIndexOf($locale.NUMBER_FORMATS.DECIMAL_SEP) + 1);
                    formattedValue = formattedValue + decimalPart;
                } else {
                    // input does not have decimal part
                    plainNumber = parseFloat(viewValue.replace(/[^\d]/g, ''));
                    formattedValue = $filter('currency', null, 0)(plainNumber);     // the 0 argument for no decimal does not work (issue with Angular)

                    if (formattedValue) {
                        // remove the decimal part
                        formattedValue = formattedValue.substring(0, formattedValue.lastIndexOf($locale.NUMBER_FORMATS.DECIMAL_SEP));
                    } else {
                        formattedValue = viewValue;
                    }
                }

                elem.val(formattedValue);
                return plainNumber;
            });
        }
    };
}]);

To use it on an input field:
<input type="text" ng-model="vm.myVariable" format-currency/>

For a working example, please see this on Plunker.

Friday, 2 September 2016

Passing Controller Function to Directive

Below is an example of how to call a controller function from inside an AngularJS directive:
myApp.directive('myDirective', function () {
    return {
        scope: { controllerFunction: '&callbackFunction' },
        link: function (scope, element, attrs) {
            scope.controllerFunction({ arg: '123' });
        },
    }
});
And the markup:
<div ng-controller="MyCtrl as vm">
      <span my-directive callback-function="vm.theControllerFunction(arg)" ></span>
</div>

We can also call the controller function whenever a value in the directive changes. An example of such directive:
myApp.directive('observeValueChange', function () {
    return {
        scope: { controllerFunction: '&callbackFunction' },
        link: function (scope, element, attrs, ngModel) {
            attrs.$observe('theValue', function (newValue) {
                scope.controllerFunction({ arg: newValue });
            })
        },
    }
});
The markup:
<input type="text" ng-model="vm.myVariable" />
      <span observe-value-change the-value="{{vm.myVariable}}" callback-function="vm.theControllerFunction(arg)" ></span>

See the working example in Plunker.

Thursday, 11 August 2016

Some Notes about SQL Index Fragmentation

This post is about a recent reading/research that I have made regarding of how to identify ‘bad’ indexes that have been defragmented much and how to fix those to be optimal again. There are many references put on this post such as useful scripts and articles for further reading.

Identifying Defragmented Indexes
Firstly, we would need to find the indexes that have much fragmentation. Below is a useful script from Microsoft Script Center site. This script shows average fragmentation for each index in all tables and indexed views.
SELECT OBJECT_NAME(ind.OBJECT_ID) AS TableName,
ind.name AS IndexName, indexstats.index_type_desc AS IndexType,
indexstats.avg_fragmentation_in_percent
FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, NULL) indexstats
INNER JOIN sys.indexes ind 
ON ind.object_id = indexstats.object_id
AND ind.index_id = indexstats.index_id
WHERE indexstats.avg_fragmentation_in_percent > 0--You can specify the percent as you want
ORDER BY indexstats.avg_fragmentation_in_percent DESC

What to Do with the Result?
From the result we can either choose to ignore, reorganise or rebuild each index.

Reorganising an index is to reorder and clean up the index with pre-existing settings. While rebuilding an index is to recreate the index from scratch. When rebuilding an index, new settings can be set as the old index will be deleted. Rebuilding an index is usually more effective than reorganising an index. However rebuilding an index will cost more.

Reorganising an index is always done online while rebuilding is offline (except if using SQL Server Enterprise edition). Stopping a rebuilding operation will make the operation to be rolled back while stopping reorganising operation will just stop the process and leave the done parts.

According to Microsoft guideline, if an index has
- less than 5 % fragmentation -> ignore
- between 5% to 30% fragmentation -> reorganise
- greater than 30 % fragmentation -> rebuild


How to Reorganise / Rebuild Index?
An index can be reorganised or rebuilt with Alter Index command. For example:
ALTER INDEX IX_MyTable_IndexName ON MyTable REORGANIZE;   
ALTER INDEX IX_MyTable_IndexName ON MyTable REBUILD;
To see all options for the command, see this MSDN documentation.


Reorganise / Rebuild all Indexes in the Database
To do this, we can use Maintenance Plan Wizard provided by SQL Server or script.

To create a maintenance plan:
1. expand the Management folder in the target database server
2. right click Maintenance Plans folder and select Maintenance Plan Wizard

For more details about using Maintenance Plan Wizard to rebuild indexes, please see this article 'Rebuilding Indexes using the SSMS Database Maintenance Wizard'.

Otherwise we can use script to reorganise/rebuild indexes. An example of simple script to rebuild all indexes of tables and indexed views in a database (source is http://www.sqlservercentral.com/blogs/juggling_with_sql/2011/06/20/rebuild-all-the-indexes-of-a-sql-database-in-one-go/):
DECLARE @tsql NVARCHAR(MAX) 
DECLARE @fillfactor INT

SET @fillfactor = 90

SELECT @tsql =
STUFF(( SELECT DISTINCT
';' + 'ALTER INDEX ALL ON ' + o.name + ' REBUILD WITH (FILLFACTOR = ' + CONVERT(VARCHAR(3),@fillfactor) + ')'
FROM
sysobjects o
INNER JOIN sysindexes i
ON o.id = i.id
WHERE
o.xtype IN ('U','V')
AND i.name IS NOT NULL
FOR XML PATH('')), 1,1,'')

--PRINT @tsql         
EXEC sp_executesql @tsql

Or we can use more sophisticated script that has been used and proven by many people like Index Defrag Script.


I have Done Rebuilt but the Index Fragmentation is Still High
Fragmentation in an index of a small table may not be reduced even after reorganising or rebuilding because they may be stored on mixed extents that are shared with different objects. We might want to check as well whether by having the index is actually helping to improve query performance or not. If not then this index can be considered to be removed.


References and further reading:
Reorganize and Rebuild Indexes
Rebuild or Reorganize: SQL Server Index Maintenance

Friday, 8 July 2016

Some Notes about ASP.NET Request Validation and Cross Site Scripting

ASP.NET framework has a built-in feature enabled by default to protect application against Cross Site Scripting (XSS) attack. It uses Request Validation feature that checks query string, cookie and posted form values to make sure that those inputs do not contain malicious script. It will throw an error if it encounters malicious script.


Request Validation version
For the recent version of ASP.NET framework, we should use the latest validation mode. In our web.config file, it should have something like:
<httpRuntime requestValidationMode="4.5" targetFramework="4.5" />
ASP.NET version 4.0 should use   requestValidationMode="4.0"


Turning off Request Validation feature in the application
We could also turn off Request Validation feature and only have this feature on some pages. To do this, in web.config set
<pages validateRequest="false" /> 
then set the validation feature on the pages that we would like to have it
<@ Page ValidateRequest="true" %> 


Disable Request Validation in a Controller
We also can disable Request Validation feature for a particular controller by setting [ValidateInput(false)] attribute:
[ValidateInput(false)] 
public ActionResult Index(MyModel m) 
{ 
    . . . 
} 


Disable Request Validation on a class property
To disable the feature at a more granular level, we can use [AllowHtml] attribute on a class property:
public class MyModel 
{ 
    public string MyProperty1 { get;  set; } 

    [AllowHtml] 
    public string MyProperty2 { get; set; } 

    . . . 

} 


Encoding input data
If we would like to allow input data that contains HTML or JavaScript codes, we can encode the input with HtmlEncode() function that are provided by ASP.NET framework. In ASP.NET version 4.5 or above, they have included AntiXss Library in the System.Web.Security.AntiXss namespace. This library prevents XSS better as it uses whitelisting approach compared to the default one which uses blacklisting approach. To set AntiXss Library as the default library for encoding, specify the encoderType in httpRuntime node in web.config:
<httpRuntime  . . .  encoderType="System.Web.Security.AntiXss.AntiXssEncoder" />


Sanitising data
When we need to accept HTML format data, then we should sanitise the data first from any malicious codes. The AntiXss Library 4.3.0 contains sanitiser functionalities however this has reached an end of life. The newer version that is included with ASP.NET 4.5 does not contain sanitiser.

We have some options to add sanitation functionality, a couple of those are:
- Write our own custom method using HTML Agility Pack. See this article for an example
- Use a third party library such as the open source HtmlSanitizer

Using the open source HtmlSanitizer library is easy. We can download the package from NuGet. Below is an example of a simple usage of it:
var sanitiser = new HtmlSanitizer(); 
sanitiser.Sanitize(inputString); 

If we would like to disallow some HTML tags, just use .AllowedTags.Remove() method. For example:
sanitiser.AllowedTags.Remove(“img”); 

Below is an example of a method that uses reflection to sanitise string properties of an object passed to it:
public object CleanStringsFromXSS(object data) 
        {  
            var sanitiser = new HtmlSanitizer(); 

            PropertyInfo[] properties = data.GetType().GetProperties(); 
            foreach (PropertyInfo property in properties) 
            { 
                Type propertyType = property.PropertyType; 
                if ((propertyType == typeof(string)) && (propertyType != null)) 
                { 
                    object value = property.GetValue(data); 
                    if (value != null && property.CanRead && property.CanWrite) 
                    { 
                        string sanitisedValue = sanitiser.Sanitize(value.ToString()); 
                        property.SetValue(data, sanitisedValue); 
                    } 
                } 
            } 

            return data; 
        } 


Request Validation vulnerabilities
There is a known vulnerability in Request Validation. Some types of JSON (JavaScript Object Notation) postback can bypass this feature. Therefore it is important to scan JSON input manually. The sanitation library above can be used to check or sanitise malicious codes.


References:
OWASP: ASP.NET Request Validation
Preventing XSS in ASP.NET Made Easy

Friday, 6 May 2016

Projecting Rows into Columns

Say we have some data in different rows of a column that we would like to project in different columns. For example we have this:

and would like to transform into this:


To do the projection we can use Pivot feature:
SELECT StudentId, DisciplineId, [1] AS Course1, [2] AS Course2, [3] AS Course3, [4] AS Course4, [5] AS Course5
FROM 
(
 SELECT  StudentId, DisciplineId, CompletionDate, CourseId
 FROM TrainingDetails
) AS T1
PIVOT 
( 
 MAX (CompletionDate) FOR CourseId IN ([1], [2], [3], [4], [5])
) AS T2
Pivot will project the rows into columns. It will also automatically apply grouping to the rest of the columns. So it is important to only feed the query with same columns that will be used in the Select result. In this example, we narrow down the source to only have columns that will be used in query (StudentId, DisciplineId, CompletionDate and CourseId) from other unrelated columns in the source (TrainingDetails table). If there is any extra column, the grouping will not be done correctly.

However, we can also achieve the same result with a more standard query:
SELECT StudentId, DisciplineId
, MAX(CASE WHEN CourseId = 1 THEN CompletionDate END) AS CompletionDateCourse1
, MAX(CASE WHEN CourseId = 2 THEN CompletionDate END) AS CompletionDateCourse2
, MAX(CASE WHEN CourseId = 3 THEN CompletionDate END) AS CompletionDateCourse3
, MAX(CASE WHEN CourseId = 4 THEN CompletionDate END) AS CompletionDateCourse4
, MAX(CASE WHEN CourseId = 5 THEN CompletionDate END) AS CompletionDateCourse5
FROM TrainingDetails
GROUP BY StudentId, DisciplineId

Tuesday, 26 April 2016

Migrating Existing TFS Database to Another TFS Server Instance

This post explains how to have a new TFS 2013 server with existing database copied from an existing TFS 2013 server.

These are the steps:
1. Backup each existing TFS database
2. Restore the databases in the new server environment. Make sure the databases names are similar like the old ones.
3. Install TFS in the new server by running TFS installer or executable file.
Choose 'Application Tier Only' or 'Upgrade' option on the wizard. Use 'Application Tier Only' if the new server has exactly the same version as the existing server. Use 'Upgrade' option if they have different versions, i.e; different version of service packs installed.
4. Once the installation is successful, go to the TFS Admin Console and change the configurations that still refer to the old server. Also change other configurations such as in Build Configuration and Backups.

Below are the screenshots of step by step running the installation wizard:
1. run the installer


2. choose 'Application Tier Only' option on left hand side menu if the TFS servers versions are similar

otherwise 'Upgrade' option if the versions are different


3. continue with the installation wizard


4. the wizard will detect local SQL instance and TFS databases


5. confirm the url and service account


6. keep progressing until the wizard finishes installation

Sunday, 10 January 2016

Config File Changes for AppFabric Upgrade

When upgrading from AppFabric 1.0 to 1.1, there are some configurations that need to be changed.

In configuration configSections node:
<!-- Old --> 
<section name="dataCacheClient" 
type="Microsoft.ApplicationServer.Caching.DataCacheClientSection, Microsoft.ApplicationServer.Caching.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" 
allowLocation="true" allowDefinition="Everywhere"/>

<!-- New -->
<section name="dataCacheClients"
type="Microsoft.ApplicationServer.Caching.DataCacheClientsSection, Microsoft.ApplicationServer.Caching.Core"
allowLocation="true" allowDefinition="Everywhere"/>

Also the dataCacheClient node can be more than one now:
<!-- Old --> 
<dataCacheClient>
    <hosts>
      <host name="oldcacheserver" cachePort="22233"/>
    </hosts>
</dataCacheClient>

<!-- New -->
<dataCacheClients> <!-- new parent node -->
    <dataCacheClient name="default">
    <hosts>
      <host name="newcacheserver" cachePort="22233"/>
    </hosts>
  </dataCacheClient>
</dataCacheClients>

If using AppFabric as session state provider, this needs to be changed as well:
<!-- Old --> 
<sessionState mode="Custom" customProvider="AppFabricCacheSessionStoreProvider">
  <providers>
 <add name="AppFabricCacheSessionStoreProvider" 
 type="Microsoft.ApplicationServer.Caching.DataCacheSessionStoreProvider, Microsoft.ApplicationServer.Caching.Client, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"                         
  cacheName="CacheName" sharedId="SomeID" />
  </providers>
</sessionState>

<!-- New -->
<sessionState mode="Custom" customProvider="AppFabricCacheSessionStoreProvider">
  <providers>
 <add name="AppFabricCacheSessionStoreProvider"
 type="Microsoft.Web.DistributedCache.DistributedCacheSessionStateStoreProvider, Microsoft.Web.DistributedCache"
 cacheName="CacheName" sharedId="SomeID"/>
  </providers>
</sessionState>


References:
How to: Configure the AppFabric Session State Provider for ASP.NET (AppFabric 1.1 Caching)
Configuration Settings for the ASP.NET 4 Caching Session State Provider (AppFabric 1.1 Caching)
Application Configuration Settings (AppFabric 1.1 Caching)
Preparing the Cache Client Development Environment (AppFabric 1.1 Caching)
Managing Security (Windows Server AppFabric Caching)

Wednesday, 30 December 2015

How to Move EF Database First Template .tt File to Another Project

On this post, we will see how to have Entity Framework .tt template file generated by an .edmx file in a different project. By default .tt files are created under the same project where the .edmx file is. Usually this is our data layer project. However, for most of the time, we would want the POCO models that are generated by a model .tt file to be put under a separate project (i.e. domain or model project) for a better practice.

There are a few steps to make this happen (I am using VS 2013 here):
1. Add a new model template .tt file in the other project through 'add a new item' then select EF DbContext Generator file type under 'Data'.


2. Open the new .tt file then change the value of 'inputFile' to point to the .edmx file in the original project.

In this case, my data project is called 'MySolution.Data.StudentBoundedContext'.

3. To ensure that all of the to be be generated POCO classes have right namespaces, we need to tell the template file the new namespace to use. Right click the .tt file and select 'Properties' then put the new namespace on 'Custom Tool Namespace' value.


4. Delete the model .tt file on the original project.

5. I prefer to leave the context .Context.tt template file on the data layer project and only move the model .tt file to a domain/model project. Therefore, I will need to tell the context template file to refer to the models in the other project.
To do this:
- add a project reference to the other project (domain/model project)
- open the .Context.tt file and add a 'using' statement referring to the models namespace


6. Regenerate the models and context files by right clicking the template files and selecting 'Run Custom Tool'.

Monday, 14 December 2015

Bulk Insert in Web SQL

Below is a snippet of how to do bulk insert of records in Web SQL:
// db is the database object that is usually initialise with openDatabase() function
db.transaction(function (tx) {  
  // insert each record
  $.each(myArray, function (i, item) {
   tx.executeSql("INSERT INTO MyTable(name, value) VALUES (?, ?)", [item.name, item.value]);
  });   
},
// error
function (error) {
 . . .
},
// success - the transaction() function does not pass any object to its success callback
function () {
 . . .
});

Web SQL does not understand the Standard SQL bulk insert syntax such as
Insert Into tbl (col1, col2) Values ('val1', 'val2'), ('val3', 'val4'), ...
but each insert statement needs to be executed using executeSql() function. A transaction is usually used to wrap these insert commands.

Tuesday, 27 October 2015

A Form Validation Example in AngularJS - Show Error Style After Submitted

Below is a basic example of a form validation in AngularJS that shows error style after the form is submitted.

To do this, we need to use novalidate attribute on the form to avoid browser validating the form but is passed to JavaScript to perform manual validation. In addition, we can utilise ng-submitted class that is added to the form by AngularJS after it is submitted.

The view:
<body ng-app="validationExample">
    <div ng-controller="MyCtrl as vm">
      <form name="myForm" novalidate ng-submit="vm.submitted(myForm, vm.input)">
        <input type="text" name="name" ng-model="vm.input.name" placeholder="please enter text" required/>
        <span ng-show="myForm.name.$error.required == true">*</span>
        <input type='text' name="value" ng-model='vm.input.value' placeholder="a number greater than 0" required ng-pattern='/^([1-9][0-9]*(\.[0-9]+)?|0+\.[0-9]*[1-9][0-9]*)$/'>
        <span ng-show="myForm.value.$error.required == true">*</span>
        <button>submit</button>
      </form>
    </div>
</body>

The script:
var myApp = angular.module('validationExample', [])

myApp.controller('MyCtrl', [function () {
    var vm = this;
    vm.submitted = function(form, input) {
      if(form.$valid) {
        alert('submitted');
      }
    }
} ]);

The stylesheet:
.ng-submitted input.ng-invalid {
  border-color:red;
}

See the example in Plunker.

To read more about form validation, please see my previous post.

Thursday, 1 October 2015

Table with Dynamic Rows Manipulation Example in AngularJS

Below is a code example of building a table with the ability to add, edit and remove rows dynamically with AngularJS. It also makes the corresponding input in focus.

The view:
<html>
  <head>
    <script src="http://code.jquery.com/jquery-2.1.4.min.js"></script>
    <script src="https://code.angularjs.org/1.2.16/angular.js"></script>
    <script src="script.js"></script>
  </head>
  <body ng-app="myApp">
    <div ng-controller="MyCtrl as vm">
      <table>
        <thead>
          <tr>
            <th>Name</th>
            <th>Value</th>
          </tr>
        </thead>
        <tbody>
          <tr ng-repeat="row in vm.rows">
            <td>
              <input type="text" ng-model="row.name" ng-readonly="row.readonly" ng-disabled="row.readonly" on-focus="!row.readonly" />
            </td>
            <td>
              <input type="text" ng-model="row.value" ng-readonly="row.readonly" ng-disabled="row.readonly" />
            </td>
            <td>
              <button ng-click="vm.editRow($index)">{{row.readonly ? "Edit" : "Save" }}</button>
              <button ng-click="vm.removeRow($index)">Remove</button>
            </td>
          </tr>
        </tbody>
      </table>
      <br />
      <input type="button" value="Add New" ng-click="vm.addNewRow('','')" />
    </div>
  </body>
</html>

The script:
angular.module('myApp', [])
.controller('MyCtrl', function () {
    var vm = this;
    vm.rows = [{"name": "aaa", "value" : 50, "readonly": true}, {"name": "bbb", "value" : 70, "readonly": true}];
    
    vm.addNewRow = function(name, value) {
      vm.rows[vm.rows.length - 1].readonly= true;
      vm.rows.push({"name":"", "value":"", "readonly": false})
    }
    
    vm.removeRow = function(index) {
      vm.rows.splice(index, 1);
    }
    
    vm.editRow = function(index) {
      vm.rows[index].readonly = !vm.rows[index].readonly;
    }
})
.directive('onFocus', function($timeout) {
    return function(scope, element, attrs) {
        scope.$watch(attrs.onFocus, function (newValue) {
            if (newValue) {
                $timeout(function () {
                    element.focus();
                }, 0, false);
            }
        }); 
      };    
});

See the example in action on Plunker.

Wednesday, 30 September 2015

[INSTALL_PARSE_FAILED_MANIFEST_MALFORMED] Error

If you encounter an issue when trying to run a Cordova app in an Android emulator with the error messages similar like the following:
  Installing app on device...
C:\Users\Me\Documents\Visual Studio 2015\Projects\MyApp\MyApp\platforms\android\cordova\node_modules\q\q.js:126
throw e;
^
ERROR: Failed to launch application on device: ERROR: Failed to install apk to device:  pkg: /data/local/tmp/android-debug.apk
Failure [INSTALL_PARSE_FAILED_MANIFEST_MALFORMED]
  Command finished with error code 1: cmd /s /c ""C:\Users\Me\Documents\Visual Studio 2015\Projects\MyApp\MyApp\platforms\android\cordova\run.bat" --nobuild --target=169.254.56.136:5555 --debug "--buildConfig=C:\Users\Me\Documents\Visual Studio 2015\Projects\MyApp\MyApp\build.json""
  ERROR running one or more of the platforms: Error: cmd: Command failed with exit code 1
  You may not have the required environment or OS to run this project
try to make sure that Package Name in config.xml is in lower case then rebuild/rerun the app.

Friday, 21 August 2015

Simpler Framework with DbContext and DbSet

I am trying to design a base framework that utilises DbContext as a unit of work and its DbSet properties as repositories. There are some voices on the Internet suggesting this approach for simplicity, performance, faster development effort and being able to keep exposing Entity Framework goodness. I also try to use bounded context approach that is based on domain driven design.

I have a base context class that is derived from DbContext:
    public abstract class BaseContext : DbContext
    {
        static BaseContext()
        {
        }
        protected BaseContext()
            : base("name=FrameworkOneDatabase")
        { }               
    }
Then some bounded contexts. Below is one of them:
    public class ArticleBoundContext : BaseContext
    {
        public ArticleBoundContext()
        {            
        }

        public virtual DbSet<Article> Article { get; set; }
        public virtual DbSet<User> Submitter { get; set; }
    }
Also a basic service class to help me calling CRUD operations on any bounded context:
    public class CRUDService
    {
        private BaseContext _context;

        public CRUDService(BaseContext context)
        {
            this._context = context;
        }

        public void Insert(dynamic entityObject)
        {
            dynamic dbset = GetDbSetFromObject(entityObject);
            entityObject.ObjectState = ObjectState.Added;
            dbset.Add(entityObject);
            _context.ApplyStateChanges();
        }

        public void InsertOrUpdate(dynamic entityObject)
        {
            dynamic dbset = GetDbSetFromObject(entityObject);
            dbset.Attach(entityObject);  
            _context.ApplyStateChanges();
        }
        
        public void Delete(dynamic entityObject)
        {
            dynamic dbset = GetDbSetFromObject(entityObject);
            dbset.Remove(entityObject);
        }

        public async Task<int> Commit()
        {
            var result = await _context.SaveChangesAsync();
            return result;
        }

        public void Dispose()
        {
            _context.Dispose();
        }

        private dynamic GetDbSetFromObject(dynamic entityObject)
        {
            // if dynamicproxies wrapper is used then get the base object
            System.Type objectType = entityObject.GetType();
            if (objectType.Namespace == "System.Data.Entity.DynamicProxies")
            {
                objectType = objectType.BaseType;
            }

            var dbset = (from p in _context.GetType().GetProperties()
                    where p.PropertyType.IsGenericType
                    && p.PropertyType.GetGenericTypeDefinition() == typeof(DbSet<>)
                    let entityType = p.PropertyType.GetGenericArguments().First()
                    where objectType == entityType
                    select p.GetValue(_context)).FirstOrDefault();

            if (dbset == null)
            {
                throw new System.ArgumentException("object type does not exist in the context");
            }

            return dbset;
        }
    }
The class has methods accepting an object then will find its corresponding DbSet member of the context. The method then call one of the DbSet operations. The GetDbSetFromObject() method is the one that will do the finding.

Then I can use all of the classes and structure above to do something like in the tests below:
    [TestClass]
    public class CRUDServiceTest
    {
        private ArticleBoundContext _context;
        private CRUDService _service;

        public CRUDServiceTest()
        {
            _context = new ArticleBoundContext();
            _service = new CRUDService(_context);
        }
        
        [TestMethod]
        public async Task CanInsertArticle()
        {
            Article article = new Article { Title = "title test " + DateTime.Now.ToString("HH:mm:ss"), Description = "desc", Url = "test.com", ObjectState = ObjectState.Added };
            article.Submitter = new User { Firstname = "first " + DateTime.Now.ToString("HH:mm:ss"), Lastname = "last", ObjectState = ObjectState.Added };
            _service.Insert(article);
            var insert = await _service.Commit();
            Assert.IsTrue(insert > 0);
        }

        [TestMethod]
        public async Task CanUpdateArticle()
        {
            Article article = _context.Article.FirstOrDefault(); 
            article.Title = "UPDATED TITLE " + DateTime.Now.ToString("HH:mm:ss");
            article.Description = "UPDATED DESCRIPTION";
            article.ObjectState = ObjectState.Modified;
            _service.InsertOrUpdate(article);
            var update = await _service.Commit();
            Assert.IsTrue(update > 0);
        }

        [TestMethod]
        public async Task CanUpdateSubmitter()
        {
            var article = _context.Article.FirstOrDefault(); 
            article.Submitter.Firstname = "UPDATED FIRSTNAME " + DateTime.Now.ToString("HH:mm:ss");
            article.Submitter.Lastname = "UPDATED LASTNAME " + DateTime.Now.ToString("HH:mm:ss");
            article.Submitter.ObjectState = ObjectState.Modified;

            _service.InsertOrUpdate(article);
            var update = await _service.Commit();
            Assert.IsTrue(update > 0);
        }
        
        [TestMethod]
        public async Task CanUpdateSubmitter_2()
        {
            var submitter = _context.Submitter.FirstOrDefault();
            submitter.Firstname = "UPDATED FIRSTNAME " + DateTime.Now.ToString("HH:mm:ss");
            submitter.Lastname = "UPDATED LASTNAME " + DateTime.Now.ToString("HH:mm:ss");
            submitter.ObjectState = ObjectState.Modified;

            _service.InsertOrUpdate(submitter);
            var update = await _service.Commit();
            Assert.IsTrue(update > 0);
        }

        [TestMethod]
        public async Task CanDeleteArticle()
        {
            var article = _context.Article.FirstOrDefault();

            _service.Delete(article);
            var result = await _service.Commit();

            Assert.IsTrue(result > 0);
        }

        [TestMethod]
        public async Task CanInsertAndDeleteSubmitter()
        {
            var submitter = new User();
            submitter.Firstname = "firstname " +DateTime.Now.ToString("HH:mm:ss");
            submitter.Lastname = "lastname " + DateTime.Now.ToString("HH:mm:ss");

            _service.Insert(submitter);
            var result = await _service.Commit();

            Assert.IsTrue(result > 0);
            var insertedSubmitter = await _context.Submitter.FindAsync(submitter.Id);
            Assert.IsTrue(insertedSubmitter.Firstname == submitter.Firstname && insertedSubmitter.Lastname == submitter.Lastname);

            _service.Delete(submitter);
            result = await _service.Commit();

            Assert.IsTrue(result > 0);
        }

        [TestMethod]
        public async Task ThrowExceptionWhenInsertingWrongObject()
        {
            try
            {
                int test = 5;
                _service.Insert(test);
                var insert = await _service.Commit();
            }
            catch (Exception ex)
            {
                Assert.IsInstanceOfType(ex, typeof(System.ArgumentException));
            }
        }
    }

Monday, 3 August 2015

Memory Issue when Building Cordova Project in Visual Studio 2015

I have just upgraded my desktop to Windows 10 and installed Visual Studio 2015 Community Edition. When trying to build my first Apache Cordova project in VS2015, I got these errors:
Could not create the Java virtual machine
Error occurred during initialization of VM
Could not reserve enough space for object heap


After trying to google for the solution, I found one that is better than the others. It told me to add a new system variable _JAVA_OPTIONS with -Xmx512M as the value.

Below is the detail:
Go to Control Panel -> System -> Advanced, then click Environment Variables. Click New on the System variables section then put:
Variable name: _JAVA_OPTIONS
Variable value: -Xmx512M

Solved my issue right away.

Monday, 27 July 2015

Automatically Generate Resized Icons and Splash Screens with Ionic Framework

Rather than manually generating or cropping different size of images of icons and splash screens for different resolutions, orientation and platforms, we can use Ionic Framework to do this work for us. Ionic will use its resizing and cropping server to do this. We will need a recent version of Ionic CLI.

Below are the steps to do this:
1. Prepare the source icon or splash screen file
For icon file:
- the file is named with 'icon' and has either .png, .psd or .ai extension type (i.e. icon.png, icon.psd or icon.ai)
- image's minimum dimensions should be 192 x 192 px
- rounded corners will be applied for specific platforms (e.g. iOS)

For splash screen file:
- the file is named with 'splash' and has either .png, .psd or .ai extension type (i.e. splash.png, splash.psd or splash.ai)
- image's minimum dimensions should be 2208 x 2208 px
- the artwork should be centered within inner 1200 x 1200 px area

2. Use a new or existing Ionic project to do this job
To have the Ionic server doing the resizing and cropping, we will need to put the file inside an Ionic project.

3. Make sure we have the intended platform on the Ionic project.
To add a platform, run ionic platform add command in the root folder of the project, for example:
C:\MyLabs\IonicTemplate\SideMenu>ionic platform add android

4. Create 'resources' folder on the root of the project

5. Put the icon or splash screen file in the folder

6. Run ionic resources command at the project root
To generate icons, run ionic resources --icon command (with double dash characters), below is a screenshot:


To generate splash screens, run ionic resources --splash command (with double dash characters), below is a screenshot:


7. Get the icons from 'resources/PLATFORM_NAME/icon' folder or the splash screens from 'resources/PLATFORM_NAME/splash' folder

Tuesday, 14 July 2015

Generating Android Release Keystore for Visual Studio Apache Cordova Project

Below are the steps to generate an Android keystore for publishing a release version of an app built using Visual Studio Apache Cordova:
1. Make sure you have Java SDK installed.

2. Run keytool command like below:
keytool -genkey -v -keystore YOUR_KEY_NAME.keystore -alias KEY_ALIAS -keyalg RSA -keysize 2048 -validity 20000
Use greater than 10,000 days (i.e. 20,000 days) validity to avoid expiration issue when uploading the package to Google App store.
For example:
C:\Program Files (x86)\Java\jdk1.7.0_55\bin>keytool -genkey -v -keystore c:\users\rical\my-release-key.keystore -alias rical_blog_app_key -keyalg RSA -keysize 2048 -validity 20000
Then you will be prompted with some questions:
Enter keystore password: *****
Re-enter new password: *****
What is your first and last name? 
  [Unknown]:  Rical Wirawan 
What is the name of your organizational unit? 
  [Unknown]:  Organisation
What is the name of your organization? 
  [Unknown]:  MyOrg
What is the name of your City or Locality? 
  [Unknown]:  MyCity 
What is the name of your State or Province? 
  [Unknown]:  NSW
What is the two-letter country code for this unit? 
  [Unknown]:  61 
Is CN=Rical Wirawan, OU=Organisation, O=MyOrg, L=MyCity, ST=NSW, C=61 correct? 
  [no]:  yes 

Generating 2,048 bit RSA key pair and self-signed certificate (SHA256withRSA) with a validity of 10,000 days 
        for: CN=Rical Wirawan, OU=Organisation, O=MyOrg, L=MyCity, ST=NSW, C=61 
Enter key password for  
        (RETURN if same as keystore password): 
[Storing my-release-key.keystore] 

3. The key file will be generated in the specified folder.

4. If you use a recent version of Cordova CLI, open build.json file in the root folder of the project. If it does not exist yet then create the file manually. Then put the details in the file, continuing our example:
{
 "android": {
     "release": {
         "keystore":"C:\\users\\rical\\my-release-key.keystore",
         "storePassword":"*****",
         "alias":"rical_blog_app_key",
         "password":"*****",
         "keystoreType":""
       }
   }
}

If your Cordova CLI version is less than 5.0, open ant.properties file (under 'your_project_name\res\cert\android' folder). Then put the key details in the ant.properties file, continuing our example:
key.store=c:\\users\\rical\\my-release-key.keystore
key.alias=rical_blog_app_key
key.store.password=*****
key.alias.password=*****

6. Build the app in Release mode with Device or one of Android emulator platforms option. Do not use any of the Ripple emulators option.

7. The package will be created inside 'bin/Android/Release' folder. The file to be deployed to Google App store is the one that does not end with (-unaligned.apk), e.g. "android-release.apk".

Wednesday, 24 June 2015

Looking Deeper into Async and Await

In this post, we will see the concepts of async and await in more details.

async keyword
A simple async method looks like this:
public async Task MyMethodAsync()
{
  . . .
}

An async keyword does not make a method to run asynchronously. What it does, it allows the method to use one or more await keyword to call other asynchronous method(s). The compiler will not generate an error if we fail to provide at least one await, it will only give a warning. We cannot use an await in a method that does not marked with async.

Secondly, it enables the method to return a value or exception into Task or Task<T> return type which will help us a lot in doing asynchronous programming.

A method that is decorated with async will actually start running its codes synchronously like a normal method until it finds an await keyword. Then it will see, if the method called by the await needs to be awaited, only then an asynchronous process will happen. We will see this in more details below.


await Keyword
Below are two examples of using await:
public async Task MyMethodAsync()
{
  // initially this runs synchronously
  . . .

  // 'await' waits for an asynchronous method to complete. It enables the process to be asynchronous,
  //    depending on how soon the method is completing.
  await RunSomethingAsync();

  // the codes past 'await' will only be executed after the awaited method completes,
  //    however 'await' does not block the thread
  . . .
}
// this second variation, shows where 'await' is used a little bit later after some other codes
public async Task MyMethodAsync()
{
  // initially this runs synchronously
  . . .

  var result = RunSomethingAsync();

  // some other codes here still run synchronously
  . . .

  // 'await' waits for an asynchronous method to complete. It enables the process to be asynchronous,
  //    depending on how soon the method is completing.
  await result;

  // the codes past 'await' will only be executed after the awaited method completes,
  //    however 'await' does not block the thread
  . . .
}

This is the keyword that enables the execution process to become asynchronous. If we do not have any await in our method, then the method will be executed synchronously even though the method is marked with async.

The keyword calls an asynchronous method. It allows the asynchronous method to be awaited until it completes then it will continue the codes after the await. While awaiting, it does not block the current thread but returns the control to the caller (i.e. MyMethodAsync() caller). This is when the asynchronous process is happening. The diagram in 'What Happens in an Async Method' section on this page explains this clearly.

Only when the called asynchronous method needs to be awaited, the process becomes asynchronous. It waits for the method to complete but also allow the caller (i.e. MyMethodAsync() caller) to execute other codes. If the outer codes would encounter an await keyword also then similar process would happen as well.

The await keyword does not need to be put together when calling an asynchronous method, it can be called later (note in our second variation example above, the await is not called straight away).

If the awaited method completes right away before the process above happens then all of those codes will be run in synchronous mode.


Task and Task<T>
We should always use either Task or Task<T> as the return type of our asynchronous method. The earlier is used when we do not have anything to be returned. Yes, when the method does not return anything, use Task instead of void (the exception will be explained below).

// below is an example of a method that returns Task<T>
public async Task<int> CalculateAsync()
{
   . . .

   // return an 'int' not 'Task<int>' type
   return 5;
}

Task should be used because this type allows the method to be awaited using await keyword which could lift up the process to be asynchronous. The return type enables the caller as well to know the progress when executing the method. Lastly, it encapsulates any exception that could occur inside the method to allow the exception to be handled easily.

Using void is also possible but it should be used for event handler only. An async void method has to use a different way to handle exception as it does not have a wrapper for the exception. It also does not provide an easy way to notify the caller when it has completed. In addition, this will be difficult to test.


Execution Thread
Our asynchronous method will begin with the caller's thread until it finds a process that needs to be awaited by await keyword (recall that not every asynchronous method that is called by await needs to be awaited). When it happens, the context of the caller's thread will be captured. The awaited process will run on a new thread from thread pool. Then when the awaited task is completed, the control is returned back to the captured context which is using the original thread (caller's thread).

If the awaited task finishes before the signing up process above is completed then the rest of the codes will be run synchronously without opening a new thread (from thread pool). Thus all the codes will always run on the initial thread which is the caller's thread.

public async Task MyMethodAsync()
{
  // initially this runs on the caller's thread
  . . .

  // since the process inside LongOperationAsync() needs to be awaited, 
  //    the original thread is captured and signed up for continuation.
  // The process is run using a new thread from thread pool 
  await LongOperationAsync();
  // after the awaiting is complete, the control is handed back to the original thread (caller's thread)

  // the codes past 'await' will be executed under original thread
  . . .
}

The process to returning back the control back to the captured context has some overhead. Moreover, the original context can suffer more if it has already had a performance issue due to other processes that it handles.

Fortunately, the codes after the await keyword can be run with a different thread unless if they really need the original context to run. For example; if the method is an ASP.NET MVC controller action method, the original context is an ASP.NET context and the codes after the await need to do some controller related stuffs then they will need to be run under ASP.NET context as well.

To enable codes after await to be run under a different context, we use ConfigureAwait(continueOnCapturedContext: false).
public async Task MyMethodAsync()
{
  // initially this runs on the caller's thread
  . . .

  // since 'ConfigureAwait(false)' is used, the process of capturing original context 
  //    and signing up for continuation can be skipped
  // the process in LongOperationAsync() is run using a new thread from thread pool 
  await LongOperationAsync().ConfigureAwait(continueOnCapturedContext: false);
  // after the awaiting is complete, the control will continue under the newly created thread,
  //    the handling back process to the original context can be skipped as well

  // the codes past 'await' will be executed under the newly created thread
  . . .
}

Whenever possible, we should always set ConfigureAwait(continueOnCapturedContext: false), unless if we know that we are going to need the original thread for the codes after the await.


References and further reading:
Async and Await Post by Stephen Cleary
Best Practices in Asynchronous Programming on MSDN
Async/Await FAQ on MSDN Blogs
Asynchronous Programming with Async and Await on MSDN

Tuesday, 23 June 2015

Getting Started with Async Await in C#

I was trying to start using async await in my data layer. After reading a few articles such as this one from MSDN, I start writing some asynchronous methods from my original synchronous methods.

A few steps that need to be done to create an asynchronous method from its counterpart synchronous method:
1. make sure System.Threading.Tasks is referred in one of the using statements
2. change the method return type to be async Task<returntype> or simply async Task if the original method was decorated with void
3. call the asynchronous version of the Entity Framework method
4. most of the time, use await keyword just before the Entity Framework asynchronous method. We could assign the result to a Task variable first then call the await keyword a little bit later if we have some operation that can be executed right away and is not dependent on the asynchronous method's result.

Below are a few simple examples:
- an example of a method that does not return anything
// original synchronous method
public void Insert(Person person)
{
 _context.Person.Add(person);
 _context.SaveChanges();
}

// the async version
public async Task InsertAsync(Person person)
{
 _context.Person.Add(person);
 await _context.SaveChangesAsync();
}

- an example of a method returning a value
// original synchronous method
public int GetTotal()
{
    return _context.Person.Count();
}

// the async version
public async Task<int> GetTotalAsync()
{
    return await _context.Person.CountAsync();
}

- an example that shows using await a little bit later:
// original synchronous method
public int GetTotalNumber()
{
    var result = _context.Person.Count();

    DoSomeOtherOperation();

    return result;
}

// the async version
public async Task<int> GetTotalNumberAsync()
{
    var result = _context.Person.CountAsync(); // use await a little bit later

    // do non-related operation to the result above
    DoSomeOtherOperation();

    return await result;
}

On the next post we will see these concepts in more details.

Monday, 18 May 2015

Using When, All, Then and Reject functions of $q Service

We will see how to use when, all, then and reject functions of the AngularJS $q service. To learn the basic usage of $q, please see my previous post.

$q.when
$q.when() can be used to wrap an object or function if we are not sure whether it returns a promise or not. If we wrap a value object or a value returned function, the value will be passed as a result argument to the next function. When we wrap a function, the when() method will ensure that the function is executed first before going to the next one.

Below are some code examples:
$q.when('aaa').then(function(result) {
    alert(result);
})

var vm = this;
vm.chars = '';

$q.when(_test()).then(function(){
 vm.chars = vm.chars + 'END'
})

_test = function() {
 for (i = 0; i <= 10000; i++) {
   vm.chars = vm.chars + '.'
 }
}


$q.all
$q.all() is used to combine promise returned functions and only continue to progress further after all of the functions complete successfully. If one of the functions fails, immediately as soon as $q.all() gets a rejected promise from one of the functions, it will end the execution and pass forward to an error callback if there is any.
vm.numbers = '';    
_promiseFunction = function(x) {
 var deferred = $q.defer();

 if (x > 0) {
   deferred.resolve(x * 100);
 } else {
   deferred.reject(x + " is smaller than 0")
 }

 return deferred.promise;
}
 
var first = _promiseFunction(1);
var second = _promiseFunction(5);
var third = _promiseFunction(10);
var four = _promiseFunction(-1);
var five = _promiseFunction(-10)

$q.all([first, second, third /*, four, five*/]).
then(function(results){
  console.log(results.length);
  for(i=0; i < results.length; i++) {
    vm.numbers = vm.numbers + ' ' + results[i];
  }
 },
 function(errors){
  // if there is any error in the sequence, the success callback will not be called
  console.log(errors);
 })


$q.then
$q.then() is useful for executing asynchronous functions (or blocks of codes) sequentially in a chain. If the function inside a then() does not return a promise, it will be wrapped as a resolved promise. Otherwise the resolved or rejected promise will be passed forward.

_promiseFunction(5).then(
 function() { alert('aaa')}
).then(
 function() { alert('bbb');}
).then(
 function() {alert('ccc');}
);

_promiseFunction(5).then(
 function() { return $q.reject('failed')}
).then(
 function(){ alert('result success')},
 function(){ alert('result fail')}
)

Note also that if we return a non-promise value in the error callback inside then(), this will be treated as a resolved promise. So we need to be careful when handling error if our intention is to pass the error forward to the next function.
$q.reject().then(
 // success
 function() {},
 // error
 function(error) {
   return 'i got the error'
 }
).then(
 //success
 function() {
   alert('success');
 },
 // error
 function() {
   alert('error')
 }
)

$q.reject
Last thing, we have $q.reject() that is useful as a quick way to create then send a rejected promise (with a value optionally) to the next function in a chain. For example, in a chain of multiple then(). We can see this in the last two examples.

See the examples in Plunker.

Friday, 15 May 2015

Setting Up One to Many Relationship with Fluent API in Code First

Let's say that we would like to have a one to many relationship between two tables, e.g. User and Article. A User can have many Articles. Below are two ways of how we could specify the models and relationship configuration with Fluent API.

1. Without Having Foreign Key Property in Model
public class Article 
{
 public int Id { get; set; }

 [StringLength(150)]
 [Required]
 public string Title { get; set; }

 [Required]
 public string Description { get; set; }

 [StringLength(500)]
 [Required]
 public string Url { get; set; }

 #region navigation properties
 public virtual User Submitter { get; set; }
 #endregion
}


public class User 
{
 public int Id { get; set; }
 
 [StringLength(100)]
 [Required]
 public string Firstname { get; set; }
 
 [StringLength(100)]
 [Required]
 public string Lastname { get; set; }

 #region navigational property
 public virtual ICollection<Article> Articles { get; set; }
 #endregion
}


protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
 #region Article table
 modelBuilder.Entity<Article>().HasKey(t => t.Id);
 // specify the relationship between the two tables
 modelBuilder.Entity<Article>().HasRequired(t => t.Submitter).WithMany(t => t.Articles);
 #endregion

 #region User table
 modelBuilder.Entity<User>().HasKey(t => t.Id);
 #endregion
}
Before we go further, I would like to say that I prefer to use Data Annotation attributes in a model for validation purpose only for its properties. The database related configurations are done inside OnModelCreating() method.

On the codes above, we do not specify any foreign key property such as 'SubmitterId'. We do this purely in 'code first perspective' only which focuses on models and 'code' relationships between them, not their relational database relationships. So we do not worry on specifying any foreign key property here, we only specify the navigational property.

Please note as well that we do not put a [Required] attribute on the navigation property even though we always want this model to have a navigational property object (i.e. an article must have a submitter). This would create an issue for us later when we want to update an article object only without worrying about submitter. It is better to specify this rule when specifying the relationship in Fluent API.

To let Code First know the relationship we want, we put this on OnModelCreating() method:
modelBuilder.Entity<Article>().HasRequired(t => t.Submitter).WithMany(t => t.Articles);
The validation rule that the navigational property (i.e. submitter) is required is set here.

Code First will generate tables like below:



2. With Foreign Key Property in Model
If we prefer to specify foreign key property in our model, then we can do this way:
// add foreign key property to the model (Article)
public int SubmitterId { get; set; }
The advantage of doing it this way is that we can name it with any name we like.

Then on OnModelCreating() method, we write:
modelBuilder.Entity<Article>().HasRequired(t => t.Submitter).WithMany(t => t.Articles).HasForeignKey(t => t.SubmitterId);

Below are the tables and columns that will be generated:



For setting other configurations using fluent API, please see this post.