Looking Deeper into Async and Await

In this post, we will see the concepts of async and await in more details.

async keyword
A simple async method looks like this:

public async Task MyMethodAsync()
{
  . . .
}

An async keyword does not make a method to run asynchronously. What it does, it allows the method to use one or more await keyword to call other asynchronous method(s). The compiler will not generate an error if we fail to provide at least one await, it will only give a warning. We cannot use an await in a method that does not marked with async.

Secondly, it enables the method to return a value or exception into Task or Task<T> return type which will help us a lot in doing asynchronous programming.

A method that is decorated with async will actually start running its codes synchronously like a normal method until it finds an await keyword. Then it will see, if the method called by the await needs to be awaited, only then an asynchronous process will happen. We will see this in more details below.


await Keyword
Below are two examples of using await:

public async Task MyMethodAsync()
{
  // initially this runs synchronously
  . . .

  // 'await' waits for an asynchronous method to complete. It enables the process to be asynchronous,
  //    depending on how soon the method is completing.
  await RunSomethingAsync();

  // the codes past 'await' will only be executed after the awaited method completes,
  //    however 'await' does not block the thread
  . . .
}

// this second variation, shows where 'await' is used a little bit later after some other codes
public async Task MyMethodAsync()
{
  // initially this runs synchronously
  . . .

  var result = RunSomethingAsync();

  // some other codes here still run synchronously
  . . .

  // 'await' waits for an asynchronous method to complete. It enables the process to be asynchronous,
  //    depending on how soon the method is completing.
  await result;

  // the codes past 'await' will only be executed after the awaited method completes,
  //    however 'await' does not block the thread
  . . .
}

This is the keyword that enables the execution process to become asynchronous. If we do not have any await in our method, then the method will be executed synchronously even though the method is marked with async.

The keyword calls an asynchronous method. It allows the asynchronous method to be awaited until it completes then it will continue the codes after the await. While awaiting, it does not block the current thread but returns the control to the caller (i.e. MyMethodAsync() caller). This is when the asynchronous process is happening. The diagram in 'What Happens in an Async Method' section on this page explains this clearly.

Only when the called asynchronous method needs to be awaited, the process becomes asynchronous. It waits for the method to complete but also allow the caller (i.e. MyMethodAsync() caller) to execute other codes. If the outer codes would encounter an await keyword also then similar process would happen as well.

The await keyword does not need to be put together when calling an asynchronous method, it can be called later (note in our second variation example above, the await is not called straight away).

If the awaited method completes right away before the process above happens then all of those codes will be run in synchronous mode.


Task and Task<T>
We should always use either Task or Task<T> as the return type of our asynchronous method. The earlier is used when we do not have anything to be returned. Yes, when the method does not return anything, use Task instead of void (the exception will be explained below).

// below is an example of a method that returns Task<T>
public async Task<int> CalculateAsync()
{
   . . .

   // return an 'int' not 'Task<int>' type
   return 5;
}

Task should be used because this type allows the method to be awaited using await keyword which could lift up the process to be asynchronous. The return type enables the caller as well to know the progress when executing the method. Lastly, it encapsulates any exception that could occur inside the method to allow the exception to be handled easily.

Using void is also possible but it should be used for event handler only. An async void method has to use a different way to handle exception as it does not have a wrapper for the exception. It also does not provide an easy way to notify the caller when it has completed. In addition, this will be difficult to test.


Execution Thread
Our asynchronous method will begin with the caller's thread until it finds a process that needs to be awaited by await keyword (recall that not every asynchronous method that is called by await needs to be awaited). When it happens, the context of the caller's thread will be captured. The awaited process will run on a new thread from thread pool. Then when the awaited task is completed, the control is returned back to the captured context which is using the original thread (caller's thread).

If the awaited task finishes before the signing up process above is completed then the rest of the codes will be run synchronously without opening a new thread (from thread pool). Thus all the codes will always run on the initial thread which is the caller's thread.

public async Task MyMethodAsync()
{
  // initially this runs on the caller's thread
  . . .

  // since the process inside LongOperationAsync() needs to be awaited, 
  //    the original thread is captured and signed up for continuation.
  // The process is run using a new thread from thread pool 
  await LongOperationAsync();
  // after the awaiting is complete, the control is handed back to the original thread (caller's thread)

  // the codes past 'await' will be executed under original thread
  . . .
}

The process to returning back the control back to the captured context has some overhead. Moreover, the original context can suffer more if it has already had a performance issue due to other processes that it handles.

Fortunately, the codes after the await keyword can be run with a different thread unless if they really need the original context to run. For example; if the method is an ASP.NET MVC controller action method, the original context is an ASP.NET context and the codes after the await need to do some controller related stuffs then they will need to be run under ASP.NET context as well.

To enable codes after await to be run under a different context, we use ConfigureAwait(continueOnCapturedContext: false).

public async Task MyMethodAsync()
{
  // initially this runs on the caller's thread
  . . .

  // since 'ConfigureAwait(false)' is used, the process of capturing original context 
  //    and signing up for continuation can be skipped
  // the process in LongOperationAsync() is run using a new thread from thread pool 
  await LongOperationAsync().ConfigureAwait(continueOnCapturedContext: false);
  // after the awaiting is complete, the control will continue under the newly created thread,
  //    the handling back process to the original context can be skipped as well

  // the codes past 'await' will be executed under the newly created thread
  . . .
}

Whenever possible, we should always set ConfigureAwait(continueOnCapturedContext: false), unless if we know that we are going to need the original thread for the codes after the await.


References and further reading:
Async and Await Post by Stephen Cleary
Best Practices in Asynchronous Programming on MSDN
Async/Await FAQ on MSDN Blogs
Asynchronous Programming with Async and Await on MSDN

Getting Started with Async Await in C#

I was trying to start using async await in my data layer. After reading a few articles such as this one from MSDN, I start writing some asynchronous methods from my original synchronous methods.

A few steps that need to be done to create an asynchronous method from its counterpart synchronous method:
1. make sure System.Threading.Tasks is referred in one of the using statements
2. change the method return type to be async Task<returntype> or simply async Task if the original method was decorated with void
3. call the asynchronous version of the Entity Framework method
4. most of the time, use await keyword just before the Entity Framework asynchronous method. We could assign the result to a Task variable first then call the await keyword a little bit later if we have some operation that can be executed right away and is not dependent on the asynchronous method's result.

Below are a few simple examples:
- an example of a method that does not return anything

// original synchronous method
public void Insert(Person person)
{
 _context.Person.Add(person);
 _context.SaveChanges();
}

// the async version
public async Task InsertAsync(Person person)
{
 _context.Person.Add(person);
 await _context.SaveChangesAsync();
}

- an example of a method returning a value

// original synchronous method
public int GetTotal()
{
    return _context.Person.Count();
}

// the async version
public async Task<int> GetTotalAsync()
{
    return await _context.Person.CountAsync();
}

- an example that shows using await a little bit later:

// original synchronous method
public int GetTotalNumber()
{
    var result = _context.Person.Count();

    DoSomeOtherOperation();

    return result;
}

// the async version
public async Task<int> GetTotalNumberAsync()
{
    var result = _context.Person.CountAsync(); // use await a little bit later

    // do non-related operation to the result above
    DoSomeOtherOperation();

    return await result;
}

On the next post we will see these concepts in more details.

Arranging Extension Methods

I like to arrange my extension methods in categories based on the functionalities and also to design them to be easily tested (mocked) as described on this post.

That way the methods are grouped nicely and also displayed neatly when showed by intellisense. Below is an example of how I do that:

public interface ICategoryOne
{
 string MethodOne();
 int MethodTwo();
 int MethodThree(string value);
}

internal class RealCategoryOne : ICategoryOne
{
 private StringBuilder sb;

 public RealCategoryOne(StringBuilder sb)
 {
  this.sb = sb;
 }

 public string MethodOne()
 {
  return "method one";
 }

 public int MethodTwo()
 {
  return sb.GetHashCode();
 }

 public int MethodThree(string value)
 {
  return value.Count();
 }
}

public static class MyExtensionMethods
{
 public static Func<StringBuilder, ICategoryOne> CategoryOneFactory = sb => new RealCategoryOne(sb);

 public static ICategoryOne CategoryOne(this StringBuilder sb)
 {
  return CategoryOneFactory(sb);
 }
}  

With intellisense, the methods will be shown like:



Then to test these extension methods with Moq:

Mock<ICategoryOne> categoryOne = new Mock<ICategoryOne>();
categoryOne.Setup(c => c.MethodOne()).Returns("Moq method 1");
categoryOne.Setup(c => c.MethodTwo()).Returns(-555);
categoryOne.Setup(c => c.MethodThree(It.IsAny<string>())).Returns(-999);

StringBuilder sb = new StringBuilder();

MyExtensionMethods.CategoryOneFactory = prm => categoryOne.Object;

var test = sb.CategoryOne().MethodOne();
Assert.AreEqual(test, "Moq method 1");

Alternatively we can create a fake/dummy class for testing:

public class FakeCategoryOne : ICategoryOne
{
 private StringBuilder sb;

 public FakeCategoryOne(StringBuilder sb)
 {
  this.sb = sb;
 }

 public string MethodOne()
 {
  throw new NotImplementedException();
 }

 public int MethodTwo()
 {
  throw new NotImplementedException();
 }

 public int MethodThree(string value)
 {
  throw new NotImplementedException();
 }
}

Then we change the factory above as:

MyExtensionMethods.CategoryOneFactory = prm => new FakeCategoryOne(prm);

Reference:
http://blogs.clariusconsulting.net/kzu/how-to-mock-extension-methods/

Grouping Data with LINQ

To group data in LINQ, we can use group ... by ... clause in query syntax or GroupBy() in method syntax. We will go through some examples and explanations along this post.


SIMPLE GROUPING
Let's start with simple grouping, below is an example:

// query syntax
var groupedData = from c in context.Customers
                  group c by c.Country;
// method syntax
var groupedData = context.Customers.GroupBy(c => c.Country);

Grouping in LINQ will result in an object of type IEnumerable<IGrouping<TKey,TSource>> which in this case is IEnumerable<IGrouping<String,Customer>>. IGrouping is a special class that has two properties; a key property and an IEnumerable<TSource> property that holds all the items corresponding to the key.

If we try to debug the 'groupedData' object, we will get something like this (you may need to click the image below to make it displayed bigger):

As we can see there's a 'Key' property and another property that contains some items corresponding to the 'Key' value.

To print out these items on screen:

foreach(var groupedItems in groupedData)
{
    Console.WriteLine(string.Format("Key: {0}", groupedItems.Key));
    foreach (var item in groupedItems)
    {
        Console.WriteLine(string.Format("{0} - {1}", item.CompanyName, item.Country));
    }
    Console.WriteLine("----------------------------------");
}

GROUPING WITH MORE THAN ONE KEY
If we want to have a grouping using two keys, we could use group x by new { x.Key1, x.Key2 } in query syntax or GroupBy( x => new { x.Key1, x.Key2 } ) in method syntax. Below is an example:

// query syntax
var groupedData2 = from c in context.Customers
                   group c by new { c.Country, c.City };
// method syntax
var groupedData2 = context.Customers.GroupBy(c => new {c.Country, c.City});

foreach (var groupedItems in groupedData2)
{
    //note that the Keys' names now become part of Key properties; ie. Key.Country and Key.City
    Console.WriteLine(string.Format("Key: {0} - {1}", groupedItems.Key.Country, groupedItems.Key.City));
    foreach (var item in groupedItems)
    {
        Console.WriteLine(string.Format("{0} - {1} - {2}", item.CompanyName, item.City, item.Country));
    }
    Console.WriteLine("----------------------------------");
}

PROJECTION
Here is an example of projecting the result into anonymous type objects:

// query syntax
var groupedData3 = from c in context.Customers
                   group c by c.Country into grp
                   select new
                   {
                       Key = grp.Key,
                       Items = grp.Select(g => new { g.CompanyName, g.Country })
                   };
// method syntax
var groupedData3 = context.Customers.GroupBy(c => c.Country).
                   Select(grp => new {
                                       Key = grp.Key, 
                                       Items = grp.Select(g => new {g.CompanyName, g.Country})
                                     }
                   );

foreach (var groupedItems in groupedData3)
{
    Console.WriteLine(string.Format("Key: {0}", groupedItems.Key));
    foreach (var item in groupedItems.Items)
    {
        Console.WriteLine(string.Format("{0} - {1}", item.CompanyName, item.Country));
    }
    Console.WriteLine("----------------------------------");
}

Below is another example that projects the result into strong typed objects.
The classes (made simple for demonstration purpose):

public class CompanyViewModel
{
    public string Name { get; set; }
    public string Country { get; set; }
}

public class GroupedCompanies
{
    public string CountryKey { get; set; }
    public IEnumerable<CompanyViewModel> Companies { get; set; }
}

Then the query:

var groupedData4 = from c in context.Customers
                   group c by c.Country into grp
                   select new GroupedCompanies
                   {
                       CountryKey = grp.Key,
                       Companies = grp.Select(g => new CompanyViewModel { Name = g.CompanyName, Country = g.Country })
                   };
foreach (GroupedCompanies groupedItems in groupedData4)
{
    Console.WriteLine(string.Format("Key: {0}", groupedItems.CountryKey));
    foreach (CompanyViewModel item in groupedItems.Companies)
    {
        Console.WriteLine(string.Format("{0} - {1}", item.Name, item.Country));
    }
    Console.WriteLine("----------------------------------");
}

GROUPING WITH MORE THAN ONE KEY + PROJECTION
Finally this example shows a combination of grouping with two keys and projection:

// query syntax
var groupedData5 = from c in context.Customers
                   group c by new { c.Country, c.City } into grp
                   select new
                   {
                       Key = grp.Key,
                       Items = grp.Select(g => new { g.CompanyName, g.City, g.Country })
                   };
// method syntax
var groupedData5 = context.Customers.GroupBy( c => new {c.Country, c.City} ).
                   Select( grp => new {
                                       Key = grp.Key, 
                                       Items = grp.Select(g => new {g.CompanyName, g.City, g.Country})
                                      }
                   );

How to Get Enum Elements' Values and Descriptions

Below is an example of how to populate a collection from an Enumeration elements' values and Description attributes (names if Description attributes are not exist).

Say that we have the following Enum:

public enum EnumStatus
{
    [Description("Not Submitted")]
    NotSubmitted = 0,

    Requested = 1,

    [Description("Pending Approval")]
    PendingApproval = 2,

    Approved = 3,

    Rejected = 4
}

Then the code to get each element's value and Description attribute (or name) is:

var type = typeof(EnumStatus);
foreach (var field in type.GetFields().Where(f => f.FieldType == type))
{
    var attribute = Attribute.GetCustomAttribute(field, typeof(System.ComponentModel.DescriptionAttribute)) 
        as System.ComponentModel.DescriptionAttribute;
    var value = field.GetValue(Activator.CreateInstance(type));
    myCollections.Add(
        new
        {
            Id = (int)value,
            Name = attribute != null ? attribute.Description : value.ToString() // or Enum.GetName(typeof(EnumStatus), value)
        }
    );
}