Microsoft has been providing APIs for Team Foundation Services (TFS) from the earliest version of TFS (since Visual Studio 2005). Using these APIs, we can create TFS clients and also write event handlers for the events raised by TFS. In the early days of TFS, we were expected to write only Microsoft .NET clients. We could give reference to the components that contained these APIs, and use the referred classes in the code to access various TFS services. Over the years, to facilitate collaboration between different platforms, we are now also expected to create clients in technologies other than Microsoft.NET; like Java, JavaScript and many others. For these technologies, the components that are written in .NET are not useful.
To assist such programming, Microsoft has recently published the APIs in the form of RESTful services that encapsulate the services of TFS and Visual Studio Online.
This article is published from the DNC Magazine for .NET Developers and Architects. Download this magazine from here [PDF] or Subscribe to this magazine for FREE and download all previous and current editions
In the case of APIs we used in the early versions of TFS, whenever we wanted to access a TFS service programmatically from a client, we had to give reference to assemblies that encapsulated calls to webservices or WCF Services of TFS. It also meant that these assemblies should be present on the computer, the compatible .NET Framework should be installed, and the assemblies should be stored in the Global Assembly Cache. In a nutshell, Team Explorer had to be installed on that computer. These prerequisites were quite restrictive.
Since we are now dealing with RESTful services, we do not have to worry about these conditions. As long as we can create a HTTP request and send it over the transport to TFS, and are able to read the response; we can write a TFS client. It can be using any technology on any operating system, and on any device. What sounds particularly interesting is a case where a TFS client is able to run on a mobile device.
Accessing Visual Studio Online (VSO) using such a client that uses RESTful services of TFS, has one issue that we have to overcome. That issue is related to security set up by VSO. While accessing through the browser, the Team Web Access application which is an ASP.NET application, uses credentials of the logged-in user. These credentials are usually the users Microsoft account credentials like those derived from Hotmail or Live account. RESTful services of TFS do not support authentication using such credentials. It either supports Basic authentication or OAuth. For each account of VSO, we can enable Basic authentication.
Let us walk through a scenario of creating a VSO account to enable Basic Authentication. Later, we will also see the limitations of Basic Authentication and how to use OAuth. This article has been co-authored by Subodh Sohoni and Manish Sharma.
Create VSO account to enable Basic Authentication
You can create a VSO account by going to http://www.visualstudio.com and then select “Visual Studio Online – Get Started for Free”. This free account works for up to 5 users. You will need to login with your Microsoft credentials like Hotmail, Live etc. After that, you should provide a name that should be unique to your account.
Now you can create your first team project on VSO. You can provide a name to that Team Project and select process template between SCRUM, Agile and CMMI. For our example, we selected the SCRUM process template. That defines the work item types that will be present in the team project. You can also choose the version control mechanism – TFVC or Git.
Now that the team project is created, we will go ahead and add the support for Basic authentication in our account. To do so, open the Settings section of the account by clicking the name and then My Profile on the right top corner. Then in the profile, select the Credentials tab.
Now enable Alternate credentials by clicking a link that says ‘Enable alternate credentials’.
Then give the User name and Password of your choice. This is the one which will be used for Basic Authentication to your account.
RESTful services to access VSO
Let us now locate the RESTFul services which we can call to access VSO services. For each account the APIs are available from the base URL https://{account}.visualstudio.com/defaultcollection/_apis/ . From here we can access various services of VSO like Projects, Work Item Tracking, Version Control, Test Management, Team Room, Shared Services and Build services. To get URL of each service API, you can visit the page https://www.visualstudio.com/en-us/integrate/api/overview. These services mainly use GET and PATCH methods.
We will focus on one of the most frequently used service and the one that requires most custom client creations, i.e. the Work Item Service. From our custom applications, we often require to create a new work item, view the data stored for that work item and update that work item back in the VSO. Functionality related to Work Item Tracking is available from the services under the URL https://{account}.visualstudio.com/defaultcollection/_apis/wits/workitems .
Let us now view a simple method to get a work item using its ID. Obviously it is a GET method with ID as a querystring parameter. So the UTI will be https://{account}.visualstudio.com/defaultcollection/_apis/wits/workitems?id=1&api-version=1.0
As you must have observed, there is no mention of Team Project name here. That is because work item ids are unique for Team Project Collection. This GET method returns a JSON object with the structure as follows:
{
"count": 3,
"value": [
{
"id": 1,
"rev": 1,
"fields": {
"System.AreaPath": "TimeS",
"System.TeamProject": "TimeS",
"System.IterationPath": "TimeS",
"System.WorkItemType": "Product Backlog Item",
"System.State": "New",
"System.Reason": "New backlog item",
"System.CreatedDate": "2015-04-29T20:49:20.77Z",
"System.CreatedBy": "Manish Sharma <[email protected]>",
"System.ChangedDate": "2015-05-09T20:49:20.77Z",
"System.ChangedBy": "Manish Sharma <[email protected]>",
"System.Title": "Customer can sign in using their Microsoft Account",
"Microsoft.VSTS.Scheduling.Effort": 8,
"WEF_6CB513B6E70E43499D9FC94E5BBFB784_Kanban.Column": "New",
"System.Description": "Our authorization logic needs to allow for users with Microsoft accounts (formerly Live Ids) - http://msdn.microsoft.com/en-us/library/live/hh826547.aspx"
},
"url": "https://technizer.visualstudio.com/DefaultCollection/_apis/wit/workItems/1"
}
}
To send this request to the VSO, we will use the instance of the class System.Net.Http.HttpClient.
HttpClient client = new HttpClient();
This client object can create HTTP Requests, Add Headers to that request, Send the request to the known URI and get the Response back.
We will first specify the request header for the type of message as JSON object. Then we specify user credentials to authenticate with VSO. The user credentials are sent in the form of Authorization header with Basic Authentication. The URL will be as mentioned earlier.
The instance of this class can now send a request and it does that asynchronously. That means, we can call Get() method on the URL as GetAsync() which returns a result of call as HttpResponseMessage object.
HttpResponseMessage response = client.GetAsync(Url).Result;
To read the contents of that response message, we need to use the await keyword.
string responseBody = await response.Content.ReadAsStringAsync();
The response is a JSON object with the structure as shown earlier. This response string will have to be typecast into a Work Item object from where we will be able to get the field values. For that, we will write a class that has all the fields or properties that match the JSON object. We created a class called WorkItemDetails for that.
public class WorkItemDetails
{
public string id;
public string rev;
public IDictionary<string, string> fields;
public string Url;
}
So now, we will deserialize the JSON object and typecast it as WorkItemDetails. We will use NewtonSoft’s JSON.Net package for that. It has a JsonConvert class which can deserialize the JSON object and also typecast it as desired.
WorkItemDetails wiDetails = JsonConvert.DeserializeObject<WorkItemDetails>(responseBody);
ID of the work item is obtained directly. But the fields are returned as a dictionary object. We can get individual fields by using a loop for key-value pair.
foreach (KeyValuePair<string, string> fld in wiDetails.fields)
{
Console.WriteLine(fld.Key + ":\t" + fld.Value);
}
The code for this entire functionality will look like the following:
static async void GetWorkItem(string username, string password, int WiId)
{
try
{
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",Convert.ToBase64String(
System.Text.ASCIIEncoding.ASCII.GetBytes(
string.Format("{0}:{1}", username, password))));
string Url = "https://ssgsonline.visualstudio.com/defaultcollection/_apis/wit/workitems?id=1&api-version=1.0";
using (HttpResponseMessage response = client.GetAsync(Url).Result)
{
response.EnsureSuccessStatusCode();
string responseBody = await response.Content.ReadAsStringAsync();
WorkItemDetails wiDetails = JsonConvert.DeserializeObject<WorkItemDetails>(responseBody);
Console.WriteLine("Work Item ID: \t" + wiDetails.id);
foreach (KeyValuePair<string, string> fld in wiDetails.fields)
{
Console.WriteLine(fld.Key + ":\t" + fld.Value);
}
}
}
Catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
The output looks similar to the following:
Create a New Work Item
To create a new work item, we will use the PATCH method for this service. PATCH method of this service accepts a parameter that indicates work item of which work item type is to be created. The URL of the method that creates a Task looks like this:
https://{account}/DefaultCollection/{TeamProject}/_apis/wit/workitems/$Task?api-version=1.0
In addition to that, this method accepts following data in the JSON format:
[
{
"op": "add",
"path": "/fields/System.Title",
"value": "JavaScript implementation for Microsoft Account"
}
]
The value of variable “op” indicates that the field is to be added, “path” is the name of the field to be given the value and “value” obviously is the value to be given to that field. It has to be sent as collection of fields. The code for creation of work item will look like this:
static async void CreateWorkItem(string username, string password)
{
try
{
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Add(new
System.Net.Http.Headers.MediaTypeWithQualityHeaderValue
("application/json-patch+json"));
client.DefaultRequestHeaders.Authorization = new
AuthenticationHeaderValue("Basic",Convert.ToBase64String(
System.Text.ASCIIEncoding.ASCII.GetBytes(
string.Format("{0}:{1}", username, password))));
WorkItemPostData wiPostData = new WorkItemPostData();
wiPostData.op = "add";
wiPostData.path = "/fields/System.Title";
wiPostData.value = "Employee edits other employees profile";
List<WorkItemPostData> wiPostDataArr = new List<WorkItemPostData> {
wiPostData };
string wiPostDataString =JsonConvert.SerializeObject(wiPostDataArr);
HttpContent wiPostDataContent = new StringContent(wiPostDataString,
Encoding.UTF8, "application/json-patch+json");
string Url =
"https://ssgsonline.visualstudio.com/DefaultCollection/
SSGS EMS SCRUM/_apis/wit/workitems/$Product%20Backlog%20Item?
api-version=1.0";
using (HttpResponseMessage response = client.PatchAsync(Url,
wiPostDataContent).Result)
{
response.EnsureSuccessStatusCode();
string ResponseContent = await
response.Content.ReadAsStringAsync();
}
}
}
catch(Exception ex)
{
Console.WriteLine(ex.ToString());
Console.ReadLine();
}
}
There are two tasks that we have to do in this case. One is to create a class that represents the data to be sent to PATCH the method.
public class WorkItemPostData
{
public string op;
public string path;
public string value;
}
Second is that the HttpClient does not by default support the PATCH method. For that, we have to create an extension method ref https://msdn.microsoft.com/en-IN/library/bb383977.aspx. This extension method is called PatchAsync() and has the following code:
public async static Task<HttpResponseMessage> PatchAsync(this HttpClient client, string requestUri, HttpContent content)
{
var method = new HttpMethod("PATCH");
var request = new HttpRequestMessage(method, requestUri)
{
Content = content
};
return await client.SendAsync(request);
}
Update Work Item
To update the work item, we have to use the same PATCH method but with different parameter. We have to now send the work item id as the parameter. The URL for the same is:
https://{account}.visualstudio.com/defaultcollection/_apis/wit/workitems/{id}?api-version={version}
The data that will be accepted, is in the JSON format as shown here:
[
{
"op": "replace",
"path": { string }
"value": { string or int, depending on the field }
}
]
The “op” variable has the values “add”, “replace”, “remove” and “test”. The last one i.e. “test” checks if the operation can be performed successfully or not; it does not save the work item actually.
Since we are sending a collection of fields, all of them can be updated in one round trip to server. The code will look quite similar to the method for creating work item but with few differences:
static async void UpdateWorkItem(string username, string password)
{
try
{
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.Accept.Add(new
System.Net.Http.Headers.MediaTypeWithQualityHeaderValue
("application/json-patch+json"));
client.DefaultRequestHeaders.Authorization = new
AuthenticationHeaderValue("Basic",Convert.ToBase64String(
System.Text.ASCIIEncoding.ASCII.GetBytes(
string.Format("{0}:{1}", username, password))));
WorkItemPostData wiPostData = new WorkItemPostData();
wiPostData.op = "replace";
wiPostData.path = "/fields/System.Title";
wiPostData.value = "Employee edits own profile in broser based app";
List<WorkItemPostData> wiPostDataArr = new List<WorkItemPostData> {
wiPostData };
string wiPostDataString =JsonConvert.SerializeObject(wiPostDataArr);
HttpContent wiPostDataContent = new StringContent(wiPostDataString,
Encoding.UTF8, "application/json-patch+json");
string Url =
"https://ssgsonline.visualstudio.com/DefaultCollection/
_apis/wit/workitems/1?api-version=1.0";
using (HttpResponseMessage response = client.PatchAsync(Url,
wiPostDataContent).Result)
{
response.EnsureSuccessStatusCode();
string ResponseContent = await
response.Content.ReadAsStringAsync();
}
}
}
catch(Exception ex)
{
Console.WriteLine(ex.ToString());
Console.ReadLine();
}
}
The only two differences are follows:
1. Variable “op” has value “replace” as we are replacing the title of work item
2. URL now changes so that team project name is removed and instead of work item type we are sending the ID of work item to be updated.
So far we have been using BASIC authentication which is useful for applications like Console Application, Windows Desktop (WinForms) application, Windows Store Applications, other technology applications and can also be used for ASP.NET web applications. One issue in Basic authentication is that password is sent over the network in plain text. Although we are using a secure protocol like https, it still is less secure compared to any other authentication mechanism that does not require us to send any password over the internet. One such mechanism is OAuth.
Using OAuth
OAuth is a service for authentication and authorization (delegation based authorization) which follows open standards. Open Standard is something which is freely adopted, can be freely used and implemented, and can be extended.
OAuth allows an application to act on user’s behalf to access server resources without providing their credentials. OAuth Protocol is designed to work with HTTP, also OAuth allows web users to log into third party sites using their available accounts.
Let’s understand Oauth through a simple example.
Suppose you are an end user for a website which provides contacts service, so that you can store contacts and later use them to send mails.
Hypothetically, say I wrote an application which can send greetings to email addresses. You want to use this application. Now my Application will use your mailing list to send greetings every morning as per scheduled. To do so, my application needs access to the website or the server which has your contacts stored.
The application I have developed can access the email address available on the server with your credentials. If you are willing to share those credentials, I am more than happy to accept them. But I am sure, you will never share your credentials with me or in fact with any third party application you are using.
So what’s the solution? This is the scenario where OAuth can help us. It uses one more party, the known and trusted authorization engine. Let us now step through the entire scenario.
You register with my application and then give me authority to send a request to your authentication provider, so that it will authenticate you and allow me to use some token as evidence of that. This is a one time task that you need to do.
Step 1: When you access my application, you are redirected to that authorization engine. You can get authorization using that trusted authorization engine, where I have no access to your credentials data.
Step 2: That engine vouches for your authenticity. It gives me some token as evidence of your authenticity.
Step 3: I will take that token as evidence of your authentication and send it with my request to the contacts server.
Step 4: That token will be acceptable to your contacts server as evidence of your authenticity and also your trust in my application. It will then provide your contact list to my application.
Hope this helps you with some OAuth concepts.
OAuth with Visual Studio Online
Let’s talk about the main components in an OAuth communication. So in accordance with the example used above:
- End User – You
- Resource – Contacts Server, it is your Resource Provider.
- Consumer – Application created by me, it is Resource Consumer
- OAuth Provider - Trusted OAuth Providers such as Microsoft
Let us now extend this example to include VSO which will be the Resource Provider. I have written an application that gives you a simple interface to view work items, create new work items and edit existing work items. It does so on VSO using RESTFul API of VSO. So you need to allow access to my application, so that it can access the VSO on your behalf, without providing your credentials to me or my application. My application demands OAuth Token issued by the trusted authorization engine, which in this case is Microsoft’s VSO Authorization Service.
So if my application wants to use OAuth, there are some steps needed:
1. First step is to register my application (Consumer) to the OAuth Server (Provider). Remember this provider also supports your resource (VSO).
2. When I register my application, I have to specify what all resources or features my application will access. These are also known as Scopes. I can restrict the scope to view work items only or allow view, create and edit work items. Along with these details, I also have to specify the Redirect URL. In the case of regular user accessing this application in future, this URL will be used by VSO Authorization Service to redirect the user to this URL.
3. Once I complete the above steps, my application will be registered with the provider and provider will provide me the following details:
- App ID – Unique ID for my application
- Secret Key – Also known as App Secret, will we used by application
4. Once we have got the details, we will use this information in the application program to get OAuth Token from the provider on behalf of end user, and use the token to Log In and access the VSO.
Following are the steps taken between End User, Provider and Consumer to complete the process:
a. End User browses the Consumer application (Consumer).
b. Consumer Application redirects the request to Authorization Server (Provider), passing the App Id, Scope and Redirect URL, through the User Agent (browser). In the application we used a login hyperlink which opens the page that has the following code:
var authorizeUrl = String.Format("https://app.vssps.visualstudio.com/oauth2/authorize?client_id={0}&response_type=Assertion&state=TestUser&scope=vso.work_write&redirect_uri={1}",
HttpContext.Application["AppId"].ToString(),
HttpContext.Application["RedirectUrl"].ToString());
return new RedirectResult(authorizeUrl);
c. If the user is not logged in to the Authorization Server (Provider), she/he has to Log In and Authorize Consumer application to access resources. The user also has a choice to deny the access.
d. If user allows the access, a single use authorization code is generated and given back to the Redirect URL specified by the consumer Application. In our application, AuthorizeCallback is the method that accepts mentioned callback with authorization code.
public async Task<ActionResult> AuthorizeCallback(string code, string state)
{
var token = await GetToken(code, false);
Session["TokenTimeout"] = DateTime.Now.AddSeconds(Int32.Parse(token.expires_in)); //Token is Valid for approx. 14 Mins
Session["AuthToken"] = token;
return RedirectToAction("GetWorkItems", "Home");
}
e. Now Consumer Application passes the authorization code, its own App Id and secret to the authorization server (Provider) and also the Redirect URL where user will receive the OAuth Token. (This token may be permanent or timestamp based, which is valid for a duration)
public async Task<AccessToken> GetToken(string code, bool refresh)
{
string tokenUrl = "https://app.vssps.visualstudio.com/oauth2/token";
string appSecret = HttpContext.Application["AppSecret"].ToString();
string redirectUrl = HttpContext.Application["RedirectUrl"].ToString();
string urlData = string.Empty;
if (refresh)
{
urlData = string.Format("client_assertion_type=urn:ietf:params:oauth:client-assertion-type:jwt-bearer&client_assertion={0}&grant_type=refresh_token&assertion={1}&redirect_uri={2}",
Uri.EscapeUriString(appSecret),
Uri.EscapeUriString(code),
redirectUrl);
}
else
{
urlData = string.Format("client_assertion_type=urn:ietf:params:oauth:client-assertion-type:jwt-bearer&client_assertion={0}&grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion={1}&redirect_uri={2}",
Uri.EscapeUriString(appSecret),
Uri.EscapeUriString(code),
redirectUrl);
}
string responseData = string.Empty;
AccessToken oauthToken = null;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(tokenUrl);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
using (StreamWriter sw = new StreamWriter(await request.GetRequestStreamAsync()))
{
sw.Write(urlData);
}
HttpWebResponse response = (HttpWebResponse)(await request.GetResponseAsync());
if (response.StatusCode == HttpStatusCode.OK)
{
using (StreamReader srResponseReader = new StreamReader(response.GetResponseStream()))
{
responseData = srResponseReader.ReadToEnd();
}
oauthToken = JsonConvert.DeserializeObject<AccessToken>(responseData);
}
return oauthToken;
}
f. After Validating the details, Authorization Server (Provider) returns an access token i.e. OAuth Token to Consumer Application (oauthToken from our earlier code).
g. Consumer application uses this token to access resources on behalf of the user, Consumer App has to send the token with every request to the Resource Provider, which in our case is VSO.
[HttpPost]
public ActionResult Create(EntityForCreateAndUpdate c)
{
if (DateTime.Parse(Session["TokenTimeout"].ToString()) <= DateTime.Now)
return RedirectToAction("RefreshToken", "Account", new { url = Request.Url.LocalPath });
if (ModelState.IsValid)
{
ViewBag.Op = "Created";
var t = helper.CreateWorkItemAsync(((AccessToken)Session["AuthToken"]), c.WorkItemTitle);
t.Wait();
return View("Details", t.Result);
}
else
return View();
}
h. Resource Provider validates the token and returns the resources needed by the Consumer Application. In this code, we created a new work item in our team project.
These steps will be easier to understand using the following visual:
Some points to remember:
- The current version of OAuth is OAuth 2.0
- OAuth 2.0 Tokens can be shared only on secured channel i.e. HTTPS. It relies on SSAL to provide encryption.
- Facebook, Google, Twitter and Microsoft Servers are some of the OAuth Providers.
Conclusion
Visual Studio Online and TFS 2015 (RC Onwards) provides RESTful APIs that allow you to extend the functionality of VSO from your apps and services. The possibilities of integration via client applications using any platform or device are endless; right from iOS, Node.js, Android and our very own Windows.
Download the entire source code from GitHub at bit.ly/dncm19-vsotfsrestapi
This article has been editorially reviewed by Suprotim Agarwal.
C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.
We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).
Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.
Click here to Explore the Table of Contents or Download Sample Chapters!
Was this article worth reading? Share it with fellow developers too. Thanks!
Subodh is a Trainer and consultant on Azure DevOps and Scrum. He has an experience of over 33 years in team management, training, consulting, sales, production, software development and deployment. He is an engineer from Pune University and has done his post-graduation from IIT, Madras. He is a Microsoft Most Valuable Professional (MVP) - Developer Technologies (Azure DevOps), Microsoft Certified Trainer (MCT), Microsoft Certified Azure DevOps Engineer Expert, Professional Scrum Developer and Professional Scrum Master (II). He has conducted more than 300 corporate trainings on Microsoft technologies in India, USA, Malaysia, Australia, New Zealand, Singapore, UAE, Philippines and Sri Lanka. He has also completed over 50 consulting assignments - some of which included entire Azure DevOps implementation for the organizations.
He has authored more than 85 tutorials on Azure DevOps, Scrum, TFS and VS ALM which are published on
www.dotnetcurry.com.Subodh is a regular speaker at Microsoft events including Partner Leadership Conclave.You can connect with him on
LinkedIn .