Bending the Windows Azure Media Services–H.264 Baseline profile

Disclaimer: What I will describe here is not officially supported by Microsoft and by Windows Azure Media Services. This means that if task fails you cannot open support ticket, nor you can complain. I discovered these hidden feature by digging deeply into the platform. Use the code and task preset at your own risk and responsibility. And note that what works now, may not work tomorrow.

Exploring the boundaries of Windows Azure Media Services (WAMS), and following questions on StackOverflow and respective MSDN Forums, it appears that WAMS has previously supported H.264 Baseline Profile and have had a task preset for Baseline Profile. But now it only has Main Profile and High Profile task presets. And because the official documentation says that Baseline Profile is supported output format, I don’t see anything wrong in exploring how to achieve that.

So what can we do, to encode a video into H.264 baseline profile if we really want? Well, use the following Task Preset at your own will (and risk ):

<?xml version="1.0" encoding="utf-16"?>
<!--Created with Expression Encoder version 4.0.4276.0-->
<Preset
  Version="4.0">
  <Job />
  <MediaFile
    WindowsMediaProfileLanguage="en-US"
    VideoResizeMode="Letterbox">
    <OutputFormat>
      <MP4OutputFormat
        StreamCompatibility="Standard">
        <VideoProfile>
          <BaselineH264VideoProfile
            RDOptimizationMode="Speed"
            HadamardTransform="False"
            SubBlockMotionSearchMode="Speed"
            MultiReferenceMotionSearchMode="Speed"
            ReferenceBFrames="True"
            AdaptiveBFrames="True"
            SceneChangeDetector="True"
            FastIntraDecisions="False"
            FastInterDecisions="False"
            SubPixelMode="Quarter"
            SliceCount="0"
            KeyFrameDistance="00:00:05"
            InLoopFilter="True"
            MEPartitionLevel="EightByEight"
            ReferenceFrames="4"
            SearchRange="32"
            AutoFit="True"
            Force16Pixels="False"
            FrameRate="0"
            SeparateFilesPerStream="True"
            SmoothStreaming="False"
            NumberOfEncoderThreads="0">
            <Streams
              AutoSize="False"
              FreezeSort="False">
              <StreamInfo>
                <Bitrate>
                  <ConstantBitrate
                    Bitrate="4000"
                    IsTwoPass="False"
                    BufferWindow="00:00:04" />
                </Bitrate>
              </StreamInfo>
            </Streams>
          </BaselineH264VideoProfile>
        </VideoProfile>
        <AudioProfile>
          <AacAudioProfile
            Level="AacLC"
            Codec="AAC"
            Channels="2"
            BitsPerSample="16"
            SamplesPerSecond="44100">
            <Bitrate>
              <ConstantBitrate
                Bitrate="160"
                IsTwoPass="False"
                BufferWindow="00:00:00" />
            </Bitrate>
          </AacAudioProfile>
        </AudioProfile>
      </MP4OutputFormat>
    </OutputFormat>
  </MediaFile>
</Preset>

You can quickly check whether it works for you by using the RunTask command line, part of the MediaServicesCommandLineTools project. The H264_BaselineProfile.xml is provided for reference in the etc folder of the project. You can tweak and Audio and Video bitrates at your will by editing the XML.

Federated Authentication–Mobile Login Page for Microsoft Live Id

Say you are developing a web site, which will have desktop users, mobile users, all kind of users. Because you respect your users, you let them login to your site using their existing credentials. One of which happens for be Microsoft Account (or formerly known as Microsoft Live ID). Also, because you really enjoy the Windows Azure platform and the fact that Azure Access Control Service is totally free with no catch, you implemented your federated login using Azure ACS. You also implemented a custom login page for you users.

Now you noticed that Microsoft Account does not recognize mobile users 100% and you have better logic for determining mobile user agents. You also want to forcibly redirect your mobile user to the mobile login page for Microsoft Account. But how?

Well, since you already implemented a custom login page, you already know what this URL is:

https://[namespace].accesscontrol.windows.net/v2/metadata/IdentityProviders.js?protocol=wsfederation&realm=[realm]&reply_to=[reply_to]&context=&request_id=&version=1.0&callback=

This is the URL where you get the JSON feed of registered Identity Providers for your relying party application. When you retrieve it, you have LoginUrl for Live ID looking similar to this one:

https://login.live.com/login.srf?wa=wsignin1.0&wtrealm=https%3a%2f%2faccesscontrol.windows.net%2f&wreply=https%3a%2f%2f[namespace].accesscontrol.windows.net%2fv2%2fwsfederation&wp=MBI_FED_SSL&wctx=[encrypted]

Now, you can one more parameter to the query string to force a very lightweight (mobile) login page for Microsoft Account. This parameter is pcexp and the value should be false. So now your LoginUrl for Microsoft Account (Live ID) will look similar to this one:

https://login.live.com/login.srf?wa=wsignin1.0&wtrealm=https%3a%2f%2faccesscontrol.windows.net%2f&wreply=https%3a%2f%2f[namespace].accesscontrol.windows.net%2fv2%2fwsfederation&wp=MBI_FED_SSL&wctx=[encrypted]&pcexp=false

That’s perfect! It works! Thanks!

But.. but you also have a WML version of your site. And you recognize and respect these user agents too. Well, there is solution to this issue too. The solution is to replace the whole domain and login page, but keep the query string intact. So, if the original login Url is this:

https://login.live.com/login.srf?wa=wsignin1.0&wtrealm=https%3a%2f%2faccesscontrol.windows.net%2f&wreply=https%3a%2f%2f[namespace].accesscontrol.windows.net%2fv2%2fwsfederation&wp=MBI_FED_SSL&wctx=[encrypted]

Replace login.live.com/login.srf? with mid.live.com/si/login.aspx?. The result is:

https://mid.live.com/si/login.aspx?wa=wsignin1.0&wtrealm=https%3a%2f%2faccesscontrol.windows.net%2f&wreply=https%3a%2f%2f[namespace].accesscontrol.windows.net%2fv2%2fwsfederation&wp=MBI_FED_SSL&wctx=[encrypted]

Done. Happy coding!

Please respect your users and their existing online identities! Do not ask them to create new usernames/password if they don’t explicitly want to!

Bending the Azure Media Services – clip or trim your media files

Disclaimer: What I will describe here is not officially supported by Microsoft and by Windows Azure Media Services. This means that if task fails you cannot open support ticket, nor you can complain. I discovered these hidden feature by digging deeply into the platform. Use the code and task preset at your own risk and responsibility. And note that what works now, may not work tomorrow.

So, we have Windows Azure Media Services, which can transcode (convert from one video/audio format to another), package and deliver content. How about more advanced operations, such as clipping or trimming. I want, let’s say to cut off first 10 seconds of my video. And the last 5 seconds. Can I do it with Windows Azure Media Services ? Yes I can, today (5 April 2013).

The easiest way to start with Media Services is by using the MediaServicesCommandLineTools project from GitHub. It has very neat program – RunTask. It expects two parameters: partial (last N characters) Asset Id and path to task preset. It will then display a list of available Media Processors to execute the task with. You chose the Media Processor and you are done!

So what task preset is for Clipping or Trimming? You will not find that type of task on the list of Task Presets for Azure Media Services. But you will find a couple of interesting task presets in the MediaServicesCommandLineTools project under the etc folder. Lets take look at the Clips.xml:

<?xml version="1.0" encoding="utf-16"?>
<!--Created with Expression Encoder version 4.0.4276.0-->
<Preset
  Version="4.0">
  <Job />
  <MediaFile>
    <Sources>
      <Source
        AudioStreamIndex="0">
        <Clips>
          <Clip
            StartTime="00:00:04"
            EndTime="00:00:10" />
        </Clips>
      </Source>
    </Sources>
  </MediaFile>
</Preset>

It is a very simple XML file with two attribute values that are interesting for us. Namely StartTime and EndTime. These attributes define points in time where to start clipping and there to end it. With the given settings (StartTime: 00:00:04, EndTime: 00:00:10) the result media asset will be a video clip with length of 6 seconds which starts at the 4th second of the original clip and ends at the 10th second of the original.

As can also see, I haven’t removed an important comment in the XML – "Created with Expression Encoder version 4.0.4276.0". Yes, I used Expression Encoder 4 Pro to create a custom job preset. You can try that too!

Tune on for more “media services bending tips”.

Identity Federation and Sign-Out

We live in 21st century, don’t we! I am a firm believer that from now on no user shall ever create a new username/password combination again. Ever! There are already enough existing online identity providers – such as Google, Yahoo, Facebook, Microsoft Account (formerly know as Live ID), Office365, OpenId, Twitter, LinkedIn, national identity providers such as NemID in Denmark, and so on, and so on.

I do believe that every single internet user has profile with at least one of these Identity Providers. And if you, dear reader, do not have any existing online profile, please do leave a comment, but be honest!

All of the developers, architects, decision makers, by all means we shall respect this fact!

I do respect it. In every single project I face I do my best to convince decision makers that it is always better to respect users and give them opportunity to use an existing online identity when there is a need to protect some parts of the application we develop. And way I do it, is by evangelizing Windows Azure Access Control Service, which is now part of Windows Azure Active Directory. I’ve written a number of articles on that subject (Introduction to Claims, Securing ASMX web services with Claims and SWT tokens, Online Identity Management via Windows Azure ACS, Unified Identity for Web Apps – the easy way,  Creating custom login page for Federated Authentication with Windows Azure ACS)  and yet I see people unaware of such service and do want to implement their own ASP.NET Membership Provider.

I also see people willing to embrace the service. They go their way through the Identity and Access Tool for Visual Studio 2012 and create their first web application with federated login. While the tool is great in its core – by doing what it is supposed to do, it yet hides a lot of process information and does not give you a complete log of what it did. There is one very neat option – create a local Controller with custom Login View:

While this option is great, it misses one very core feature – the log off feature! So you happily created your federated sign in, configured Identity Providers, etc. Now you login to test. Next you click the default [log off] link in your web app. And … you are still logged in! What the hack? You will ask.

Well, when using Federated Log-in, we also have to use a Federated Log-Off (or Sign Out). For this, we have to edit our default log-off method and add one single line. Imagine the default Log Off method looks like:

[HttpPost] 
[ValidateAntiForgeryToken] 
public ActionResult LogOff() 
{ 
    WebSecurity.Logout(); 
    return RedirectToAction("Index", "Home"); 
}

We only have to add:

    FederatedAuthentication.WSFederationAuthenticationModule.SignOut();

So the final Log Off will be like this:

[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult LogOff()
{
    WebSecurity.Logout();
    FederatedAuthentication.WSFederationAuthenticationModule.SignOut();
    return RedirectToAction("Index", "Home");
}

And voliah! We are done. Now we can also successfully log off the web application. Note that FederatedAuthentication type is part of the System.IdentityModel.Services assembly and you must add a reference to it.

Couple of things to pay attention to and remember:

• Identity And Access menu item (result of Identity and Access tool installation) will only be visible for web projects targeting 4.5 Framework!
  
• You have to reference System.IdentityModel.XX (4.0.0.0) assemblies and not Microsoft.IdentityModel.XX (3.5.0.0) assemblies in your project. Failing to do so, you may see unexpected behavior and even errors and failures. Very often if you upgrade your project from .NET Framework prior 4.5 to .NET Framework 4.5 there are references left to Microsoft.IdentityModel.XX – remove them explicitly!
  
• Do respect your users’ existing online identities! The users will respect you, too!

A journey with Windows Azure Media Services–Smooth Streaming, HLS

Back in January Scott Gu announced the official release of Windows Azure Media Services. It is amazing platform that was out in the wild (as a CTP, or Community Technology Preview) for less then an year. Before it was RTW, I created a small project to demo out its functionality. The source code is public on GitHub and the live site is public on Azure Web Sites. I actually linked my GitHub repo with the Website on Azure so that every time I push to the Master branch, I got a new deployment on the WebSite. Pretty neat!

At its current state Windows Azure Media Services  does support the VOD (or Video On Demand) scenario only. Meaning that you can upload your content (also known as ingest), convert it into various formats, and deliver to audience on demand. What you cannot currently do is publish Live Streaming – i.e. from your Web Cam, or from your Studio.

This blog post will provide no direct code samples. Rather then code samples, my aim is to outline the valid workflows for achieving different goals. For code samples you can take a look at the official getting started guide, my code with web project, or the MediaServicesCommandLineTools project on GitHub, which I also contribute to.

With the current proposition from Azure Media Services you can encode your media assets into ISO-MP4 / H.264 (AVC) video with AAC-LC Audio, Smooth Streaming format to deliver greatest experience to your users, or even to Apple HTTP Live Streaming format (or just HLS). Everything from the comfort of your chair at home or in the office. Without the big overspend in expensive hardware. Getting the results however may be tricky sometime, and the platform does not help you with very detailed error messages (which I hope will change in the very near future).

You can achieve different tasks (goals) in different ways sometime. Windows Azure Media Services currently works with 4 Media Processors:

• Windows Azure Media Encryptor
• Windows Azure Media Encoder
• Windows Azure Media Packager
• Storage Decryption

When you want to complete some task you always provide a task preset and a media processor which will complete the given task. It is really important to pay attention to this detail, because giving a task preset to the wrong processor will end up in error and task failure.

So, how to get (create/encode to) a Smooth Streaming Content?

Given we have an MP4 video source - H.264 (AVC) Video Codec + AAC-LC Audio Codec. The best will be if we have multiple MP4 files representing same content but with different bitrates. Now we can use the Windows Azure Media Packager and the MP4 To Smooth Streams task preset.

If we don’t have MP4 source, but we have any other supported import format (unfortunately MOV is not a supported format), we can use Windows Azure Media Encoder to transcode our media into either an MP4 (H.264) single file, or directly into Smooth Streaming Source. Here is a full list of a short-named task presets that can be used with Windows Azure Media Encoder. To directly create a Smooth Streaming asset, we can use any of the VC1 Smooth Streaming XXX task presets, or any of the H264 Smooth Streaming XXX task presets. That will generate a Smooth Streaming asset encoded with either VC-1 Video profile, or H.264(AVC) Video Codec.

OK, how about Apple HTTP Live Streaming (or HLS)?

Well, Apple HLS is similar to Smooth Streaming. However, there is a small detail, it only supports H.264 Video codec! The most standard way of creating Apple HLS asset is by using Windows Azure Media Packager and the XML task preset for “Convert Smooth Streams to Apple HTTP Live Streams”. Please take a note on the media processor – it is the Windows Azure Media Packager. This also will accept an input asset to be valid Smooth Streaming Asset encoded with H.264 (AVC) video codec! Do not forget that you could have created Smooth Streams with VC-1 Video Profile codec, which are totally valid and running Smooth Streams, but they will fail to convert to Apple HTTP Live Streams.

Hm, can’t we get all-in-one?

I mean, can’t I have a single media asset and deliver either Apple HTTP Live Streams or Smooth Streams, depending on my client? Sure we can. However this is CPU intensive process. It is called “dynamic packaging”. The source must be a multi-bitrate MP4 asset. This one consists of multiple MP4 files of same content with different bitrates. And it requires an on-demand streaming reserved units from Media Services. You can read more about dynamic packaging here.