Viewpoint : Silverlight

How to use Silverlight Pivot Viewer with SharePoint Lists

Steve Samnadda — Thu, 01 Sep 2011 20:20:00 GMT

The Internet provides us with an ever-increasing amount of information. Despite this fact, most people are still not very good at multi-tasking. Unlike a computer, human context-switching wastes a tremendous amount of time and energy. As a result we have become better at filtering out the information we choose to spend our time on. For instance, a typical person only revisits about 6-7 websites on a daily basis. And when browsing the web a typical user spends less than one minute on each site.

For that reason, when designing websites with large amounts of data it is necessary to present that information in such a way that users can quickly find and consume the items they are looking for. (Remember you have less than 60 seconds!)

Microsoft's Pivot Viewer Control uses Silverlight to provide faceted searches in a fluid and visually appealing manner. This tool has tremondous potential for running searches of any kind. Deep Zoom Image technology allows high-resolution images to represent each search item. And faceted search items can be selected, sort, and reshaped in very quick, intuitive manner.

There are several great examples on the web already using Microsoft Pivot Viewer (Netflix, Hitched, Hard Rock, SFMOMA). Also, there are class libraries (Deep Zoom Tools, Pivot Server Tools) available to dynamically generate the images and supporting xml needed to feed this control. These tools run well on a computer or off an normal ASP.NET application with direct access to the hard drive. But how easy would it be to integrate this control with Microsoft's premier platform for business intelligence, document management, and collaboration: SharePoint. The answer was not the easy, especially generating Deep Zoom Images from SharePoint. Once we figured out how to map these files the result was rock-solid search tool for our team sites.

For our purposes, we wanted to use the Pivot Viewer to search team sites based on metadata associated with that site’s purpose. Our metadata was specific to the project: Country, Department, Created Date, Keywords.

Screenshots below:

There were four main components used to make the Pivot Viewer work with SharePoint

Landing Page Site – This site is the parent to each or our team sites. It contains the following lists and pages:
- “Sites” list – This list is used to auto-create team sites and track metadata within them
- “Site Images” list – Contains images associated with each team site
- “XAP library” – Document library used to store PivotViewer.xap.
- Search.aspx - Search page with a Content Editor with Silverlight <object> tag
Sites.cxml – This is an XML file is passed in as a parameter to PivotViewer.xap. It is generated from the "Sites" list and contains a listing of all our team sites and the metadata or “facets” associated with each team site. An event receiver is attached to the “Sites” list to generate this file any time a metadata column is updated.
dzc_output.xml – A catalog of all images uploaded to “Site Images” list. Each image is assigned an id# that is referenced by an item inside Site.cxml.
Deep Zoom Tiles – Each image added to “Site Images” fires an event receiver to process the photo into Deep Zoom Tiles. Deep zoom tiles allow for a quick initial load time and the ability to quickly zoom very deep into an image. This is achieved by creating a ‘pyramid’ of images where each level of the pyramid is written to a folder and contains successively more slices of the photos at higher zoom levels.

So between the “Sites” metadata and “Site Images” lists and their respective event handlers there is everything needed to generate a Pivot View. There is a specific folder structure that these files should be contained in (which is all documented). Instead of explaining that, I will go straight into the technical challenges involved in generating these files programmatically.

Overview: Deep Zoom Tools (documentation)

DeepZoomTools handles writing dzc_output.xml and each corresponding image tile using the ImageCreator and CollectionCreator classes. The biggest challenge was getting DeepZoomTools.dll to write file to an SPFile stream instead of to the hard drive. The filenames would always correspond to local path while we wanted to write to a SharePoint folder. Luckily, their are many events we can attach to before and after a file stream request is being made. At this point we can supplant the filestream with a memorystream of our own making

First Issue - Switching the stream dzc_output_images

We overrode the OutputNeeded event to translate the filename, ensure the proper SharePoint folder/file exists, and create a MemoryStream to write.

Then overrode the OutputCompleted event to stuff the resulting file from the MemoryStream into a SPFile.

The magic happens with PrepFilename method which converts a local path file to the Sharepoint file we want:

Examples

source

file:///c:/windows/system32/inetsrv/dzc_output_images/ourimage.xml

destination

/sites/Projects/DeepZoomImages/SiteImages/dzc_output_images/ourimage.xml

source

c:\windows\system32\inetsrv\dzc_output_images\ourimage_files\0\0_0.png

destination

/sites/sis/projects/deepzoomimages/projectimages/dzc_output_images/ourimage_files/0/0_0.png

Second Issue - Switching the stream dzc_output.xml

The dzc_output.xml is a catalog of all the images created during the last step. It was easy enough to tell it what to generate but when it actually came time to write it wasn't as simple as the solution in the first issue. This time the Collection Creator Wrapper was entirely disposing of the file before we had a chance to write to it. To solve this issue we overrode the MemoryStream class, attached an event to the Dispose event and handled it in our code to get the contents of the XML. Problem solved!

Third Issue - Creating the Faceted Search Collection

Oh this wasn't really an issue. Thanks to PivotServerTools, we just passed in the data and it did all the work for us. OK, there may have been some more wiring needed, but we can't give up all our secrets. ;)

Kidding! If anyone would like some more details on implementation just shoot us an email!

Cheers.

Getting Silverlight Drag And Drop Support For FireFox 5 On A Mac

Chris Domino — Wed, 10 Aug 2011 22:33:00 GMT

I've been working on a massive cross browser, cross platform Silverlight application for the last year or so. One of the major components is file drag and drop (heretofore D&D, not to of course be confused with Dungeons and Dragons). Getting this to work on Windows is trivial, and there are myriad resources out there to get you started. However, over on the Mac, things were much more difficult.

FireFox wasn't actually too bad, but Safari required us to wire up some JavaScript to help convince the browser to pass the dropped file's bits along to Silverlight. But all-in-all, when we were done, I felt that it was still pretty amazing that we had D&D working in Silverlight cross-platform with what turned out to be a lot less effort than I expected.

Our app went into beta, and the Mac users immediately started logging bugs about D&D not working on FireFox. So we dusted off our test Mac, and everything seemed fine. The disconnect turned out to be, after some quick investigation, a version conflict. We were running one of the last builds of FireFox 3; the users were all on 5 (if you recall, version 5 came out rather quickly after 4 shipped).

So we upgraded, and quickly saw the problem: although drag still worked, drop was completely dead. FireFox version 4 and up had broken support for Silverlight D&D (or, technically, just the second D). And I use the term "support" loosely because official backing from Microsoft wasn't really there. (It was kind of like Silverlight 4 "not supporting" Chrome, although it works just fine.)

What I want to discuss here is how to get Silverlight D&D working on Macs running FireFox 5.

We went the full nine yards when we built our Silverlight D&D infrastructure: using a Silverlight Behavior for the infrastructure that allowed us to attach the functionality to any UIElement. It used a VisualStateManager to provide cues to the user that a dragged file could be dropped at that particular area (by "lighting" up the text and animating a glowing DropShadow around the boarder). Additionally, we had a cursor control that hid the mouse pointer and showed different images in its place that provided additional visual indications where D&D was enabled. Finally, a static helper class provided common functionality, such as flags to track if we were dragging and if a drop action, based on the current cursor location, was possible.

Check out my colleague Jonathan Rupp's post on how he got us this far. I'm going to take the next step here and get our Silverlight D&D logic working for FireFox 5 on a Mac. The basic approach is to leverage the fact that drag still works, and use a combination of HTML 5, the Silverlight HTML bridge, and jQuery to basically "fake" a drop. Do read Jonathan's post, as I'll be referring to the code he presents.

The idea is to use Silverlight's ability to still subscribe to the drag events raised by FireFox 5 on a Mac to position a transparent div over the drop zone, wire up HTML 5 drop events on it, handle the file processing in JavaScript, and then pass the raw data back to Silverlight. I know that I just presented like half a dozen different technologies in that one sentence, so let's break the procedure down step by step.

The first thing to do is create a div in the ASPX page that hosts our Silverlight control. This is what we'll be using as our drop surface; there's nothing too special about it (yet):



	<div id="divDropSurface" style="background: transparent; z-index: 50; position: absolute; top: 0px; left: 0px; width: 0px; height: 0px;">

As you can tell, the styling implies that this div will be absolutely positioned and dimensioned to cover a certain portion of the screen. One thing to note: this technique will only work on a Mac, where Silverlight automatically sets the windowless mode to true, allowing the Silverlight region to participate in HTML Z-indexing. If you set windowless to true on a PC, you'll break D&DN altogether, as well as introduce some performance hits. I wrote more about this here.

Next we need to hook up the HTML 5 D&D events on our drop surface. Call the following method in jQuery's document ready event:



	function HookFFDropEvents() { //ff on mac if (window.navigator.userAgent.indexOf("Mac OS") >= 0 && $.browser.mozilla) { //get drop surface var ds = document.getElementById("divDropSurface"); //hook D&D events ds.addEventListener("drop", FFDrop, false); ds.addEventListener("dragenter", FFDragEnter, false); ds.addEventListener("dragleave", FFDragLeave, false); ds.addEventListener("dragover", FFClearEvent, false); } }

We'll talk about the FFDragLeave and FFDrop methods later. FFDragEnter is not used (but will be, I'm sure, if FireFox 6 breaks drag as well). FFClearEvent is the standard JavaScript that blocks the browser's default handling of a file drop and allows the code on the page to handle it.



	function FFClearEvent(e) { //override the browser's default behavior for file drops e.stopPropagation(); e.preventDefault(); }

The next step is to position this div directly on top of a particular area of your Silverlight control when a drag is detected. (Recall that in Silverlight, you hook drag events on a particular UIElement, not the entire application.) Our Silverlight D&D behavior keeps a reference to the UIElement that it's supporting. So when a file is being dragged over a valid drop zone, an event is fired that basically updates the VisualStateManager for that UIElement. What I added to this was logic to get the HTML coordinates of this UIElement, and position our drop surface div over it.



	//determine if we are on mac on FF5 if (FileDragDropHelper.IsMacFF5) { //get the drop target and the drop surfce Control target = this.GetCurrentDNDBehavior().Target; HtmlElement element = HtmlPage.Document.GetElementById("divDropSurface"); if (element != null) { //set flag to indicate that we're currently dragging FileDragDropHelper.IsHovered = true; //position drop surface over drop target Point position = target.TransformToVisual(null).Transform(new Point()); element.SetStyleAttribute("top", string.Concat(position.Y, "px")); element.SetStyleAttribute("left", string.Concat(position.X, "px")); element.SetStyleAttribute("width", string.Concat(target.ActualWidth, "px")); element.SetStyleAttribute("height", string.Concat(target.ActualHeight, "px")); } }

The flag in Line #2 is maintained in the aforementioned D&D helper utility. Line #5 is another helper method that takes in the current mouse position (gotten from the drag event), gets the UIElement at that location, and grabs the associated behavior (made possible by the gluey nature of attached properties). Everything else is pretty straight forward, utilizing the beauty of the Silverlight HTML bridge.

Now if the user were to release the mouse button, the drop would happen. But before we drop, we need to handle the case were the user drags off the control without dropping. This interaction is important, as it not only mimics what our D&D behavior is doing for us automatically in other environments, but can be reused upon a drop (since the UI needs to be reset properly).

First, we create a hidden HTML button that will be used to facilitate communication from JavaScript to Silverlight.



	<input type="button" id="hidFileDropInfo" style="display: none;" />

Then we hook its click event in Silverlight (as part of the behavior).



	HtmlPage.Document.GetElementById("hidFileDropInfo").AttachEvent("click", HandleFileReceived);

We'll see what HandleFileReceived looks like in a bit. First, let's revisit the FFDragLeave JavaScript method.



	function FFDragLeave(e) { //reset our button, telling silverlight to update the UI that we're no longer over a drop surface $("#hidFileDropInfo").val("CLEAR").click(); }

We're using the Silverlight HTML bridge and jQuery chaining to set the value of the hidden HTML button to "CLEAR" and then click it. That click event will be handled in Silverlight by the aforementioned HandleFileReceived method, which we're still not quite ready to discuss. First, let's talk about the drop workflow.

While the drop surface is positioned over the drop zone, Silverlight won't be receiving any mouse input. But that's okay, since the visual state won't need to change until the drop surface receives either a drag leave or a drop event. We've covered what happens in the former case, so without further ado, let's talk about drop.

Here's the FFDrop method in all its glory:



	//globals var _xml; var _fileCounter; function FFDrop(e) { //override the browser's default behavior for file drops FFClearEvent(e); //get files var files = e.dataTransfer.files; if (typeof files == "undefined" \|\| files.length == 0) return; //initialize global variables _xml = "<files>"; _fileCounter = files.length; //process each file for (var n = 0; n < files.length; n++) ProcessFile(files[n]); }

The idea is to get the HTML 5 drop event metadata, process each file, and build up some XML that acts as a data contract between JavaScript and Silverlight. Lines #13 and #14 initialize the _xml global variable to the root node of our XML markup, and _fileCounter to the number of dropped files.

Most of the magic happens in ProcessFile, which follows.



	function ProcessFile(file) { //create reader var reader = new FileReader(); //client side error handling reader.onerror = function (error) { //determine error switch (error.target.error.code) { //show appropriate error case 1: alert(file.name + " was not found."); break; case 2: alert(file.name + " has been modified since it was dropped."); break; case 3: alert(file.name + " has been cancelled."); break; case 4: alert(file.name + " could not be read."); break; case 5: alert(file.name + " is too large."); break; } } //read data reader.readAsDataURL(file); reader.onloadend = function (dropped) { //get raw data from file var data = dropped.target.result; if (data.length > 128) { //build xml representation of each file _xml = _xml + "<file><name>" + file.name + "</name><size>" + file.size + "</size><data>" + data + "</data></file>"; //decrement the counter... _fileCounter--; //...and when there are no more files to upload... if (_fileCounter == 0) { //...close the xml... _xml = _xml + "</files>"; //...and send it to silverlight $("#hidFileDropInfo").val(_xml).click(); } } else { //error alert(file.name + " has invalid data. It may be corrupted."); } } }

In Line #4, we new up an HTML 5 FileReader, which when fed the metadata of a drop event, gives us a nice OO representation of a file. The error event is hooked in Line #6, and processing is done according W3C protocol. Line #'s 30 and 31 load the file; the fact that this is an asynchronous process is what forces us to use global variables. In Line #38, we build the XML representation of the file array we're going to be passing to Silverlight. The _fileCounter variable is decremented in Line #40 each time a file is successfully collected, so that the check in Line #42 can determine if all asynchronous operations have completed, the XML can be capped off, and finally be sent to Silverlight via the aforementioned bridge-and-change method in Line #47.

So we have both our drag leave and our drop methods setting the value of a hidden HTML button and then clicking it. The click event is handled by Silverlight, and deals with both cases.



	private void HandleFileReceived(object sender, HtmlEventArgs args) { //initialization FileDragDropHelper.IsHovered = false; string value = HtmlPage.Document.GetElementById("hidFileDropInfo").GetProperty("value").ToString(); //get value if (string.IsNullOrEmpty(value) \|\| value.Equals("CLEAR")) { //process a reset HtmlElement element = HtmlPage.Document.GetElementById("divDropSurface"); if (element != null) { //reset drop surface element.SetStyleAttribute("top", "0px"); element.SetStyleAttribute("left", "0px"); element.SetStyleAttribute("width", "0px"); element.SetStyleAttribute("height", "0px"); } } else { try { //build xml doc to hold D&D raw data XDocument doc = XDocument.Load(new StringReader(value)); List<FileDragDropHelper.DroppedFileWrapper> files = new List<FileDragDropHelper.DroppedFileWrapper>(); //iterate all dropped files... foreach (XElement node in doc.Root.Nodes()) { //...and poor-man deserialize them List<XElement> nodes = node.Nodes().Cast<XElement>().ToList(); string name = nodes[0].Value; long size = Convert.ToInt64(nodes[1].Value); //strip heading off of base64 file header data string data = nodes[2].Value; data = data.Substring(data.IndexOfThatDoesntBreakMacs(";base64,") + 8); //collect files files.Add(new FileDragDropHelper.DroppedFileWrapper(name, size, Convert.FromBase64String(data))); } //pass file along to behavior var b = this.GetCurrentDNDBehavior(); if (b != null) b.OnDrop(files.ToArray()); } catch (Exception ex) { //error MessageBox.Show(ex.ToString()); } finally { //pass drag leave to the rest of the D&D infrastructure this.DoLeave(); FileDragDropHelper.DoHangledDragLeave(); } } }

Once again, Line #41 gets the D&D behavior, and fires the drop event (with the files) on the target UIElement. Line #7 determines if this is a "CLEAR" event (in which case the drop surface is hidden) or if this is an actual drop. Drag leave still works normally in FireFox 5 on a Mac, so the behavior itself can update the VisualStateManager and reset any "dragging" flags in either case. The rest is not too exciting: Line #31 builds an XDocument from the raw XML and iterates the child "file" nodes. It then pulls out the file name and size properties, as well as converts the data from a base64 string to a byte array, and feeds it to our DroppedFileWrapper DTO (which is a wrapper around a FileInfo object).

If time allowed, this would have all been baked into our behavior and done proper: implementing XML deserialization for the file bits, not using buttons to pass data around, and generally less piece-wising these technologies together. But with the timeline I had, the complexity of the solution, and the existing infrastructure it had to fit into, (which itself was already quite complex due to cross-browser, cross-platform Silverlight D&D support) this is how it was born.

To summarize, if you're not using a behavior or any other existing D&D implementation, here's what you need to do to get Silverlight D&D working in FireFox 5 on a Mac:

Create a "hidden" (no length or width and positioned at 0,0) absolutely-positioned div to act as a drop surface.
Wire up the HTML 5 D&D events on the drop surface.
Position it over the "drop zone" (the Silverlight UIElement to be dropped onto) when it's dragged over.
Use HTML 5 JavaScript to process the drop event and read each file.
Pass the raw data to Silverlight.
"Hide" the drop surface (remove its length and width and reposition to 0,0) after the drop or upon the drop surface's drag leave event.

And there you have it: Silverlight drag and drop in FireFox 5 on a Mac. Have fun!

Using oEmbed To Generate Thumbnails From Content On YouTube And Other Social Media Sites

Chris Domino — Sun, 10 Apr 2011 13:21:00 GMT

In my second-to-last post, I talked about showing Flash superimposed over Silverlight without the need to enable IsWindowless mode on the plugin. Something I foreshadowed but didn't discuss was where the Flash content came from, so I wanted to continue along the topic and elucidate this detail.

There is a new protocol out there called oEmbed. This spec, supported by all the major media providers out there (YouTube, Flickr, Hulu, Vimeo, etc.) is a REST-ish API that takes in a URL to a video or image, and responds with an XML or JSON object containing all the metadata you'll need to integrate this content into your application.

It's actually really easy. Check out the link above for the details, but all you need to do is construct a URL to the provider's specification (all are a bit different), and serialize the result into a .NET object. You can then persist it in a database or bind it to your UI. Here's an example of what a YouTube request will be (start by loading a page and copy-and-pasting the URL; I'm using "http://www.youtube.com/watch?v=6E3znZoFnN8"):

http://www.youtube.com/oembed?url=http://www.youtube.com/watch?v=6E3znZoFnN8&format=xml

The response will be a nice XML document containing several nice pieces of metadata:

Title
Flash object tag's HTML
Thumbnail Image URL
Height (both of plugin and thumbnail)
Width (both of plugin and thumbnail)

And much more. If you prefer JSON (which is faster to deserialize than XML), the APIs will have a way to specify the format for the response, either as a query string parameter or a part of the path. As you can see, you don't even need to encode the raw URL!

Some weirdness I ran into was the concept of "protected videos." These are proprietary pieces of content on the web that the author does not want displayed anywhere else except for where it was originally published. My guess is that it has something to do with copyrights or distribution or promotions or advertisements; money in other words.

It's difficult to detect this situation from the response, but an oEmbed request for protected content will fail; make sure you have some good error handling around this. I've found your best shot is to use a Try/Catch block, catching both WebExceptions and standard Exceptions. My fail-case contingency flow was to still persist the URL, display a message to the user (in lieu of the Flash content) when it was needed in my app, and provide a link to pop the video up in a new Window navigated to its home on YouTube or wherever.

Here is an example of a wrapper method I used to take in a URL and return the raw response from a provider's oEmbed endpoint:



	public string GetRawoEmbed(string url) { try { //initialization WebClient wc = new WebClient(); //get base url Uri uri = new Uri(url, UriKind.Absolute); string baseUri = string.Format("{0}://{1}", uri.Scheme, uri.Authority).ToLower().Replace("www.", string.Empty); wc.BaseAddress = baseUri; //get provider oEmbedProvider provider = this.ObjectContext.oEmbedProviders.Where(o => o.BaseURL.ToLower().Replace("www.", string.Empty).Equals(baseUri)).FirstOrDefault(); if (provider != null) { //get data return ASCIIEncoding.ASCII.GetString(wc.DownloadData(string.Concat(provider.oEmbedFormattedURL, url))); } else { //not supported return "UNSUPPORTED"; } } catch (WebException) { //protected return "PROTECTED"; } catch (Exception ex) { //error return "ERROR"; } }

Notes:

Line #12 is just a call into our database where we store the "supported" oEmbed providers. This is a "system" table in our schema that has the URL format string, friendly name, etc. for each endpoint.
Line #16 uses standard ASCII encoding to get the raw data as a string out of the response bits.
Errors: I know that returning strings like "ERROR" or "UNSUPPORTED" are not the most elegant way to allow your application to fail gracefully. But for our situation, this whole feature was a last minute enhancement, and we had no choice but to cut corners. However I feel that it's a small price to pay for getting the URL to a thumbnail image for a YouTube video essentially for free.

Once we get the data back from the oEmbed endpoint of our provider, the last step is to deserialize the string into an object and databind it to the UI. Here's that logic:



	private OEmbed DeserializeOEmbed(string result) { //get result if (result.Equals("ERROR")) { //handle error return null; } else if (result.Equals("PROTECTED")) { //handle protected return null; } else if (result.Equals("UNSUPPORTED")) { //handle unsupported return null; } else { //deserialize video metadata try { //open stream using (MemoryStream ms = new MemoryStream(Encoding.Unicode.GetBytes(result))) { //deserialize DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(OEmbed)); return (OEmbed)serializer.ReadObject(ms); } } catch (Exception ex) { //error return null; } } }

Assuming there are no errors along the way, most of the magic is in Line #28, where we use the trusty WCF DataContractJsonSerializer to turn the JSON we get from the providers into our business object. Hmm. The term "Business Object" might be a bit too stuffy to use for something as cool as dynamically displaying Flash content over Silverlight, but I'll get over it.

Well that's about it, except for one final note. In case you're wondering if there's a more dynamic way to call the oEmbed providers based on the input URL than checking a database or configuration file of "supported" endpoints, well then you're right. There is a clearing house service out there that does just that: takes a URL, and tries to parse it against dozens and dozens of providers.

Check out http://api.embed.ly for the API. It's free. Just use it as you would any other oEmbed provider's API. The endpoint looks like this:

http://api.embed.ly /1/oembed?url=[copy-and-pasted URL from the browser here]

<Rant>

If you are like me, you'll be awesome and spend hours refactoring your code until it is pristinely organized into reusable methods, has the absolute minimum amount of external dependencies it needs to physically run, and is as dynamic and beautiful as possible. If you're not like me, then you're probably not even reading this, which means I'm really just talking to myself. Anyway, I waged a fierce battle against my team in favor of using the clearing house. Why introduce a new database table to our schema that has nothing to do with our data model, only to effectively reduce the number of providers we support?

I was overruled, in favor of the argument that if the clearing house went away, the feature would be dead, and we'd have to do it more statically anyway. I see the wisdom and foresight of this, (I know you're reading, Jonathan) but I personally dislike architecting against the apocalypse. Whenever a healthy debate ensues in the project room, I hate the arguments that begin with "Well, what if..."

There's always a "what if" when it comes to software development.

Until we get to whatever is waiting for us after the Internet, we will ultimately be dependent on it for almost every app we build. And guess what? The Internet is made out of plastic and wires and other people's code. Everything is dependent on something else. We depend on electricity not going down. We depend on routers not going down. We depend on the .NET Framework not going down. We depend on YouTube not going down. So although the clearing house is indeed the weakest link in our logical chain of dependencies, we can't eliminate it based on that alone.

If the clearing house goes down, then the feature will break. But until we have contingency plans for the rest of the components failing, I am comfortable taking that risk. Sometimes building beautiful, dynamic software requires a more ballsy architecture. I'd rather have to explain to the client why the feature went down (since it won't be my fault, technically) than why the hot new media website on the Internet isn't supported by our application.

</Rant>

Silverlight IsWindowless-ness, And How You Don’t Need It To Display Flash!

Chris Domino — Sat, 12 Mar 2011 00:13:00 GMT

Now, a lot has been written about Silverlight interop with regard Flash, not to mention HTML, JavaScript, WCF, and even other Silverlight controls. Like my last few postings on the topic, I want to call out that even though there's much blogging that's been done on this topic, I've stumbled onto an ancillary scenario seemingly along the same lines as others. However, it's distinct enough to be worth mentioning; distinct enough to not be yet another of the same thing as the previous six links you probably just glazed over on Bing or Google.

When you consider a browser with plugins (specifically plugins that are part of the content, verses "add-ins" which are tools that may or may not interact with the content, but are definitely not part of it [Silverlight is a plugin, FireBug is an add-in]) there are two dimensions at play when it comes to physically rendering the page. First there's the HTML, and second there's the plugin. And generally, they stay separate. FireFox, for example, goes so far as to run its plugins in a different process.

I use the word "dimension" specifically in the preceding paragraph. HTML is in one dimension, and Silverlight or Flash or Java is in another. Even though plugins appear to be embedded within the HTML, they really aren't part of it at all. The browser tells the plugin where it will be displayed, and the plugin does the rest: its own execution, app domain management, rendering, etc. Despite the HTML bridge that exists in Silverlight, plugins and HTML are not aware of each other. In fact, that's why such a bridge exists at all.

To continue this spatial metaphor, each dimension has its own Z-indexing system, much like the axes of the Cartesian plane. The Z-index of a div has no bearing on the Z-index of a Canvas. I've seen people in the forums making this mistake time and again. There is no magical combination of Z-Indexes of div tags, object tags, or Silverlight controls that will, for example, allow you to have a long Silverlight ComboBox hang below its object tag's content region and superimpose itself over the surrounding HTML. Not possible. That would be like simultaneously standing on both sides of a wall with no doorway or threshold.

The confusion deepens with Silverlight's introduction of the IsWindowless property into the mix. This is probably the least understood (or most misunderstood) Silverlight concept. Some seem to regard this as a panacea for Silverlight oddities. Does your non-rectangular control have a weird black background? Set IsWindowless to true. Can't get HTML to show up on top of Silverlight? Set IsWindowless to true. Those who have been through the trenches know that there's a performance hit to IsWindowless-ness, but it's hard to stop at that when this magical Boolean makes your bizarre issues go away.

What's really happening is that IsWindowless acts like an interdimensional worm hole between the HTML and Silverlight worlds. Even though the dimensions stay separate, IsWindowless forces Silverlight to do the diligence of rendering itself as though the worlds could be combined. I assume this is where the extra work comes in causing the much-warned-against performance ramifications. But it still doesn't inject itself into the HTML Z-indexing scheme. Now, that said, the div that the object tag lives will be rendered in the specified Z order, as in the following example:

A div with a Z-index of 2 will be rendered beneath a div with a Z-index of 3 that contains Silverlight content. A div with a Z-index of 4 (and the proper absolute positioning) will be rendered topmost.

However, any Silverlight Z-indexes will stay in their own dimension. So although you can now have HTML on top of Silverlight, you still can't shuffle Canvases and divs together. But at least we can get this multidimensional beast of a webpage acting as though its constituent parts were cohesively working together to render unified content.

But this worm hole can lead us to more harm than good. For example, such blending of HTML and Silverlight breaks drag & drop. Since Silverlight now lives "within-ish" the HTML, it stops receiving its own mouse events, as the browser intercepts them. Our application depended on drag & drop, so IsWindowless was out, as was our requirement to be able to display Flash videos over our Silverlight control.

So that brings me to the point: you can still superimpose Flash over Silverlight without the crutch (and ensuing sluggishness) of IsWindowless.

In both HTML and Silverlight, any ties between two elements with the same Z-index values would be broken by the order in which they were added to their parent container. For example, if two TextBlocks were added to a Canvas in xaml and had no Z-indexes set, both values would default to zero, the the TextBlock add last would display over top the of the first one. What if this principle guided the Z-index ordering of plugins as it did elements?

When I had IsWindowless turned on, an absolutely-positioned div I created dynamically via a Silverlight button click (and is therefore added to the DOM after the Silverlight) would position itself correctly topmost on the page with the proper Z-index. When I turned IsWindowless off, the div, as expected, disappeared. Upon inspecting the page with FireBug, out wayward div was indeed still there, just loitering in the wrong dimension!

Now what if this div contained the Flash content (by way of an HTML object tag)? Even though the div wouldn't be displayed, since the Flash plugin was technically added to the DOM after Silverlight's, it should still show up, on top of the app, positioned according to its div. And guess what: it worked! Well, almost; IE needed some coaxing. Here's the JavaScript that worked cross-browser.



	var _div; function ShowVideo(html, width, height) { //get middle x and y var x = ($(window).width() / 2) - (width / 2); var y = ($(window).height() / 2) - ((Number(height) - 30) / 2); //show div _div = $("#divVideo"); _div.css({ "height": height, "width": width, "top": y, "left": x }); _div.html(html); //set timeout fixes an ie issue setTimeout("_div.show();", 100); } function HideVideo() { //kill flash var div = $("#divVideo"); div.hide(); div.html(""); }

In Line #2, we're pulling a chunk of HTML out of the database with the object tag all ready to go, along with the height and width of the video, and passing those to the JavaScript method via Silverlight. (How you can easily get these wonderful values with nothing more than a URL to a YouTube clip will be the topic of a future post.)

Line #6 is an adjustment for some Silverlight chrome we display around where the div will show up. We animate a popup with a close button, and then fire the JavaScript that creates the Flash content. The video displays perfectly in the middle of the popup, giving no indication of how many technologies had to be entangled to get it to work. The close button calls "HideVideo" in Line #14.

Line #12 is the fix for IE wonkiness. Basically, nothing would happen when the popup first showed. However, merely clicking on the page would immediately bring the Flash content into position. It was as though IE needed to be kicked like a stubborn horse to recalculate the Z-indexes of the plugins. This is indeed a hack, but it's worked for weeks now. My hunch is that the slower JavaScript engine on IE is to blame. By showing the div, pausing, and then populating it with Flash, it had time to catch up.

Chrome is now my primary browser. Don't tell Microsoft.

But like I said, it works, without having to resort to IsWindowless-ness. Calling HideVideo() in Line #14 properly "disposes" of the Flash by wiping it off the DOM, which will stop playing the current movie before closing it.

That's it! Have fun Flashing your Silverlight!

A Color Comparison Algorithm

Chris Domino — Wed, 02 Mar 2011 03:29:00 GMT

Every once in a while you are presented with an opportunity to write something that is truly cool. And by "cool" I don't mean programmatically interesting or logically fun. I've done cool things with SharePoint list event handlers and cool things with .NET RIA Services, for example. But what I mean here is something, literally, cool; something that someone who is not in the world of software development would overhear and think to themselves, "Wow. That sounds...cool."

We are building a Facebook-ish Silverlight application for a group of designers. Part of the data model for this app is the storage of about two thousand "Defined Colors," which they use in their brands. Each color is stored in a bit-shifted integer representation (four 8-bit bytes smashed together) of the component aRGB values. The requirement was to allow users to be able to select an image from their machine, and have it pop open on the page. They could then click anywhere within in, and based on the shade of the selected pixel, I needed to return the closest color to it from among the palette of Defined Colors.

Yeah. That's cool.

My brain immediately whirred with how this could be done. I had played with WritableBitmap in the past for some lightweight photo-cropping functionality I needed, and knew this was the correct vehicle to use. Not only did it serve as an ImageSource, but also provided an aperture into the underlying array of pixels beneath the image, not unlike a microscope exposing the atoms making up a flower. However, this was not the cool part, not the part that exited me.

The cool part was the algorithm that flew into my head to compare two colors for similarity. The math wasn't complicated at all (as you'll see); it was the careful dissection of the colors that I found fascinating. Any time something abstract can be mapped to a system of numbers, I feel like anything is possible. That's how technologies like voice recognition or handwriting-to-text conversion probably came about: whittle it down to a number, something you can work with, something that follows rules, and we can make it do anything.

Suddenly my excitement waned and quickly turned to anxiety. The lines of code that I had already begun subconsciously composing in my head seemed so simplistic - someone had certainly done something like this before...right? But a little bit of Binging put my mind back at ease. If this has been done, and then had been written up, it wasn't well tagged; I couldn't find anything quite like what I came up with. A lot of things were close: calculating luminosity, converting hex to RGB, etc. But there was nothing exactly hitting home regarding the comparison of two colors. Perhaps it had been deemed too obscure or unusable to be worth the time to blog. Either way, it was a rare occurrence for me to be glad not to find what I was looking for; this was my story to tell.

So let's get to it. My idea was to first lobotomize the "input" color (corresponding to the clicked-on pixel from the image) and store its constituent aRGB parts as four integer variables. Next, spin through all of the Defined Colors, performing the same operation on them. In each iteration, we need to obtain a new color, the literal "difference" between the current Defined Color and selected input color, by subtracting the aRGB values from one another, component by component. In other words: for each comparison, subtract alpha from alpha, red from red, green from green, and blue from blue. I'll refer to this "new" color as the "offset color."

Now that we have each "difference," we need to know which one is the least different. This is done by summing the four parts of each offset color, (which I'll call the "constituent sum" because that sounds awesome) and tracking the smallest such sum as we iterate through our palette. Think of these aRGB values as vectors with the same origin and length. If we have an input vector and a finite set of source vectors, the one pointing in the "closest" direction to our input is the one with the smallest magnitude of difference from it; the one least offset from it.

Turns out it's the same story with colors. Basically, after enumerating the source Defined Colors, we grab the one corresponding to the offset color whose constituent sum produced that smallest delta from the input color's sum. And upon testing, it totally worked! I think the code is much more straightforward than the narrative, so let's take a look:



	private Color GetClosestColor(Color color) { //initialization int minA = byte.MaxValue; int minR = byte.MaxValue; int minG = byte.MaxValue; int minB = byte.MaxValue; Color closest = Colors.Transparent; //iterate all colors foreach (DefinedColor dc in DataAccessLayer.GetAllDefinedColors()) { //get each .net color Color c = Utilities.GetColorFromDefinedColorRGB(dc.RGB); //subtract the components int a = Math.Abs(c.A - color.A); int r = Math.Abs(c.R - color.R); int g = Math.Abs(c.G - color.G); int b = Math.Abs(c.B - color.B); //determine if the sum of the differences of this color is the least of all so far if ((a + r + g + b) < (minA + minR + minG + minB)) { //if so, this is the closest color minA = a; minR = r; minG = g; minB = b; //store this color closest = c; this.ExistingColor = dc; } } //return return closest; }

Some notes:

The "DefinedColor" object in Line #10 is simply a wrapper object around what we have stored in the database for each source Defined Color: basically the "int" value (byte representation of the aRGB and a friendly name).
Line #29 stores the closest Defined Color in a global variable for use in data binding (which is outside the scope of this discussion).
You'll see a call to a utility method in Line #13. This converts the integer representation of a color ("raw" aRGB) into a .NET color. Here's that method, along with its corollary (which isn't used here, but I wanted to include it for your reference).



	public static Color GetColorFromDefinedColorRGB(int definedColor) { //convert color return Color.FromArgb(255, (byte)(definedColor >> 16 & 0xFF), (byte)(definedColor >> 8 & 0xFF), (byte)(definedColor & 0xFF)); } public static int GetIntFromColor(Color color) { //convert color return color.A << 24 \| color.R << 16 \| color.G << 8 \| color.B; }

In Line #4, you'll see that I "hard coded" 255 for the alpha channel. This simplification is due to fact that the application doesn't support transparent colors (although it easily could).

That's about it for my color comparison algorithm. Although certainly not an everyday thing, it's interesting to understand how to extract the constituent parts of colors in general. This has come up more times than I thought it would, especially using technologies like Silverlight.

Have fun coloring!

Silverlight Navigation with 100% Custom Transitional Animations

Chris Domino — Sat, 26 Feb 2011 20:16:00 GMT

Something that's always impressed me about Silverlight (and has made my life a lot easier) has been the browser integration, both in terms of the HTML bridge and the navigation history functionality. I want to focus on the latter; the former has been beaten to death, and at this point, it's hard for me to get excited about yet another way to sling JavaScript.

Clicking the "back" browser button is one of the handiest UX innovations I've ever seen. At the same time, it creates quite an onus for the web developer, as it can easily put a poorly-designed application into an indeterminate state. This can be a rude awakening (it was for me) the first time you need to start messing with Page.IsPostBack and Control.AutoPostBack. Then throw some AJAX into the mix. If you are not diligent with your page transitions, you could get a poor user experience, or worse, garbage data when the user clicks the seemingly harmless back button.

Now in Silverlight applications (verses a web page that has some Silverlight on it for purposes of sex appeal alone), the back button takes on a different role. If you aren't using the Silverlight Navigation Framework, then much like a hard page refresh, "backing" to a Silverlight page will cause the control to reload from scratch, as if it was the first time it had been visited. However, the nav framework will nestle itself into the browser's history quite nicely, using hashed "relative" URLs, making your Silverlight app behave as though it were a standard website comprised of distinct physical pages.

This enables SEO, browser forward/back navigation, true MVC-like architecture with the UriMapper, quasi-master pages, and lots of other enterprise-ish application features. Behind the scenes, all that's happening is that URLs are mapped to relative folder locations for XAML files within the XAP, and the corresponding file is loaded into a particular content frame. There are a bunch of posts out there regarding how to transition smoothly between pages while remaining within the Silverlight Navigation Framework, but none quite worked out for our situation.

We were building our second Silverlight application for a client. The first one was a SharePoint intranet, with lots of isolated Silverlight controls (ie, many XAPs) on the page. There were some major performance ramifications to this, as we had to pay a price for the spin up of each XAP's app domain. In other words, a separate version of the Silverlight runtime was loaded into memory for each control on the page. Yeah. Ouch.

So when we began architecting the second app, which was pure Silverlight, the first thing we decided upon was one XAP that contains the entire UI. This made start up times faster, which is a good first step in a development project where performance was considered a feature, not an optimization. Additionally, the app demanded very intense styling and animations, so almost every control we created has a custom template of some sort. Basically, we had to do more, and do it faster. For the navigation, pages fly in and out of the screen and the background scrolls in the opposite direction behind the "chrome" of the application. Here's a Paint masterpiece of what we had to do. (The blue rectangles represent the viewable area of the app in the browser, the solid-colored boxes are stacked images, and the back rectangles are user controls.)

As the user navigates to different "pages," the user control that corresponds to it animates "down" into view, while the appropriate background slides "up" behind it. (The word "pages" above is in quotes because, remember, everything is physically on the same page, within the same XAP.) However, the magic that happens is that browser back/forward navigation, browser tab titles, and deep linking all work!

In other words, the technique I will show allows us to essentially browser navigate around a single user control in Silverlight!

No idea what I'm talking about? Here's an example. Even though the application seems to navigate normally, notice how the pages animate feely (instead of one disappearing and another appearing, a la some cheesy PowerPoint slide transition) and smoothly. The source code will be included at the end of this post.

Since we're not actually moving to different pages or loading different controls, the sky is the limit for animated transitions; we don't need to rely on any third party controls or limited OOTB functionality. Instead all we need is the chrome (the main user control; the RootVisual of the Silverlight application itself), the child user controls that act as "pages" (but aren't Pages a la Silverlight 3 Navigation), and a custom Navigation Loader that does the magic of making the browser think it's going to different pages when it's really staying in the same place.

The "chrome" of the application is more or less like a master page: it contains all the UI that remains stationary while the content pages animate around it. This includes menus, tab strips, footer links, mastheads, etc. It also includes the Silverlight Navigation Framework pieces: the aforementioned UriMapper and Navigation Loader, as well as the Frame, which, in conventional Silverlight navigation terms, is the area where the content for each actual page is loaded inside the chrome. Think of it just like an HTML iframe whose source is set with JavaScript.

Inside the chrome is some sort of Panel that contains all of the controls that act as our pages. This panel can be as simple as a Canvas wherein its children have their Y coordinates animated, or a custom panel (as was our case) that arranged pages all over the place and used interfaces and attributes to facilitate communication and positioning. This detail, however, is out of the scope of this post. What's important is that all the pages live inside a single panel in the chrome. Alongside this panel is the navigation frame, which, as its children, has the UriMapper busily mapping URIs to XAML file relative paths, a navigation loader, and the content of the page being loaded.

I hope that last sentence sounded strange. If we have a panel that's animating child controls around to show different pages, then what content are we loading into the frame itself? The answer: nothing. The things we need the frame for are to fire navigation events and have its Navigate method called. This method allows, for example, a button click on one page to kick off a nav to another. But there's no rule that the frame's Content property ever needs to be set! That's right: we basically have a collapsed frame on the page that does nothing except listens for URL changes and fires events. It doesn't have to display anything!

How can this be so? Because frames have these navigation loader things we've been talking about. Their actual job is to asynchronously turn a URI into, essentially, a UserControl. Frames have a default navigation loader: an instance of PageResourceContentLoader. This puppy cranks out actual Silverlight pages reflectively instantiated from the passed in Uri, and sets them to the content of the frame. Here is where conventional frame transition animations will be implemented. However, like I said, the fact that these animations are linked to the frame, and not the pages themselves, makes them not flexible enough. So my thinking: simply don't show the frame at all and handle the UI myself.

(Besides, if I'm animating a transition from one page to another, it makes my head hurt conceptualizing that fact that I only have one page at time in the traditional transition model: the current page is destroyed and then the new page is materialized. With my technique, we actually have all of the physical page UserControls sitting next to each other; it's much easier to conceive and work with.)

The only problem is that it doesn't work. That is, we need to do a little more than simply ignore the frame's content. Without a custom loader in place, Silverlight's default frame loader will still "try" to navigate around, and with no content, blank pages will be displayed. So what I did was create a custom loader that EXPLICITLY loads nothing. It's sort of like the different between a null string and an empty string where nulls are not supported. Leaving the frame "null" didn't work; using a custom loader to ensure the frame that it was okay to be "empty" did the trick! Here's what the loader looks like:



	using System; using System.Threading; using System.Windows.Controls; using System.Windows.Navigation; namespace UI.Infrastructure { public class NavigationListener : INavigationContentLoader { #region Members private class LoadAsyncResult : IAsyncResult { #region Members public object AsyncState { get; set; } public WaitHandle AsyncWaitHandle { get { return null; } } public bool CompletedSynchronously { get { return true; } } public bool IsCompleted { get { return true; } } #endregion } #endregion #region Public Methods public IAsyncResult BeginLoad(Uri targetUri, Uri currentUri, AsyncCallback userCallback, object asyncState) { //oblige callback userCallback(new LoadAsyncResult() { AsyncState = asyncState }); //fake result return null; } public bool CanLoad(Uri targetUri, Uri currentUri) { //TODO: security? return true; } public void CancelLoad(IAsyncResult asyncResult) { //TODO: cancellation? } public LoadResult EndLoad(IAsyncResult asyncResult) { //fake result return new LoadResult(new Page()); } #endregion } }

As you can tell, this is more or less a fake class. The only notable lines are #35, where we actually return null for the result (which is okay) and #49, where we return a harmless new Page as our content. You'll see why we need something there later, even though the frame in XAML is collapsed. But that's it. The rest of it is fluff to embody the interface. Even our implementation of IAsyncResult with LoadAsyncResult (Line #'s 10-27) is fake; it's the bare minimum we need to get the meaningless IAsyncResult we need to pass our fake Page. It's actually kind of depressing how abusive I am to this sub-system; all this work to nullify its functionality.

So right now, we have a frame that's properly updating the browser's history as we nav around the app, but isn't showing anything. Next we need to wire in the event handlers so that we can tell our panel how to animate our pages. On your frame, handle any of the following events, per your requirements:

Navigating
Navigated
NavigationFailed
NavigationStopped

The first two adhere to the familiar .NET event "-ing" and "-ed" paring pattern. Navigating fires first, and like other "-ing" events, gives you the opportunity to cancel the navigation. The event args (NavigatingCancelEventArgs) contain the new Uri. If you keep track of the current Uri in your code behind (or custom navigation panel, as I did) and have a mechanism to associate your page user controls to URIs (via attribution on the user control itself, as I did), you can pretty easily come up with a nice little "page lifecycle" model for your controls. In fact, all of my pages (which, as I will keep reminding you, are normal UserControls, not Silverlight Pages) have a custom attribute that provides metadata containing which URL each responds to, an ordinal to make calculating coordinates easier, etc.

The code behind of each such page control looks like this:



	[NavPage("/NewCustomerForm", 1)] public partial class NewCustomerPage : UserControl { }

So when Navigating fires, I will use the current Uri to pull the corresponding UserControl from my panel via reflection. (Basically, find everything decorated with "NavPage" in my panel, and give me the one whose Uri matches.) Then I could do things like error validation, clean up, etc. on the current control, and, using the Uri from the event args, pull the destination control and start loading data. Finally, we kick off the animation.

Here's what the frame looks like in XAML:



	<navigation:Frame x:Name="frmContent" Visibility="Collapsed" Navigating="frmContent_Navigating" NavigationFailed="frmContent_NavigationFailed"> <navigation:Frame.ContentLoader> <infrastructure:NavigationListener /> </navigation:Frame.ContentLoader> <navigation:Frame.UriMapper> <uriMapper:UriMapper> <uriMapper:UriMapping Uri="" MappedUri="/Views/Home.xaml" /> <!-- other page mappings here --> </uriMapper:UriMapper> </navigation:Frame.UriMapper> </navigation:Frame> <Canvas x:Name="canNavPanel"> <Canvas.Clip> <!-- something here to hide controls that are "off the screen" --> </Canvas.Clip> <!-- page here at (X,Y) --> <!-- page here at (X,Y + 500 or however much needed to be off screen) --> </Canvas>

Line #'s 6 - 8 tell the frame to use our custom navigation content loader (an instance of "NavigationListener" defined arbitrarily with the XAML namespace "infrastructure"). And notice Line #3 that makes the frame itself invisible, allowing the Canvas to take its place in the UI.

It should be pointed out that I didn't handle the frame's Navigated event in this case. (Note that this is what fires the OnNavigatedTo event in standard Silverlight Pages.) If you do use this event, make sure that any initialization work you do in your destination control isn't intense enough to make your transition animation choppy. It's for this reason that I hooked the Completed event of my animation's Storyboard and used that to tell the destination control that it's currently on stage.

Now that all of this is wired up, when the URL changes (including deep linked, SEO-happy URLs) the navigation logic will flow as follows:

The frame detects the URL change and gets a relative path to the destination XAML file from the UriMapper. It then forwards this information to its INavigationContentLoader and raises the Navigating event. The navigation content loader has a few opportunities to cancel the navigation, and if it does, the NavigationStopped event is fired. (This will also fire if the user changes the URL again during the processing of the new page.) Otherwise, it will asynchronously load the new page, and the frame will raise Navigated when the new content is ready. Finally, if anything is amiss, the NavigationFailed event occurs.

So that's how browser integration works. What if we wanted one page to be able to navigate to another? No sweat. The navigation frame has a lot of HyperlinkButton behavior built into it, most notably, its Navigate method. Pass this puppy a Uri that the UriMapper can deal with and our navigation logic works as advertised. All we have to do is expose this as a utility method either directly in chrome (hacky) or have it all abstracted into a navigation behavior (elegant). I went with the hacky approach, as this entire mess evolved over time in Visual Studio as requirements changed on us, instead of being born fully mature on a whiteboard.



	public bool NaviateToPage(string relativeUri) { //navigate return this.frmContent.Navigate(new Uri(relativeUri, UriKind.Relative)); }

Just one problem: as all of this browser integration goodness was happening, I found a bug; the browser tab's title was being set to the relative path of the page! Since most users won't notice the URL changing (no doubt transfixed on the beautiful page transitions) then we really lose the UX of being on a different page, browser-wise, after each animation. Even the clicking sound your browser makes when a link is followed adds to this experience. I was able to trace the problem down to the fake Page we were returning from the custom nav content loader. Silverlight Pages (which extend UserControl directly) add a Title property, which ultimately has the same functionality as the "title" tag in an HTML page's "header." The fake page I returned to the navigation frame had a blank title, which the Silverlight Navigation Framework must default to the URL. So the solution was simple: grab the frame, then grab the frame's content (which is the instantiated fake page), and then grab the frame's content's title and set it. Set it to what? Whatever you want. You can add the title of each page to the custom attribute decorating our user controls, infer it from the Uri, etc.

Here's a hypothetical idea of what the frame's Navigating event could look like, assuming all of our page user controls implement some interface named "ISomePageUserControlInterface," for example. (This is another option you can do to go along with the custom attribution of these controls we discussed earlier.)



	private void frmContent_Navigating(object sender, NavigatingCancelEventArgs e) { //[pseudo-code example validation logic of current page] ISomePageUserControlInterface currentPage = this.MethodToGetPageUserControlByURL(this._variableHoldingCurrentURL); if (!currentPage.Validate()) { //cancel e.Cancel = true; return; } //... //[pseudo-code example load logic of new page] ISomePageUserControlInterface newPage = this.MethodToGetPageUserControlByURL(e.Uri.ToString()); newPage.Load(); //... //begin invoke this so that the content is loaded before we nav to it (needed when app doesn't start on home page (if we have been deep linked)) this.Dispatcher.BeginInvoke(() => { //set new page title on content of frame ((Page)this.frmContent.Content).Title = newPage.Title; }); }

Additionally, you'd kick off your animation in this event handler, as well as perform any other cleanup / maintenance tasks (like setting "this._variableHoldingCurrentURL" to "e.Uri.ToString()," etc.).

Well that's about it: 100% custom animations for transitions between pages navigated to both internally and externally. And it's really not all that much work; just a lot if indirection. It always amazes me how much a layer of abstraction can affect any system. The Silverlight Navigation Framework fools the browser into thinking it's loading different pages. My custom navigation content loader fools the page into thinking it's loading different controls. At this point, the poor user has no idea what's going on behind the scenes (which is a good thing, probably, as lengthy and stunning animations can buy us up to two seconds (anything longer won't fly) to load data). A colleague of mine always quotes the famous David Wheeler saying "All problems in computer science can be solved by another level of indirection." This is good stuff; it is why, for example, I named my navigation content loader "NavigationListner" - it does just that: listens. It uses the navigation frame as a puppet to fire the events our UI needs to animate to new controls. It abstracts a layer of abstraction.

Have fun navigating! (Or, more accurately, "navigating.")

The SUESS Lifecycle: Stage 1 - Upload

Chris Domino — Sat, 17 Jul 2010 05:57:00 GMT

The SUESS Series

Introduction
Stage 1 - Upload
Stage 2 - Encode
Stage 3 - SmoothStreaming

Our in-depth media adventure begins, well, at the beginning of the SUESS "lifecycle" with Uploading. This stage contains two sub-components: the Uploader Silverlight control, and the WCF service on the Web Server that facilitates the file upload process. The only prerequisite required here is to have your data tier (Component 3) in place (which can simply be a folder on the Web Server).

Of course, there is nothing new about a file upload control; it's been in HTML longer than I have. Over the years, I've seen a lot of modifications to the familiar read only textbox and "Browse..." button to get the effects people are starting to expect in this whole Web 2.0 craze: progress bars, client-side file type filtering, large file support, etc.

To achieve these nice features, a lot of magic needs to happen client side. If you've read anything I've written before, you'll know that I don't consider AJAX to be magic; it's voodoo at best compared to the wizardry of Silverlight. The argument remains the Silverlight needs to be installed on the client's machine while AJAX comes down for free from the server along with the rest of the page. However, when it comes to programmatic benefits such as a first-class development experience (IntelliSense, compilation, UI designer, etc.), robustness of code, and extent of possibilities of what you can build, you cannot argue against Silverlight.

<RANT>

Now don't get me wrong: AJAX is a really cute technology that has a lot of niche uses. The difference, as far as I'm concerned, falls in the mindset of the developer. Take animating progress bars for example. In AJAX/HTML, you'd need to arrange a few nested divs, set all their widths and background colors, and then when the upload mechanism reports progress, update the width of the inner-most div. To me, that's not really a true progress bar; it's HTML kludginess.

Now, in Silverlight, you see, you literally just animate a ProgressBar.

THAT'S the point.

</RANT>

The example Uploader control I'll show you shortly lacks all the bells and whistles with which it could be adorned to truly make the life of a content manager much better. These features include drag and drop support, multi-file uploads, a slick UI, etc. However, the big ones, like progress bars and cancellation, will demonstrate how uploading large files (such as media) can be a pleasant experience for a user.

First of all, what does it look like? Well, not much, unfortunately. A pretty admin UI was not a requirement of the project through which I came up with SUESS, since this was all back end functionality. But here's what we have:

The title and description fields are watermarked textboxes with some gradient borders to implement the comps of the project. I really don't want to bore you with the details of the code that updates the visual state of the UI; it's hard to make hiding buttons and displaying error messages sexy. You'll be able to download SUESS in its entirety and comb through all the details yourself. Instead, let's take a quick trip through it, and then dive into the cool stuff.

The only input we actually need from the user is the file itself. Depending on your data tier, you could accept more metadata. (SUESS was actually born into SharePoint, so the sky was the limit for metadata.) But for now, we'll just have two optional textboxes for the name of the media and a description. If the name is blank, we'll use the file name. (As you'll see later, with the way Encoder outputs its SmoothStreaming formatted files, we don't have to worry about filename collisions; it's only a consideration during uploading.)

Next we have the "Encoding Quality" drop down. Once again, this is optional, and included only as an example to show another aspect of what you can do with SUESS in terms of explicitly controlling a myriad of Encoder options. (You'll see what I'm talking about in the Encoding post about SUESS.) These are hard-coded (barf, I know) values mimicking the Video Complexity (no documentation available) enumeration in the Encoder SDK. This is the best we can do to this extent, since we can't actually reference the DLL in our Silverlight project and reflect each enum member.

The purpose of including this in the example Uploader is to allow the user the ability to "throttle" the encoding process, which, as previously mentioned, could be very processor (and, of course, time) intensive. Unfortunately, I don't have a lot of metrics regarding just how much quality is sacrificed in terms of time taken to encode. However, I can say that media encoded with the highest quality settings indeed takes much longer than with the lowest.

Finally, we have the media itself. Silverlight gives us the OpenFileDialog control that is more or less identical to what you'd find in Windows Forms. The one immediate advantage it gives us over what is available in the HTML version is the ability to filter the file types (by extension) that the user is allowed to select in the dialog. This not only gives us a more elegant user experience (since we can "label" our filter with something like "Media Files,") but also makes the development much easier, since we don't have to go through the "hand slapping" input validation experience.

<RANT>

Back in the HTML/AJAX version of a file upload control, if you need to enforce a certain type (or types) of file(s), the best you can do is inspect the file the user selected, check its extension, and display an error message, forcing the user to have to go through the entire exercise again. Instead, with Silverlight, we can simply hide invalid file extensions in the folders they navigate within the dialog via the Filter property. This probably doesn't seem like a big deal, but I think it's huge: small UX improvements like these are what Web 2.0 is all about; with Silverlight, you are really using an application, not just a webpage.

</RANT>

Let's talk about a few of the more interesting code samples. The first one simply sets the OpenFileDialog to only allow the files that Encoder supports to be uploaded. It also has a hard coded value (which can easily be made to be configurable) to disallow files over 1GB (purely for sanity purposes).



	private void btnSelectFile_Click(object sender, RoutedEventArgs e) { //initialization OpenFileDialog ofd = new OpenFileDialog(); //show open file dialog ofd.Multiselect = false; ofd.Filter = "Media Files\|.asf;.avi;.bmp;.gif;.jpg;.jpeg;.m2t;.m2ts;.mov;.mp4;.mpeg;.mpg;.png;.tif;.tiff;.ts;.vob;.wmv;"; bool? fileSelected = ofd.ShowDialog(); //check for file if (fileSelected.HasValue && fileSelected.Value) { //check legth if (ofd.File.Length > this._maxFileSize) { //(omitted UI code) //file too large this.tbProgress.Text = "You cannot upload a file larger than 500 MB."; return; } //(omitted UI code) //start upload this.StartUpload(ofd.File); } }

Line #7 above sets the Filter property of the OpenFileDialog to the Encoder 3 supported file types. This is what ensures that Encoder will be able to handle whatever the user throws at it. Line #15 shows that "UI" code has been omitted. You'll see this in a lot the code samples throughout SUESS. Like I said, I don't want to bore you with the details of dealing with VisualStates in the application. Here are some more action shots:

Next let's look at the logic that implements the recursive "chunky" upload. If we send the entire file up to the server, we can't show progress bars, and really don't take advantage of client side functionality at all. Instead, the SUESS Uploader sends 1MB of the file up to the server at a time. After each call, the UI is updated with the current progress, and then the next "chunk" is uploaded.

Since all Silverlight service communication is done asynchronously, we need to daisy chain the "Completed" callback for each of these calls so that they are executed serially within it. Normally, if service calls can be done in parallel (such as downloading images or assembling the content of unselected tabs), you can kick off them all off at the same time; when they're done, they're done, and the corresponding part of the UI lights up.

Otherwise, if you need to call service A and then pass its result on to service B (or a subsequent call back to A, like the Uploader), then there will be some chaining going on. Here's a quick diagram that demonstrates these two paradigms.

In the top half, where we show "parallel" calls, all of the service references are explicit. We invoke a service, and when the associated completed event handler is fired, we do something back on the UI. What if we need to call the same service multiple (and in an unknowable amount of) times in a specific order? This is the case with the Uploader. Since we obviously can't know the size of the file, then we further don't know how many chunks we'll be dealing with. We need to make these calls serially.

The answer is recursion, as shown in the bottom half of the diagram. (Of course, this is only logical recursion, since the same physical method isn't actually calling itself.) I experimented with some crazy iterative algorithms, but they got real messy real quick, and were all ultimately besmirched by the asynchronousity of Silverlight. Instead, I created a method that uploads a chunk (array of bytes) of a file, and when it's done, increments a counter and progress bars. Finally, the same service is called again with the next chunk.

This algorithm, other than being sweet at uploading files, makes two additional features of the Uploader trivial: real time progress bars and upload cancellation. We'll discuss those next. First, however, let's look at this code. This piece is the StartUpload method that the above code block calls to kick off the recursion.



	private void StartUpload(FileInfo file) { //(omitted UI code that resets progress bars, clears errors, etc.) //start recursive upload this._index = 0; this._fileName = string.Format("{0}-{1}", Guid.NewGuid(), file.Name); this.UploadFile(file, true); }

The Boolean passed as the second parameter to UploadFile in Line #7 simply tells the algorithm this is the first chunk. UploadFile actually doesn't care; it just passes this along to the service so that it knows whether to create a new file on the data tier or open an existing one. This can probably be inferred on the server rather than made explicit, but I didn't want to have to burn an extra trip to the disk for each chunk if I didn't have to.

Here's the algorithm:



	private void UploadFile(FileInfo file, bool isFirstChunk) { //initialization byte[] buffer = new byte[this._bufferSize]; MediaServiceSoapClient svc = Utilities.GetMediaClient(); //get chunk using (Stream data = file.OpenRead()) { //write to buffer data.Seek(this._index, SeekOrigin.Begin); data.Read(buffer, 0, this._bufferSize); } //upload chunk svc.UploadFileChunkAsync(this._fileName, buffer, isFirstChunk); svc.UploadFileChunkCompleted += (sender, args) => { //determine result if (args.Cancelled \|\| args.Error != null \|\| !string.IsNullOrEmpty(args.Result)) { //(error code very much truncated here; the above if statement should check each OR'ed condition separately and update the UI appropriately) } else if (this._index > -1) { //update text progress double progress = (Convert.ToDouble(this._index) / Convert.ToDouble(file.Length)) * 100; this.pbUpload.IsIndeterminate = false; this.tbProgress.Text = string.Format("{0} / {1} MB uploaded...", Convert.ToDouble(this._index) / Convert.ToDouble(this._bufferSize), this.GetTotalSize(file.Length)); //animate value Storyboard sb = new Storyboard(); DoubleAnimation da = new DoubleAnimation(); Storyboard.SetTarget(da, this.pbUpload); Storyboard.SetTargetProperty(da, new PropertyPath("Value")); sb.Children.Add(da); da.To = progress; //determine if we've reached the end of the file if (file.Length > this._index) { //pause and upload next chunk Thread.Sleep(200); this._index += this._bufferSize; //recurse this.UploadFile(file, false); } else { //(omitted UI code) //we're done - start encoding this.EncodeCompletedFile(); } //animate progress sb.Begin(); } else { //update canceled this.ShowCancel(); return; } }; }

The first thing to point out here is the utility method in Line #5. This method is a nice Silverlight helper that alleviates the need to have to deal with the "ServiceReferences.ClientConfig" files that store the Service URLs and other WCF settings. Using this method allows you to promote your control from development through production without having to worry about maintaining it.



	public static MediaServiceSoapClient GetMediaClient() { //return return new MediaServiceSoapClient(Utilities.GetBinding(), Utilities.GetAddress("Media")); }

Here are the two internal methods that build the WCF binding and get the URL of the service dynamically:



	private static BasicHttpBinding GetBinding() { //initialization BasicHttpBinding binding = new BasicHttpBinding(BasicHttpSecurityMode.None); //set timeout binding.SendTimeout = TimeSpan.FromHours(1); binding.OpenTimeout = TimeSpan.FromHours(1); binding.CloseTimeout = TimeSpan.FromHours(1); binding.ReceiveTimeout = TimeSpan.FromHours(1); //set messags sizes binding.MaxBufferSize = int.MaxValue; binding.MaxReceivedMessageSize = int.MaxValue; //return return binding; }

And the piece to get the service endpoint. (Note that since this example was lifted from a SharePoint implementation of SUESS, I had to be a bit creative with the URL.)



	private static EndpointAddress GetAddress(string name) { //window.location.href Uri uri = new Uri(((ScriptObject)HtmlPage.Window.GetProperty("location")).GetProperty("href").ToString()); //return return new EndpointAddress(string.Format("{0}://{1}/_vti_bin/{2}Service.asmx", uri.Scheme, uri.Host, name)); }

The rest of the main algorithm is actually pretty straight forward. The recursiveness happens in lines 39 through 42. We pause a bit in Line #39 so that Silverlight doesn't try to open the file from a new thread before the previous one properly closed it, increment the index of where we "are" in the file, and then recursively call the service.

How do we break out of the recursion? Three things can happen. First of all, if something goes terribly wrong on the server or some other exception is thrown, it will be caught on Line #20, handled, and then we'll hard return out of the method. (Again, since this isn't physical recursion, we don't have any stack trace "depth" to worry about.) Otherwise, we use the index. If it's greater than or equal to the length of the file, we know we've uploaded all the bytes: break out here, and start encoding.

The final way is through cancelation, which is one of the big features the Uploader. I know this isn't anything amazing, but it's another example of something that's pretty easy in Silverlight and probably pretty tough in HTML. As someone who's been in and around SharePoint for years, I've seen a lot of large file uploads quietly timeout after watching the page spin for ten minutes. That sort of behavior is not good enough; we need a big red self-destruct button to make sure we can cleanly stop an upload.

So as all this asynchronous uploading is happening on background threads, how do we cancel it from the click event of a cancel button on the UI thread? It turns out that it just works. Silverlight will automatically fire the completed event for a service call on the correct thread, alleviating the need for any dispatching. Since we don't have to worry about any cross threading, we can jump right in with the logic.

The Uploader cancel button click does two things. First, it simply sets the aforementioned index to -1, which basically throws a wrench in the recursive gears. Since all of our calls are on the same thread, we can check this index, see that we've been cancelled, and, well, stop making calls. This is all the housekeeping we need on the client. But what about the server?

The second cancellation task is to make one more call that tells the server that this upload has been cancelled so it can clean up the file. This is one of the reasons for the intermediate "upload" folder on the server: we never had to worry about IIS serving fragments of files. Other than cancellation, dropped connections will also leave broken files on the server. If the upload connections die, then we obviously won't be able to make another call to tell the server to clean up this file. Here is where the cleanup job finishes up for us.

This is a good transition to start looking at server code. We'll begin with the CancelFile method:



	[WebMethod] public string CancelFile(string path) { //initialization string result = string.Empty; path = Path.Combine(this.GetTempUploadPath(), path); //impersonation SPSecurity.RunWithElevatedPrivileges(() => { try { //delete if (File.Exists(path)) File.Delete(path); } catch (Exception ex) { //(error code omitted) } }); //return return result; }

The interesting thing going on here is in Line #8 where I elevate to run as the SharePoint app pool account. This is important for two reasons (neither of which, of course, are what the RunWithElevatedPrivileges delegate is designed to do). First of all, we don't have to assign "Everyone" permissions on our folders. Second, it gets us around a potential IIS double hop issue, in case the GetTempUploadPath method (which is a wrapper around a config file call) returns a UNC path. (Kerberos is the right way to deal with IIS double hops, but that seems to be more configuration than most people - including me - are willing to deal with).

Next we have the method that accepts a chunk from the client and builds a file on the destination server:



	[WebMethod] public string UploadFileChunk(string path, byte[] data, bool isFirstChunk) { //initialization string result = string.Empty; Action<object> uploadCode = null; path = Path.Combine(this.GetTempUploadPath(), path); //impersonation SPSecurity.RunWithElevatedPrivileges(() => { try { //determine if this is the first request if (isFirstChunk) { //delete if (File.Exists(path)) File.Delete(path); } //determine if file exits if (File.Exists(path)) { //add chunk to existing file uploadCode = (o) => { //open file using (FileStream fs = File.Open(path, FileMode.Open)) { //write chunk this.WriteChunk(fs, data); } }; } else { //create new file uploadCode = (o) => { //open file using (FileStream fs = File.Create(path)) { //write chunk this.WriteChunk(fs, data); } }; } //upload Utils.ForceRetryFunction<object, Exception>(() => { uploadCode(null); return null; }, "MediaService.UploadFileChunk", string.Concat("The following error occured while uploading ", path)); } catch (Exception ex) { //(error code omitted) } }); //return return result; }

There are a couple of things to note here. First of all, you'll notice that I use an Action delegate to pass the blocks of code that do the file writing (WriteChunk will be displayed shortly) to a method in Line #48 called ForceRetryFunction. This method takes in a Func (that is passed via an anonymous method in the invocation) which is basically processed in a while loop until it doesn't throw the type of exception (that is generically passed in as well). There are few other operations in the system that spawned SUESS, so I refactored it into ForceRetryMethod.

But WHY? Simply because certain operations need to be kicked in the ass to work. In this example, with I/O happening on several threads really fast, .NET can step on itself and open the file before it's properly closed (just like on the client). I'll put the code here for fun because it's a pretty cool algorithm, but the details are outside the scope of SUESS.



	public static T ForceRetryFunction<T, E>(Func<T> code, string sender, string description) where E : Exception { //initialization int sleep = 500; bool worked = false; T result = default(T); DateTime now = DateTime.UtcNow; //keep trying while (!worked) { try { //run the code result = code(); //code successful worked = true; } catch (E ex) { //only try for one minute if (DateTime.UtcNow.Subtract(now).TotalMinutes > 1) { //unable to save //(error code omitted) worked = true; } else { //method blew up: sleep and try again lock (Utils._random) sleep *= Convert.ToInt32((1.5 + Utils._random.NextDouble())); //sleep exponentially Thread.Sleep(sleep); } } } //return return result; }

After all the trying, retrying, locking, and checking, the actual work that the server performs can be boiled down the simplest method in this section: WriteChunk. Here's the little guy:



	private void WriteChunk(FileStream fs, byte[] buffer) { //seek to end of the file fs.Seek(0, SeekOrigin.End); //write chunk fs.Write(buffer, 0, buffer.Length); }

Very straight forward, and unfortunately, a rather anticlimactic way to end our discussion of the SUESS Uploader. Once the file is up on the server, the Uploader's only remaining task is to kick off the Encoder stage. This is where all the really cool stuff happens, so stay tuned for the next post!

Have fun!

SUESS: A Silverlight Uploader, Encoder, and Smooth Streamer

Chris Domino — Sun, 13 Jun 2010 19:18:00 GMT

The SUESS Series

Introduction

Stage 1 - Upload

Stage 2 - Encode

Stage 3 - SmoothStreaming

When Silverlight first came out, I was initially excited simply because, as I told my colleagues, "Now, we can do all that Flashy stuff too!" Growing up a .NET developer, my task lists contained to-do items such as building reports, optimizing algorithms, and modeling business objects; designing animations, playing videos, and encoding media were simply not part of my life.

I remember when media first started popping up on websites years ago. "How the hell do you write a movie player?" I thought to myself, mesmerized by whatever banner ad first showed off the possibilities of the technology to me. Little did I ever think that the ability to do cool stuff like that (back when people actually tried to shoot the dancing monkey in ads to win a free XBox) would find its way to the Microsoft stack upon which I teetered uncomfortably, at times, looking out at an uninterrupted future of stored procedures and service references.

Oh but it did! And in true Microsoft fashion, they not only made this stuff possible, they made it easy. So what's the answer to the exasperated question I asked above? It takes one little line of XAML to write a movie player. I was almost disappointed at how simple it was; the ease of such an implementation was a little anticlimactic. When I first learned basic programming concepts such as OO and recursion in high school, it was hard! When I taught myself SharePoint (2003), for example, I felt as though I had really endured a rite of passage.

But one line of XAML? Ok, I guess. Looks like I'm a Web 2.0 developer now...

How quickly I forget what it really means to be in the Microsoft development paradigm. Version (n) of a technology makes the old selling points of version (n-1) trivial, and the future selling points of version (n+1) possible. So now that playing media is easy, we can spend more time solving the ancillary problems that are now feasibly solvable.

Where do the movies that we can now play come from? Where do we put them? Does Silverlight support them? How do we make the viewing experience decent for all users across the Internet? Once we make it possible to play media, we need to keep right on going and implement every step of what I call SUESS or the Silverlight Uploader, Encoder, and Smooth Streamer.

SUESS is a process that handles not only playing media, but uploading and encoding it as well. It takes advantage of several cutting edge technologies to do this: Silverlight 4, WCF, Expression Encoder, and IIS Media Services (both of the later were in version 3 at the time of this writing). The details of building each piece of SUESS are scattered around the Internet in various blogs and forums. My goal here is to bring them all together and show how these technologies interoperate to deliver a true, full featured media solution.

Architecturally speaking, SUESS has a lot of moving parts, but they all fit together very nicely. Let's start with everyone's favorite Visio architecture diagram:

Following is a description of all of SUESS's constituent components, and the interactions among them.

Components

The "components" section describes the system from the perspective of each machine's logical role. Grammatically, these are the nouns, where the following "interactions" section outlines the verbs.

Component 1: Client - The Client obviously is the user's machine, where all Silverlight code runs. SUESS begins and ends here, starting with the file upload, and ending with viewing the media. The Uploader control is straight Silverlight and .NET on the client, using WCF to communicate with the Web and Media Servers. Both of these interactions (uploading and encoding) could be very long running, and employ progress bars to keep user's blood pressures down. The later uses Silverlight 4's new support for the WCF Polling Duplex (bi-directional) binding, enabling the server to initiate calls and beam data down to the client!
The media viewer is another story. The out-of-the-box MediaElement does not support the IIS SmoothStreaming format. I was pretty surprised to discover that, after building this entire infrastructure to convert media into a format that was designed specifically for Silverlight, Silverlight didn't even support it out-of-the-box! The answer is actually part of the IIS SmoothStreaming SDK, which gives us the SmoothStreamingMediaElement. It's currently still in beta, and I'll talk about this more later. Click here to download it (this page has some links to other great resources for ramping up on IIS SmoothStreaming).
As with any other Silverlight solution, the Silverlight 4 plugin needs to be installed on the user's browser. This requirement is of course the fuel for many heated Silverlight vs. Flash debates, but that is a topic for a different thought.
Component 2: Web Server - The Web Server hosts the actual web site, as well as the WCF services that are used to upload the file from the client. This server has a pretty typical configuration, requiring only IIS, ASP.NET, and WCF. Basically, once the Client initiates an upload, the file is sent over the wire in 1 MB chunks (which is easily configurable depending on your infrastructure). The Web Server then creates a new file on the File Server, and simply accumulates each chunk into it until the file is completely uploaded. "Chunking" files not only makes progress bars easy to implement, but also doesn't crush the bandwidth between the server and client. I also have some tricks in the code that harden it against periodic network burps, allowing large files to upload uninterrupted.
In addition, these services also facilitate file cancellation, which is concurrently easy within the chunking paradigm. A separate call is made to the server that stops the upload, deletes the temp file, and resets the progress bar. Finally, (and this is out of scope for this article but well worth mentioning) we could implement an optional clean up job to delete any file fragments resulting from uploads that do fail (since, let's face it, Internet connections still go down).
Component 3: File Server - Regardless of how your site is hosted, this is your data tier. Although there's no reason why the uploaded files couldn't live in SQL server, I prefer to store them on the file system for both performance and ease of serving purposes. To this end, the File Server has two shared folders: one for temp uploads and one for encoded files ready to be served. Basically, all file chunks are routed through the Web Server and end up in the temp folder.
Once a file is completely uploaded, the client turns around and calls the Media Server, directing it to grab the file from its temp location on the Files Server, and encode it. The new IIS SmoothStreaming-encoded version is saved in the final destination folder and the original file is deleted from the temp folder. The aforementioned clean up job runs at night (when no one is presumably uploading media) and deletes all files from the temp folder.
Component 4: Media Server - Here is where the magic happens. This server has IIS Media Services 3.0 installed, which enables IIS SmoothStreaming. A virtual directory in IIS is mapped to the file destination folder on the File Server, and serves these files to Silverlight 4 clients. Additionally, Expression Encoder 3 (the "full" version with IIS SmoothStreaming support) is installed here. In order the use the SDK, the product must be licensed and installed. The Encoder SDK is hosted in a WCF service and does the actual work to encode native media into the SmoothStreaming format.
You might be wondering why we have two IIS servers in this farm. The main reason is because media encoding is super processor-intensive, and will suck up every last CPU cycle it can find, spiking all cores. In order to keep your web site humming along, it's a good idea to offload this work if at all possible. A dedicated Media Server also gives us a nice separation of concerns: your Web Server deals with many concurrent users consuming your site's pages; your Media Server deals with fewer people viewing movies and fewer yet uploading and encoding content. However, if you are constrained by physical (or virtual) hardware, there's no technical reason (other than performance) why SUESS couldn't grow very decent media tomatoes in a web garden.

Interactions

Before diving deeply into each component's details and code in SUESS, I'd like to make another pass through architecture from the perspective of each interaction. As with any distributed system, there is a lot of communication that needs to happen to facilitate each server's role. For example, the client needs to tell the Media Server when a file is done uploading, which it can't know until the Web Server informs it that it's properly been copied to the File Server.

Interaction A: File Upload - The "media conversation" begins when the client's Uploader control has a valid file selected. It calls a WCF service on the Web Server and sends the first "chunk" (which is an array of bytes) of the file, as well as the file name with a uniqueness-enforcing Guid appended to it.
The Web Server then reads the File Server's temp upload path from a config file, builds the full file name, and simply checks if such a file already exists. If so, it opens it, seeks to the end, and writes the chunk to the file; otherwise it creates a new file with the uploaded chunk.
The fact that Silverlight service calls are asynchronous does some favors for us, but makes the rest of the uploading algorithm more difficult. The favor is making progress bars easy. When each async call finishes, we know exactly how much of the file we've uploaded, (chunk size times number of chunks uploaded over total file size) and can simply use a Storyboard to animate a ProgressBar, since we'll always be on the UI thread.
However, being asynchronous, we can't simply "loop" through the file, and upload the next chunk when the current one finishes. So what the Uploader does is keeps track of the position in the file we are in, then recursively uploads the next chunk in the completed event of each pervious chunk's service call. This way, we don't keep the Browser's UI thread locked up, and can maintain a very accurate progress bar. We'll go into more detail about this algorithm when we look at the code for the Uploader later.
Interaction B: File Storage / Cancellation / Clean Up Job - The File Upload interaction is pretty straight forward. On the surface, it would appear that File Storage is as well. After the Web Server finishes copying the last chunk to the File Server, the client then kicks off the Encoding process. It's the same story with cancellation. The client breaks the recursion on its end to stop uploading, and tells the Web Server to delete the file from the File Server. Pretty easy, right?
Well, there's one little (huge) problem: the IIS double hop issue. If it weren't for this, there would be no reason to break this interaction out into its own discussion. Basically, under NTLM, IIS cannot be a client and a server at the same time. What this means is that the code behind a service call (IIS being the "server") cannot then turn around and make a call of its own (making IIS a "client") using the same credentials it was passed. This plugs a security hole in which your credentials could otherwise be used to access resources elsewhere on the network you don't explicitly have rights to.
To get around this (as circumventing security continues to accumulate an obnoxiously high percentage of time in my career) we need to either implement a delegation-based security model such as Kerberos, or, more realistically, explicitly impersonate the calls to the File Server. So from this interaction-based perspective, the Uploader is free to call the Web Server or Media Server directly, but these calls cannot be propogated along to the File Server without suffering though a second hop. (It is with great restrain that I did not include pictures of rabbits jumping around the servers in my architectural diagram above: an image the double hop issue always puts in my head.) So make sure you plan out your security model carefully (unless you are using anonymous access or a web garden, in which case this isn't a problem). You'll appreciate this when it's 1 AM, you have all kinds of access denied exception red ink in your Event Viewer, and are staring cross-eyed at the security tab of your upload folder and wondering how giving "Everyone" "Owner" permissions couldn't possibly be working!
Interaction C: File Encoding - The next step is to encode the media into IIS's SmoothStreaming format. In order for SUESS to be successful, we cannot constrain users into a narrow array of files they can upload. To this end, we can use the Expression Encoder SDK to convert almost any native media a user might have into the proprietary IIS SmoothStreaming format. Interaction-wise, all we need to do is make a service call to the Media Server and tell it which file in the temp uploads path to encode. I'll defer the more in depth discussion about SmoothStreaming for now; there is, however, one important technical footnote to this interaction.
Since the Encoder SDK is 32 bit only, we need to be careful on Windows Server 2008 R2, which is 64 bit (or any 64 bit OS). Although your code will compile just fine, the 64 bit wpw3 process cannot call the 32 bit Encoder DLLs. This will happen even if you compile against the "Any CPU" platform in Visual Studio. To work around this, make sure your app pool in IIS runs in 32 bit mode. (To set this, right click an app pool in IIS, select "Advanced Properties," and set "Enable 32 Bit Applications" to True in the "(General)" section.)
Interaction D: Media Encoding Process - The final interaction is the actual encoding process, which is kicked off in the previous section. Our Media Server's WCF encoder service is basically a wrapper around the Encoder SDK. The way it works is that the encoder service takes the file name from the client (along with other optional metadata, such as video quality) and creates an Encoder Job (which is the main workhorse of the SDK).
The Job object has a few events that we can hook, primarily "EncodeProgress." This, combined with WCF duplexing, allows a truly bi-directional communication for this interaction. The Job object fires the event when the percentage complete changes, and WCF actually calls a method on the Silverlight Uploader down on the client, which uses the information to update a progress bar! This is huge for this interaction for several reasons. First of all, encoding in general is slow, and real time progress bars give users a visual indicator that things are indeed chugging along on the server. Also, Silverlight 4's native support for the WCF Polling Duplex binding means no kuldges for accurately reporting progress. I've seen (and experimented with myself) a lot different polling schemes to simulate bi-directional communication between WCF and Silverlight. Now that it's all wrapped up nicely for us, we can present our users with a much richer experiment much easier. Finally, there's also a performance benefit on the network side, since, like the chunky upload, we don't have one massive service call sucking up all our bandwidth.
The rest of this interaction is between the Media Server and the File Server. The IIS SmoothStreaming format isn't just a single file with a particular encoding; it's an entire file structure with manifests, data files, etc. So after Encoder reads the source file from the File Server, it does its thing and dumps all the resultant files into the destination folder. Finally, when it's done, it deletes the originally uploaded file, and fires the "EncodeCompleted" event on the client. Since the destination folder on the File Server is already mapped to a virtual directory under the Media Server's IIS with Media Services installed, the new SmoothStreaming media is automatically ready to be smoothly streamed to Silverlight clients!

That does it for the architectural overview of SUESS. Like I said, the main goal here is to wrap all aspects of media serving on the web into one system: uploading, encoding, storing, and viewing. The Microsoft stack of IIS Media Services, Silverlight, WCF, and Expression Encoder provides an unprecedented media platform to deliver rich, "Web 2.0" content to our users. SUESS is merely the glue that sticks all these technologies together.

Look for future posts from me (and soon, I promise!) that will dive into each component with deep technical discussions and code samples. Until then!

Silverlight vs. Flash: An SEO Perspective

rfreedman — Sun, 06 Jun 2010 20:53:00 GMT

Microsoft has done a great job with Silverlight 4, but many people don't know what the primary differences are, specifically as it relates to search engine optimization (SEO). The purpose of this blog post is to illustrate the primary differences between Silverlight and Flash (from an SEO perspective).

The bottom line is that Silverlight is more SEO Friendly than Adobe Flash

Silverlight 3 is more SEO friendly becasue it permits deep linking URLs to point to places within the application or website. To make things even more exciting for search engine crawlers, an ASP add-on mirrors dynamic content into HTML for easy indexing. Microsoft has included an XML presentation layer in Silverlight to address a major problem with Flash. With Flash, you need to use the SWFObject to provide copy to search bots. Google chooses to overlook XML that is application-specific, as would be the case with Silverlight XML.

Silverlight treats text as a separate entity on a web server, and subsequently applications are fully searchable and indexed by search engines. Though Flash has traditionally not been very SEO friendly, Adobe is working in conjunction with Google and Yahoo, have had some success in making Flash more friendly to search engines. But for now, Silverlight is the clearn winner.

Programmatically Consuming Silverlight Styles

Chris Domino — Thu, 29 Apr 2010 00:38:00 GMT

Hard coding, as anyone who has been development software for more than 20 minutes knows, is bad news. Yet it happens over and over, but probably more as a result of laziness than anything else. And I'm not afraid to say that I've been guilty too. A lot of times, for example, when building a new web site, I am extremely diligent at putting as much styling as possible in my CSS files; my pages' HTML is so pristine that they won't even have style tags!

But then, after staying up all night to meet a dead line, or when a persistent bug regarding an IE 6 compatibility issue refuses to go away, or basically if anything else among the myriad of challenges that building software deals with comes up, it's easy to say "Screw it; I'm hard coding." But for the most part, I think developers are pretty good about using configuration files, style sheets, XSLT; anything that doesn't need to be compiled to keep our software nice and flexible.

These mechanisms are great as long as we can easily pull data out of them. Of course, with components like config files, .NET has the means to consume them, very easily, baked right in. But when it comes to style sheets, things aren't so easy. In fact, I've never needed to programmatically "consume" a style sheet before; it's always simply been welded to my page's HTML.

But with Silverlight, I've started to need this functionality, mainly because we can do so much more with it than HTML. But as is the case whenever code runs on the client, the more we can do means the more we have to do. For example, in the site I'm working on now, I built a pager control that automatically creates buttons for each page, and dynamically resizes itself to fit nicely along with the Canvas it is paging.

One of the problems I had to solve was to show horizontal scrolling for my pager if the number of pager buttons would be beyond the Silverlight control's content region. Here's what it looks like:

If the horizontal scroll bar is visible, I need to make the pager taller. Normally, I would just use ScrollViewer.ComputedHorizontalScrollBarVisibility to do the job. However, with all the padding, dynamic controls, and resizing I do, this property didn't always return value I was expecting. So instead of kludging it, I decided to compute the width myself. This at first sounded daunting, but the logic wasn't too bad. Basically, I sum the widths of all the button controls, and add on the left and right Margin and Padding for each.

The problem is that the margins and paddings were set in styles. (The widths had to be calculated dynamically, since page "10" is wider than page "9" - TextBlock.ActualWidth came to the rescue here.) Now of course this isn't a problem because I could always iterate and accumulate the controls in my pager and get the "actual" width of the control (and therefore determine the need to have a horizontal scrollbar) that way.

However, meh, I thought I could do something a little more elegant. Besides, all it would take is one little UI tweak that could change the behavior of my calculation. So instead, I decided to "pull" these values from the style used on the button one time, store them globally, and then reuse them as needed. This way, the style "owned" these values, and if they ever changed, my logic would continue to work just fine. Yay me.

But then, Silverlight punched me in the face. Basically, the way Style and Setter objects work internally is sort of kludged. I stopped researching when I discovered that my idea wasn't going to work. (Basically Setters only ever set their values on the objects they style exactly one time in the control's lifespan. I believe this is performance enhancement - like how you can't databind to the Position property of a MediaElement even though it's a DependencyProperty.) So to enforce these performance gains, the Style and Setter classes expose very little to us in the API. The Silveright team won't let us hurt ourselves; it's like a child being forced to wear a safety helmet in a dodge ball game.

So did I stop at that? Well, yeah, but it stayed in the back of mind, bothering me all the while I built my pager control. When I finished it early, I decided to use my extra time to go back and try to figure out a way to programmatically access a style in Silverlight.

And I got it! My idea was to use the fact that there is nothing magical AT ALL about styles! You can tell by the XAML and the names of the Setter object's properties ("Property" and "Value") that all they do is reflectively set properties on objects. That's why you can put attached properties and other junk in a style - not just "style" attribute values like in HTML and CSS.

So I wrote a little diddy that, well, basically just uses styles! You give me a style and a property, and I'll instantiate on object of the Style's TargetType type, (until now I've been annoyed that you have to tell a XAML Style what type of object it styles, but this makes my code work!), apply a style, and just reflectively grab that object's property value!

Here's what it looks like:



	public static T GetStylePropertyValue<T>(Application app, string styleName, string propertyName) { //get style Style style = app.Resources[styleName] as Style; if (style == null) return default(T); //instantiate an obejct that is the type of this style's target type FrameworkElement element = (FrameworkElement)style.TargetType.Assembly.CreateInstance(style.TargetType.FullName); if (element == null) return default(T); //apply this style to the new object element.Style = style; //use reflection to get the value of the property PropertyInfo pi = element.GetType().GetProperties().Where(p => p.Name.Equals(propertyName)).FirstOrDefault(); if (pi == null) return default(T); //return return (T)pi.GetValue(element, null); }

Basically, "T" is the .NET type of the property, "app" is usually App.Current, and along with the name of the style and property, I've got it. So for my pager situation, "T" is "Thickness" and "propertyName" is "Margin." Finally, consuming the margin's dimensions gives me what I need to calculate the total "actual" width of my pager.

Have fun!