jvance.com

Jarrett's Tech Blog - Browsing C#

1. Unzip Nested Zip Files While Streaming

   I recently encountered a scenario where I needed to unzip all the files in a zip file and also any files from internal zip files.  The source data is streaming in through an HTTP POST via IIS into BizTalk. The zip files can be large (up to 200 MB) and there can be multiple posts happening at the same time.  This is too much data to fit in memory.  Also, I needed to avoid unnecessary network traffic so using temporary files is not an optimal solution.  Therefore, I needed a forward-only streaming solution.

   To accomplish this, I turned to #ziplib. The ZipInputStream object looked like the perfect solution to this situation. Here is an example of how to use this class:

   using ( ZipInputStream s = new ZipInputStream(stream)) {
     ZipEntry theEntry;
     while ((theEntry = s.GetNextEntry()) != null) {
       int size = 2048;
       byte[] data = new byte[2048];
       size = s.Read(data, 0, data.Length);
       if (size > 0) {
         Console.Write(new ASCIIEncoding().GetString(data, 0, size));
       } else {
         break;
       }
     }
   }
   

   As the raw data is streamed through the ZipInputStream, it gets unzipped.  The GetNextEntry() method sets the position to the beginning of the next file.  Then we just read from the ZipInputStream to get the unzipped file data.  So to unzip nested zip files, I came up with a function I could call recursively:

   public static void NestedUnzip(Stream stream, string targetPath)
   {
     ZipInputStream s = new ZipInputStream(stream);
     ZipEntry entry;
     while ((entry = s.GetNextEntry()) != null) {
       //when internal zip file, unzip it
       if (Path.GetExtension(entry.Name).ToLower() == ".zip") {
         NestedUnzip(s,
           Path.Combine(targetPath, Path.GetFileNameWithoutExtension(entry.Name)));
       } else {
         //make sure target path exists
         string path = Path.Combine(targetPath, entry.Name);
         Directory.CreateDirectory(Path.GetDirectoryName(path));
   
         //write the data to disk
         using (FileStream fs = File.Create(path)) {
           byte[] buffer = new byte[1024];
           int read = buffer.Length;
           while (true) {
             read = s.Read(buffer, 0, buffer.Length);
             if (read > 0) fs.Write(buffer, 0, read);
             else break;
           }
         }
       }
     }
   }
   

   Now this would work great for my needs as it process the data as a forward-only read-only stream.  However, whenever a nested zip runs out of entries (i.e. GetNextEntry() == null) the ZipInputStream calls close on the underlying stream.  This results in the unzip process ending prematurely.

   To fix this, I commented out the Close() call within the GetNextEntry() method of the ZipInputStream class:

   if (header == ZipConstants.CentralHeaderSignature ||
     header == ZipConstants.EndOfCentralDirectorySignature ||
     header == ZipConstants.CentralHeaderDigitalSignature ||
     header == ZipConstants.ArchiveExtraDataSignature ||
     header == ZipConstants.Zip64CentralFileHeaderSignature) {
     // No more individual entries exist
     // -jv- 11-Jun-2009 Removed close so it can support nested zips
     //Close();
     return null;
   }
   

   Of course, the calling method should properly close the source stream so this is a safe change to make. For example:

   using (Stream s = inmsg.BodyPart.GetOriginalDataStream()) {
     NestedUnzip(s, unzipLocation)
   }
   

   The result is a perfect streaming solution with low memory usage and no need for temporary files.

   Posted by JarrettV on June 13 at 11:33 AM

   Comments (1)
   
   

2. LINQ Syntax Choice

   Which LINQ syntax do you prefer?  The fluent style or the query style?

   

   I personally prefer the Fluent style as it appears to come natural to me.  However, I have yet to deal with some of the more complicated joins that may work better with the query style syntax.

   Posted by Jarrett on October 17 at 10:32 PM

   Comments (0)
   
   

3. Using the New DateTime Support in .NET 3.5 via MVC & jQuery

   I've added New Global Date and Time Support  to BlogSvc by utilizing the new expanded support for date times with proper time zone support.  Some highlights in the MSDN documentation:

   The DateTimeOffset structure represents a date and time value, together with an offset that indicates how much that value differs from UTC. Thus, the value always unambiguously identifies a single point in time. A DateTimeOffset value is not tied to a particular time zone, but can originate from any of a variety of time zones. The TimeZoneInfo class makes it possible to work with dates and times so that any date and time value unambiguously identifies a single point in time. Taking advantage of time zone support in the .NET Framework is possible only if the time zone to which a date and time value belongs is known when that date and time object is instantiated.

   So the MSDN documentation is not clear on which class can make a date/time unambiguous.  However, the last sentence is the best clue.  We must capture both the UTC value and an originating time zone.

   For an ASP.NET MVC application we can add configuration to associate all date/times to our preferred time zone. 

   

   With this configuration, we can now write an HtmlHelper extension to display a DateTimeOffset in our preferred time zone.

   

   Call the extension method and pass it either a DateTimeOffset or a DateTime

   .

   This will result in the follwing html:

   <abbr title='Tuesday, October 07, 2008 10:01 PM (GMT-06:00) Central Time (US &amp; Canada)'>10/7/2008 10:01 PM - CST</abbr>
   

   This gives an abbreviated date/time display at your preferred time zone.

   However, you may want to display time relative to the user browsing your website. There are two ways to accomplish this:

   • Ask and store each user's time zone
   • Automatically determine user's time zone using javascript

   The first option is commonly used with forums and requires the user to register and choose their preferred time zone. The second option makes much more sense for content websites (such as a blog) since it can occur automatically without registration.

   There is already a great plugin for jQuery that can display fuzzy times that are directly relative to the user.  It is called the Time Ago Plugin.  To use it, lets create another helper extension that utilizes a micro-format:

   

   This creates abbreviations in the html, that when activated through jQuery show times that will be automatically updated even after the user has loaded the page.

   

   This DateTime display is much more recognizable and personable to the casual user.

   Head on over to the BlogSvc website to download the code.

   Update: here is a bonus function that will allow you to format the date (with access to the time zone information) anyway you'd like.

   

    

   Call it like

   :

   Posted by Jarrett on October 09 at 2:24 AM

   Comments (1)
   
   

4. New jQuery Rater Plugin for Star Ratings

   The jQuery Star Rating widget is a neat control to add to your website.  It has many options that allow you to easily customize it.  However, it's usage is based on a fully degradable form submission model which means you must use markup containing an option list.  Unfortunately, this means it becomes quite complex to support a rating model that fires off an ajax request to submit a new rating.  Therefore, the developer is left to worry about:

   • Writing code on the server to generate the option list only for users that have not already rated
   • If the user has already rated, the server must generate different markup, or set the widget to disabled
   • The developer must write javascript to handle the widget callback and build an ajax request
   • No built in support for rating count and updated rating result
   • No built in support for a failed rating

   To make life easier, I wrote the jQuery Rater Plugin that should reduce the complexity of implementing an ajax rating scenario. 

   

   jQuery Rater Plugin Demo

   Features

   • Auto Ajax posting
   • Supports rating update and rating count
   • Shows previous rating before user has rated
   • Markup is same for rated and unrated
   • Supports step ratings (partial stars)

   View the Demo Page

   First, lets take a look at the markup which is contained in an ASP.NET user control.  However, you could write this in your server language of choice.  There are three things being set below.  The width of the stars, the current rating, and the count of ratings.

   

   The only markup elements required by the plugin are the "ui-rater-starsOff" and "ui-rater-starsOn" spans.  Also, they must be contained within an element that has an id you can reference.

   We only need to activate the plugin when the user has not already rated the entry.

   

   The only option I've set is the address to post the rating to.  When the user performs a rating, the plugin will automatically build an ajax request to the postHref.  It includes the id and chosen rating value in the form post data.

   This plugin does more with less code than the existing jQuery Star Rating Widget because it foregoes the degrade-ability in favor of ajax.

   • Previous Javascript size: ui.core.packed.js 4KB + ui.stars.pack.js 4KB + rate.js 2KB = 10KB
   • New Javascript size: jquery.rater.js 3KB (unpacked)

   View the source code

   To see a full server implementation, see the source code of AtomSite.net.

   Download jQuery Rater Plugin (javascript + demo html + image + sample server page)

   Found this plugin useful? Please kick it -

   Posted by Jarrett on September 29 at 1:37 AM

   Comments (33)
   
   

5. New BlogSvc Release 0.2

   This release includes an implementation of Atom Publishing Protocol on WCF 3.5.  In the words of Tim Bray:

   An Atompub implementation lets you create, retrieve, update, and delete (CRUD) Web Resources. ... Atompub starts with a Service Document, which contains one or more named Workspaces, which contain Collections, which are what you actually POST to in order to start up the CRUD process.  So the idea is simple; have a collection that when you POST to it, creates a new publication.

   The object model is based off of the Atom Syndication Format and the AtomPub specs.  All of the objects are based off of Xml or the new XElement.  Propeties are used to support strongly typed access to the data.

   Atom Syndication Format	Atom Publishing Protocol
   AtomCategory	AppCategories
   AtomContent	AppCollection
   AtomEntry*	AppControl
   AtomFeed	AppService
   AtomGenerator	AppWorkspace
   AtomLink*
   AtomPerson	Atom Threading Extension
   AtomSource	ThreadInReplyTo
   AtomText	* Extended

    

   This release should work in IIS6 or IIS7 with .NET 3.5.  Also the SVC handler must support all verbs.  Since AtomPub is RESTful, you'll need PUT and DELETE to go along with the usual GET and POST verbs.

   The WCF service is built using the new Web Programming Model available in 3.5.  However, it is designed to support normal web services as well (more on this in a future post).  A neat WCF feature with this release is the support of media entries allowing a user to post images to a collection.  I found the trick to supporting raw data on Carlos' blog.  However, there is a catch. Anytime you want to accept unknown content types and known content types, you must only deal with Stream objects.  For example, although CreateEntry will always return an AtomEntry document you must specify a Stream because the input could be an AtomEntry or say a JPG image.

   [ServiceContract]
   public interface IAtomPub
   {
       [WebGet(BodyStyle = WebMessageBodyStyle.Bare, UriTemplate = "{workspaceName}/{collectionName}/{entryName}/media")]
       Stream RetrieveMedia(string workspaceName, string collectionName, string entryName);
       
       [WebInvoke(BodyStyle = WebMessageBodyStyle.Bare, UriTemplate = "{workspaceName}/{collectionName}")]
       Stream CreateEntry(string workspaceName, string collectionName, Stream stream);
   
       [WebGet(UriTemplate = "{workspaceName}/{collectionName}/{entryName}")]
       Stream RetrieveEntry(string workspaceName, string collectionName, string entryName);
   
       [WebInvoke(BodyStyle = WebMessageBodyStyle.Bare, UriTemplate = "{workspaceName}/{collectionName}/{entryName}", Method = "PUT")]
       Stream UpdateEntry(string workspaceName, string collectionName, string entryName, Stream stream);
   
       [WebInvoke(UriTemplate = "{workspaceName}/{collectionName}/{entryName}", Method = "DELETE")]
       void DeleteEntry(string workspaceName, string collectionName, string entryName);
   
       [OperationContract]
       [WebGet(UriTemplate = "service")]
       AppService RetrieveService();
   
       [WebGet(UriTemplate = "{workspaceName}/{collectionName}/category?scheme={scheme}")]
       AppCategories RetrieveCategories(string workspaceName, string collectionName, string scheme);
   
       [WebGet(UriTemplate = "{workspaceName}/{collectionName}")]
       AtomFeed RetrieveFeed(string workspaceName, string collectionName);
   }

   You can direct it to a strongly typed implementation by checking the content type.

   public Stream CreateEntry(string workspaceName, string collectionName, Stream stream)
   {
       string contentType = WebOperationContext.Current.IncomingRequest.ContentType;
       AtomEntry entry;
       if (contentType == Atom.ContentType || contentType == Atom.ContentTypeEntry)
       {
           entry = new AtomEntry();
           XmlReader reader = new XmlTextReader(stream);
           entry.Xml = XElement.Load(reader);
           entry = CreateEntry(workspaceName, collectionName, entry);
       }
       else entry = CreateMedia(workspaceName, collectionName, stream);
       return GetStream(entry);
   }

    

   IDs and Hrefs

   • Blog.svc WCF Service
   • UriTemplates
   • Handling Entry or Media Resources
   • WebLinks
   • Object Model over .Net 3.5 SP1 Object Model

   Test AtomPub and Atom and Threading auto links.

   Posted by Jarrett on August 04 at 8:30 PM

   Comments (1)
   
   

6. WCF Adds Root Node on IXmlSerializable Object

   Has anybody else had trouble with WCF adding a root node on their objects that implement IXmlSerializable?

   Here is the setup:

   • Create a class that implements IXmlSerializable

   public class Entry : IXmlSerializable
   {
     public XElement Xml { get; set; }
     public void ReadXml(XmlReader reader)
     {
       Xml = XElement.Load(reader, LoadOptions.SetBaseUri);
     }
     public void WriteXml(XmlWriter writer)
     {
       Xml.WriteTo(writer);
     }
   }

   • Create a WCF service that returns the object.

   [ServiceContract]
   [XmlSerializerFormat]
   public interface IService
   {
     [OperationContract]
     [WebGet(BodyStyle = WebMessageBodyStyle.Bare, UriTemplate = "{a}/{b}/{c}")]
     Entry RetrieveEntry(string a, string b, string c);
   }
   
   public class Service: IService
   {
     public Entry RetrieveEntry(string a, string b, string c)
     {
       return new Entry
       {
         Xml = new XElement("test", a + "/" + b + "/" + c);
       }
     }
   }

   When hitting the service located at http://localhost/EntryService.svc/one/two/three, I expect to get the following xml:

   <test>one/two/three</test>

   Instead, I always get the above xml wrapped in a root node as shown below:

   <Entry xmlns="http://schemas.datacontract.org/2004/07/EntryTest">
     <test>one/two/three</test>
   </Entry>

   1. I've set the BodyStyle to Bare
   2. I've set the [XmlSerializerFormat] attribute
   3. I've tried putting [XmlRoot(null)] on the Entry class

   The only thing that seems to work is changing the return type to XElement. This is really odd behavior of WCF. I can't find anything in the documentation as to why it is doing this.

   Update (7/27): It appears you must use the XmlRootAttribute to accomplish this. Your WriteXml(XmlWriter writer) method can check to see if the root node was already added by the serializer, and if not add it:

   public void WriteXml(XmlWriter writer)
   {
   
       //only start document if needed
       bool start = false;
       if (writer.WriteState == WriteState.Start)
       {
           start = true;
           writer.WriteStartDocument();
           writer.WriteStartElement("root", "http://example.com");
       }
       //TODO: custom serialization here
   
       if (start)
       {
           writer.WriteEndElement();
           writer.WriteEndDocument(); 
       }
   }

   This allows the WriteXml to be usable even when the object isn't being serialized by the WCF serializer.
   Posted by Jarrett on July 26 at 10:27 PM

   Comments (0)
   
   

7. BlogSvc - New Blogging Service for .NET 3.5

   Update: BlogService now has it’s own website, see http://blogsvc.net

   I've started a new project on CodePlex called BlogService.  Eventually, it will have it's own website at blogsvc.net.  The point of this project is to provide a robust back-end for a blog site.  It uses the provider model design pattern for abstracting the data layer similar to ASP.NET providers.  I've uploaded a design diagram of the major classes. It also contains a WCF service that will expose multiple endpoints (soap, rest, json).

   To kick things off, I created a community preview release that includes the following functionality:

   1. BlogProvider
   2. XmlBlogProvider
   3. Business (Blog, BlogSite, BlogComment, etc.)
   4. Configuration
   5. Unit Tests
   6. WCF Service
   7. Multi-site support

   This release provides most of the functionality needed to support a front-end blogging site. It does not include web pages yet.

   BlogSvc Release 0.1

   Posted by Jarrett on July 21 at 11:15 PM

   Comments (0)
   
   

8. Comments Security Hole in BlogEngine.NET 1.4

   I found a hole in BlogEngine.NET that allows anyone to delete and approve comments.

   UPDATE 7/11: Fixed in 1.4.0.12

   Verify the Hole

   • Add a test comment to your blog

   

   • Refresh the blog posting to retrieve latest source
   • View source of blog posting
   • Find guid of comment to delete (see line 103 below)

   <div id="id_9c2b8578-1dde-421e-94ff-5ea7e0d82012" class="comment">
     <p class="date">7/10/2008 4:13:35 PM</p>
     <p class="gravatar"><img src="http://www.gravatar.com/avatar/b642b4217b34b1e8d3bd915fc65c4452.jpg?s=80&amp;d=..." alt="Test3" /></p>
     <p class="content">asfasdfafdasd sa afsdfdsaas </p>
     <p class="author">
       Test3
       <img src="/blogengine/pics/flags/us.png" class="flag" alt="us" />
     </p>
   </div>
   

   • Request a POST to BlogPostUrl.aspx?deletecomment=guid

   

   • Refresh the page and notice comment is deleted

   Patch the Hole

   You can patch the hole by updating the Page_Load event in the CommentView.ascx.cs file by checking for an authenticated user (lines 117,118, & 127)

   protected void Page_Load(object sender, EventArgs e)
   {
     if (Post == null)
       Response.Redirect(Utils.RelativeWebRoot);
     
     if (!Page.IsPostBack && !Page.IsCallback)
     {
       if (Page.User.Identity.IsAuthenticated)
       {
         if (Request.QueryString["deletecomment"] != null)
           DeleteComment();
         
         if (!string.IsNullOrEmpty(Request.QueryString["approvecomment"]))
           ApproveComment();
   
         if (!string.IsNullOrEmpty(Request.QueryString["approveallcomments"]))
           ApproveAllComments();
       }
   
     string path = Utils.RelativeWebRoot + "themes/" + BlogSettings.Instance.Theme + "/CommentView.ascx";
   

   Repeat the steps given above to verify that the hole has been patched.

   In the process of adding OpenID support to the comment system in BlogEngine.NET I found myself deep in a rabbit hole of refactoring. This comment security issue is just one of the things I've found during my journey. I've reported the issue on the BlogEngine.NET Issue Tracker.  I think it is important to point out that the patch above is just a quick fix.  The proper solution is to put authorization checks in the business layer (the BlogEngine.Core.Post business object in this case).

   Posted by Jarrett on July 10 at 6:06 PM

   Comments (0)
   
   

9. Add OpenID to BlogEngine.NET 1.4 Part 1

   Update : there is no Part 2 as I switched to AtomSite for my blog engine.

   Here is a guide on how to add OpenID support to BlogEngine.NET 1.4. This first blog post will detail how to use an OpenID account to login as an existing blog user via impersonation.  I've used Andrew's blog post How to add OpenID support to BlogEngine.NET as a starting place. 

   Pros

   • Quick, minimum changes required
   • Retains support of blogging tools (such as Live Writer)
   • Easy to understand and backwards compatible

   Cons

   • Not "true" OpenID integration
   • Accounts are maintained separately
   • Impersonation file is required

   Step 1: Download latest release of dotnetopenid library

   Step 2: Place DotNetOpenId.dll into Bin folder (if compiling BlogEngine.Web then add reference to project)

   Step 3: Update Login.aspx page

   • Add

   <%@ Register Assembly="DotNetOpenId" Namespace="DotNetOpenId.RelyingParty" TagPrefix="RP" %>
   

   • Replace

   <asp:Login ID="Login1" runat="server" class="loginbox" />
   

   with

   <RP:OpenIdLogin ID="OpenIdLogin1" runat="server" onloggedin="OpenIdLogin1_LoggedIn" />
   

   Step 4: Update Login.aspx.cs code behind file

   • Replace

   protected void Page_Load(object sender, EventArgs e) 
   { 
     if (Page.User.Identity.IsAuthenticated) 
     { 
       changepassword1.Visible = true; 
       changepassword1.ContinueButtonClick += new EventHandler(changepassword1_ContinueButtonClick); 
       lsLogout.Visible = true; 
       Login1.Visible = false; 
       Page.Title += Resources.labels.changePassword; 
     }  
     else 
     { 
       Login1.LoggedIn += new EventHandler(Login1_LoggedIn); 
       Login1.FindControl("username").Focus(); 
     } 
   }
   

   with

   protected void Page_Load(object sender, EventArgs e) 
   { 
     if (Page.User.Identity.IsAuthenticated) 
     { 
       changepassword1.Visible = true; 
       changepassword1.ContinueButtonClick += new EventHandler(changepassword1_ContinueButtonClick); 
       lsLogout.Visible = true; 
       OpenIdLogin1.Visible = false; 
       Page.Title += Resources.labels.changePassword; 
     } 
   }
   

   • Replace

   void Login1_LoggedIn(object sender, EventArgs e) 
   { 
     if (!Roles.IsUserInRole(Login1.UserName, BlogEngine.Core.BlogSettings.Instance.AdministratorRole)) 
       Response.Redirect(BlogEngine.Core.Utils.RelativeWebRoot, true); 
   }
   

   with

   protected void OpenIdLogin1_LoggedIn(object sender, DotNetOpenId.RelyingParty.OpenIdEventArgs e) 
   { 
     //don't allow login control to login, we will do that manually 
     e.Cancel = true; 
     //get impersonated user 
     string user = Impersonation.Impersonate(e.ClaimedIdentifier); 
     if (!Roles.IsUserInRole(user, BlogEngine.Core.BlogSettings.Instance.AdministratorRole)) 
       Response.Redirect(BlogEngine.Core.Utils.RelativeWebRoot, true); 
     else 
       FormsAuthentication.RedirectFromLoginPage(user, OpenIdLogin1.RememberMe); 
   }
   

   Step 5: Add Impersonation.xml file to App_Data folder

   <impersonation> 
     <impersonate> 
       <id>http://jarrettv.myopenid.com/</id> 
       <user>Jarrett</user> 
     </impersonate> 
   </impersonation>
   

   Step 6: Add Impersonation.cs to BlogEngineCore project and compile

   using System;
   using System.Linq;
   using System.Xml.Linq;
   using System.Web;
   using System.Web.Hosting;
   
   namespace BlogEngine.Core
   {
       public class Impersonation
       {
           public static string Impersonate(string id)
           {
               string user = id;
               string path = BlogSettings.Instance.StorageLocation + "impersonation.xml";
               
               if (!VirtualPathUtility.IsAppRelative(path))
                   throw new ArgumentException("xmlFileName must be app-relative");
   
               string fullyQualifiedPath = VirtualPathUtility.Combine
                   (VirtualPathUtility.AppendTrailingSlash
                   (HttpRuntime.AppDomainAppVirtualPath), path);
   
               string actualPath = HostingEnvironment.MapPath(fullyQualifiedPath);
   
               XDocument xDoc = XDocument.Load(actualPath);
               user = xDoc.Element("impersonation").Elements("impersonate")
                   .Where(x => x.Element("id").Value == id).Select(x => x.Element("user")).Single().Value;
               return user;
           }
       }
   }
   

   Step 7: Deploy following changed files

   • Login.aspx
   • Login.aspx.cs
   • \Bin\BlogEngine.Core.dll
   • \Bin\DotNetOpenId.dll
   • \App_Data\Impersonation.xml

   

   In the next part, I will add OpenID support to the comment system.

   Technorati Tags: openid,asp.net,c#,blogengine
   Posted by Jarrett on July 05 at 8:26 PM

   Comments (4)