Sunday, 12 July 2009

Validating XML Files against XSD Schemas (especially for files that don’t reference the schema)

This post has moved to http://www.scottleckie.com/2009/07/validating-xml-files-against-xsd-schemas-especially-for-files-that-don%e2%80%99t-reference-the-schema/

Wow, XML is a pig, isn’t it? Don’t get me wrong; it does everything I need it to do in describing multi-faceted data, but it’s a pretty steep learning curve.
At first, I used XPath and a lot of coded validation. Then I finally invested time in learning XML Schemas (XSDs) and that helped a lot because I could validate the entire document and, only when I knew it was valid, start pulling data out of it. At around the same time, I stumbled upon LINQ to XML and this shortened the code substantially.  Now, all I had to do was validate the document against the XSD and, if it passed, get to decoding it via LINQ and we’re done and dusted in a few lines of code.
The XSD validation, by the way, looks like this;
private static readonly string SCHEMA = "http://schemas.axiossystems.com/DDI/SnmpMappings/";
private static bool ValidateFile(string file, string schemaFile)
{
if (file == null || schemaFile == null)
throw new ArgumentNullException("Must supply non-null file and Schema file to MappingFilesParser.ValidateFile()");
XmlSchemaSet schemas = new XmlSchemaSet();
schemas.Add(SCHEMA, schemaFile);
XDocument doc = XDocument.Load(file);
doc.Validate(schemas, (o, e) =>
{
throw new XmlSchemaValidationException(string.Format("{0} validating {1}", e.Message, file));
});
return true;
}


So, here you define the schema used in your XML file in the static readonly string called “SCHEMA”, then call ValidateFile(name of XML File, name of schema file).


The function either returns or throws ArgumentNullException or XmlSchemaValidationException.


All good so far. Then I realised that, if you load any random XML file that does not reference your schema file then the .Validate() method will still complete successfully.


What I mean by this is that every XML file has to have an xmlns namespace declaration similar to this;


<?xml version="1.0" encoding="utf-8"?>
<xs:schema targetNamespace="http://schemas.axiossystems.com/DDI/SnmpMappings/"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:ddi="http://schemas.axiossystems.com/DDI/SnmpMappings/"
elementFormDefault="qualified"
attributeFormDefault="unqualified">


If your XML file does not refer to the XSD then the validation passes, which is not what I intended. The solution, as outlined in Scott Hansleman’s blog is to check the namespaces in the XML file and confirm that it does reference our XSD, then run through the validation. So, now the ValidateFile() method looks like this;


private static readonly string SCHEMA = "http://schemas.axiossystems.com/DDI/SnmpMappings/";
private static bool ValidateFile(string file, string schemaFile)
{
if (file == null || schemaFile == null)
throw new ArgumentNullException("Must supply non-null file and Schema file to MappingFilesParser.ValidateFile()");
logger.InfoFormat("Validating {0} against schema; {1}", file, schemaFile);
XmlSchemaSet schemas = new XmlSchemaSet();
schemas.Add(SCHEMA, schemaFile);
XPathDocument x = new XPathDocument(file);
XPathNavigator nav = x.CreateNavigator();
nav.MoveToFollowing(XPathNodeType.Element);
IDictionary<string, string> schemasInFile = nav.GetNamespacesInScope(XmlNamespaceScope.All);
bool foundOurSchema = false;
foreach (KeyValuePair<string, string> namespaces in schemasInFile)
{
if (namespaces.Value.CompareTo(SCHEMA) == 0)
foundOurSchema = true;
}
if (!foundOurSchema)
throw new XmlSchemaValidationException(string.Format("The file {0} does not reference the required schema; {1}",
file, SCHEMA));
XDocument doc = XDocument.Load(file);
doc.Validate(schemas, (o, e) =>
{
throw new XmlSchemaValidationException(string.Format("{0} validating {1}", e.Message, file));
});
return true;
}


Here, we first confirm that the XML file references the XSD, then validate against the XSD.

Monday, 8 June 2009

FileTreeView – a SequioaView-like Application

This post has moved to http://www.scottleckie.com/2009/06/filetreeview-%e2%80%93-a-sequioaview-like-application/

I've long been a huge fan of the SequoiaView application released by Technische Universiteit Eindhoven, which displays disk utilization in a beautiful squarified cushion treemap format.
This was released in 2002 and does a great job of showing exactly what's eating the space on your disk, but it has one major drawback; if you point it at a 2TB volume with a million files, but you only want to see what's taking the space in a small corner of the disk, it reads the entire volume before displaying what you originally asked it to.
So, I decided to write a C# alternative to SequoiaView, partly to help us find the big files in specific folders really quickly, and partly just as an exercise.
Article
You can download the setup from here. Do be aware that this requires the dotNet Framework 2.0 or higher and, while the setup is supposed to go fetch this if required, it does seem a little flakey. In other words, it’s probably safer to ensure you have the .Net Framework 2.0 or higher installed before starting.
You also need to install the Microsoft Data Visualization Components, which are available here.

Source

I’m going to be uploading the source, plus a background article, to CodeProject shortly. I’ll come back and update this link then.

Using FileTreeView

Refer to the following picture for each of the components within FileTreeView;
application_guide
  1. Enter the path or folder that you want to display here, or press the "..." button to browse, then press "Go"
  2. The number of folders and files found so far will be displayed here. You can press "Cancel" if you get bored and it will display what it has so far
  3. This rather nifty set of colours (actually seven little label controls) allows you to set the node colours
  4. By default, the names of the files and folders will be displayed at all depths (which means from the top folder down to the deepest folder). Use this slider to de-clutter the display by displaying labels only to the depth of your choosing
  5. This is the TreeMap control, that displays all discovered folders and their contents, grouped by the relative size of each folder
  6. Each folder or file is a node. You can hover over a node to see its details, double-click to drill down or right-click for more options
  7. Right clicking a node allows you to display it in Windows Explorer, open it, or zoom in and out of the tree structure

Licence

The FileTreeView application is open source, under the CodeProject Open License. Note that the Microsoft Data Visualization Components are free to use for non-commercial use only. See here for specific licence terms.

Help

If you have any suggestions or issues, please post a comment to this article.

Friday, 15 May 2009

Horrible error running against a 32bit .Net Library from a 64bit application

This post has moved to http://www.scottleckie.com/2009/05/horrible-error-running-against-a-32bit-net-library-from-a-64bit-application/

So, this is my first new dev project on my shiny new Windows 7 64bit machine (having come from a 32bit Vista box). As it happens, I want to use the Microsoft Data Visualization Components, which are only available as a set of 32bit DLLs compiled sometime in 2006.
OK, all good so far – loaded ‘em up, referenced them, threw a TreeMap control on my form, compiled fine and then ran it, and… Bang!
image
Got a message;
Could not load type 'Microsoft.Research.CommunityTechnologies.Treemap.NodeColor' from assembly 'TreemapGenerator, Version=1.0.1.38, Culture=neutral, PublicKeyToken=3f6121a52ebf7c82' because it contains an object field at offset 0 that is incorrectly aligned or overlapped by a non-object field.
Turns out the problem is that I’m referencing a 32bit component from a 64bit application, and that there is a union of objects on a 32bit boundary, not a 64bit one. Think that’s right, but anyhoo, the cure is obvious in hindsight – my application needs to be compiled as 32bit.
As this is my first foray into development on a 64bit machine I didn’t actually know where to set this! Found it eventually, though; on the Project Properties form, go to the Build tab and set “Platform Target” to “x64”. I’ll need to remember to make a conscious decision in the future whether a new app is supposed to be 32 or 64 bit!
image

Downloaded CHM help file shows “Navigation to the webpage was cancelled”

This page has moved to http://www.scottleckie.com/2009/05/downloaded-chm-help-file-shows-%e2%80%9cnavigation-to-the-webpage-was-cancelled%e2%80%9d/

Huh – that was an odd one. I copied a bunch of controls over to my Windows 7 box and the DLLs work, but the help files are stuffed;
image
It seems this is common problem, fixed by right-clicking the CHM file and clicking on an “Unblock” button which I’ve never seen before!
image
I can’t claim credit for this – found it at Rob Chandler’s blog here. Weird sense of priorities, huh? OK to copy and run any old DLL that you found lying in the bin, but a helpfile? Woooohhhh… no… that’s much more dangerous!

Thursday, 14 May 2009

Microsoft Data Visualization Components on Windows 7

this post has moved to http://www.scottleckie.com/2009/05/microsoft-data-visualization-components-on-windows-7/

I’ve been playing around with the Data Visualization Components recently (looking to incorporate the TreeMap control with a SequioaView-a-like disk space analyser) but ran into problems getting the toolkit installed on Windows 7 RC1. Running the setup from the official page at http://research.microsoft.com/en-us/downloads/dda33e92-f0e8-4961-baaa-98160a006c27/default.aspx gets stuck looking for .Net Framework 1.1.4322;
image
Of course, being Windows 7, .Net 3.5 is already installed which should include .Net 1.1 but it looks like the Components installer is hopelessly confused. I couldn’t get the installer to believe we had something better than 1.1 already installed and I didn’t want to try and hack .Net 1.1 on top of Windows 7.
So, all I really needed were the “bunch of files” that come in the Component setup so I installed the package on an XP machine and copied them across to a folder on Windows 7, then referenced the DLL by hand.
I doubt there’s any .Net developers out there, running the RC of Windows 7, who doesn’t have a spare XP machine lying around too(!) but, just in case there is, I’ve placed the Zip here for your convenience. The licence states that it’s OK to distribute for non-commercial use and doesn’t say it needs to be in the original MSI format, so I don’t see a legal problem with this.

Sunday, 12 April 2009

“The breakpoint will not currently be hit. No symbols have been loaded for this document.” - VS2008

This post has been moved to http://www.scottleckie.com/2009/04/%e2%80%9cthe-breakpoint-will-not-currently-be-hit-no-symbols-have-been-loaded-for-this-document-%e2%80%9d-vs2008/

I’d been hacking around with Sharp Architecture (#Arch)a few months back but haven’t touched it recently. However, the release of 1.0 to coincide with the formal release of ASP.NET MVC 1.0 got me interested again, so I downloaded the latest and greatest to see what’s adoing…
There’s a great community around #Arch and it’s pretty easy to get your head around (assuming you’ve a basic grounding in MVC and NHibernate) and it even comes with a sample/tutorial app based on the ubiquitous NorthWind database. Now, being a cautious sort I figured I’d start by getting the NorthWind sample up and running, as this would prove I had all the dependencies installed and wired up.
All you (should) need to do is restore the standard NorthWind sample database, point the nhibernate.config at your local SQL Server, and you’re up and running. Except I wasn’t; all I was getting was “Internet Explorer cannot display the webpage”. OK; double check nhibernate.config and assemblies, turn up NHibernate logging, and stick a breakpoint on Global.asax.cs – that’ll pinpoint the error, right?
Wrong. The files are not being loaded, and my breakpoints have gone a funny shade of yellow, instead of red. Hovering over them displays the message “The breakpoint will not currently be hit. No symbols have been loaded for this document.” – Crikey! Much googling and assuming that the root cause must be astoundingly complex led me up several dead ends until I figured out that the cause was much simpler. Have a look at my (truncated) IE dialogue;
image
It’s talking to “localhost” which obviously(!) is my PC. But check out the zone display at the bottom; Internet! What’s it doing thinking my PC is in the Internet zone (with attendant security restrictions)? I tried adding localhost to the Intranet zone but this doesn’t appear to work either. In desperation, I thought “let’s ping it” even though I did not expect that to fail. Well, it didn’t fail, but look at this;
image
Looks ok. WAIT! what the heck is “::1”? Well, the more new-fangled amongst you will recognise this as the IPv6 version of localhost. Quite what IPv6 actually is, is beyond the scope of this article, but the main point is that 99.999% of the Internet and attendant applications expect IPv4 addresses, not IPv6. The IPv4 version of localhost should be the much more recognisable “127.0.0.1”.
So, I went back to my IE page and tried http://127.0.0.1:2386/ instead of the localhost version and, lo and behold;
image
It works! Final bit of the jigsaw is, where on earth is it getting ::1 from? Well, it was getting it from the hosts file, which lives in your <windows>\System32\Drivers\etc folder. Mine contained this;
image
So, I overwrote the ::1 entry (the IPv6 version of localhost) to be 127.0.0.1 (the IPv4 version);
image
Retried my web browser and all is now good with “localhost”;
image
Here’s my question, though; I didn’t change the Hosts file, and localhost certainly did work last time I was playing with #Arch, so what changed? Did a Microsoft security update overwrite the Hosts’ localhost entry?
Oh, one other thing. If you are using User Access Control (UAC) then access to the Hosts file will be restricted. The solution is to go find Notepad in your Start Menu and don’t left click on it; right-click it, and select “Run as administrator” then you will be able to save the Hosts file.

Auditing access to a file on Vista

This post has been moved to http://www.scottleckie.com/2009/04/auditing-access-to-a-file-on-vista/

Every so often, when I open Outlook, it tells me that my PST file was not closed properly and it’s being checked for errors. It seems from a spot of googling that this isn’t an Outlook problem as such, more that another app opened my PST and did not close it properly.
Task Manager or Anvir Task Manager (excellent app) shows that, whoever the culprit is, it’s not running the next time I start Outlook. My suspicions are firmly on Skype, especially as there are a ton of postings on it keeping the PST open, but I’ve checked the obvious settings and they don’t seem to be the culprit.
Right, I thought, let’s turn on file auditing and I can see which process accesses the PST file… Wow! What a faff this is, in Vista. Prior to Vista, auditing was really easy but now things have got way more “powerful” (according to Microsoft) or “difficult” (according to me). What is particularly galling is that you can even go and enable access audits on a specific file, and Vista won’t give you a hint that File System auditing isn’t even enabled! So, after a bit of digging around, this is how you audit access to a file;

Turn on File System auditing

You need to use a horrid app called AuditPol.Exe – the semantics of this are a bit too esoteric for me, but here’s what you need to enable file system auditing;
1: D:\Windows\system32> auditpol /set /subcategory:"file system" /success:enable /failure:enable


Note that the whole “auditpol” command is on one line.




Enable Auditing for a file



Now, go and right click the file you want to be audited, right click and go to properties. Go to the “Security” tab, and then click on “Advanced”. Now that you are in the Advanced dialogue go to the second tab, which should be called “Auditing”. You may have a button marked “Continue” if you are still using UAC – if so, click this and accept the next dialogue;


image


Finally, you get the auditing dialogue. Clock on “Add” and enable the auditing that you’re looking for. If you just want to know what / who opens the file, you just need “List folder / read data” auditing.


image


Very important; the “Name” entry defaults to your user ID, which means that it will only audit access by you. If you want to record access by anyone (and remember services may be running under the SYSTEM or other account) then you need to click the “Change” button and set it to audit access by “Everyone”.






Checking the event log for audit records



Last stage; you can check the Windows Event Log (via the “Administrative Tools” group or by running EventVwr.exe) to see audit results. Check the Windows Logs / Security log for event ID 4663, or the Task Category of “File System”;


image




Remember to disable when you fixed your problem



Best advice suggests that, once you’ve got to the bottom of your file access problems, you should disable auditing. To do this, follow these steps in reverse order; remove the audit options from the file, then disable auditing with AuditPol.ex;


1: D:\Windows\system32> auditpol /set /subcategory:"file system" /success:disable /failure:disable