XML through PowerShell

Thomas Lee Read more October 7, 2009

XML is fast becoming a must-know technology for IT professionals: under Windows Vista and Longhorn server, it’s a core technology, used for event log entries, unattended setup files and group policy settings. 2007 is the year you should learn more about XML. In this month’s column, I’ll look at what XML is and how to use it, and instead of the usual C# code I’ll be illustrating using PowerShell, since you can access .NET directly from within it. Its built-in Reflection capability (via the Get-Member cmdlet) lets you use PowerShell to explore the XML capabilities of .NET.

XML is a mark-up language for encoding information and services that both computers and humans can understand. It enables you to create documents that contain structured information, and it looks superficially similar to HTML, since both derive from the older Standard Generalized Mark-up Language (SGML). An XML document consists of some textual data, together with mark-up tags that describe the structure of that data: each tag identifies some data item with a name, followed by the data itself. For example, an XML document containing information about a book might look like this:

<Book>

<Title>Windows Server 2003 TCP/IP Protocols and Services</Title>

<Author>Joseph Davies</Author>

<Author>Thomas Lee</Author>

</Authors>

<Publisher>Microsoft Press,U.S</Publisher>

<Language>English </Language>

<ISBN-10>0735612919</ISBN-10>

<ISBN-13>978-0735612914 </ISBN-13>

</Book>

As you can see, the basic syntax of XML is straightforward: information about a specific book is delineated by a series of tags that define the contents and structure of the document. As in HTML, tags are enclosed in angle brackets and generally come in pairs of opening tag followed by a closing tag formed by prefixing / to the tag name. XML tags also work roughly the same way as they do in HTML. In the example, the title of the book is an element enclosed by an opening <Title> tag and a closing </Title> tag. XML tag names are case sensitive, so the closing tag must be </Title> not </title> (which would leave the tag still open). Tags can also stand on their own, as with that <InPrint/>, denoted by the name being followed by a /. Standalone tags are used as flags, in this case to denote the book is still in print: it could have been written instead as <InPrint>Yes</InPrint>.

Tags can be nested to indicate deeper structure, as where the <Authors> tag holds two <Author> tags, a structure that expresses the fact that a book can have more than one author. Tags may also contain attributes, as illustrated in the <Price> tag, whose attribute “currency” is given the value “sterling”.

XML documents are composed of a series of one or more document elements, where each element may possess zero, one, or more child elements. A well-formed document must have just a single root element – in the example, this is the <Book> element. But there’s far more to XML syntax than I have space for here, and for a set of good tutorials check out www.w3schools.com/default.asp.

XML schemas

So much for XML syntax, but what do the individual elements mean? For example, what does <Price> represent in the book document? Is it an amount of currency and, if so, which one? Or is it some internal index number? An XML schema is where you describe the meanings to be attached to the contents of an XML document. The schema is another XML document, which specifies what elements are allowed to appear in the document, the order of those elements, their numbers of child elements, and so on. For my book example, I could create a schema that defines what a book document must look like (it has a root element called <Book> and so on) and what individual elements represent (<Price> is the selling price in pounds sterling). Using a strict XML schema to validate the document would have made that use of the currency attribute unnecessary.
One final important aspect of XML regards formatting and document transformation. An XML Style Sheet enables you to transform an XML description into a new document, for example an HTML document. HTML employs predefined tags whose meaning is well understood by all web page authors and browsers: for example, the <table> element starts a new table, which the user’s browser knows how to display. You can easily add styles to an HTML document by supplying a Cascading Style Sheet. However, XML doesn’t employ predefined tags and you’re free to choose any tag names you like A <table> element in XML might mean an HTML table or a piece of furniture: it’s the XML Style Sheet that describes how each tagged element should be displayed.

XML Style sheets can also be used to transform one XML document into another, possibly breaking down some elements or moving them around, and such manipulations aren’t uncommon in systems that exchange XML documents.

Processing XML in code

An XML document looks like a tree with a single root element and multiple child elements. You can process such a tree either on a node-by-node basis or by pulling the entire tree into memory and processing its various elements. Java developers will no doubt be familiar with the Simple API for XML (SAX), which supports node-by-node parsing. The whole-tree approach is known as the Document Object Model (DOM), and is flexible because it lets you navigate any element in the tree as needed. On the other hand, SAX is faster because it doesn’t need to load the entire document into memory, looking at just a single node at a time and reading only forwards. Both approaches work, but .NET doesn’t implement SAX directly.

I’m going to use PowerShell to illustrate the manipulation of XML documents, for a couple of reasons. First, PowerShell provides a great set of tools for exploring XML documents, since it utilises the .NET Framework’s XML capabilities directly, including all of the System.XML namespace; second, with XML becoming so important to IT professionals, you may find some of the techniques I describe useful when managing XML documents. A word of warning, though: it may take a bit of time to get comfortable with XML processing within PowerShell. It took me a while, but it quickly became obvious how powerful PowerShell can be in processing XML documents.

Open up a PowerShell window now and try out some of these commands (I’ll put the XML documents I use in the column onto my website at www.reskit.net/pcpro). To open up an XML document in PowerShell, issue the following command:

$doc= [xml] (cat book.xml)

This is a neat bit of shorthand that in effect opens the file and creates an XML document based on any text in the file. For those more used to C#, the following PowerShell syntax would also work and might be more familiar to you:

$doc= New-Object xml

$doc.load(“book.xml”)

In this case, I first create a new XML document object, and only then use a built-in method to load the file. In both cases, PowerShell creates a variable called $doc, which is an XML document, and you can access $doc to get at the information contained in the XML document. Opening the document this way uses the DOM, and PowerShell creates an in-memory representation of the document you can then manipulate.

To see the details of all the properties and methods implemented by the $doc object, you can use PowerShell’s Get-Member cmdlet, as follows:
$doc | Get-Member

When you try this, you’ll see that the output includes 90 individual methods you can invoke on the object. In addition, you can see a document-specific property (Book). The methods available within $doc are those implemented by .NET for the System.Xml.XmlDocument class.

Manipulating an XML document

Once you have this document loaded into PowerShell, you can address its individual elements directly. For example, to get the book’s price, simply type $doc.book.price; to get the book currency attribute, type $doc.book.price.currency; or to obtain the book’s language, type $doc.book.language. The screenshot above shows how you can directly address the elements in the XML document.

I first open the XML document, then display the book’s price element, which is in fact an XML element (and is being formatted using PowerShell’s default formatter). I can also directly address the currency attribute, but addressing the price amount is more tricky: since PowerShell uses the “#” symbol as a comment, the parser can’t interpret $doc.book.price.#text, nor can I use PowerShell’s escape character ($doc.book.price.`#text won’t work either). You could argue this is a limitation of PowerShell’s V1 parser, but with a bit of trickery it’s possible to get around the limitation and get to the price. The last PowerShell command shows how to iterate through a repeating group of XML elements (the <Authors> element) and obtain the individual authors.

Using these techniques, the following is a simple PowerShell script to print out the book details to the console:

#book1.ps1

#Simple Script to print detail of a book

$doc= [xml] (cat book.xml)

$a = [string] “”

$au = [string] $doc.book.authors.author

$t = [string] “#text”

$price=[double] $doc.book.price.$t

$bookprice=$price.tostring(“C”)

#Write Details

Write-Host “Book Details: $($doc.book.title)”

Write-Host “Publisher: $($doc.book.publisher)”

Write-Host “Price: $bookprice ‘($($doc.book.price.currency)’)”

Write-Host “Author(s): $au”

If ($doc.book.inprint.length -ge 0) {

Write-Host “Book in print: Yes”

}

This script retrieves information about the book, including the book’s price from the document, and then writes the information to the console in a formatted fashion.

The script is certainly not “production ready”, since there’s no error handling in it – it will work fine so long as the XML document is well formed and contains those elements named in the script. But if, for example, the XML had no <Authors> tag, it wouldn’t work properly.

Writing scripts like this one might be useful when working on Vista Infrastructure, since group policies, event logs, deployment scripts, IIS configuration and much more are now based on the use of XML. PowerShell provides you a simple way into the richness of XML and .NET’s XML support.

<strong>PowerShell and XPath</strong>

XPath is a language for finding information within an XML document and navigating through its contents. XPath is another W3C standard you can use with .NET and PowerShell: see www.w3schools.com/xpath/xpath_intro.asp for an introduction to XPath and www.w3.org/TR/xpath20 for a formal definition. With XPath, you create an expression that identifies the XML element(s) you wish to process, and then call SelectSingleNode or SelectNodes methods using that expression. In the simple book document above, you might specify $doc.selectnodes(“//Author”) to select the Author elements from the book document:
PSH [D:\foo]: $doc= [xml] (cat book.xml)

PSH [D:\foo]: $doc.selectnodes(“//Author”)

#text

—–

Joseph Davies

Thomas Lee

PSH [D:\foo]:

Using XPath within PowerShell is probably more appropriate for larger XML documents. To demonstrate this, I’ll use an XML version of Shakespeare’s Romeo and Juliet, which you can download from www.ibiblio.org/xml/examples/shakespeare/r_and_j.xml. Using the same method of opening up the XML file I showed you earlier, you can open up the play in PowerShell:

$r=[xml] (cat r_and_j.xml)

When you run this statement, you’ll notice a slight delay as the file is read and PowerShell creates a DOM object for you. The file itself occupies around 225KB on disk, and consumes just over 500KB in PowerShell memory. You can use XPath on this document, for example, to list the players as follows:

$r.selectnodes(“//PERSONA”)

Note that the XPath expression is case sensitive, whereas normal PowerShell commands aren’t. Thus, in PowerShell, the same XPath statement could also be achieved using $r.play.personae.persona, which isn’t case sensitive. As with my earlier document, you can use PowerShell to extract parts of the document for manipulation, as shown in the screenshot above, right.

Persisting objects using XML

More neat features of PowerShell’s support for XML are the Export-Clixml and Import-Clixml cmdlets. Export-Clixml is able to create an XML-based representation of some object and then export it to a file, while Import-Clixml imports a Clixml file and creates corresponding objects within PowerShell. You could create and export a simple integer variable, as follows:

$i = [int] 21

$i | export-clixml int.txt

The contents of the file int.txt look like this:

</Objs>

You can re-import this saved XML file using Import-Clixml:

$j=import-clixml int.txt

These two cmdlets enable you to use XML documents to store persistent PowerShell objects, which can, of course, be more complex than a simple integer. For example, you could persist the directory listing for a folder in XML using this syntax:

ls D:\foo | export-clixml files.xml

And, of course, once you have this information in an XML document, you can use all the techniques described above to process that document.

Extending XML support

The XML support in PowerShell V1 is a great start, but it’s by no means complete. In parallel with the completion of PowerShell V1, a community effort has been under way to provide some useful extensions. These PowerShell Community Extensions (PSCX) are intended to provide extra cmdlets, providers, aliases, filters, functions and scripts for Windows PowerShell, which you can download from www.codeplex.com/PowerShellCX.

The PSCX extensions provide two useful XML-related cmdlets called Convert-XML and Format-XML. Convert-XML is used to perform XSLT transformations on XML documents, while Format-XML is used to tidy up XML for better display. For example, you could use Convert-XML to prettify books.xml:

PSH [D:\foo]: $doc=get-xml book.xml

PSH [D:\foo]: Format-xml -inputobject $doc -attribute -indentstring ” ”

<Book>

<Title>Windows Server 2003 TCP/IP Protocols and Services</Title>

<Author>Joseph Davies</Author>
<Author>Thomas Lee</Author>

</Authors>

<Publisher>Microsoft Press,U.S</Publisher>

<Language>English</Language>

<ISBN-10>0735612919</ISBN-10>

<ISBN-13>978-0735612914 </ISBN-13>

<Price

currency=”sterling”>36.99</Price>

</Book>

PSH [D:\foo]:

I’ve shown you some of the simpler aspects of XML and how to use PowerShell to manipulate them. If you know the structure of an XML document, you can easily manipulate it in PowerShell, and if you don’t know its structure you can use PowerShell to discover and exploit it.