SchemaLightener turns 4 years old ... and still free!

The initial release of the SchemaLightener was four years ago this month.  And I've been getting requests for it every week since.  When I started, I didn't think it would last this long, but its taken on a life of its own.  I simply wanted to work with libraries of schemas easier.  I'd been frustrated at the inability of tools to lighten or flatten them.  So I made my own.  Four years on, I'm still happy to be getting requests and great feedback.  It's being used in over 19 countries and is still free.

It was a simple design, built with XSLT 1.0.  And now, its been updated and includes a nice GUI interface, sample data, batch files, and an ANT task as well.  The most important developments of course have been the expansion into three tools in one.  The Lightener surely does lighten schemas, providing sample based profiling of a larger data model.  But the addition of the SchemaFlattener makes working with schemas much easier as well.  It takes a tangled web of includes and condenses them down into the smallest number of files necessary (one per namespace).  The WSDLFlattener, a third tool, does the same thing for WSDL files.  Merging files to the minimum so they can be managed more easily.

The tool has been tested with many consortium libraries including  OAGi – Open Application Group – http://www.oagi.org, Swift – Society for Worldwide Interbank Financial Telecommunication – http://www.swift.com, NIEM – National Information Exchange Model – http://www.niem.gov, HR-XML – Human Resources XML – http://www.hr-xml.org, OTA – Open Travel Alliance – http://www.opentravel.org, ACORD – Association for Cooperative Operations Research and Development  – http://www.acord.org, STAR – Standards for Technology in Automotive Retail – http://www.starstandard.org.  Plus countless company libraries and extensions.

So if you're interested in getting a copy, just send me an email.  No spam or mailing list risk - so don't worry about getting hit with unwanted mail.  And thanks for all the coffee and support over the years!

Profiling Xml Schema

I've gotten some recent offline comments about the article I wrote called "Profiling Xml Schema" on xml.com.  Its been quite a while since I wrote the piece.  At the time I was attempting to cull a "best practices" guide on how schema developers are actually using this technology.  I've run into too many tool bugs with the more advanced aspects of the spec. Looking back, the article holds up pretty well.  Most of the recommended practices are still the norm today.
  

An overdue apology to Pink Floyd's Roger Waters

When Roger Waters left Pink Floyd, sides were drawn.  I was a huge admirer of David Gilmour's playing.  He had the ability to transform, transport, and transcend via auditory travels.  His hypnotic playing had captured me for a long time.  And I'd moved with him and the band.
I'd also been admiring of the lyrical prowess of the band.  Interestingly most attributable to Roger Waters.  But as Waters and Gilmour split their working relationship, I chose my sides according to my musical muse.  Gilmour was the shaman and he was still in the band called Pink Floyd.  Anyone leaving the band must have been the problem.  Well, there was no need for the jury to retire.  Waters was guilty and hence the band was "saved" by his exit.  Judgment passed and all was again well in the land.
But a nagging disconnect existed.  My admiration of the written, lyrical word was always strong and only continued to grow over the years.  The lyrics that I most loved in Pink Floyd's music were mostly written by Waters.  How could I disavow the contribution of the lyrical poet?  Like a pebble in my shoe, I walked on in my life with uncomfortable steps not knowing how to accept this disconnection.  Each step accumulating its nagging bite.
Specifically, I can talk of the pinnacle of Pink's work, The Wall.  (While I worship at the altar of numerous works by the band, one has to admit that this is the seminal work they produced.)  An anthem for the young male psyche, its words and music match and meld.  And the movie added yet another infallibility to the work of art.  There is neither an imperfect sound nor word in the piece, I held.  I knew every eighth note and syllable.  And I knew each small difference between what was on the record (vinyl at first) and what was in the movie.
So given the esteem with which I had elevated this combination of word, note, and image, how could I disavow its most responsible architect?  The pebble grew into a thorn.
Fast forward to 1990.  The Berlin Wall had recently come down and Roger Waters was producing a concert in Berlin performing The Wall.  While admittedly historic, I had chosen my path long ago and I was not open to receive it.  It wasn't Pink Floyd, it was Roger Waters.  It wasn't David Gilmour, Richard Wright and Nick Mason.   So it must be imperfect.  I did not watch it nor get the CD or DVD of the performance.  Out of a sense of loyalty I believed.
But an impostor? ... the man most responsible for the work?  The thorn had been affecting my reasoning.
Forward again to 2011.  I was invited to a friend's house to partake in beer and The Wall movie.  I'd lost count as to how many times I'd seen it.  Dozens for sure.  But this time was different.  After the movie, my friend talked about the Berlin show and put the DVD into the player.  Skeptically, I placated him by sitting through a taste, believing it couldn't be the perfection I'd known for so long.  I only saw a portion, but took it home to view in full, which I did the next week.
The DVD shined the brightest light on my aching foot.  Each song.  The show.  The performance.  It was all amazing.  It repeatedly brought chills to my skin.  I am after all a history major, and the significance of this piece at the Berlin Wall in 1990 was another window into this work.  Inspired casting (i.e. Thomas Dolby as the Professor).  It was ... well ... outstanding.  How can this be?  Waters was supposed to be the problem.  And yet, this show was incredible.  Had "I" been guilty all this time?  This will not do.
I finally removed the obstruction and opened up to a new ending to the story.  Pink's story.  Waters really is an architect of The Wall's greatness.  He isn't an impostor or the problem.  He is as worthy as Gilmour to the immortal legacy of Pink Floyd's music.  (Not just The Wall, but all of it.)
And so I offer Roger Waters a heart felt and belated apology.  An apology for eschewing your work and contributions all these intervening years.  An apology for judging.  And for letting that judgment interfere with the appreciation of your wonderful art.  The art that has touched my life and made me feel like few other things.  I hereby apologize.
The bleeding hearts and artists make their stand.

ID3v2 tag editing - how I wish I could use Xml

Trying to edit various tags in my music collection.  The good news is that there is a standard data format, namely ID3v2.  The bad news is that the files can't be edited easily.  Specialized tag editors are needed to embed the artist name, album, genre etc into the mp3.  There is an awesome one I've used called Mp3tag.  Its free and works very well.  But things like embedding the album cover art is impossible to do.  Let alone other automated tasks.
I find myself wanting to get into the details of the tagging and perhaps write some scripts to automate some tagging tasks.  And I keep thinking I'd like to be able to use Xml to manage the metadata.  I could get the tools and edits I want done very fast.  And no need to get some specialized editor that has only some of the features I want.  Xml is just text, so any text editor would work.  And it manages delimited metadata very well.
Until the standard changes, I'm stuck using someone else's tool.

MISMO Trimester Meeting mentions use of Schema Lightener / Flattener

Was recently contacted by Greg Alvord, the MISMO Architecture Committee Chairman about the SchemaLightener/SchemaFlattener.  He was doing a presentation that included extensions and wanted to mention this tool.  I've included MISMO in the testing of the tool, so I was happy to have them make use of it.
Below is the title and direct link to the presentation given at the MISMO Trimester Meeting May 23 - 26, 2011 Las Vegas, Nevada.


Greg Alvord
Understanding and Implementing Version 3.x
http://www.mismo.org/mismo-files/meetings/Version3_1DeepDive.pdf

latest SchemaLightener feedback

"It worked like a charm on my schemas :)  I bought you a few cups of coffee for the effort. Keep up the good work."

"Brilliant thanks Paul.  I've downloaded your tool and it works really well."

"thank you for sending me the tools. The WSDL flattener helped me a lot and I will tell everybody in our office about it."

"Great job on the xml tool"

"Paul, thank you so much!  I’ll definitely buy you a coffee! ;)"

"Please  could you let me know how to get hold of your WSDL flattener – those xs:includes are driving me nuts!"

"Thanks a lot. I did test the sample data and it works beautifully."

"Your XML Schema Lightener and Flattener tool sounds like just what I need."

"Thanks Paul, this worked like a charm. A little bit of hacking at it but we now have the schema we were after! Thanks for your contribution, it would have been a nightmare without your XML Lightner tool."

Schema Lightener / Schema Flattener / WSDL Flattener now updated - and still free

The Schema Lightener / Schema Flattener / WSDL Flattener tool has been recently upgraded. 

And of course it is still absolutely free.

A summary of enhancements:

1. Schema Flattener (and WSDL Flattener) now handle recursive includes.  This has been an ongoing request, and it’s finally done!   See below for details.
2. Schema Lightener can use multiple Xml instances files.  Details on how to do it are blogged here.  A sample multi-file is included in the download.
3. ANT task files are included along with Apache Ant itself.  You can customize this to suit your needs.
4. Updated GUI with fixes and ability to clear logs

As always, if you'd like to help support this effort and keep these tools free, you may choose to buy some coffee here:
http://www.xmlhelpline.com/tools/BuyMeACoffee.php
And thank you!

Try it and if you like it, please blog or post about it so we can get the word out.

To get it, simply send me an email and I'll reply with the download link. (Don't worry, I won't spam you with unwanted email.)

Here is a link to background information:
http://www.xmlhelpline.com/tools/

**********************

Regarding the issue with recursive includes (see post): some of you have replied to me with error messages similar to the one below.  This issue was resolved in the Lightener some time ago but was a known limitation in the other tools.  I finally addressed it reusing this code in both the Schema Flattener and the WSDL Flattener.  So now no matter which tool you use, it can handle recursive includes.
------------------------------
Processing ...
Exception: net.sf.saxon.trans.XPathException: Too many nested template or function calls. The stylesheet may be looping.
Too many nested template or function calls. The stylesheet may be looping.; SystemID: file:/C:/tinyurl/xmlHelplineFreeTools-3.1-working/SchemaFlattenerF.xslt; Line#: 566; Column#: -1
------------------------------