Invisible XML is a language for describing the implicit structure of data, and a set of technologies for making that structure explicit as XML markup. It allows you to write a declarative description of the format of some text and then leverage that format to represent the text as structured information.
Specification
Invisible XML (ixml) 1.0 was published on 10 June, 2022. The same specification was published as a Final Community Group Report by the W3C on 12 December, 2023. The grammar of Invisible XML is defined with Invisible XML. The ixml grammar for ixml is available in ixml format or in xml format. Invisible XML was developed by the Invisible Markup Community Group (CG) at the W3C.
The current working draft is also available. This draft includes the resolved errata and any additional proposals adopted since the 1.0 specification was published.
Discussion
If you’re interested in following along in the discussion, please join the community group! You can also:
- Browse the archives of the mailing list (and subscribe!)
- Review the issues list.
- Keep track of current proposals on the dashboard.
Tutorials
- Steven Pemberton’s
hands-on tutorial
is a great way to begin learning how to write your own grammars.
- His advanced tutorial explores some more complex Invisible XML parsing challenges.
- Norm Tovey-Walsh has written introductory material about ixml and about writing grammars on xml.com.
Implementations
- Aparecium is Michael Sperberg-McQueen’s proof-of-concept implementation in XQuery.
- CoffeePot is Norm Tovey-Walsh’s Java implementation.
- EarleyBird is M. Joel Dubinko’s Rust implementation (in development)
- Hywel is Bethan Tovey-Walsh’s Python implementation (in development)
- ixampl is Steven Pemberton’s implementation (it is available online as a web service).
- JayParser is Tomos Hillman’s XSLT implementation (in progress).
- jωiXML is John Lumley’s browser-based JavaScript implementation.
- Markup Blitz is Gunther Rademacher’s Java implementation.
- Fredrik Öhrström has added Invisible XML support to XMQ.
- Alain Couthures is adding Invisible XML support to XsltForms.
Test suite
The ixml GitHub repository includes a comprehensive test suite. It is also available as a browsable catalog.
Schemas
The XML versions of Invisible XML grammars conform to the RELAX NG schemas provided.
Example grammars
A selection of sample grammars are available including:
- ABNF, the grammar notation used in IETF Requests for Comments, as defined in RFC 5234.
- ISBN, International Standard Book Numbers.
- ISO 8601, the standard ISO date, time, datetime, duration, interval, and recurrence formats, as defined in ISO 8601:2004.
- BCP 47, tags for identifying languages, as defined in BCP 47.
In addition, Gunther Rademacher's grammar converter is able to convert Invisible XML to W3C-style EBNF. This can be used to produce syntax diagrams using the RR tool.
Other resources
- Gingersnap is a library of XSLT modules for processing (the XML form) of Invisible XML grammars.
- Steven Pemberton discussed Invisible XML in a podcast for BrightTALK.
Other things named “ixml”
If you found your way to this page with a web search for “ixml” and you weren’t looking for Invisible XML, you might have been looking for one of these:
- iXML, a standard for embedded metadata in production media files.
- ixml, an iterative event-driven XML parser with a standard Python iterator interface.
- IXmlNode, a Microsoft DOM interface.
- iXML Library, an SAP API for handling XML documents in DOM format. (sic)
Bibliography
iXML in papers and presentations
Additional papers and presentations on the development of Invisible XML
- Hillman, Tomos. XSLT Earley: First Steps to a Declarative Parser Generator. Presented in two halves at XML Prague, 2020 and Declarative Amsterdam, 2020.
- Lumley, John. “Variations on an Invisible Theme: Using iXML to produce XML to produce iXML to produce …” Presented at Balisage: The Markup Conference 2024, Washington, DC, July 29 - August 2, 2024. In Proceedings of Balisage: The Markup Conference 2024. Balisage Series on Markup Technologies, vol. 29 (2024). https://doi.org/10.4242/BalisageVol29.Lumley01.
- Pemberton, Steven. Invisible XML: introduces the concepts, and develops a notation to support them.
- Pemberton, Steven. Data just wants to be (format) neutral: discusses issues with automatic serialisation, and the relationship between Invisible XML grammars and data schemas.
- Pemberton, Steven. Parse Earley, Parse Often: How to parse anything to XML: discusses issues around grammar design, and in particular parsing algorithms used to recognise any document, and converting the resultant parse-tree into XML, and gives a new perspective on a classic algorithm.
- Pemberton, Steven. On the Descriptions of Data: discusses the Usability of Notations: Discusses changes to the design following experience with using it, giving examples of its use to develop data descriptions, and in passing, suggests other output formats.
- Pemberton, Steven. On the Specification of Invisible XML: describes decisions made during the production of the specification of ixml.
- Pemberton, Steven. “Round-tripping Invisible XML”, in Proc. XML Prague 2024, Prague, Czechia, 2024, pp 153-164, ISBN 978-80-907787-2-6.
- Sperberg-McQueen, C. M. “Aparecium: An XQuery / XSLT library for invisible XML.” Presented at Balisage: The Markup Conference 2019, Washington, DC, July 30 - August 2, 2019. In Proceedings of Balisage: The Markup Conference 2019. Balisage Series on Markup Technologies, vol. 23 (2019). https://doi.org/10.4242/BalisageVol23.Sperberg-McQueen01.
- Tovey-Walsh, Norm. “Ambiguity in iXML: and how to control it.” Presented at Balisage: The Markup Conference 2023, Washington, DC, July 31 - August 4, 2023. In Proceedings of Balisage: The Markup Conference 2023. Balisage Series on Markup Technologies, vol. 28 (2023). https://doi.org/10.4242/BalisageVol28.Tovey-Walsh01.
Related literature
Important and/or interesting references. This list is not exhaustive. Suggestions for additions welcome.
- Brabrand, Claus, Robert
Giegerich, and Anders Möller. “Analyzing Ambiguity of Context-Free Grammars.”
Science of Computer Programming, volume 75, number 3.
Elsevier. (2010).
https://cs.au.dk/~amoeller/papers/ambiguity/.
doi:
10.1016/j.scico.2009.11.002 - Grune, Dick, et. al.
Modern Compiler Design, second edition.
Springer Publishing Company. ISBN:
978-1-4939-4472-9. 01 May 2016. - Loup. Earley Parsing Explained. https://loup-vaillant.fr/tutorials/earley-parsing/parser.
- Schwarz, Keith. Advanced Parsing Techniques. CS143: Compilers. Stanford University. https://web.stanford.edu/class/archive/cs/cs143/cs143.1128/lectures/07/Slides07.pdf
- Scott, Elizabeth. SPPF-Style Parsing From Earley Recognisers. Electronic Notes in Theoretical Computer Science. 203, 2, p. 53-67, Apr 2008. https://pure.royalholloway.ac.uk/en/publications/sppf-style-parsing-from-earley-recognisers
- Scott, Elizabeth and Adrian Johnstone. GLL Parsing. Electronic Notes in Theoretical Computer Science. 253, p. 177-189, 2010. https://dotat.at/tmp/gll.pdf
- Scott, Elizabeth, Adrian Johnston, and L. Thomas van Binsbergen. Derivation representation using binary subtree sets. Science of Computer Programming. 175, p. 63-84 22 p. 15 Apr 2019. https://pure.royalholloway.ac.uk/en/publications/derivation-representation-using-binary-subtree-sets