OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

 


 

   RE: [xml-dev] text files & xsd & regex

[ Lists Home | Date Index | Thread Index ]

Hi Rick,

  Thanks for the response ...

> 1) SGML. It allows you to specify regular expressions
> (content models) together with the delimiters used,
> to read in text, parse it to SGML, then output as events.
> If you have many documents with lots of these, run them
> through an SGML processor.

  I will take a look and maybe steal some ideas but I don't have 
  SGML processor for platform I have to run on ...
  (wrapping SP is unfortunately not an option)

> 2) Check out Xpath2. It seems that it will have some kind of
> syntax for this kind of thing, see 
> http://www.w3.org/TR/xquery-operators/#func-matches
> and the reference to captured substrings.   I guess the
> most consistant thing would be to follow them in some way.

  http://www.w3.org/TR/xquery-operators/#func-tokenize seems to be
  even more usefull but I'm afraid that I couldn't produce more
  nested structures ... (please see below for further elaboration)

> 3) Probably ISO DSDL will have something like this.
> In particular, which features in addition to Regular Fragmentations
> are you interested in?

  as far as I can see RF supports only simple "shallow" structures
  (please see below for further elaboration)

> 4) In the meantime, you can tokenize many kinds of strings 
> and check them for various constraints using Schematron,
> which can be embedded in <appinfo> now and extracted
> using a stylesheet. That does not give you full regular 
> expressions.  (Schematron 1.6 will be out within a month,
> with <let> statements that help you do consecutive substring
> capturing from strings, though this is not as powerful as
> full regular expressions.)  We have a free Windows tool that
> supports embedded Schematron in XML Schemas.

  I'm afraid that Schematron won't help for parsing itself ...

  Maybe better description: annotated XML schema defines structure 
  ("template") to fill with the text matched and splitted by regular 
  expression ... Processor traverses schema object model and when finds 
  an instruction (annotation) it reads line from input file and applies 
  that instruction and emmits XML elements ...

  To better illustrate my aim there is an example (rather simple) of 
  schema (simplified and annotated purchase order from xml schema primer), 
  input file and  result (serialized to file) ... (files are attached too)

    * input file (po.txt) :

ST:Alice Smith;123 Maple Street;Mill Valley;CA;90952;US
BT:Robert Smith;8 Oak Avenue;Old Town;PA;95819;US
I:Lawnmower	1	148.95		872-AA
I:Baby Monitor	1	39.98	1999-05-21	926-AA

    * result file (po.xml) :

<?xml version="1.0"?>
<purchaseOrder orderDate="1999-10-20">
    <shipTo>
        <name>Alice Smith</name>
        <street>123 Maple Street</street>
        <city>Mill Valley</city>
        <state>CA</state>
        <zip>90952</zip>
	<country>US</country>
    </shipTo>
    <billTo>
        <name>Robert Smith</name>
        <street>8 Oak Avenue</street>
        <city>Old Town</city>
        <state>PA</state>
        <zip>95819</zip>
	<country>US</country>
    </billTo>
    <items>
        <item>
            <productName>Lawnmower</productName>
            <quantity>1</quantity>
            <USPrice>148.95</USPrice>
	    <partNum>872-AA</partNum>
        </item>
        <item>
            <productName>Baby Monitor</productName>
            <quantity>1</quantity>
            <USPrice>39.98</USPrice>
            <shipDate>1999-05-21</shipDate>
	    <partNum>926-AA</partNum>
        </item>
    </items>
</purchaseOrder>

    * schema file (po.xsd) :

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema";
xmlns:xyz="http://uri.org";>

 <xsd:element name="purchaseOrder" type="PurchaseOrderType"/>

 <xsd:complexType name="PurchaseOrderType">
  <xsd:sequence>
   <xsd:element name="shipTo" type="USAddress">
<xsd:annotation><xsd:appinfo><xyz:match>^ST:</xyz:match></xsd:appinfo></xsd:
annotation>
   </xsd:element>
   <xsd:element name="billTo" type="USAddress">
<xsd:annotation><xsd:appinfo><xyz:match>^BT:</xyz:match></xsd:appinfo></xsd:
annotation>
   </xsd:element>
   <xsd:element name="items"  type="Items"/>
  </xsd:sequence>
 </xsd:complexType>

 <xsd:complexType name="USAddress">
  <xsd:annotation>
    <xsd:appinfo>
    <xyz:split>
<![CDATA[:(?<name>[^;]*);(?<street>[^;]*);(?<city>[^;]*);(?<state>[^;]*);(?<
zip>[^;]*);(?<country>[^;]*)]]>
    </xyz:split>
    </xsd:appinfo>
  </xsd:annotation>
  <xsd:sequence>
   <xsd:element name="name"   type="xsd:string"/>
   <xsd:element name="street" type="xsd:string"/>
   <xsd:element name="city"   type="xsd:string"/>
   <xsd:element name="state"  type="xsd:string"/>
   <xsd:element name="zip"    type="xsd:decimal"/>
   <xsd:element name="country" type="xsd:string"/>
  </xsd:sequence>
 </xsd:complexType>

 <xsd:complexType name="Items">
  <xsd:sequence>
   <xsd:element name="item" minOccurs="0" maxOccurs="unbounded">
<xsd:annotation><xsd:appinfo><xyz:match>^I:</xyz:match></xsd:appinfo></xsd:a
nnotation>
    <xsd:complexType>
      <xsd:annotation>
        <xsd:appinfo>
          <xyz:split>
<![CDATA[:(?<productName>.*?)\t(?<quantity>.*?)\t(?<USprice>.*?)\t(?<shipDat
e>.*?)\t(?<partNum>.*)]]>
          </xyz:split>
        </xsd:appinfo>
       </xsd:annotation>
     <xsd:sequence>
      <xsd:element name="productName" type="xsd:string"/>
      <xsd:element name="quantity">
       <xsd:simpleType>
        <xsd:restriction base="xsd:positiveInteger">
         <xsd:maxExclusive value="100"/>
        </xsd:restriction>
       </xsd:simpleType>
      </xsd:element>
      <xsd:element name="USPrice"  type="xsd:decimal"/>
      <xsd:element name="shipDate" type="xsd:date" minOccurs="0"/>
      <xsd:element name="partNum" type="xsd:string"/>
     </xsd:sequence>
    </xsd:complexType>
   </xsd:element>
  </xsd:sequence>
 </xsd:complexType>

</xsd:schema>

  Thank you again for ideas ...

Regards

Martin

-- 
Martin Krumpolec <krumpo@pobox.sk>

ST:Alice Smith;123 Maple Street;Mill Valley;CA;90952;US
BT:Robert Smith;8 Oak Avenue;Old Town;PA;95819;US
I:Lawnmower	1	148.95		872-AA
I:Baby Monitor	1	39.98	1999-05-21	926-AA

po.xml

po.xsd





 

News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 2001 XML.org. This site is hosted by OASIS