OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help



   Processing huge XML files, generating XML from databases

[ Lists Home | Date Index | Thread Index ]
  • From: Simon Carter <simon_carter@splwg.com>
  • To: xml-dev@lists.xml.org
  • Date: Sat, 15 Jul 2000 22:42:44 +1100

I've developed a Windows GUI stream editor called TextPipe that works in a
similar manner to Unix sed, perl, grep, head, tail, awk, dos2ux, ux2dos etc.
Multiple operations can be chained together, such as end of line conversions
from Unix to DOS, search/replace (exact match, pattern match, sounds like,
edit distance), adding and removing lines and columns etc.

TextPipe is unique in a number of ways:

1. It can select a subset of text and transform the text in the subset eg.
sort lines 3-17, or search and replace in columns 5-50, or MIME encode lines
600-700. Subsets can be search/replace results, groups of lines, groups of
columns, groups of CSV fields, groups of Tab-delimited fields and more.

This means that you can have a high level search for <table>.*</table>, and
inside that search result a new search for <tr>.*</tr> and inside that
search result a new search for <td>.*</td>. Inside this search result you
might convert any found text to Title Case. This means that you can be
extremely selective about the type of table/row/column you match.

2. TextPipe can send SQL commands to multiple databases (via ADO/ODBC), with
the result sets output as CSV, fixed width or XML (TextPipe can generate a
DTD). This can be used to perform lookups based on a selected column, or
data cleansing, or data extract, transform and load (ETL) processes.

3. If TextPipe can't do exactly what you want you can write code in VBScript
or JScript.

4. TextPipe has no limitation on the size or number of files/folders. It
processes all files in approx 5 MB of memory.

If you'd like more information it's available from the URL below. Please let
me know if you find TextPipe useful for XML-related work!


End Hours Wasted Manually Editing Data / Text with TextPipe

Other products: LFNit!, DirSize, BabyShield, DirDate, ZeroIn, Arc Menu,
ClipSize, Clean 'n' Go


News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 2001 XML.org. This site is hosted by OASIS