Re: [xml-dev] Parsing bad HTML

["Steven J. DeRose" <sderose@acm.org>]

I'd say there are two questions you have to decide before you can 
tell the best approach:

1: What do you *want* that to parse to?

2: What other cases do you need to deal with?

For #1, I'd assuming that you want "67<eight" to be treated as if it 
were "67&lt;8", and you want the pointies inside the alt attribute to 
really be there, presumably also via &lt; (you might instead want to 
strip the "b" pseudo-tags entirely, or insert an "<eight>" tag for 
some reason, or....

For #2, is there a recurring pattern/problem you need fixed? If not, 
then there's no point in *programming* a solution of any kind. If so, 
what *is* the pattern? From this example it might be that you want to 
treat a less-than as literal data instead of a tag-start in any one 
of these situations:

a) after a sequence of digits

b) before "eight" (or before "e", or before any spelled-out name of a 
number, or....)

c) whenever the following text can't form a well-formed tag (this 
might be too draconian, though)

d) whenever the following XML Name characters (in this case "eight") 
don't form a known HTML tag name.

As is always true, a finite number of examples can be accounted for 
by an infinite number of different models; but to implement something 
you have to take your pick. If there's a lot of data involved, you 
would be well advised to scan around it and determine what kinds of 
errors there are,  then solve the most frequent ones first.

On the whole, this looks to me like the sort of thing you either fix 
by hand (if there are few enough examples, or if the examples are 
totally unpredictable); or by some simple regexes, such as:

sed 's/<\([a-zA-Z]*\)</\&lt;\1</g

Steve

At 12:22 PM -0800 11/13/08, Paul M wrote:
>I use tidy to clean up  bad html docs. It does a pretty good job of 
>converting html => strict xthml
>
>However, the following is a bit too much
>
><p>
><sub>123</sub>4567<eight<img src="file.gif" alt="<b>hello</b>">
></p>
>
>The problem is with 7<eight. Stray < and > seem to make tidy choke. 
>What is the best method of handling this? I am leaning toward perl 
>and regexp, but am hoping to avoid this. Maybe a Java solution? And 
>tidy solutions?
>
>-thanks


-- 

Steve DeRose -- http://www.derose.net, email sderose@acm.org