Comments on: Looking for a java html parser (or groovy)

By: Tom Copeland

Tom Copeland — Wed, 14 Jan 2009 20:51:09 +0000

Sure, yup, I used this web page (e.g., http://wavyx.net/2009/01/13/looking-for-a-java-html-parser-or-groovy/), and the line causing the problem seems to be this one:

… which seems quite limited, so I should probably import some of this POJA (Plain Old Java API). Or maybe you can point me other references?

Oddly, the tokenizing exception seems to be happening on the first space after the period. Hm. Weird. Must be some hidden character there or something.

If you want, email me at tom@infoether.com and I’ll email you the source to Brian’s HtmlParser; it comes with a build.xml so all you need to do is run Ant and it’s all set to go. It also comes with a sample HtmlScrubber class that normalizes element name cases, strips optional quotes, and so forth.

By: Eric Rodriguez

Eric Rodriguez — Wed, 14 Jan 2009 15:06:30 +0000

I’m having difficulties to download the jar version of your parser or to access quiotix website.
Could you tell me which full line of the html source is causing the problem?

By: Eric Rodriguez

Eric Rodriguez — Wed, 14 Jan 2009 14:25:33 +0000

On first guess, I would suggest some problem with UTF8 encoding.
But I should definitely try by myself. Which page did you use, or could you send me the html file to test?

By: Tom Copeland

Tom Copeland — Wed, 14 Jan 2009 14:11:32 +0000

Since it uses JavaCC HtmlParser is very fast and flexible, but it fails to parse some HTML – for example, when presented with the HTML of this blog post, it raised an exception:

$ java com.quiotix.html.parser.HtmlParser < blog.html
Exception in thread “main” com.quiotix.html.parser.TokenMgrError: Lexical error at line 101, column 107. Encountered: “\u2020″ (8224), after : “”
at com.quiotix.html.parser.HtmlParserTokenManager.getNextToken(HtmlParserTokenManager.java:2046)
at com.quiotix.html.parser.HtmlParser.jj_ntk(HtmlParser.java:571)
at com.quiotix.html.parser.HtmlParser.ElementSequence(HtmlParser.java:46)
at com.quiotix.html.parser.HtmlParser.HtmlDocument(HtmlParser.java:35)
at com.quiotix.html.parser.HtmlParser.main(HtmlParser.java:27)

Ah well.