Comments (1)
Hi Felix,
if I understand correctly you called DefaultExtractor#getText(URL) with an URL
like "http://www.äöü.xyz/".
This seems to be unsupported by Java 6 (see
http://java.sun.com/docs/books/tutorial/i18n/network/iri.html
). In particular, what you passed was then an IRI, not a URL.
A workaround for now could be creating the URLs like this
URL u = new URL("http://"+IDN.toASCII("www.äöü.xyz")+"/");
However, since the getText(URL) method is explicitly marked as "show case
only", you might also consider
using a dedicated HTTP client library, such as HttpClient
(http://hc.apache.org/) and call getText(InputSource)
instead.
I would not recommend using getText(URL) in a production setup. You will sooner
or later run into problems
that are out of scope for boilerpipe (robots.txt, broken servers, proxies, ...)
Marking as WontFix.
Best,
Christian
Original comment by ckkohl79
on 24 Jan 2010 at 4:09
- Changed state: WontFix
- Added labels: Priority-Low, OpSys-All
- Removed labels: Priority-Medium
from boilerpipe.
Related Issues (20)
- BoilerplateBlockFilter ignores labelToKeep
- [deleted issue]
- Program does not terminate for badly formatted/syntactically incorrect HTML input
- How to use boilerpipe to get some text with a hyperlink from the web page? HOT 1
- Incomplete extraction of text with special characters
- Server returned HTTP response code: 403 for URL (SOLVED) please use this codeline. HOT 2
- Limit the parsing depth of the html parsing to avoid out of memory situations HOT 1
- Extract article from non-english text HOT 1
- Missing Maven 1.2.0
- Xerces for andorid jar file needed HOT 2
- its not working for a news site HOT 1
- Incomplete extraction of article
- Fail to extract main content on some page, get footnote instead
- IllegalArgumentException for many web pages
- Missing ImageExtractor in downloabale 1.2 jar file
- Performance issues with UnicodeTokenizer
- Boilerpipe is conflicting with CyberNeko library HOT 1
- Unsupported content type: null HOT 1
- Different result when using Web Api and the source api?
- How to debug the result?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from boilerpipe.