florent37 / rxretrojsoup Goto Github PK

View Code? Open in Web Editor NEW

488.0 13.0 48.0 2.44 MB

A simple API-like from html website (scrapper) for Android, RxJava2 ready !

License: Apache License 2.0

Java 99.91% Shell 0.09%

html java jsoup retrofit rx observable css android query api

rxretrojsoup's Introduction

RxRetroJsoup

RxJava2 ready !

Create an interface with @Select annotated method

public interface TutosAndroidFrance {

    @Select("article")
    Observable<Article> articles();

}

Annotate your model with Jsoup queries ( https://jsoup.org/cookbook/extracting-data/selector-syntax )

public class Article {

    @JsoupText(".entry-title a")
    String title;

    @JsoupHref(".read-more a")
    String href;

    @JsoupSrc(".entry-thumb img")
    String image;

    @JsoupText(".entry-content p")
    String description;

    public Article() {
    }
    ...

}

Build a RetroJsoup

final TutosAndroidFrance tutosAndroidFrance = new RetroJsoup.Builder()
                .url("http://tutos-android-france.com/")

                //optionally
                .client(your_okhttp_client)

                .build()
                .create(TutosAndroidFrance.class);

Fetch your objects !

tutosAndroidFrance.articles()
                .toList()

                .subscribeOn(Schedulers.newThread())
                .observeOn(AndroidSchedulers.mainThread())

                .subscribe(items -> );

RxJsoup

RxJsoup.with(url)
    .select("article")
    .flatMap(element -> rxJsoup.text(element, ".entry-title a"))
    .subscribe(text -> {});


    .flatMap(element -> rxJsoup.src(element, ".entry img"))
    .flatMap(element -> rxJsoup.href(element, ".entry a"))
    .flatMap(element -> rxJsoup.attr(element, ".entry h1", "id"))

RxJsoup.connect(

           Jsoup.connect("www.thewebsite.com")
               .userAgent(MY_USER_AGENT)
               .data("credential", email)
               .data("pwd", password)
               .cookies(loginForm.cookies())
               .method(Connection.Method.POST)

           )
           .subscibe(response -> {})

Download

In your module

dependencies {
  compile 'com.github.florent37:retrojsoup:1.0.4'
  compile 'com.github.florent37:rxjsoup:1.0.4'
  annotationProcessor 'com.github.florent37:retrojsoup-compiler:1.0.4'

  //don't forget to include jsoup & rxjava
  compile 'org.jsoup:jsoup:1.10.2'
  compile 'io.reactivex.rxjava2:rxjava:2.0.7'

  //optionaly
  compile 'com.squareup.okhttp3:okhttp:3.6.0'
}

repositories {
  jcenter()
}

Credits

Author: Florent Champigny

Blog : http://www.tutos-android-france.com/

Fiches Plateau Moto : https://www.fiches-plateau-moto.fr/

License

Copyright 2017 florent37, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

rxretrojsoup's People

Contributors

Stargazers

Watchers

Forkers

narakai susyimes sdtm1016 repoforks huoshulei llpj replive coder698 futabooo tanlocle0495 hudawei996 ternence naseemakhtar994 hystdyorg androiddream keven0403 szn0212 muhammadammad jamesliandroid noeurng jiejunping shelbyyu chenyi2013 linheimx itang01 ranchohxk pythonzz hram sunnyline exhange matrixy ttm56p itsjw puujee0238 rayworks nerdfaisal404 oleminhquang winterdl brospack varunkashyapks virendersran01 jiangbaotian zhangshu345 xcatx9527 liamdgray iq-scm

rxretrojsoup's Issues

Nullable JsoupText

How can I set field property as nullable.
Im my case on some web page in list of items one field in not requaired.
If I add this field I recieve not all available items.

RxJava 2

RxJava 2 revision pattern is 'io.reactivex.rxjava2:rxjava:x.y.z'
In README.md is compile 'io.reactivex:rxjava:2.0.7'

Friends, this project can not open any ghosts

why？

There any sample apk?

I download your sample apk in android-arsenal, but i think its not work. i cant see image

#Bug

Error:(24, 52) error: cannot access Observable
class file for rx.Observable not found

I found error this one.

Kotlin support

Any plans on supporting kotlin? it seems like it doesn't like kotlin pojo class.
It will be really great if you could add this to your milestone.

Compilation failed

public class TrendingRetroJsoup implements Trending {
private final RxJsoup rxJsoup;

public TrendingRetroJsoup(String url, Boolean exceptionIfNotFound) {
this.rxJsoup = new RxJsoup(url, exceptionIfNotFound);
}

Error:(15, 20) error: constructor RxJsoup in class RxJsoup cannot be applied to given types;
required: String,boolean,OkHttpClient
found: String,Boolean
reason: actual and formal argument lists differ in length

compile 'com.github.florent37:retrojsoup:1.0.2'
compile 'com.github.florent37:rxjsoup:1.0.2'
compile 'org.jsoup:jsoup:1.10.2'
compile 'io.reactivex:rxjava:1.2.7'
compile 'io.reactivex:rxandroid:1.2.1'
compile 'com.squareup.okhttp3:okhttp:3.6.0'
annotationProcessor 'com.github.florent37:retrojsoup-compiler:1.0.0'

help wanted.