mattbdean/jraw

The Java Reddit API Wrapper

jraw's Issues

Add documentation on SubredditPaginator about getComments()

Several issues (#29, #44, and #66) have already been opened about Submission.getComments() returning null if it was retrieved by a SubredditPaginator. Some documentation should be added to the SubredditPaginator class explaining why this happens and what the user should expect from this method.

See here for an explanation.

Java 7 support

Having support for Java 7 would make the library usable for Android apps and other projects that aren't ready to take the switch to Java 8.

Pros:

• Support for Android
• According to Plumbr, Java 7 is the most popular version as of May 8, 2014
• Test both Oracle JDK 7 and OpenJDK 7 on Travis-CI as opposed to just Oracle JDK 8
• (IMHO) better looking Javadoc

Cons:

• No default methods for interfaces (Created, Distinguishable, etc.)
• No lambdas/streams (although the standard for loop seems superior)
• No java.time classes (e.g. LocalDateTime in RestClient)

[Android] Json null object reference

I use the following code in Android:

//In Activity:
        ConnectionTask task = new ConnectionTask();
        Credentials credentials = Credentials.standard("myusername", "mypassword");
        task.execute(credentials);


//Custom Async:
public class ConnectionTask extends AsyncTask<Credentials, Void, LoggedInAccount> {
    private final String TAG = getClass().getName();

    @Override
    protected LoggedInAccount doInBackground(final Credentials... params) {
        RedditClient reddit = new RedditClient("MY-JRAW-REDDIT-ANDROID-APP");

        try {
            LoggedInAccount loggedInAccount = reddit.login(params[0]);

            Log.e(TAG, "getfulname: " + loggedInAccount.getFullName());
            Log.e(TAG, "com karma: " + loggedInAccount.getCommentKarma());
            Log.e(TAG, "link karma: " + loggedInAccount.getLinkKarma());

            return loggedInAccount;
        } catch (Exception ne) {
            ne.printStackTrace();
            return null;
        }
    }

    @Override
    protected void onPostExecute(final LoggedInAccount loggedInAccount) {
        if (loggedInAccount != null) {
            Log.e(TAG, "getfulname: " + loggedInAccount.getFullName());
            Log.e(TAG, "com karma: " + loggedInAccount.getCommentKarma());
            Log.e(TAG, "link karma: " + loggedInAccount.getLinkKarma());
        }
    }
}

and get the following Exception:

01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.models.JsonModel.data(JsonModel.java:65)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.models.JsonModel.data(JsonModel.java:43)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.models.Thing.getFullName(Thing.java:37)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at net.dean.jraw.RedditClient.login(RedditClient.java:160)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at com.natieklopper.appreddit.util.ConnectionTask.doInBackground(ConnectionTask.java:21)
01-05 15:41:08.410    2990-3006/com.natieklopper.appreddit W/System.err﹕ at com.natieklopper.appreddit.util.ConnectionTask.doInBackground(ConnectionTask.java:13)

Debug Object (me) in RedditClient at line 160:

LoggedInAccount {getCommentKarma()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getCreated()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'org.codehaus.jackson.JsonNode org.codehaus.jackson.JsonNode.get(java.lang.String)' on a null object reference], getCreatedUtc()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'org.codehaus.jackson.JsonNode org.codehaus.jackson.JsonNode.get(java.lang.String)' on a null object reference], getFullName()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getId()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getLinkKarma()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getModHash()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], getType()="ACCOUNT", hasGold()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], hasMail()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], hasModMail()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], hasVerifiedEmail()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], isFriend()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], isMod()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference], isOver18()=[threw java.lang.NullPointerException: Attempt to invoke virtual method 'boolean org.codehaus.jackson.JsonNode.has(java.lang.String)' on a null object reference]}

Anything I'm missing here?

Provide a way to access RestClient's HttpLogger

Currently there is no way to use the customizability provided by the HttpLogger class because RestClient does not expose it. A simple getter method should do.

Cookie authentication is now deprecated

According to this thread, /u/kemitche has now deprecated authentication via POST /api/login.

Use of the API when authenticated via cookies is deprecated and slated for removal. All API clients MUST convert to authenticating to the reddit API via OAuth 2 by August 3, 2015. After that date, reddit.com will begin heavily throttling and/or blocking API access that is not authenticated with an OAuth 2 access token. Yes, this applies to "logged out" access to the API. For API access without a reddit user, please use Application Only Authentication to get an access token.

I can foresee a few changes to the codebase in response:

1. RedditClient and RedditOAuth2Client will become merged.
2. Credentials will have a new factory method for Application Only authentication (using the API without a user)
3. RedditClient.login(Credentials) will become mandatory, since some form of OAuth2-based authentication must happen (whether that's an app or application-only) before the API can be reached.

---

The other topic mentioned in this post is the naming restrictions on API clients.

1. We're asking API clients to not use the word "reddit" in their name except in the phrase "for reddit", e.g., "My cool app for reddit"
2. We're asking "commercial" API consumers to register with us.

Since this is an open source wrapper and the licensing page defines "commercial" as "if you are earning money from it, including via in-app advertising or in-app purchases. Open source use is generally considered non-commercial," so the project should still be able to maintain the same name.

submission.getComments() returns null?

For some reason this code throws a NullPointerException at for(Comment comment : s.getComments()) even though the submission has comments.

SubredditPaginator subreddit = new SubredditPaginator(reddit, "all");
            subreddit.setLimit(100);
            subreddit.setSorting(Sorting.RISING);
            while(subreddit.hasNext()){
                Listing<Submission> submissions = subreddit.next();
                for(Submission s : submissions)
                {
                    System.out.println("Submission: "+s.getTitle());
                    for(Comment comment : s.getComments())
                    {
                        System.out.println("-Comment: "+comment.getAuthor());
                    }
                }
            }

MoreComments

The method "loadMoreComments(...) doesn't work correctly. The new comments that are loaded aren't integrated into the tree.

(apologies if this is not the right place for this kind of feedback )

How to turn off logging?

This is the code i've tried:

HttpLogger logger = new HttpLogger(JrawUtils.logger());
logger.disable(HttpLogger.Component.REQUEST);
logger.disable(HttpLogger.Component.RESPONSE);

I've added the above snippet at various points in the code with no luck (INFO JRAW messages are still displayed)

I'm using the latest release build v0.6.1

More descriptive Listing data in JsonModel.toString()

In JsonModel's toString() method, if a return value of a @JsonInteraction method is a subclass of JsonModel, the representation of that model is printed as "[ClassName]". For example:

CustomModel {getFoo()="Object@12345", getListing()=[Listing]}

This isn't very descriptive. It would be a lot more helpful if Listings were viewed as Listing[size] or some other variant.

A sample output could look like this:

CustomModel {getFoo()="Object@12345", getListing()={Listing getSize()="25"}}

Use the www subdomain for HTTPS requests

According to this, ssl.reddit.com should be used for "logging in, OAuth authorizations, and preferences". Using https://www.reddit.com is preferred to https://ssl.reddit.com for "normal" requests.

Ability to disable caching

I'd like to be able to disable caching. When I poll for "new" comments, I only see an updated response every 2-3 minutes while using JRAW. If I hit the same URL in my browser I see it continually updated.

Looks like OKHttp's CacheControl should allow for this, but not exactly sure where that should be used in JRAW. If you have any tips let me know, I'd be glad to create a pull request.

More secure Credentials class

Currently usernames and passwords alike are stored in plaintext strings. According to this StackOverflow answer, credentials should be encrypted, "only decrypting them temporarily during the authentication process." Using something as simple as a debugger on an instance of Credentials could give an attacker an all-access pass to whatever account is being used.

Missing "Week" timeperiod

Reddit lets you choose a week for a time period, but JRAW only has HOUR, DAY, MONTH, YEAR, and ALL

Make using HTTP not an option for accounts that require HTTPS

Right now, it would be easy to ignore the user's wishes about requiring HTTPS by calling RestClient.setHttpsEnabled(boolean). Disabling HTTPS on an account like this will cause errors to be sent back to the client.

invalid_scope error

https://ssl.reddit.com/api/v1/authorize?state=hg1nscn1581r1hseddf1ig5ovr&duration=permanent&scope=identity%2C&response_type=code&client_id=N2Lpad7lBpi4kA&redirect_uri=oauth%3A%2F%2Fasne

caused by redundant %2C (encoded ',') at end of scope=identity parameter.

Remove username and password in Credentials factory methods where appropriate

The only time it is necessary for the client to ask for a username/password is when the client is using application-only OAuth or the app type is 'script'. Otherwise, the browser should handle authentication.

The new methods signatures should be

public static Credentials installedApp(String clientId, String redirectUrl)

and

public static Credentials webapp(String clientId, String clientSecret, String redirectUrl)

Use fixtures and mocking instead of network requests

By using a framework such as Mockito to mock certain classes, we can cut down on the amount of requests we have to send over the network and reduce testing time. Granted, we will still have to send some requests to properly test them. Fixtures can be created in the src/test/resources/ directory and will be returned when mocked classes request a certain resource. See here for an example.

This will require a huge undertaking and might best be accomplished in smaller chunks.

Forbidden Access When Getting Listing from SubredditPaginator

I'm having an issue where I get a Network exception when running

SubredditPaginator sub= new SubredditPaginator(reddit, "subreddit");
return sub.next();


Stacktrace:

Caused by: net.dean.jraw.http.NetworkException: Request returned non-successful status code: 403 Forbidden
at net.dean.jraw.http.RestClient.execute(RestClient.java:134)
at net.dean.jraw.RedditClient.execute(RedditClient.java:152)
at net.dean.jraw.paginators.Paginator.next(Paginator.java:107)
at net.dean.jraw.paginators.SubredditPaginator.next(SubredditPaginator.java:44)
at net.dean.jraw.paginators.Paginator.accumulate(Paginator.java:127)
at net.dean.jraw.paginators.Paginator.accumulateMerged(Paginator.java:135)


From some googling, it seems as if OAuth was messed up somewhere, even though I have not implemented logging in/OAuth (yet). What could be the issue?

Thanks!

EDIT: Just FYI, I ran the code with "all" instead of "subreddit"

Change where @EndpointImplementations are placed

Due to the switch from referencing the source to the Javadoc, @EndpiontImplementation annotations should be limited to methods that have a privacy of at least protected, since private methods aren't shown in Javadoc.

getComments() always returns null

I am getting the front page perfectly with a SubredditPaginator, but for every submission in the Listing, getComments() returns null.

Reddit responds with 'DEVELOPER_ALREADY_ADDED' improperly

This issue is similar to #7, except that addDeveloper() is returning an ApiException whose code is DEVELOPER_ALREADY_ADDED, even though removeDeveloper() was executed right before that call.

Here is the relevant code.

Always use HTTPS for specific users

Now that Reddit supports HTTPS sitewide, a new field is returned in the /api/login response: "need_https". If this value is true, any test that is executed while a user is logged in will fail because it is executed over plain HTTP.

Here is a sample response:

{
    "json": {
        "errors": [],
        "data": {
            "need_https": false,
            "modhash": "...",
            "cookie": "..."
        }
    }
}

Gradle task to set up a new user

A task in build.gradle could easily help new devs get started with JRAW. It should:

1. Take a username, password, password verification, optional email, and a captcha attempt and register a new user.
2. Create a OAuth2 app with type 'script' (AccountManager.createOrUpdateApp())
3. Write the username, password, client id, and client secret to credentials.json using Jackson
4. Create a subreddit
5. Create a multireddit whose name is not jraw_testing
6. Submit a selfpost to /r/jraw_testing2

JRAW could be included into the buildscript's dependencies to make this easier.

Only host the latest Git Javadocs

Hosting all of the docs for every single commit creates an unnecessarily large Git repo. Most people won't even look at these. At the very minimum, we should only host Javadoc for the latest passing commit.

The new URL to access these docs could be /git/ instead of /git/latest.

Doesn't load all comments

Some comment trees have a "continue this thread" option. Currently JRAW doesn't expand the full comment tree:

see e.g.

http://www.reddit.com/r/test/comments/2onit4/test_depth_0/

(scroll down to depth 10)

This is a known issue and has recently been fixed on PRAW, see e.g.

http://www.reddit.com/r/redditdev/comments/2onkd8/praw_maximum_depth_of_subtrees/

Mixed method styles

In AccountManager, some methods have names that are just the action (vote(), hide(), delete()), and others mimic the style of a setter method (setSubscribed(), setSticky(), setSaved()). There should only be one style of method in this class.

Include type parameters in ENDPOINTS.md

For example, under the "Implemented?" header, ENDPOINTS.md lists RedditClient.login(String, String) as simply RedditClient.login(). This could give anyone who isn't familiar with the library the idea that providing a username and password is done somewhere else in the code.

Travis builds for pull requests fail

As matt5188 said,

It appears to be due security restrictions that Travis imposes on secure environment variables when handling pull requests from a forked repo. Not sure the best way to handle this scenario - this is the first time I've worked with Travis. I'll read into it.

A solution to this would be to run a custom TestNG suite if [ "${TRAVIS_PULL_REQUEST}" = "x" ] where x is the PR number.

If this method is used, then the PR would still have to be tested locally before merging to make sure that all the features that depend on authentication still work.

See here for Travis' docs on security in PRs

Manager calls to the API should not return RedditResponses

The fact that many methods in AccountManager return a RedditResponse means that whomever is using the library might feel the need to do something with this response, when that is not the case at all. Whatever data that is shown in these responses should be gleaned and the methods should return void.

Receiving "MULTI_EXISTS" when using MultiRedditManager.create()

In MultiRedditManagerTest's testCreate() method, a multireddit of a given name is deleted by calling manager.delete(name) if a multireddit of the same name already exists. This call throws no ApiException, so logically we can assume that the multireddit has been deleted.

However, after the code flow gets to manager.create(), the call throws an ApiException whose code is MULTI_EXISTS. This is strange because we have already insured that that specific multireddit did not exist before executing the request.

See here for the relevant code.

Make links in ENDPOINTS.md use the OAuth2 version of the docs

Currently, all links to endpoints use the format https://www.reddit.com/dev/api#{endpoint}. Since ENDPOINTS.md now categorizes endpoints by their OAuth2 scope, the path should be /dev/api/oauth instead of /dev/api

Use a char array for passwords

According to this StackOverflow question, char arrays are preferred over Strings for password handling because it reduces the window that an attacker could obtain the password before GC kicks in.

To implement this, one might separate TestUtils.getCredentials() into String getUsername() and char[] getPassword() methods and then make a login(String, char[]) method in RedditClient.

Bad Request when Refreshing Token 0.7.0.3

I'm getting the following error
net.dean.jraw.http.NetworkException: Request returned non-successful status code: 400 Bad Request

when running the following code:

OAuthHelper helper = reddit.getOAuthHelper();
helper.setRefreshToken("Token received from UserChallenge");

OAuthData finalData = helper.refreshToken(Credentials.installedApp("my key", "http://ccrama.me"));
reddit.authenticate(finalData);

Is this an issue with JRAW or an issue with how I'm trying to refresh my token?

Thanks!

Better way to check if the latest git docs are already uploaded

In upload_docs.sh, a mechanism is in place to make sure that the docs for a specific commit aren't uploaded twice to the same location. This is currently being done by cloning gh-pages and checking if a specific folder exists. Instead, an HTTP request could be sent to the github.io site at the beginning of the script to prevent the Travis build from doing unnecessary work.

Here is how the request could be sent in cURL:

curl -s -o /dev/null -w "%{http_code}" -L -I http://thatjavanerd.github.io/JRAW/docs/git/{short_hash}/index.html

(source)

This sends a HEAD request to the index.html page of the commit to test and will result with the output of "404" or "200".

If the code is a 404 Not Found, then we know not to continue. If else, it's all good and we can upload the docs.

Http Client read timeout not set / cannot set said timeout without extending RedditClient. Without it application hangs.

Hello,

I encountered a bug where without read timeout set my application hangs on socketRead0. I could not set this timeout without extending your class, because OkHttpClient is not accessible from outside.

Have Jackson ignore unused (null) properties

This will fix net.dean.jraw.test.AccountManagerTest.testUpdatePreferences().

Login with other OAuth2 app types

Currently only 'script' apps are allowed, although Credentials provides a way to create instances for web and installed apps.

This should be relatively painless to implement in OAuth2RedditClient's login(Credentials) method.

See here for an overview of the requests needed to accompilsh this.

Use Jackson databind instead of manually mapping properties

The current method for parsing JSON into Objects is really clunky (see here). This could be drastically improved by using Jackson databind and annotations libraries.

Use Previously Authenticated Details to Re-Login (Application Use)

The OAuth2 wiki page doesn't go into much detail on how to re-gain access to the previously logged-in user without having to have the user re-allow the application access to their profile.

Currently, I ask the user for a username + password, and if the UserChallengeTask completes correctly, I store the two values to re-authenticate on the next time the application is opened. I can create a new credentials object using the username + password, but how do you tell the RedditClient to log in using those details and skip the permissions screen (that they have already accepted)?

Thanks!

Remove RenderStringPair

RenderStringPair's only purpose is to contain two versions of the same data - one raw Markdown text, and the other the HTML version. The HTML version is useless to the API, so that can be safely discarded.

NoClassDefFound - java.nio.charset.StandardCharsets

Hi there,

I'm trying to use JRAW on android, but it crashes on any device older than android 4.4 with a NoClassDefFoundError, as it can't find the StandardCharsets Class (Which was added in API 19 https://developer.android.com/reference/java/nio/charset/StandardCharsets.html).

Is there any way to fix this?

Best Regards,
Michael.

Update AccountPreferences(Editor)

Some new preferences have been added since the class was last updated, such as forcing HTTPS and monitoring username mentions.

Configurable timeout lengths

A constant of 5 seconds is fine for testing purposes, but since this is a library revolving around networking, this value should be configurable.

Ability to request a new OAuth token

See here for how this can happen.

Flair support

There is a shocking lack of flair support in this library. The basics (at a minimum) need to be covered.

• POST /r/{subreddit}/api/flairselector List username flairs
• POST /r/{subreddit}/api/selectflair Choose flair
• POST /r/{subreddit}/api/setflairenabled Enable/disable user flair

Any endpoints under the modflair are helpful, but not required.

Ability to override Host

I'm using JRAW in an Android Wrapper, it is a lib to my Android app I'm working on.
I would like to add tests to my JRAW Android Wrapper Library using Wiremock, for this I would need to be able to set the host via RedditClient.setHost() and setSpecialHost(). Also to set the Port and HTTP/HTTPS.

Can these please be added?
I don't wan't to have the source of JRAW and edit that, at the moment I'm just using it via Gradle as a library.

Thank you.

setTimePeriod() on SubmissionSearchPaginator seems not to work

When I run this code:
submissionSearchPaginator.setTimePeriod(TimePeriod.valueOf("DAY")); //I do valueOf because the value might be different//
I still get results from hundreds of days before. The method seems not to work.

Submission.getComments() always returns null

Even when I know that the submission has comments and getCommentCount() tells me the right amount of comments, getComments() always returns null.

Submission.getShortUrl() should not return java.net.URL

Returning java.net.URL does two things:

• Opens up the possibility of MalformedURLExceptions when the post's ID has characters that are not allowed in a URL. This could happen when when submission.getShortUrl() is called directly, or when submission.toString() is called.
• Deviates from the standard of returning strings for any URIs or URLs returned by the API

No way to get Subscriptions?

I can't seem to find any method to get subscribed subreddits (preferably returning a List of sub names). Does one exist?

Thanks!