speric/kindle-highlights

Get your Kindle highlights.

kindle-highlights's Issues

Unable to sign in

I'm using v1.0.2 of this gem on macOS 10.12.3. My ruby version is: ruby 2.3.0p0 (2015-12-25 revision 53290) [x86_64-darwin15].

My test code:

require 'kindle_highlights'

kindle = KindleHighlights::Client.new(email_address: "[email protected]",
                                      password: "password")
puts kindle.books

I know my email address and password are correct because I'm copying and pasting the exact values I use when logging into amazon.com manually.

I'm receiving this error:

$ ruby kindle.rb
/Users/me/.rvm/gems/ruby-2.3.0/gems/kindle-highlights-1.0.2/lib/kindle_highlights/client.rb:55:in `conditionally_sign_in_to_amazon': Unable to sign in, received error: 'Enter a valid email or mobile number' (KindleHighlights::Client::AuthenticationError)
from /Users/me/.rvm/gems/ruby-2.3.0/gems/kindle-highlights-1.0.2/lib/kindle_highlights/client.rb:63:in `load_books_from_kindle_account'
from /Users/me/.rvm/gems/ruby-2.3.0/gems/kindle-highlights-1.0.2/lib/kindle_highlights/client.rb:16:in `books'
from kindle.rb:6:in `<main>'

I've tried passing different values for mechanize_options: { user_agent_alias: 'Mac Safari' } but I still recieve the same error.

Have you seen this issue before? TIA

ntlm: RDoc is not a full Ruby parser and will fail when fed invalid ruby programs: undefined method `[]' for nil:NilClass

First and foremost, many thanks for this library! 🙇 👍
FYI and in case something can be done about it, I got an error related to ntlm's documentation during installation:

$ gem install kindle-highlights
Fetching unf_ext-0.0.7.7.gem
Fetching unf-0.1.4.gem
Fetching domain_name-0.5.20190701.gem
Fetching webrobots-0.1.2.gem
Fetching net-http-persistent-3.1.0.gem
Fetching ntlm-http-0.1.1.gem
Fetching http-cookie-1.0.3.gem
Fetching connection_pool-2.2.2.gem
Fetching kindle-highlights-2.0.1.gem
Fetching mime-types-data-3.2019.1009.gem
Fetching mime-types-3.3.1.gem
Fetching net-http-digest_auth-1.4.1.gem
Fetching mechanize-2.7.6.gem
Building native extensions. This could take a while...
Successfully installed unf_ext-0.0.7.7
Successfully installed unf-0.1.4
Successfully installed domain_name-0.5.20190701
Successfully installed webrobots-0.1.2
Successfully installed ntlm-http-0.1.1
Successfully installed http-cookie-1.0.3
Successfully installed connection_pool-2.2.2
Successfully installed net-http-persistent-3.1.0
Successfully installed mime-types-data-3.2019.1009
Successfully installed mime-types-3.3.1
Successfully installed net-http-digest_auth-1.4.1
Successfully installed mechanize-2.7.6
Successfully installed kindle-highlights-2.0.1
Parsing documentation for unf_ext-0.0.7.7
Installing ri documentation for unf_ext-0.0.7.7
Parsing documentation for unf-0.1.4
Installing ri documentation for unf-0.1.4
Parsing documentation for domain_name-0.5.20190701
Installing ri documentation for domain_name-0.5.20190701
Parsing documentation for webrobots-0.1.2
Installing ri documentation for webrobots-0.1.2
Parsing documentation for ntlm-http-0.1.1
Before reporting this, could you check that the file you're documenting
has proper syntax:

  ${HOME}/.rbenv/versions/2.6.3/bin/ruby -c lib/net/ntlm.rb

RDoc is not a full Ruby parser and will fail when fed invalid ruby programs.

The internal error was:

	(NoMethodError) undefined method `[]' for nil:NilClass

ERROR:  While executing gem ... (NoMethodError)
    undefined method `[]' for nil:NilClass

$ which ruby
${HOME}/.rbenv/shims/ruby

$ ruby -v
ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-darwin19]

$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 3.0.3
  - RUBY VERSION: 2.6.3 (2019-04-16 patchlevel 62) [x86_64-darwin19]
  - INSTALLATION DIRECTORY: ${HOME}/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0
  - USER INSTALLATION DIRECTORY: ${HOME}/.gem/ruby/2.6.0
  - RUBY EXECUTABLE: ${HOME}/.rbenv/versions/2.6.3/bin/ruby
  - GIT EXECUTABLE: /usr/local/bin/git
  - EXECUTABLE DIRECTORY: ${HOME}/.rbenv/versions/2.6.3/bin
  - SPEC CACHE DIRECTORY: ${HOME}/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: ${HOME}/.rbenv/versions/2.6.3/etc
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86_64-darwin-19
  - GEM PATHS:
     - ${HOME}/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0
     - ${HOME}/.gem/ruby/2.6.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - ${HOME}/.rbenv/versions/2.6.3/bin
     - /usr/local/Cellar/rbenv/1.1.2/libexec
     - ${HOME}/.gem/ruby/2.6.0/bin
     - ${HOME}/.rbenv/shims
     - /usr/local/bin
     - /usr/local/sbin
     - ${HOME}/bin
     - /usr/bin
     - /bin
     - /usr/sbin
     - /sbin
     # <redacted>

It didn't seem to prevent from importing the library though:

$ irb
irb(main):001:0> require 'kindle_highlights'
=> true

mechanize hits captcha but not detected by gem

Description

Captcha error not printed when mechanize encounters one.

Expected Behaviour

When captcha is shown on login, application should print:

KindleHighlights::Client::CaptchaError: Received a CAPTCHA while attempting to sign in to your Amazon account. You will need to resolve this manually at <link to remedy captcha errors>

Turns out Amazon changed the id tag they use for the captcha image from ap_captcha_img to auth-captcha-image:

...
<div id="auth-captcha-image-container" class="a-section a-text-center">
  <img alt="Visual CAPTCHA image, continue down for an audio option." src="https://opfcaptcha-prod.s3.amazonaws.com/f3eb1f1041fa4a3f91ccdba0fa5fd578.jpg?AWSAccessKeyId=AKIAIRWCIOSGWYRDLHHQ&amp;Expires=1547779432&amp;Signature=QS936h%2B0t1uJqlmTzRHmZ3fNpRw%3D" data-refresh-url="/ap/captcha?appAction=SIGNIN&amp;captchaObfuscationLevel=ape:aGFyZA==&amp;captchaType=image" id="auth-captcha-image">
</div>
...

Make more newbie friendly

Happy to pair with someone on how to make this more newbie friendly. Interested in this gem, but cannot get it to work.

Likely other polyglots who use highlights for learning languages are interested and are even less technical than I am. Shout out if you are interested in making this more broadly accessible. Will to help with doing that. Just gotta get it to work first...

I think this is broken

Hi, This gem looks great. I was trying to use tobi's highlights gem, which leverages this, but I think amazon moved the url you use as a constant "http://kindle.amazon.com/login" to "https://read.amazon.com/notebook". I'm assuming the page they return now is totally different so the parsing would have to be rewritten as well. I'm happy to fix and submit a PR if it's just the login that's broken, but my guess is this needs to be redone from scratch?

mechanize.rb:372:in `click': undefined

With:

require 'kindle_highlights'

# pass in your Amazon credentials. Loads your books (not highlights)
# on init, so might take a while
kindle = KindleHighlights::Client.new("[email protected]", "password")

Getting an error message of:

/home/jason/.rvm/gems/ruby-2.3.0/gems/mechanize-2.7.4/lib/mechanize.rb:372:in click': undefined method[]' for nil:NilClass (NoMethodError)
from /home/jason/.rvm/gems/ruby-2.3.0/gems/kindle-highlights-0.0.8/lib/kindle_highlights/client.rb:30:in load_books_from_kindle_account' from /home/jason/.rvm/gems/ruby-2.3.0/gems/kindle-highlights-0.0.8/lib/kindle_highlights/client.rb:10:ininitialize'
from main.rb:5:in new' from main.rb:5:in

'

Any ideas on what I may be doing wrong?

Enter a valid email or mobile number error after running smoothly for a while.

rake aborted!
KindleHighlights::Client::AuthenticationError: Unable to sign in, received error: 'Enter a valid email or mobile number'
/Users/cggaurav/.rvm/gems/ruby-2.1.2/gems/kindle-highlights-1.0.1/lib/kindle_highlights/client.rb:55:in `conditionally_sign_in_to_amazon'
/Users/cggaurav/.rvm/gems/ruby-2.1.2/gems/kindle-highlights-1.0.1/lib/kindle_highlights/client.rb:63:in `load_books_from_kindle_account'
/Users/cggaurav/.rvm/gems/ruby-2.1.2/gems/kindle-highlights-1.0.1/lib/kindle_highlights/client.rb:16:in `books'
/Users/cggaurav/Projects/apps/play/highlights/Rakefile:30:in `update'
/Users/cggaurav/Projects/apps/play/highlights/Rakefile:56:in `block in <top (required)>'
/Users/cggaurav/.rvm/gems/ruby-2.1.2/gems/rake-11.2.2/exe/rake:27:in `<top (required)>'
Tasks: TOP => default => download
(See full trace by running task with --trace)

Any plan to release a version with root_url support?

2.0.1, currently the latest available version, doesn't support root_url.
While one can clone and work with latest master locally, it would be a lot more convenient to be able to directly install a version supporting this feature.
Is there any plan to do so in the near future? 🙂 🙏

Nothing downloaded and no error message

Hello,

I'm modifying this script to work with v2.0.1 of kindle-highlights, but nothing happens. I've got a small test script, and it returns a book count of 0:

require 'kindle_highlights'
require 'htmlentities'

html = HTMLEntities.new
kindle = KindleHighlights::Client.new(email_address: ENV["AMAZON_USER"],
                                          password: ENV["AMAZON_PASS"])
@highlights = []

p kindle.books.count

kindle.books.each do |book|
    kindle.highlights_for(book.asin).each do |highlight|
        highlight["book"] = html.decode(book.title)
        highlight["highlight"] = html.decode(highlight["highlight"])
        @highlights << highlight
    end
end

p @highlights

I'm 100% that my Amazon email and password are correct. I ran the login + book count part in irb, no error is returned there.

Do you know if Amazon made further changes?

Thanks,

Clement

Is there a way to know the name of the chapter where the highlight comes from?

Avoiding 2fa sign-in

I get an empty results from kindle.books and received a text asking me to verify the authenticity of the login attempt.

Any ideas on how to work around this? I was able to get it working once.

Is the gem outdated?

Hi @speric Is the gem outdated? I'm running into this error when I try to initialize the client and print a list of books. Do you have any pointers?

> /home/runner/RubyKindle/.local/share/gem/ruby/3.2.0/gems/kindle-highlights-2.0.1/lib/kindle_highlights/client.rb:88:in `login_via_mechanize': undefined method `email=' for nil:NilClass (NoMethodError)
> 
>       signin_form.email = email_address
>                  ^^^^^^^^
>     from /home/runner/RubyKindle/.local/share/gem/ruby/3.2.0/gems/kindle-highlights-2.0.1/lib/kindle_highlights/client.rb:65:in `conditionally_sign_in_to_amazon'
>     from /home/runner/RubyKindle/.local/share/gem/ruby/3.2.0/gems/kindle-highlights-2.0.1/lib/kindle_highlights/client.rb:54:in `load_books_from_kindle_account'
>     from /home/runner/RubyKindle/.local/share/gem/ruby/3.2.0/gems/kindle-highlights-2.0.1/lib/kindle_highlights/client.rb:22:in `books'
>     from main.rb:19:in `<main>'

Here's the code I'm running. I've removed my actual email and password in this post.

client = KindleHighlights::Client.new(
  email_address: "[email protected]",
  password: "password"
)

puts client.books