winebarrel/faraday_middleware-aws-sigv4

Faraday middleware for AWS Signature Version 4 using aws-sigv4.

FaradayMiddleware::AwsSigV4

Faraday middleware for AWS Signature Version 4 using aws-sigv4.

Required dependency version

version	ruby	faraday
>= 1.0.0	>= 2.5	>= 2.0
>= 0.6.0	>= 2.4	>= 1.8
>= 0.5.0	>= 2.4	>= 0.15
>= 0.3.0	>= 2.3	>= 0.15
< 0.3.0	-	< 0.15

Installation

Add this line to your application's Gemfile:

gem 'faraday_middleware-aws-sigv4'

And then execute:

$ bundle

Or install it yourself as:

$ gem install faraday_middleware-aws-sigv4

Usage

require 'faraday_middleware/aws_sigv4'
require 'pp'

conn = Faraday.new(url: 'https://apigateway.us-east-1.amazonaws.com') do |faraday|
  faraday.request :aws_sigv4,
    service: 'apigateway',
    region: 'us-east-1',
    access_key_id: ENV['AWS_ACCESS_KEY_ID'],
    secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
  # see http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Sigv4/Signer.html

  faraday.response :json, content_type: /\bjson\b/

  # NOTE: If you need to send case sensitive headers like `x-amz-access-token` in `SP API`.
  # cf. https://github.com/lostisland/faraday/issues/747#issuecomment-439864181
  faraday.adapter Faraday.default_adapter
end

res = conn.get '/account'

pp res.body
#=> {"_links"=>
#     {"curies"=>
#       {"href"=>
#         "http://docs.aws.amazon.com/apigateway/latest/developerguide/account-apigateway-{rel}.html",
#        "name"=>"account",
#        "templated"=>true},
#      "self"=>{"href"=>"/account"},
#      "account:update"=>{"href"=>"/account"}},
#    "throttleSettings"=>{"rateLimit"=>10000.0, "burstLimit"=>5000}}

Upgrading from faraday_middleware-aws-signers-v4

If you previously provided the service_name option, you need to rename it service

Test

bundle install
bundle exec appraisal install
bundle exec appraisal faraday-2.7 rake

Related Links

• http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Sigv4/Signer.html

faraday_middleware-aws-sigv4's Issues

Support Faraday >= 2.x

Hi,

Is there any plan to support Faraday 2.x ? This is the last dependency in my Gemfile who prevents me to upgrade.

Regards

New release?

❤️ Thank you for supporting Faraday 2! ❤️

I would like to use this in the rubygems.org website, since then we could go all the way to Faraday 2+ there.

If you need anything to make a release, just ask! (I could write a changelog, celebrate, or otherwise cheer on!)

Changes since last release v1.0.1.

v1.0.1...master

Confirm Ruby 3 support

As Ruby 2.7 EOL is supposed to be end of March 2023, would you be able to confirm Ruby 3 compatibility? Thanks.

Faraday default adapter capitalize headers and makes requests broken for some APIs

Hi, thank you for the awesome work on this Gem. It saved me a lot of time while developing connection with Amazon Selling Partner API.

There is one tricky thing I think we could mention in the Readme - in the SP API we have to send x-amz-access-token in request headers. This header is for some reason case sensitive. I've lost few hours getting Access Denied error until I found out that Faraday default adapter Net::HTTP capitalizes header names by default: lostisland/faraday#747 (comment)

Once I've switched the adapter to excon adapter it just started working.

Wanted to add such comment in the Readme to probably save some time for someone else meeting this problem in future:

  ...
  faraday.response :json, content_type: /\bjson\b/
  faraday.response :raise_error

  # NOTE: If you need to send case sensitive headers like `x-amz-access-token` in `SP API` I strongly
  # recommend you don't use the default Net::HTTP adapter as it's capitalizing header names:
  # https://github.com/lostisland/faraday/issues/747#issuecomment-439864181
  faraday.adapter Faraday.default_adapter
end

Don't have access to open a pull request so writing it as an issue.

Elasticsearch::Transport::Transport::Errors::Forbidden: [403]

Elasticsearch::Transport::Transport::Errors::Forbidden: [403] {"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}

I have tried using both e elasticsearch-rails and searchkick gems to try and do elasticsearch indexing for our project. We are making creation and editing of e-courses online.

When we have a small amount of data its fine but for example if I increase this part beyond the words in the section. Its around 1200 then I get the error.

{
      "sections":
      "Picture removal detract earnest is by. Esteems met joy attempt way clothes yet demesne tedious. Replying an marianne do it an entrance advanced. Two dare say play when hold. Required bringing me material stanhill jointure is as he. Mutual indeed yet her living result matter him bed whence.By spite about do of do allow blush. Additions in conveying or collected objection in. Suffer few desire wonder her object hardly nearer. Abroad no chatty others my silent an. Fat way appear denote who wholly narrow gay settle. Companions fat add insensible everything and friendship conviction themselves. Theirs months ten had add narrow own.Was justice improve age article between. No projection as up preference reasonably delightful celebrated. Preserved and abilities assurance tolerably breakfast use saw. And painted letters forming far village elderly compact. Her rest west each spot his and you knew. Estate gay wooded depart six far her. Of we be have it lose gate bred. Do separate removing or expenses in. Had covered but evident chapter matters anxious.Abilities or he perfectly pretended so strangers be exquisite. Oh to another chamber pleased imagine do in. Went me rank at last loud shot an draw. Excellent so to no sincerity smallness. Removal request delight if on he we. Unaffected in we by apartments astonished to decisively themselves. Offended ten old consider speaking.Improved own provided blessing may peculiar domestic. Sight house has sex never. No visited raising gravity outward subject my cottage mr be. Hold do at tore in park feet near my case. Invitation at understood occasional sentiments insipidity inhabiting in. Off melancholy alteration principles old. Is do speedily kindness properly oh. Respect article painted cottage he is offices parlors.Am no an listening depending up believing. Enough around remove to barton agreed regret in or it. Advantage mr estimable be commanded provision. Year well shot deny shew come now had. Shall downs stand marry taken his for out. Do related mr account brandon an up. Wrong for never ready ham these witty him. Our compass see age uncivil matters weather forbade her minutes. Ready how but truth son new under.Sigh view am high neat half to what. Sent late held than set why wife our. If an blessing building steepest. Agreement distrusts mrs six affection satisfied. Day blushes visitor end company old prevent chapter. Consider declared out expenses her concerns. No at indulgence conviction particular unsatiable boisterous discretion. Direct enough off others say eldest may exeter she. Possible all ignorant supplied get settling marriage recurred.Dashwood contempt on mr unlocked resolved provided of of. Stanhill wondered it it welcomed oh. Hundred no prudent he however smiling at an offence. If earnestly extremity he he propriety something admitting convinced ye. Pleasant in to although as if differed horrible. Mirth his quick its set front enjoy hoped had there. Who connection imprudence middletons too but increasing celebrated principles joy. Herself too improve gay winding ask expense are compact. New all paid few hard pure she.Agreed joy vanity regret met may ladies oppose who. Mile fail as left as hard eyes. Meet made call in mean four year it to. Prospect so branched wondered sensible of up. For gay consisted resolving pronounce sportsman saw discovery not. Northward or household as conveying we earnestly believing. No in up contrasted discretion inhabiting excellence. Entreaties we collecting unpleasant at everything conviction.Difficulty on insensible reasonable in. From as went he they. Preference themselves me as thoroughly partiality considered on in estimating. Middletons acceptance discovered projecting so is so or. In or attachment inquietude remarkably comparison at an. Is surrounded prosperous stimulated am me discretion expression. But truth being state can she china widow. Occasional preference fat remarkably now projecting uncommonly dissimilar. Sentiments projection particular companions interested do at my delightful. Listening newspaper in advantage frankness to concluded unwilling. Extremely we promotion remainder eagerness enjoyment an. Ham her demands removal brought minuter raising invited gay. Contented consisted continual curiosity contained get sex. Forth child dried in in aware do. You had met they song how feel lain evil near. Small she avoid six yet table china. And bed make say been then dine mrs. To household rapturous fulfilled attempted on so.Ferrars all spirits his imagine effects amongst neither. It bachelor cheerful of mistaken. Tore has sons put upon wife use bred seen. Its dissimilar invitation ten has discretion unreserved. Had you him humoured jointure ask expenses learning. Blush on in jokes sense do do. Brother hundred he assured reached on up no. On am nearer missed lovers. To it mother extent temper figure better.Rendered her for put improved concerns his. Ladies bed wisdom theirs mrs men months set. Everything so dispatched as it increasing pianoforte. Hearing now saw perhaps minutes herself his. Of instantly excellent therefore difficult he northward. Joy green but least marry rapid quiet but. Way devonshire introduced expression saw travelling affronting. Her and effects affixed pretend account ten natural. Need eat week even yet that. Incommode delighted he resolving sportsmen do in listening.She literature discovered increasing how diminution understood. Though and highly the enough county for man. Of it up he still court alone widow seems. Suspected he remainder rapturous my sweetness. All vanity regard sudden nor simple can. World mrs and vexed china since after often.Luckily friends do ashamed to do suppose. Tried meant mr smile so. Exquisite behaviour as to middleton perfectly. Chicken no wishing waiting am. Say concerns dwelling graceful six humoured. Whether mr up savings talking an. Active mutual nor father mother exeter change six did all.Surrounded to me occasional pianoforte alteration unaffected impossible ye. For saw half than cold. Pretty merits waited six talked pulled you. Conduct replied off led whether any shortly why arrived adapted. Numerous ladyship so raillery humoured goodness received an. So narrow formal length my highly longer afford oh. Tall neat he make or at dull ye.Wrong do point avoid by fruit learn or in death. So passage however besides invited comfort elderly be me. Walls began of child civil am heard hoped my. Satisfied pretended mr on do determine by. Old post took and ask seen fact rich. Man entrance settling believed eat joy. Money as drift begin on to. Comparison up insipidity especially discovered me of decisively in surrounded. Points six way enough she its father. Folly sex downs tears ham green forty.Sportsman delighted improving dashwoods gay instantly happiness six. Ham now amounted absolute not mistaken way pleasant whatever. At an these still no dried folly stood thing. Rapid it on hours hills it seven years. If polite he active county in spirit an. Mrs ham intention promotion engrossed assurance defective. Confined so graceful building opinions whatever trifling in. Insisted out differed ham man endeavor expenses. At on he total their he songs. Related compact effects is on settled do.Wrote water woman of heart it total other. By in entirely securing suitable graceful at families improved. Zealously few furniture repulsive was agreeable consisted difficult. Collected breakfast estimable questions in to favourite it. Known he place worth words it as to. Spoke now noise off smart her ready.Dispatched entreaties boisterous say why stimulated. Certain forbade picture now prevent carried she get see sitting. Up twenty limits as months. Inhabit so perhaps of in to certain. Sex excuse chatty was seemed warmth. Nay add far few immediate sweetness earnestly dejection. Extremely we promotion remainder eagerness enjoyment an. Ham her demands removal brought minuter raising invited gay. Contented consisted continual curiosity contained get sex. Forth child dried in in aware do. You had met they song how feel lain evil near. Small she avoid six yet table china. And bed make say been then dine mrs. To household rapturous fulfilled attempted on so.Ferrars all spirits his imagine effects amongst neither. discover"}

This is our config for our initializer for searchkick:
ENV["ELASTICSEARCH_URL"] = "https://*.eu-west-2.es.amazonaws.com:443"

Searchkick.aws_credentials = {
access_key_id: Rails.application.credentials.dig(:aws, :access_key_id),
secret_access_key: Rails.application.credentials.dig(:aws, :secret_access_key),
region: 'eu-west-2'
}

I can index from my dev machine using curl without signing and also I can index through kibana sitting on the instance. I can index 1000s of words. I test around 10,000

If you can shed any light on the matter I would be most grateful.

[Question]Why is faraday_middleware a development dependency?

Although require faraday_middleware is required according to usage document, why is its dependency is development_dependency? I'm wondering why faraday_middleware is not a runtime dependency.

Experience request signing issue after 12 hours, and seeing "security token included in the request is expired"

I appear to have a similar issue to #17

My requests are signed and everything is completely fine and dandy, for a while.
Then I experience a "security token included in the request is expired" exception after my application lives for about 12 hours in aws.

I am in aws, using opensearch, with the ruby searchkick and opensearch-ruby ruby gems.

The devil is in the details, and my app is very large and proprietary, however...

Currently my setup is not too pretty
What follows is some more details about my setup, I have a Gemfile:

# Search (Opensearch)
gem 'searchkick', '>= 5.0.3' # https://github.com/ankane/searchkick/tree/v5.0.1#getting-started
gem 'graphql-searchkick', '>= 1.0.0' # graphql pagination for searchkick integration
gem 'searchjoy', '~> 0.5.1' # trending search
gem 'opensearch-ruby', '>= 2.0.0' # search http client transports for opensearch
gem 'swearjar', '>= 1.4.0' # Profanity detection
gem 'oj' # Recommended for JSON parsing performance by ankane/searchkick

And a code snippet like this in my config/initializers/searchkick.rb:

Searchkick.aws_credentials = credentials_stanza

def credentials_stanza
  {
    service: 'es',
    credentials: instance_profile_creds.credentials,
    region: AppName.aws_region
  }
end

def instance_profile_creds
  @instance_profile_creds ||= Aws::InstanceProfileCredentials.new(
    http_read_timeout: TIMEOUTS.aws.instance_profile.read,
    retries: 3,
    region: AppName.aws_region
  )
end

Possible solution

I believe my solution is to use the credentials_provider and not credentials hash key

As per the docs https://github.com/aws/aws-sdk-ruby/blob/c2cf749b9fad60a40aaccd35958f143ef3b38b0b/gems/aws-sigv4/lib/aws-sigv4/signer.rb#L57-L61

So I will change my def credentials_stanza definition to:

    def credentials_stanza
      {
        service: 'es',
        credentials_provider: instance_profile_creds,
        region: AppName.aws_region
      }
    end

Caveat

It appears everyone is just passing along the hash keys and values. So it makes the issue hard to point a finger at.
Currently my config hash goes from:

• my rails initializer
• into searchkick gem
• into this gem https://github.com/winebarrel/faraday_middleware-aws-sigv4
• Into the aws gems https://github.com/aws/aws-sdk-ruby

Conclusion

I thought I'd document this here, and close this issue once I can confirm my issue is resolved.