sharperad / cake.aws.s3 Goto Github PK
View Code? Open in Web Editor NEWAmazon S3 addin for Cake
Home Page: http://cakebuild.net
License: MIT License
Amazon S3 addin for Cake
Home Page: http://cakebuild.net
License: MIT License
We performed an automated audit of your Cake addin and found that it does not follow all the best practices.
We encourage you to make the following modifications:
.csproj
should have a line similar to this: <PackageReference Include="Cake.Core" Version="0.33.0" PrivateAssets="All" />
.csproj
should have a line like this: <PackageIconUrl>https://cdn.jsdelivr.net/gh/cake-contrib/graphics/png/cake-contrib-medium.png</PackageIconUrl>
.Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.
This issue was created by a tool: Cake.AddinDiscoverer version 3.12.1
Add support for transfer acceleration, which would just involve adding a property to the UploadSettings and applying it when creating the S3 client.
We've started using GitHub Discussions as the preferred communication channel moving forward (instead of Gitter) because it makes it easier to keep track of discussions in a structured way, especially if multiple discussions are happening at the same time. It also allows to search for previous questions/answers, which can be a helpful resource.
As such, we're recommending addin maintainers to update any links to Gitter to point to the GH discussions.
Template:
## Discussion
For questions and to discuss ideas & feature requests, use the [GitHub discussions on the Cake GitHub repository](https://github.com/cake-build/cake/discussions), under the [Extension Q&A](https://github.com/cake-build/cake/discussions/categories/extension-q-a) category.
[![Join in the discussion on the Cake repository](https://img.shields.io/badge/GitHub-Discussions-green?logo=github)](https://github.com/cake-build/cake/discussions)
I have my deployment vm set up with Instance Credentials using IAM roles, and access key/secret is not (shouldn't be) needed. However, it looks like this library has a hard requirement on the AccessKey
existing in a UploadSettings
object.
The expected behavrior would be that it falls back to the default AWS authentication methods if accesskey/secrets isn't explicitly set.
I have a cake script that looks like this:
#addin nuget:?package=AWSSDK.Core&version=3.3.101.9
#addin nuget:?package=AWSSDK.S3&version=3.3.101.9
#addin "Cake.AWS.S3&version=0.6.8"
// ... and eventually
var uploadSettings = CreateUploadSettings();
As you can see, I am trying to call CreateUploadSettings.
Unfortunately, it fails with this message:
error CS0103: The name 'CreateUploadSettings' does not exist in the current context
Do you know what I am doing wrong?
My goal of using CreateUploadSettings
is to use my aws credentials that are stored in my aws credentials file. Could you please also let me know if this will take these credentials?
I am trying to setup this so that I don't have to set the AccessKey
and SecretKey
in my UploadSettings
. I would expect Cake.AWS.S3
to pick these credentials automatically just like the aws cli does.
This is a very useful addin for Cake, thanks! ๐
One problem I hit was that the AWS environment variables used by CakeEnvironmentExtensions don't match what's used by the AWS SDK:
Was this a deliberate choice to differentiate them from the fallback credentials used by the SDK?
Normally I use the ctors which leverage the FallbackCredentialsFactory:
... which gives you a few ways to automatically inject the credentials - via an instance profile when running inside AWS or via the standard environment variables when running outside AWS.
Not a major problem obviously, but it does require more setup than expected for the common case, so this addin should ideally use the same standard conventions already established by the AWS SDK.
Hi @SharpeRAD,
I am using Cake.AWS.S3 version 1.0.0 which references version 1.0.6 of MimeTypesMap package. But it throws a missing package error for the above version of MimeTypesMap.
I have checked in NuGet for MimeTypesMap but couldn't find 1.0.6 version in the version history for this package.
Do we know if it's an existing issue and if we have any fix for the same?
Thanks
We performed an automated audit of your Cake addin and found that it does not follow all the best practices.
We encourage you to make the following modifications:
.csproj
should have a line similar to this: <PackageReference Include="Cake.Core" Version="0.28.0" PrivateAssets="All" />
Apologies if this is already being worked on, or if there are existing open issues, this issue was created based on what is currently published for this package on NuGet.org and in the project on github.
versions Cake 0.32.1
versions Cake.AWS.S3 0.6.8
Operating system I am on Windows 10.
The file uploads fine, but the headers appear to not be getting set.
It may be that I am missing something, however I am not seeing it would appreciate any guidance.
The file is just a index.html.
Here is the relevant part of my cake script.
var sync = new SyncSettings()
{
AccessKey = accessKey,
SecretKey = secretKey,
BucketName = bucketName,
CannedACL = S3CannedACL.PublicRead,
CompressContent = true,
DefaultContentType = "text/html",
GenerateContentType = true,
Region = RegionEndpoint.APSoutheast2,
};
sync.Headers.CacheControl = "private, no-cache, no-store, proxy-revalidate, no-transform";
sync.Headers["Pragma"] = "no-cache";
await S3SyncUpload(file, sync);
issuing a dotnet cake
Could not load D:\source\psnbooster\build\tools\Addins\Cake.AWS.S3.0.6.6\lib\netstandard2.0\Cake.AWS.S3.dll (missing AWSSDK.S3, Version=3.3.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
D:/source/psnbooster/build/build.cake(212,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(213,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(218,19): error CS0103: The name 'S3CannedACL' does not exist in the current context
D:/source/psnbooster/build/build.cake(222,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(223,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(228,19): error CS0103: The name 'S3CannedACL' does not exist in the current context
Error: Error(s) occurred when compiling build script:
D:/source/psnbooster/build/build.cake(212,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(213,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(218,19): error CS0103: The name 'S3CannedACL' does not exist in the current context
D:/source/psnbooster/build/build.cake(222,9): error CS0103: The name 'S3Upload' does not exist in the current context
D:/source/psnbooster/build/build.cake(223,8): error CS0246: The type or namespace name 'UploadSettings' could not be found (are you missing a using directive or an assembly reference?)
D:/source/psnbooster/build/build.cake(228,19): error CS0103: The name 'S3CannedACL' does not exist in the current context
any ideas?
I get the following error:
Error: The assembly 'Cake.AWS.S3, Version=0.4.6.0, Culture=neutral, PublicKeyToken=null' is referencing an older version of Cake.Core (0.17.0). This assembly need to reference at least Cake.Core version 0.22.0. Another option is to downgrade Cake to an earlier version. It's not recommended, but you can explicitly opt-out of assembly verification by configuring the Skip Verification setting to true (i.e. command line parameter "--settings_skipverification=true", envrionment variable "CAKE_SETTINGS_SKIPVERIFICATION=true", read more about configuration at https://cakebuild.net/docs/fundamentals/configuration)
Thanks again for creating this Cake Addin, we really appreciate the effort that you have put in to creating it.
We, the Cake Team, recently announced a new Cake Contrib Icon, details of which can be found here:
http://cakebuild.net/blog/2017/06/new-cake-contrib-icon
Would you consider changing the nuspec file for your NuGet Package to use this new Cake Contrib Icon? If so, the recommended URL to use is:
Details of the above URL can be found in the repository here:
https://github.com/cake-contrib/graphics
Please let me know if you have any questions.
First of all, I wanted to thank you for adding to the Cake community by adding this addin.
I was just wondering if you had seen this blog post:
http://cakebuild.net/blog/2016/08/cake-contribution-organization
We are currently going through a process of increasing the visibility of addins, and also trying to ensure their long term maintainability.
To that end, we are asking addin creators to add the cake-contrib user on NuGet as a co-owner (this can be done through the NuGet website by clicking on Manage Owners
on the package page).
Would you be interested in doing this for Cake.AWS.S3? If you have any questions about this, please let me know. There was some initial concern that the Cake Team were trying to "take over" packages, and that couldn't be further from the truth, and if you have this concern, or others, I would like to address them.
Thanks!
When using SyncUpload method to sync a local directory, no matter what KeyPrefix I specified. The local folder will always be put in the bucket root folder on S3.
The GetKey method in S3Manager.cs seems not using the keyPrefix, it just use the local folder path as the S3 key.
string key = this.GetKey(file, fullPath, settings.LowerPaths);
I am trying the set CacheControl to "no-cache" but no matter what I do CacheControl always reverts to "public, max-age=864000"
var settings = CreateSyncSettings(awsAccessKey, awsSecretKey)
.SetBucketName(s3Bucket)
.SetRegion(regionName)
.SetLowerPaths(false);
settings.CacheControl = "no-cache";
var invalidationList = await S3SyncUpload((DirectoryPath)"site", settings);
In 0.6.6, along with Cake.AWS.S3.dll the following dependencies were included in the lib/ folder:
AWSSDK.S3.dll
AWSSDK.Core.dll
MimeTypesMap.dll
In 0.6.7 the only library present is Cake.AWS.S3.dll and cake script now immediately fails with the following errors:
Could not load [repo]\tools\Addins\Cake.AWS.S3.0.6.7\lib\net46\Cake.AWS.S3.dll (missing AWSSDK.S3, Version=3.3.0.0, Culture=neutral, PublicKeyToken=885c28607f98e604)
error CS1061: 'ICakeContext' does not contain a definition for 'CreateDownloadSettings' and no extension method 'CreateDownloadSettings' accepting a first argument of type 'ICakeContext' could be found (are you missing a using directive or an assembly reference?)
error CS0103: The name 'S3Download' does not exist in the current context
The NuGet Gallery will soon include a "Cake" tab with instructions on how to use NuGet packages in Cake build scripts. I'd like to suggest that you include the tag cake-addin
in the NuGet package of this addin, so that the NuGet Gallery can display the correct instructions to install this addin.
The UI will look similar to this:
Documentation on best practices on tags for Cake addins: https://cakebuild.net/docs/extending/addins/best-practices#tags
The comments for uploading mention utilization of the multi-part upload capability, but no usage of the defined AWS SDK API (https://docs.aws.amazon.com/AmazonS3/latest/dev/HLuploadFileDotNet.html?shortFooter=true) is seen in the code that I can find.
Cake 2.0.0 has been released ๐
I would love to be able to do the following and use S3's AES256 server-side encryption with AWS-managed (or customer managed) KMS keys:
var settings = Context.CreateUploadSettings();
settings.BucketName = myBucket;
settings.ServerSideEncryption = ServerSideEncryptionMethod.AES256;
await S3Upload(myFile, myS3Key, settings);
or
var settings = Context.CreateUploadSettings();
settings.BucketName = myBucket;
settings.ServerSideEncryption = ServerSideEncryptionMethod.AWSKMS;
settings.ServerSideEncryptionKeyManagementServiceKeyId = "MY_KMS_KEY_ARN";
await S3Upload(myFile, myS3Key, settings);
Since S3 now supports these types of server-side encryption (as opposed to customer-managed encryption), adding support for stuff like this would be very helpful. If I find time, I will see if I can put together a pull request. The reference guide for these encryption methods lives here: https://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html
Task("Download-File-Fallback")
.Description("Download a file from S3 using AWS Fallback credentials")
.Does(() =>
{
S3Download("C:/Files/test.zip", "test.zip", Context.CreateDownloadSettings()
{
BucketName = "cake-s3"
});
});
gets error CS1026: ) expected
pointing at the end of the line:
S3Download("C:/Files/test.zip", "test.zip", Context.CreateDownloadSettings()
If I code like this it works:
Task("Download-File-Fallback")
.Description("Download a file from S3 using AWS Fallback credentials")
.Does(() =>
{
var settings = Context.CreateDownloadSettings();
settings.BucketName = "cake-s3";
S3Download("C:/Files/test.zip", "test.zip", settings);
});
In order to use Cake.AWS.S3 in a Cake 0.26+ environment without having to suppress warnings, etc., it would be useful to update references.
C:\code> .\build.ps1
Preparing to run build script...
Running build script...
Error: The assembly 'Cake.AWS.S3, Version=0.6.0.0, Culture=neutral, PublicKeyToken=null'
is referencing an older version of Cake.Core (0.22.0).
This assembly need to reference at least Cake.Core version 0.26.0.
Another option is to downgrade Cake to an earlier version.
It's not recommended, but you can explicitly opt-out of assembly verification
by configuring the Skip Verification setting to true
(i.e. command line parameter "--settings_skipverification=true",
environment variable "CAKE_SETTINGS_SKIPVERIFICATION=true",
read more about configuration at https://cakebuild.net/docs/fundamentals/configuration)
Will have a PR shortly
We've recently released new icons for Cake extensions, to help more easily differentiate Cake Addins, Modules, Recipes, etc.
More information: https://github.com/cake-contrib/graphics
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.