max-ieremenko/thirdpartylibraries

This command line tool helps to manage third party libraries and their licenses in .net applications.

thirdpartylibraries's Issues

Packages from Private GitHub package repos are being reported with `nuget.org` Fake URLs

Describe the bug
We have proprietary packages in nuget.pkg.github.com/MYCORP. When generating our FOSS list with your tool all packages are output to the readme.md with https://www.nuget.org/packages/MYPACKAGE/VERSION which doesn't exist. No license information is gathered either. Right now this scenario only pertains to NUGET but it might very likely pertain to NPM in the future for us. My GitHub PAT has access to these repositories as well for the restore and also has package:read permissions in the PAT as well. No errors are reported during the runtime of the tool.

To Reproduce
Create a CSPROJ with a private package coming from GitHub.

Expected behavior
The proper URL and other things such as licensing should be output.

Environment

• OS: MacOS 13.3.1
• .net framework version: 6.0.100

Thank you :)

Make github "Forbidden: rate limit exceeded" more informative

When github API complains about 403 rate limit exceeded response contains detailed information in headers:

X-RateLimit-Limit: 60
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1633713483
X-RateLimit-Used: 60

Provide details in the exception message.

License could not be resolved from https://raw.github.com

Nuget package example: Newtonsoft.Json 10.0.1
https://raw.github.com/JamesNK/Newtonsoft.Json/master/LICENSE.md

Export to CSV Instead of readme.ms

In order to work with other tools for other package managers (python, go, etc.) that export to CSV I would like to request that the table structure that you create for the licenses themselves (not the rollups) into a CSV file.

Export the CSV with the following columns
package, version, license type, used by, internal (build dep)

A post-processor could be created I suppose but since you have a Repository design pattern it should be fairly straight forward to impl a CSV export option

Error -> Type requires a JSON object to deserialize correctly

Describe the bug

To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List<T> that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Path '', line 1, position 1.

To Reproduce
ThirdPartyLibraries update -appName MyApp -repository ./3rdParty -github.com:personalAccessToken $GITHUB_PAT $source

Expected behavior
An updated readme file

Environment

• OS: macOS (latest)
• .net framework version net6/7
• npm for web side logic as well

Additional context
Had no problems in 3.0.0 but quite a few libs have been added.

Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'ThirdPartyLibraries.Repository.Template.LibraryIndexJson' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List<T> that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Path '', line 1, position 1.
Newtonsoft.Json.JsonSerializationException: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'ThirdPartyLibraries.Repository.Template.LibraryIndexJson' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List<T> that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Path '', line 1, position 1.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureArrayContract(JsonReader reader, Type objectType, JsonContract contract)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
   at ThirdPartyLibraries.Shared.JsonSerializerExtensions.JsonDeserialize[T](Stream content) in /_/Sources/ThirdPartyLibraries.Shared/JsonSerializerExtensions.cs:line 15
   at ThirdPartyLibraries.Repository.StorageExtensions.ReadLibraryIndexJsonAsync[TModel](IStorage storage, LibraryId id, CancellationToken token) in /_/Sources/ThirdPartyLibraries.Repository/StorageExtensions.cs:line 55
   at ThirdPartyLibraries.Suite.Remove.Internal.PackageRemover.RemoveFromApplicationAsync(LibraryId id, String appName, CancellationToken token) in /_/Sources/ThirdPartyLibraries.Suite/Remove/Internal/PackageRemover.cs:line 38
   at ThirdPartyLibraries.Suite.Update.UpdateCommand.RemoveFromApplicationAsync(ILogger logger, IPackageRemover remover, HashSet`1 references, CancellationToken token) in /_/Sources/ThirdPartyLibraries.Suite/Update/UpdateCommand.cs:line 133
   at ThirdPartyLibraries.Suite.Update.UpdateCommand.ExecuteAsync(IServiceProvider serviceProvider, CancellationToken token) in /_/Sources/ThirdPartyLibraries.Suite/Update/UpdateCommand.cs:line 43
   at ThirdPartyLibraries.CommandChain.ExecuteAsync(IServiceProvider serviceProvider, CancellationToken token) in /home/runner/work/ThirdPartyLibraries/ThirdPartyLibraries/Sources/ThirdPartyLibraries/CommandChain.cs:line 24
   at ThirdPartyLibraries.Program.RunConsoleAsync(CommandLine commandLine, ConsoleLogger logger, CancellationToken token) in /home/runner/work/ThirdPartyLibraries/ThirdPartyLibraries/Sources/ThirdPartyLibraries/Program.cs:line 88

License Link in Report - Shouldn't it link to the published license for the Library

Having a copy of the MIT license isn't all that helpful. What if the actually linked MIT license has some modifications?

Can we have it so that it links to the LICENSE.txt file in the repo for the library if possible?

npm ArgumentNullException : Value cannot be null. (Parameter 'version')

Describe the bug
npm package.json with dependency with empty version leads to ArgumentNullException : Value cannot be null. (Parameter 'version')

{
  "dependencies": {
    "bootstrap": "^4.4.1"
  },
  "devDependencies": {
    "jasmine": ""
  }
}

Expected behavior
According to specification empty string is the same as *

.net 6.0 support

"Dedicated To The Public Domain" License is broken into 5 different licenses

Describe the bug
I'm not sure if this is isolated to this specific license or any license that has whitespace in it. Dedicated To The Public Domain is broken into several individual licenses.

To Reproduce
Include a library that is dedicated to the public domain.

Expected behavior
This should be one license.

Environment

• OS: Mac/Linux
• 6/7

Additional context
Add any other context about the problem here.

Getting error looking for Chef cookbooks (nodejs_test) when running `Update-ThirdPartyLibrariesRepository`

I run the following command (pwsh on Mac)

Update-ThirdPartyLibrariesRepository -AppName ThirdPartyLibraries -Source ./ -Repository ./3rdParty/ -InformationAction Continue

and get this error (anonymized)

Update-ThirdPartyLibrariesRepository: Directory /Users/me/src/./awn/chef/cookbooks/nodejs/test/cookbooks/nodejs_test/templates/default/node_modules not found.

How to prevent - Target net20 not found - on Win 10 with .net 3.5 Feature enabled

hi
i try to execute for an old solution but i can not find a way to get it run -

ThirdPartyLibraries update -appName Test -source . -repository c:\temp_repo

Target net20 not found in .NETFramework,Version=v2.0, .NETFramework,Version=v2.0/win, .NETFramework,Version=v2.0/win-x64, .NETFramework,Version=v2.0/win-x86.
System.InvalidOperationException: Target net20 not found in .NETFramework,Version=v2.0, .NETFramework,Version=v2.0/win, .NETFramework,Version=v2.0/win-x64, .NETFramework,Version=v2.0/win-x86.
at ThirdPartyLibraries.NuGet.ProjectAssetsParser.ParseTarget(String frameworkName) in C:...\ThirdPartyLibraries\Sources\ThirdPartyLibraries.NuGet\ProjectAssetsParser.cs:line 200
at ThirdPartyLibraries.NuGet.ProjectAssetsParser.GetReferences(String targetFramework) in C:...\ThirdPartyLibraries\Sources\ThirdPartyLibraries.NuGet\ProjectAssetsParser.cs:line 74

any suggestion/hint what i am doing wrong here?

CI build

Linux Support

Is this supposed to work under linux, too? I face the following problems:

$  dotnet tool install --global ThirdPartyLibraries.GlobalTool
You can invoke the tool using the following command: ThirdPartyLibraries
Tool 'thirdpartylibraries.globaltool' (version '2.0.0') was successfully installed.

$  ThirdPartyLibraries refresh -appName MyApp -repository .
Error: Could not find a part of the path '/home/marc/.dotnet/tools/.store/thirdpartylibraries.globaltool/2.0.0/thirdpartylibraries.globaltool/2.0.0/tools/netcoreapp3.1/any/packages'.
System.IO.DirectoryNotFoundException: Could not find a part of the path '/home/marc/.dotnet/tools/.store/thirdpartylibraries.globaltool/2.0.0/thirdpartylibraries.globaltool/2.0.0/tools/netcoreapp3.1/any/packages'.
   at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerator.Unix.cs:line 80
   at System.IO.Enumeration.FileSystemEnumerator`1.Init() in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerator.Unix.cs:line 40
   at System.IO.Enumeration.FileSystemEnumerator`1..ctor(String directory, Boolean isNormalized, EnumerationOptions options) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerator.cs:line 43
   at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerable.cs:line 39
   at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerableFactory.cs:line 128
   at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options) in /_/src/System.IO.FileSystem/src/System/IO/Directory.cs:line 191
   at ThirdPartyLibraries.Repository.FileStorage.GetAllLibrariesAsync(CancellationToken token) in D:\Work\ThirdPartyLibraries\Sources\ThirdPartyLibraries.Repository\FileStorage.cs:line 41
   at ThirdPartyLibraries.Suite.Internal.PackageRepository.UpdateAllPackagesReadMeAsync(CancellationToken token)
   at ThirdPartyLibraries.Suite.Commands.RefreshCommand.ExecuteAsync(CancellationToken token) in D:\Work\ThirdPartyLibraries\Sources\ThirdPartyLibraries.Suite\Commands\RefreshCommand.cs:line 29
   at ThirdPartyLibraries.Program.Main(String[] args) in D:\Work\ThirdPartyLibraries\Sources\ThirdPartyLibraries\Program.cs:line 49

Downloaded the relased package, unzipped it to /home/marc/tmp/3rdlic and started:

$  dotnet ThirdPartyLibraries.dll refresh -appName MyApp -repository ~/code/repo
Error: Could not find a part of the path '/home/marc/tmp/3rdlic/packages'.
System.IO.DirectoryNotFoundException: Could not find a part of the path '/home/marc/tmp/3rdlic/packages'.
   at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerator.Unix.cs:line 80
   at System.IO.Enumeration.FileSystemEnumerator`1.Init() in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerator.Unix.cs:line 40
   at System.IO.Enumeration.FileSystemEnumerator`1..ctor(String directory, Boolean isNormalized, EnumerationOptions options) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerator.cs:line 43
   at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerable.cs:line 39
   at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options) in /_/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerableFactory.cs:line 128
   at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options) in /_/src/System.IO.FileSystem/src/System/IO/Directory.cs:line 191
   at ThirdPartyLibraries.Repository.FileStorage.GetAllLibrariesAsync(CancellationToken token) in D:\Work\ThirdPartyLibraries\Sources\ThirdPartyLibraries.Repository\FileStorage.cs:line 41
   at ThirdPartyLibraries.Suite.Internal.PackageRepository.UpdateAllPackagesReadMeAsync(CancellationToken token)
   at ThirdPartyLibraries.Suite.Commands.RefreshCommand.ExecuteAsync(CancellationToken token) in D:\Work\ThirdPartyLibraries\Sources\ThirdPartyLibraries.Suite\Commands\RefreshCommand.cs:line 29
   at ThirdPartyLibraries.Program.Main(String[] args) in D:\Work\ThirdPartyLibraries\Sources\ThirdPartyLibraries\Program.cs:line 49

Command line tool net5.0 out of support

The command line tool and .net tool for net5.0 will be removed in future releases.

How it could affect me?

• no attachment ThirdPartyLibraries.*-net5.0.zip on release pages for future releases
• dotnet tool install --global ThirdPartyLibraries.GlobalTool on the environment with only installed .NET 5 SDK will not be supported

If any objections, please comment here.

Feature Request: Ability to consolidate reports from multiple repositories/sources

I would like to have a merge of multiple projects (//*.csproj) into a single report. Is there a way to do this currently?