Unhandled exception for ignored packages from nuget private feeds

[digitalcoyote]

Unhandled exception for ignored packages from nuget private feeds

[digitalcoyote]

I have a feeling I know what caused that. Side effect of trying to work on too little sleep. I probably won't get a chance to look at it tonight (working on that sleep thing). I expect I'll get to it the next day though. And hopefully I'll be able to run it through enough tests to catch anything else.

Thanks for the help

[marco-junge]

Is it possible to exclude ignored packages already here?

[digitalcoyote]

That seems appropriate. I'll look further into it tonight and try to have a fix ready soon.

[digitalcoyote]

29801a9 should take care of it. It does change the functionality a bit, but I think this early in the life of the package this is acceptable. I also added some try/catch to add warnings/errors in msbuild when things fail and allow it to continue when it can (with reduced functionality).

I'm too tired to test it at the moment and I no longer have a private feed I can use to test it (I'll have to manually throw an exception there). If you want to try the code out, this is likely the version that will be published tomorrow.

[marco-junge]

Awesome, that was blazing fast! I'll try it out later today and give you feedback. Thanks, man.

[marco-junge]

Ignoring packages works perfect now. Verified from a local build of NuGetDefense.
But I recognized another exception after adding jquery 1.5.1 to my net core project file just to check a breaking build for a vulnerable package.

'CheckForVulnerableNuGetPkgs:
dotnet "C:....nuget\packages\nugetdefense\1.0.2\build..\tools\netcoreapp3.1\NuGetDefense.dll" C:...\MyProject.csproj netcoreapp3.1
1>C:...\MyProject.csproj : warning : NuGetDefense : OSS Index scan failed with exception: System.AggregateException: One or more errors occurred. (The JSON value could not be converted to NuGetDefense.Core.Vulnerability+AccessVectorType. Path: $[0].vulnerabilities[0].cvssVector | LineNumber: 0 | BytePositionInLine: 841.)
---> System.Text.Json.JsonException: The JSON value could not be converted to NuGetDefense.Core.Vulnerability+AccessVectorType. Path: $[0].vulnerabilities[0].cvssVector | LineNumber: 0 | BytePositionInLine: 841.
at System.Text.Json.ThrowHelper.ThrowJsonException()
at System.Text.Json.Serialization.Converters.JsonConverterEnum1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) at System.Text.Json.JsonPropertyInfoNotNullable4.OnRead(ReadStack& state, Utf8JsonReader& reader)
at System.Text.Json.JsonPropertyInfo.Read(JsonTokenType tokenType, ReadStack& state, Utf8JsonReader& reader)
at System.Text.Json.JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
at System.Text.Json.JsonSerializer.ReadCore(JsonReaderState& readerState, Boolean isFinalBlock, ReadOnlySpan1 buffer, JsonSerializerOptions options, ReadStack& readStack) at System.Text.Json.JsonSerializer.ReadAsync[TValue](Stream utf8Json, Type returnType, JsonSerializerOptions options, CancellationToken cancellationToken) at NuGetDefense.OSSIndex.Scanner.GetReportsForPackagesAsync(NuGetPackage[] pkgs) --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task1.get_Result() at NuGetDefense.OSSIndex.Scanner.GetVulnerabilitiesForPackages(NuGetPackage[] pkgs, Dictionary2 vulnDict)
`

[digitalcoyote]

Well, without being so tired I've noticed a few issues so it's taking longer than expected to work through it all. I probably won't get a full code review in yet (waiting on CI and unit tests).

[digitalcoyote]

I pushed up what I've done so far. I was able to get JQuery 1.5.1 to spit out 7 vulnerabilities. I've found debugging is frustrating if you don't remove the nuget package references and replace the with project references. Hopefully that shouldn't be necessary after everything is properly unit tested.

[marco-junge]

This works as excepted:

cd Src/NugetDefense/testfiles
dotnet "..\bin\Release\netcoreapp3.1\NuGetDefense.dll" test.csproj
...
packages.config(3,4) : Error : 7 vulnerabilities found for jQuery @ 1.5.1

But I got no vulnerabilities when I create a new net core application like this:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>netcoreapp3.1</TargetFramework>
    </PropertyGroup>
    <ItemGroup>
      <PackageReference Include="jQuery" Version="1.5.1" />
    </ItemGroup>
    <ItemGroup>
      <ProjectReference Include="..\NuGetDefense\NuGetDefense.csproj" />
    </ItemGroup>
    
    <PropertyGroup>
      <NuGetDefenseExe Condition="'$(OS)' == Unix">dotnet "$(MSBuildThisFileDirectory)../NuGetDefense/bin/Release/netcoreapp3.1/NuGetDefense.dll"</NuGetDefenseExe>
      <NuGetDefenseExe Condition="'$(OS)' == 'Windows_NT'">dotnet "$(MSBuildThisFileDirectory)..\NuGetDefense\bin\Release\netcoreapp3.1\NuGetDefense.dll"</NuGetDefenseExe>
    </PropertyGroup>
  
    <Target Name="CheckForVulnerableNuGetPkgs" AfterTargets="Build">
      <Exec Command="$(NuGetDefenseExe) $(MSBuildProjectFullPath) $(TargetFramework)" IgnoreExitCode="true" />
    </Target>
</Project>

cd Src/ConsoleApp1
cp ../NugetDefense/NugetDefense.json ./
dotnet restore
dotnet build
...
ConsoleApp1 -> ...\Src\ConsoleApp1\bin\Debug\netcoreapp3.1\ConsoleApp1.dll
  https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.zip
  https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-recent.json.zip
  CVE: CVE-2009-1605
  Product: 0.1
  CWE: CWE-119
  Product: sumatrapdf
  Descritption: Heap-based buffer overflow in the loadexponentialfunc function in mupdf/pdf_function.c in MuPDF in the mupdf-20090223-win32 package, as used in SumatraPDF 0.9.3 and earlier, allows remote attackers to execute arbitrary code via a crafted PDF file.  NOTE: some of these details are obtained from third party information.
  Vendor: sumatrapdfreader
  CSVSS Score: 
  CSVSS Vector: 
  There is already an entry for sumatrapdf with CVE: CVE-2009-1605. Changing Version to: 0.1
  CVE: CVE-2009-1605
  Product: 0.2
  CWE: CWE-119
...
(many CVEs)
...
0 Warnings
0 Errors

[digitalcoyote]

I'm setting up the unit tests so I can investigate this at a more granular level. I haven't. Committed everything yet, but I have some fixes for the Core package that may handle this. I'm also setting up private feeds through appveyor to make testing these easier.

[marco-junge]

Please have a look at PR #20. I added some refactorings and tests to verify reading package dependencies from different sources.

[digitalcoyote]

Thanks for the contribution. I'll take a look at the PR as soon as I can.

[digitalcoyote]

I'm going to remove this feature (dependency vulnerability checking) for now. The more I think about it, I feel like depending on metadata to determine the versions of packages that were restored is flawed. Depending on the version of NuGet used (or the behavior of a similar tool), different versions could be restored than what this NuGetDefense could reasonable expect.

I'm considering trying to look and see exactly what versions were restored instead. I'm going to leave your PR open for now, as I may revisit it after I get a method of determining the versions locally. If I still need to pull metadata regarding which dependencies are dependencies of other packages from a NuGet feed, I'll end up needing to ensure support for Private Feeds as well.

[digitalcoyote]

1.0.2.0 removed dependency checking. I added the "Hold" label to the pull request so it can be considered when dependency checking is revisited.