gplumb/netcrawlerdetect

A .net standard port of JayBizzle's CrawlerDetect project (https://github.com/JayBizzle/Crawler-Detect).

netcrawlerdetect's Issues

A conflict with System.ValueTuple package

Hi,

I've tried to add the NetCrawlerDetect in our solution, but it caused multiple dll conflicts and cycled dependencies.

Problem description

Here is a repro:
https://github.com/ShadowJack/NetCrawlerDetectConflictExample

The project is compiled successfully after the first commit, but the build is broken when I install the NetCrawlerDetect package.

Here is the build output:

1>------ Rebuild All started: Project: NetCrawlerDetectConflict, Configuration: Debug Any CPU ------
1>C:\Users\KPA\Documents\Visual Studio 2019\Projects\NetCrawlerDetectConflict\NetCrawlerDetectConflict\TupleFactory.cs(11,23,11,46): error CS8137: Cannot define a class or member that utilizes tuples because the compiler required type 'System.Runtime.CompilerServices.TupleElementNamesAttribute' cannot be found. Are you missing a reference?
1>C:\Users\KPA\Documents\Visual Studio 2019\Projects\NetCrawlerDetectConflict\NetCrawlerDetectConflict\TupleFactory.cs(11,23,11,46): error CS8179: Predefined type 'System.ValueTuple`2' is not defined or imported
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

Also Visual Studio reports another error in a BotDetector class:

Severity	Code	Description	Project	File	Line	Suppression	State
Error	CS0731	The type forwarder for type 'System.Text.RegularExpressions.MatchCollection' in assembly 'System' causes a cycle	NetCrawlerDetectConflict	C:\Users\KPA\Documents\Visual Studio 2019\Projects\NetCrawlerDetectConflict\NetCrawlerDetectConflict\BotDetector.cs	14	Active

Environment

Microsoft Visual Studio Community 2019
Version 16.0.3
VisualStudio.16.Release/16.0.3+28803.352
Microsoft .NET Framework
Version 4.8.03752

ASP.NET and Web Tools 2019   16.0.12313.64372
ASP.NET and Web Tools 2019
ASP.NET Web Frameworks and Tools 2019   16.0.12313.64372
C# Tools   3.0.0-beta4-19218-01+18b0cfb66884840b40491fa0896a8854d68b3959
NuGet Package Manager   5.0.0
...

Possible issue

Many references to .NET Standard dlls are added to the project when I install the NetCrawlerDetect package. All of them have a hintPath to ..\packages\NetCrawlerDetect.0.2.3\lib\netstandard2.0\ folder.
I guess normally it shouldn't be the case, as all .NET standard 2.0 dlls are already present in .NET Framework 4.6.2

Please advise.

WebHeaderCollection mismatch in keys style.

I might just be doing something wrong here but in all the things that I've tried, my WebHeaderCollection has keys of the form User-Agent which doesn't match USER_AGENT in your list.

It's the difference between - and _ .

Now obviously this must work for some people, otherwise it would be obvious that the library never worked, so a quicker pointer would be useful.

Thanks.