Giter Site home page Giter Site logo

rssfeeder's People

Contributors

azure-pipelines[bot] avatar dependabot[bot] avatar mgnoonan avatar

Stargazers

avatar

Watchers

avatar avatar

rssfeeder's Issues

Issue with EXCEPTION System.NullReferenceException

Account Id:
ActionId: 00a8540d-7a8d-4edc-9d55-58bc8ad537e8
ActionName: RssFeeder.Mvc.Controllers.RssController.Get (RssFeeder.Mvc)
Application version:
Auth user Id:
Browser version:
City: Washington
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A35AF
Cloud role name:
Country or region: United States
Device model:
Device type: PC
Elapsed: 40.7145
Event time: 6/18/2020, 8:42:18 AM
Exception type: System.NullReferenceException
Failed method: RssFeeder.Mvc.Controllers.RssController+d__5.MoveNext
MessageTemplate: HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms
Operating system:
Operation Id: f315678b7acf554dab1939e3724f3b12
Operation name:
Parent Id: cfa75ce20a5e2345
ParentId: 0000000000000000
Problem Id: System.NullReferenceException at RssFeeder.Mvc.Controllers.RssController+d__5.MoveNext
RequestId: 80008229-0000-f400-b63f-84710c7967bb
RequestMethod: GET
RequestPath: /api/rss/drudge-report
Sample Rate: 100
SDK version: dotnetc:2.11.0-21474
Session Id:
Severity level: Error
Source of synthetic traffic:
SourceContext: Serilog.AspNetCore.RequestLoggingMiddleware
SpanId: cfa75ce20a5e2345
State or province: Virginia
StatusCode: 500
Telemetry type: exception
TraceId: f315678b7acf554dab1939e3724f3b12
User Id:

View this instance in the Azure portal<\a>
View similar exceptions in this time range<\a>

Handle crawler response status codes that indicate failure

Message View Event

Response status code = "Forbidden", "https://www.realclearpolitics.com/video/2021/12/27/president_biden_there_is_no_federal_solution_to_covid_test_shortage.html"

Properties

* statusCode : "Forbidden"
* uri : "https://www.realclearpolitics.com/video/2021/12/27/president_biden_there_is_no_federal_solution_to_covid_test_shortage.html"
* urlHash : "b5ca5bbcb1266bf3689ec35033653a74"
* url : "https://t.co/6IuKDr4z8a"
* collectionName : "bad-blue"

Issue with HEAD /api/rss/drudge-report

MS.ProcessedByMetricExtractors: (Name:'Requests', Ver:'1.1')
Application version: 1.0.0.0
AspNetCoreEnvironment: Production
City: Redding
Client IP address: 0.0.0.0
Country or region: United States
Device type: PC
Event count: 1.00
Event time: 8/29/2019 9:12:40 AM
Operation ID: 760b4eecb2496e4c9df3a060e5c4b493
Operation name: HEAD /api/rss/drudge-report
Operation parent ID: 760b4eecb2496e4c9df3a060e5c4b493
Performance: <250ms
Request ID: |760b4eecb2496e4c9df3a060e5c4b493.cb857c11
Request name: HEAD /api/rss/drudge-report
Request URL: https://rssfeedermvc.azurewebsites.net/api/rss/drudge-report
Response code: 404
Response time: 0ms
Role instance: RD00155D8A4145
Role name: rssfeedermvc
SDK version: aspnet5c:2.7.0
State or province: California
Successful request: False

Add logic to clean up files from working folder

The working folder will grow with files over time, need to add some logic to clean them up. I recommend anything older than 7 days, since that's the same purge rate in use for the CosmosDB.

Issue with EXCEPTION System.ArgumentException

Account Id:
ActionId: a944368a-752e-44bf-9370-0c4934f2ef39
ActionName: RssFeeder.Mvc.Controllers.RssController.Get (RssFeeder.Mvc)
Application version:
Auth user Id:
Browser version:
City: Washington
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A89A5
Cloud role name:
ContentType: null
Country or region: United States
Device model:
Device type: PC
Elapsed: 1636.8255
EndpointName: RssFeeder.Mvc.Controllers.RssController.Get (RssFeeder.Mvc)
Event time: 10/25/2020, 12:22:19 PM
Exception type: System.ArgumentException
Failed method: RssFeeder.Mvc.Controllers.RssController+d__7.MoveNext
Host: rssfeedermvc.azurewebsites.net
MachineName: RD00155D8A89A5
MessageTemplate: HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms
Operating system:
Operation Id: 8dd1951ead0dd641bb0b0d2603c939cb
Operation name:
Parent Id: 20ba2232628c7244
ParentId: 0000000000000000
Problem Id: System.ArgumentException at RssFeeder.Mvc.Controllers.RssController+d__7.MoveNext
Protocol: HTTP/1.1
RequestId: 800054fe-0001-f200-b63f-84710c7967bb
RequestMethod: GET
RequestPath: /api/rss/eagle-slant
Sample Rate: 100
Scheme: https
SDK version: dotnetc:2.14.0-17971
Session Id:
Severity level: Error
Source of synthetic traffic:
SourceContext: Serilog.AspNetCore.RequestLoggingMiddleware
SpanId: 20ba2232628c7244
State or province: Virginia
StatusCode: 500
Telemetry type: exception
TraceId: 8dd1951ead0dd641bb0b0d2603c939cb
User Id:

View this instance in the Azure portal<\a>
View similar exceptions in this time range<\a>

Add support for embedded YT player

If the link is direct to a YouTube video, it would be nice to have an embedded player in the news reader rather than shelling out to the browser. Although that is viable alternative, it's jarring to have to leave the dedicated news reader when an embedded player will allow for video playback without leaving the current environment.

ScriptParser occasionally fails

Need to go through and audit the failures to see why we sometimes get a null response. It's a regex parsing scheme so this doesn't surprise me.

Issue with HTTP "GET" "/robots.txt" responded 404 in 0.3884 ms

Account Id:
Application version:
Auth user Id:
Browser version:
City: Washington
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A4F4F
Cloud role name:
ContentType: null
Country or region: United States
Device model:
Device type: PC
Elapsed: 0.3884
Event time: 6/27/2020, 6:36:09 PM
Host: rssfeedermvc.azurewebsites.net
MachineName: RD00155D8A4F4F
Message: HTTP "GET" "/robots.txt" responded 404 in 0.3884 ms
MessageTemplate: HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms
Operating system:
Operation Id: 50d9d001a2e16c489f52080866d41e52
Operation name:
Parent Id: b31f53b31b059940
ParentId: 0000000000000000
Protocol: HTTP/1.1
RequestId: 8000a28c-0001-f800-b63f-84710c7967bb
RequestMethod: GET
RequestPath: /robots.txt
Sample Rate: 100
Scheme: https
SDK version: dotnetc:2.14.0-17971
Session Id:
Severity level: Informational
Source of synthetic traffic:
SourceContext: Serilog.AspNetCore.RequestLoggingMiddleware
SpanId: b31f53b31b059940
State or province: Virginia
StatusCode: 404
Telemetry type: trace
TraceId: 50d9d001a2e16c489f52080866d41e52
User Id:

View this instance in the Azure portal<\a>

upgrade to .NET 6.0

Still in preview, but getting close enough we can go ahead and do the upgrade.

Add article parser to handle script-based article content

Describe the issue

As described in #155 some content sites are lazy loading the articles via JS techniques (apnews.com is one example). The HttpClient download as currently configured will strip out script tags from the saved content.

Resolution

  • If possible, continue to use the HttpClient to download the content, but add an option to leave certain script tags in place.
  • If not possible to use HttpClient, then implement #155 first to force a Selenium download of the content.
  • Create a new article parser to handle the article content coming from the script element, or an embedded JSON object within the script element.

GetHostName fails with encoded URLs

Message View Event

PARSE_ERROR: UrlHash '"1b9521d0a0869447378b7829e2785c97"':'"https://t.co/6bCh6H04GW"'

Exception

System.UriFormatException: Invalid URI: The format of the URI could not be determined.
   at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind, UriCreationOptions& creationOptions)
   at System.Uri..ctor(String uriString)
   at RssFeeder.Console.ArticleParser.GetHostName(RssFeedItem item) in C:\Projects\RssFeeder\RssFeeder.Console\ArticleParser.cs:line 62
   at RssFeeder.Console.ArticleParser.Parse(RssFeedItem item) in C:\Projects\RssFeeder\RssFeeder.Console\ArticleParser.cs:line 58
   at RssFeeder.Console.WebCrawler.ParseAndSave(RssFeed feed, RssFeedItem item) in C:\Projects\RssFeeder\RssFeeder.Console\WebCrawler.cs:line 139

Properties

* urlHash : "1b9521d0a0869447378b7829e2785c97"
* url : "https://t.co/6bCh6H04GW"
* collectionName : "bad-blue"

Do not crawl/parse home pages

When crawling for articles, it is unlikely that a link to a homepage is going to yield any meaningful article text. In an attempt to isolate such links, we need to break down the Uri components and if there is nothing in the AbsolutePath value other than maybe a slash / then treat that link as uncrawlable.

There are several cases where we are getting such links, perhaps the regex for the FeedBuilder is too greedy. Or maybe it is intentional on the part of the site authors to slip in some generic site pages.

Error crawling rumble.com

Looks like maybe rumble does not always have the target element. Need to handle this more cleanly.

Example data:

{
    "@t": "2021-10-30T15:06:26.6477545Z",
    "@mt": "Attempting HTML tag parsing using body selector '{bodySelector}' and paragraph selector '{paragraphSelector}'",
    "@m": "Attempting HTML tag parsing using body selector '#videoPlayer' and paragraph selector 'video'",
    "@i": "2a4d07d6",
    "bodySelector": "#videoPlayer",
    "paragraphSelector": "video",
    "siteName": "rumble",
    "urlHash": "039c7752ada8ad6de5c62c257ed9b0aa",
    "url": "https://t.co/Ou9HZQXgRu",
    "collectionName": "bad-blue"
}

image

Add ability to force download using the Chrome WebDriver

Describe the issue

By default, the web crawler will download the target page using HttpClient, and if that fails for some reason the crawler will retry using the Chrome Selenium WebDriver. The problem is, the HttpClient does not setup to run any JS scripts and some sites are now lazy loading content via JS. The apnews.com website is a good example.

Resolution

Add an option to force the crawler to always use WebDriver on certain hostnames. The list of hostnames should be driven by configuration, either in the appSettings.json, the feeds.json, or in the RavenDB store.

Unexpected encoding in the og:url value

Operation Name:

Timestamp: 2021-11-05T14:27:24.5613800Z

Problem Id: System.UriFormatException

Assembly: RssFeeder.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

Exception Type: System.UriFormatException

Method: RssFeeder.Mvc.Handlers.GetFeedHandler+d__7.MoveNext

Outer Exception Type: System.UriFormatException

Outer Exception Message: Invalid URI: The format of the URI could not be determined.

Outer Exception Assembly: System.Private.Uri, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

Outer Exception Method: System.Uri.CreateThis

Innermost Exception Type:

Innermost Exception Message:

Country or region: United States

State or province: Virginia

City: Washington

Link: https://portal.azure.com/#blade/AppInsightsExtension/DetailsV2Blade/ComponentId/%7B%22SubscriptionId%22:%2249f15ff4-ec14-4ec7-b6e0-45254b71cdc6%22,%22ResourceGroup%22:%22Default-Web-EastUS%22,%22Name%22:%22RssFeederMvc%22%7D/DataModel/%7B%22eventId%22:%2285884012-3e44-11ec-a8f8-eb4fe7b4d769%22,%22timestamp%22:%222021-11-05T14:27:24.56138Z%22%7D

Exported URL missing protocol and hostname

Operation Name:

Timestamp: 2021-12-21T20:43:44.9371977Z

Problem Id: System.UriFormatException

Assembly: RssFeeder.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

Exception Type: System.UriFormatException

Method: RssFeeder.Mvc.Handlers.GetFeedHandler+d__7.MoveNext

Outer Exception Type: System.UriFormatException

Outer Exception Message: Invalid URI: The format of the URI could not be determined.

Outer Exception Assembly: System.Private.Uri, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

Outer Exception Method: System.Uri.CreateThis

Innermost Exception Type:

Innermost Exception Message:

Country or region: United States

State or province: Virginia

City: Washington

Link: https://portal.azure.com/#blade/AppInsightsExtension/DetailsV2Blade/ComponentId/%7B%22SubscriptionId%22:%2249f15ff4-ec14-4ec7-b6e0-45254b71cdc6%22,%22ResourceGroup%22:%22Default-Web-EastUS%22,%22Name%22:%22RssFeederMvc%22%7D/DataModel/%7B%22eventId%22:%22b2b6290f-629e-11ec-b964-bbcf8d469ceb%22,%22timestamp%22:%222021-12-21T20:43:44.9371977Z%22%7D

Issue with rssfeedermvc GET Rss/Get [id]

_MS.ProcessedByMetricExtractors: (Name:'Requests', Ver:'1.1')
Account Id:
Application version: 1.0.0.0
AspNetCoreEnvironment: Production
Auth user Id:
Browser version:
City: St. Petersburg
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A35AF
Cloud role name: rssfeedermvc
Country or region: United States
Device model:
Device type: PC
Event time: 3/10/2021, 10:57:00 AM
Operating system:
Operation Id: 7c5ab4e057e81849879041de0fb8ce39
Operation name: GET Rss/Get [id]
Parent Id: 7c5ab4e057e81849879041de0fb8ce39
Request Id: 02dac10359aceb4c
Request name: GET Rss/Get [id]
Request source:
Request URL: https://rssfeedermvc.azurewebsites.net/api/rss/liberty-daily
Response code: 500
Response time: 644.61 ms
Sample Rate: 100
SDK version: aspnet5c:2.17.0+c9d95e701e2474b7eb3b46ae7953b6c7570356ab
Session Id:
Source of synthetic traffic:
State or province: Florida
Successful request: false
Telemetry type: request
User Id: opT1R

View this instance in the Azure portal<\a>

Issue with rssfeedermvc GET Rss/Get [id]

_MS.ProcessedByMetricExtractors: (Name:'Requests', Ver:'1.1')
Account Id:
Application version: 1.0.0.0
AspNetCoreEnvironment: Production
Auth user Id:
Browser version:
City: Des Moines
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A77F8
Cloud role name: rssfeedermvc
Country or region: United States
Device model:
Device type: PC
Event time: 1/28/2021, 2:55:47 PM
Operating system:
Operation Id: e6ebc373960e41fa923ce6852163b291
Operation name: GET Rss/Get [id]
Parent Id: |e6ebc373960e41fa923ce6852163b291.0
Request Id: c5aaf0de853a2a46
Request name: GET Rss/Get [id]
Request source:
Request URL: https://rssfeedermvc.azurewebsites.net/api/rss/eagle-slant
Response code: 500
Response time: 1.85 s
Sample Rate: 100
SDK version: aspnet5c:2.16.0+4d6ef1104436a5df0cc0d3eafed1691b5ed56f79
Session Id: e6ebc373960e41fa923ce6852163b291
Source of synthetic traffic: Application Insights Availability Monitoring
State or province: Iowa
Successful request: false
Telemetry type: request
User Id: us-fl-mia-edge_e6ebc373960e41fa923ce6852163b291

View this instance in the Azure portal<\a>

Add site name meta data to description

Sometimes the article will list an og:site_name tag in the meta data. Add additional logic to place this meta data into the Description for the article.

If the site does not have a meta tag for the name, then parse the URL for the article and list the base domain name for site name. For example: nytimes.com

Issue with EXCEPTION System.ArgumentException

Account Id:
ActionId: 13eba08b-8010-48eb-b8c8-f5121000a7c1
ActionName: RssFeeder.Mvc.Controllers.RssController.Get (RssFeeder.Mvc)
Application version:
Auth user Id:
Browser version:
City: Washington
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A35AF
Cloud role name:
Country or region: United States
Device model:
Device type: PC
Elapsed: 405.5059
Event time: 6/8/2020, 8:02:20 PM
Exception type: System.ArgumentException
Failed method: RssFeeder.Mvc.Controllers.RssController+d__7.MoveNext
MessageTemplate: HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms
Operating system:
Operation Id: f73a48f40e0f274f92cfe0bc3de3c154
Operation name:
Parent Id: 46094c592972a54f
ParentId: 0000000000000000
Problem Id: System.ArgumentException at RssFeeder.Mvc.Controllers.RssController+d__7.MoveNext
RequestId: 80000e93-0003-d600-b63f-84710c7967bb
RequestMethod: GET
RequestPath: /api/rss/drudge-report
Sample Rate: 100
SDK version: dotnetc:2.11.0-21474
Session Id:
Severity level: Error
Source of synthetic traffic:
SourceContext: Serilog.AspNetCore.RequestLoggingMiddleware
SpanId: 46094c592972a54f
State or province: Virginia
StatusCode: 500
Telemetry type: exception
TraceId: f73a48f40e0f274f92cfe0bc3de3c154
User Id:

View this instance in the Azure portal<\a>
View similar exceptions in this time range<\a>

Issue with deleting documents with a defined partition key

System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: PartitionKey value must be supplied for this operation.
at Microsoft.Azure.Documents.Client.DocumentClient.d__326.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Documents.Client.DocumentClient.d__197.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.Documents.BackoffRetryUtility1.<ExecuteRetryAsync>d__5.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException) at Microsoft.Azure.Documents.BackoffRetryUtility1.d__5.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at RssFeeder.Console.Program.DeleteDocument(String databaseName, String collectionName, String documentID) at RssFeeder.Console.Program.BuildFeedLinks(RssFeed feed, String databaseName) at RssFeeder.Console.Program.ProcessWithExitCode(Options opts) ---> (Inner Exception #0) System.InvalidOperationException: PartitionKey value must be supplied for this operation. at Microsoft.Azure.Documents.Client.DocumentClient.<AddPartitionKeyInformationAsync>d__326.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Documents.Client.DocumentClient.<DeleteDocumentPrivateAsync>d__197.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Azure.Documents.BackoffRetryUtility1.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.Documents.ShouldRetryResult.ThrowIfDoneTrying(ExceptionDispatchInfo capturedException)
at Microsoft.Azure.Documents.BackoffRetryUtility`1.d__5.MoveNext()<---

Reduce file storage requirements

Proposal

Examine ways to reduce the file storage requirements in Seq, RavenDB and CosmosDB. Target areas include:

  • Reduce logging
  • Reduce file retention
  • Remove file attachments (not used in practice)
  • Purge documents after initial retention but keep only a bare minimum of metadata, probably the FeedAttributes section

Value statement

The first three are obvious benefits. The fourth has a benefit of keeping the original meta data around longer but without any of the bulk information such as parser results and Open Graph values. This will allow the crawler to avoid crawling old links (some of the sites the crawler hits keep links around for a very long time).

Add health checks

Need to stop using the drudge rss feed as the health check. While it is a good test, it's also skewing the metrics because we can't distinguish between monitor traffic and actual rss reader traffic.

The health check should include a call to the Cosmos DB to get some metrics. I'm thinking the article count for each feed and the last article added timestamp.

Issue with Value cannot be null. Parameter name: Name

{OriginalFormat}: An unhandled exception has occurred while executing the request.
Application version: 1.0.0.0
AspNetCoreEnvironment: Development
Assembly: RssFeeder.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
CategoryName: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware
Client IP address: 0.0.0.0
ConnectionId: 0HLPBV8RQD67V
DeveloperMode: true
Device type: PC
Event count: 1.00
Event time: 8/28/2019 4:56:50 PM
EventId: 1
EventName: UnhandledException
Exception type: System.ArgumentNullException
Failed method: RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
FormattedMessage: An unhandled exception has occurred while executing the request.
Operation ID: d24b309d6b0ca04dbf25fd1e3d5c2d55
Operation name: GET Rss/Get [id]
Operation parent ID: |d24b309d6b0ca04dbf25fd1e3d5c2d55.abd97a3b_
Outer exception assembly: Microsoft.SyndicationFeed.ReaderWriter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
Outer exception message: Value cannot be null.
Parameter name: Name
Outer exception method: Microsoft.SyndicationFeed.Atom.AtomFormatter.CreateContent
Outer exception type: System.ArgumentNullException
Problem ID: System.ArgumentNullException at RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
RequestId: 0HLPBV8RQD67V:00000002
RequestPath: /api/rss/drudge-report
Role instance: office-dell
SDK version: il:2.10.0-41092
Session ID: Pl4v9
Severity level: Error
User ID: LVaVP

Stack Trace
System.ArgumentNullException: Value cannot be null.
Parameter name: Name
at Microsoft.SyndicationFeed.Atom.AtomFormatter.CreateContent
at Microsoft.SyndicationFeed.Atom.AtomFormatter.CreateContent
at Microsoft.SyndicationFeed.Atom.AtomFormatter.Format
at Microsoft.SyndicationFeed.XmlFeedWriter.Write
at RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult at RssFeeder.Mvc.Controllers.RssController+<Get>d__5.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.TaskAwaiter1.GetResult
at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor+TaskOfIActionResultExecutor+d__0.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at System.Threading.Tasks.ValueTask`1.get_Result
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__12.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__10.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__13.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__23.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__18.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__16.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Builder.RouterMiddleware+d__4.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware+d__6.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware+d__7.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware+d__7.MoveNext

Issue with LastUpdated

{OriginalFormat}: An unhandled exception has occurred while executing the request.
Application version: 1.0.0.0
AspNetCoreEnvironment: Development
Assembly: RssFeeder.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
CategoryName: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware
Client IP address: 0.0.0.0
ConnectionId: 0HLPBV44G810O
DeveloperMode: true
Device type: PC
Event count: 1.00
Event time: 8/28/2019 4:48:54 PM
EventId: 1
EventName: UnhandledException
Exception type: System.ArgumentException
Failed method: RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
FormattedMessage: An unhandled exception has occurred while executing the request.
Operation ID: 2008a92cd3b45f408e3821da51b2ac15
Operation name: GET Rss/Get [id]
Operation parent ID: |2008a92cd3b45f408e3821da51b2ac15.e2a5f187_
Outer exception assembly: Microsoft.SyndicationFeed.ReaderWriter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
Outer exception message: LastUpdated
Outer exception method: Microsoft.SyndicationFeed.Atom.AtomFormatter.CreateContent
Outer exception type: System.ArgumentException
Problem ID: System.ArgumentException at RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
RequestId: 0HLPBV44G810O:00000002
RequestPath: /api/rss/drudge-report
Role instance: office-dell
SDK version: il:2.10.0-41092
Session ID: Pl4v9
Severity level: Error
User ID: LVaVP

Stack Trace
System.ArgumentException: LastUpdated
at Microsoft.SyndicationFeed.Atom.AtomFormatter.CreateContent
at Microsoft.SyndicationFeed.Atom.AtomFormatter.Format
at Microsoft.SyndicationFeed.XmlFeedWriter.Write
at RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult at RssFeeder.Mvc.Controllers.RssController+<Get>d__5.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.TaskAwaiter1.GetResult
at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor+TaskOfIActionResultExecutor+d__0.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at System.Threading.Tasks.ValueTask`1.get_Result
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__12.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__10.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__13.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__23.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__18.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__16.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Builder.RouterMiddleware+d__4.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware+d__6.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware+d__7.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware+d__7.MoveNext

Update to version 3 of the Azure Cosmos DB .NET SDK by 31 August 2024

Version 2.x of the Azure Cosmos DB .NET SDK will be retired.

Update to version 3 of the Azure Cosmos DB .NET SDK by 31 August 2024

You're receiving this email because you've used version 2.x of the Azure Cosmos DB .NET SDK.

Because version 3 of the Azure Cosmos DB .NET SDK includes updated features and improved performance, we'll retire version 2.x of this SDK on 31 August 2024. As a result, you'll need to update your SDK to version 3 by that date.

Version 3 includes:

  • Improved performance through stream API support.
  • Built-in support for change feed processor library and bulk operations.
  • Mockable APIs for easier unit testing.

Required action

To avoid service disruptions, migrate to Azure Cosmos DB .NET SDK version 3 by 31 August 2024.

Help and support

If you have questions, ask the community experts in Microsoft Q&A or post your them on our GitHub. If you have a support plan and require technical support, create a support request:

  1. For Summary, type Cosmos DB .NET SDK.
  2. For Issue type, select Technical.
  3. For Subscription, select your subscription.
  4. For Service, select My services, then select Cosmos DB.
  5. For Resource, select your resource.
  6. For Problem type, select SQL API SDK – NET.
  7. For Problem subtype, select Error or unexpected result.

Account Information

Subscription ID: redacted
Azure Cosmos DB account names: redacted
Azure Cosmos DB .NET SDK package name: DOCUMENTDB-NETCORE-SDK/2
Azure Cosmos DB .NET SDK versions: 2.12.0

Add support for HEAD request

We get a few sporadic HEAD requests to the API, which should return the headers of the requested resource but no body. The call should also load the cache if it has expired.

See this post for additional information.

Add article link to description

Articles that failed to download wind up with a blank description. For example:

{
  "id": "2192dfb7-398b-4a5f-8830-bc3433ac9050",
  "FeedId": 0,
  "Url": "https://nypost.com/2018/07/14/assange-could-soon-be-evicted-from-london-embassy/",
  "UrlHash": "3269eb32b8940060985ef409de7448aa",
  "ImageUrl": null,
  "Title": "Assange faces embassy eviction...",
  "Description": null,
  "DateAdded": "2018-07-15T08:01:24.5711968-04:00",
  "FileName": null,
  "SiteName": null
}

It would be helpful to include the link to the article in the Description field, so the user will know what site they are linking to.

Proposal: Use Route Templates for matching routes to article parser definitions

Proposal

Allow the site parser entries to specify a list of URL routes partial matches where the attempt to parse for article text is not skipped (or maybe parsed by the adaptive parser) rather than the specified parser.

Value statement

News sites will sometimes divide into different areas. For example, they could have a video only area, or a live feed area, or live stock quotes. The main difference with these areas is that they rarely contain any article text. If a matching URL is found, either do not attempt to parse for the article text, or run it through the adaptive parser for an attempt to normalize any format differences. Most likely we should just return the basic info and the user can click through to the site link for additional information. There is no article to read, so clicking through to the site is the best option.

Variations

We could support a few different variations on the route matching to try to cover as many use cases as possible.

{
    "SiteName": "cnbc",
    "ArticleSelector": "div[class^='ArticleBody-']",
    "ParagraphSelector": "p",
    "Parser": "generic-parser",
    "IgnoreRoutes": {
        "StartsWith": [ "quotes", "breaking" ],
        "Contains": [ "live" ],
        "EndsWith": []
    }
    "@metadata": {
        "@collection": "SiteArticleDefinition"
    }
}

UriFormatException when adding the RssFeedItem to the Syndication collection

Operation Name: GET Rss/Get [id]

Timestamp: 2021-11-01T21:21:20.3901578Z

Duration (ms): 809.9008

Country or region: United States

State or province: Florida

City: Lakeland

Link: https://portal.azure.com/#blade/AppInsightsExtension/DetailsV2Blade/ComponentId/%7B%22SubscriptionId%22:%2249f15ff4-ec14-4ec7-b6e0-45254b71cdc6%22,%22ResourceGroup%22:%22Default-Web-EastUS%22,%22Name%22:%22RssFeederMvc%22%7D/DataModel/%7B%22eventId%22:%22b9c9eff6-3b59-11ec-8bb5-fd4a5ccb0ada%22,%22timestamp%22:%222021-11-01T21:21:20.3901578Z%22%7D

Issue with rssfeedermvc GET /wp-content/plugins/wp-file-manager/lib/php/connector.minimal.php

_MS.ProcessedByMetricExtractors: (Name:'Requests', Ver:'1.1')
Account Id:
Application version: 1.0.0.0
AspNetCoreEnvironment: Production
Auth user Id:
Browser version:
City: Washington
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A89A5
Cloud role name: rssfeedermvc
Country or region: United States
Device model:
Device type: PC
Event time: 10/27/2020, 8:00:39 AM
Operating system:
Operation Id: 0d8704d6a542b34691687c9e3cce75b0
Operation name: GET /wp-content/plugins/wp-file-manager/lib/php/connector.minimal.php
Parent Id: 0d8704d6a542b34691687c9e3cce75b0
Request Id: 7da7217fc9eefc4a
Request name: GET /wp-content/plugins/wp-file-manager/lib/php/connector.minimal.php
Request source:
Request URL: https://rssfeedermvc.azurewebsites.net/wp-content/plugins/wp-file-manager/lib/php/connector.minimal.php
Response code: 404
Response time: 955.3 μs
Sample Rate: 100
SDK version: aspnet5c:2.14.0
Session Id:
Source of synthetic traffic:
State or province: Virginia
Successful request: false
Telemetry type: request
User Id:

View this instance in the Azure portal<\a>

The HTTP request to the remote WebDriver server for URL timed out after 60 sec

Message View Event

WebDriverUrlToDisk: '"b24bac4707bc52c6a4390c6afa3c4463"':'"https://t.co/8oAZwZxU2R"' - Unexpected error '"The HTTP request to the remote WebDriver server for URL http://localhost:50359/session/264f704684280bc44901a9b64273774d/url timed out after 60 seconds."'

Exception

OpenQA.Selenium.WebDriverException: The HTTP request to the remote WebDriver server for URL http://localhost:50359/session/264f704684280bc44901a9b64273774d/url timed out after 60 seconds.
 ---> System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 60 seconds elapsing.
 ---> System.TimeoutException: The operation was canceled.
 ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
 ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request..
 ---> System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at OpenQA.Selenium.Remote.HttpCommandExecutor.MakeHttpRequest(HttpRequestInfo requestInfo)
   at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute)
   --- End of inner exception stack trace ---
   at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute)
   at OpenQA.Selenium.Remote.DriverServiceCommandExecutor.Execute(Command commandToExecute)
   at OpenQA.Selenium.WebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
   at OpenQA.Selenium.WebDriver.set_Url(String value)
   at OpenQA.Selenium.Navigator.GoToUrl(String url)
   at RssFeeder.Console.Utility.WebUtils.WebDriverUrlToDisk(String url, String urlHash, String filename) in C:\Projects\RssFeeder\RssFeeder.Console\Utility\WebUtils.cs:line 119

Properties

* urlHash : "b24bac4707bc52c6a4390c6afa3c4463"
* url : "https://t.co/8oAZwZxU2R"
* message : "The HTTP request to the remote WebDriver server for URL http://localhost:50359/session/264f704684280bc44901a9b64273774d/url timed out after 60 seconds."
* collectionName : "bad-blue"

DOWNLOAD_ERROR: UrlHash '"12efde62fc592e6640cc3021bd2bcf37"':'"https://archive.fo/aOwrS"'

Message View Event

DOWNLOAD_ERROR: UrlHash '"12efde62fc592e6640cc3021bd2bcf37"':'"https://archive.fo/aOwrS"'

Exception

System.ArgumentNullException: Value cannot be null. (Parameter 'uriString')
   at System.Uri..ctor(String uriString)
   at RssFeeder.Console.WebCrawler.ParseAndSave(RssFeed feed, RssFeedItem item) in C:\Projects\RssFeeder\RssFeeder.Console\WebCrawler.cs:line 139
   at RssFeeder.Console.WebCrawler.DownloadList(RssFeed feed, String workingFolder, List`1 list) in C:\Projects\RssFeeder\RssFeeder.Console\WebCrawler.cs:line 199

Properties

* urlHash : "12efde62fc592e6640cc3021bd2bcf37"
* url : "https://archive.fo/aOwrS"
* collectionName : "revolver-news"

Issue with rssfeedermvc GET /api/rss/

MS.ProcessedByMetricExtractors: (Name:'Requests', Ver:'1.1')
Account Id:
Application version: 1.0.0.0
AspNetCoreEnvironment: Production
Auth user Id:
Browser version:
City:
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A4145
Cloud role name: rssfeedermvc
Country or region: United States
Device model:
Device type: PC
Event time: 8/30/2019, 1:55:16 PM
Operating system:
Operation Id: 76ddeef6c3310c46be36db817857d32b
Operation name: GET /api/rss/
Parent Id: 76ddeef6c3310c46be36db817857d32b
Request Id: |76ddeef6c3310c46be36db817857d32b.e3ef4d69
Request name: GET /api/rss/
Request source:
Request URL: https://rssfeedermvc.azurewebsites.net/api/rss/
Response code: 404
Response time: 215.9 μs
Sample Rate: 100
SDK version: aspnet5c:2.7.0
Session Id:
Source of synthetic traffic:
State or province: Washington
Successful request: false
Telemetry type: request
User Id:

View this instance in the Azure portal<\a>

Export step does not include all articles from the current session

Message View Event

Query: "from RssFeedItems 
                   where FeedAttributes.DateAdded >= '2022-01-04T18:00:04.0657768Z'
                   and FeedAttributes.FeedId = 'freedom-press'"

Properties

* query : "from RssFeedItems \r\n                   where FeedAttributes.DateAdded >= '2022-01-04T18:00:04.0657768Z'\r\n                   and FeedAttributes.FeedId = 'freedom-press'"
* collectionName : "freedom-press"

Add support for videos from alternate sites

As a matter of identification, it is easy enough to target the hostnames for YouTube or Rumble as being a home for streaming videos. But in certain cases, other hosts will add open graph tags that indicate the presence of a video. Here is one example:

    "OpenGraphAttributes": {
        "og:type": "article",
        "og:title": "{}",
        "og:url": "https://citizenfreepress.com/{article_route}/",
        "og:description": "{description}",
        "og:site_name": "CITIZEN FREE PRESS",
        "og:image": "http://img.youtube.com/vi/{hash}/0.jpg",
        "og:image:secure_url": "https://img.youtube.com/vi/{hash}/0.jpg",
        "og:image:width": "512",
        "og:image:height": "512",
        "og:locale": "en_US",
        "og:video:type": "text/html",
        "og:video": "http://www.youtube.com/embed/{hash}",
        "og:video:secure_url": "https://www.youtube.com/embed/{hash}"
    }

Again, easy enough because og:video attribute indicates a YT stream. But there are other examples configurations. This one is a direct link to an MP4 file (note the mime type):

    "OpenGraphAttributes": {
        "og:site_name": "Mail Online",
        "og:title": "{title}",
        "og:type": "article",
        "og:description": "{description}",
        "og:url": "https://www.dailymail.co.uk/{article_route}",
        "og:image": "https://i.dailymail.co.uk/1s/2021/10/29/16/49799865-0-image-a-{hash}.jpg",
        "og:image:width": "636",
        "og:image:height": "382",
        "og:video": "https://videos.dailymail.co.uk/video/mol/2021/10/21/{hash}/1024x576_MP4_{hash}.mp4",
        "og:video:type": "video/mp4",
        "og:video:width": "636",
        "og:video:height": "358"
    }

Any initial solution will need to account for at least the YT video, and maybe ignore the MP4 scenario as a first out. There may be other configurations I have not run across.

I would use the og:video:secure_url if it is specified, then fallback to og:video if not.

Stop forcing feed URLs to lowercase

During the UrlHash calculation, the URL is forced to lowercase to prevent any case sensitive changes from changing the hash calculation. So posting the same URL with different casing will still come up with the same hash value and will not trigger a crawl.

However, some sites (most notably YouTube) use case sensitive elements in either the route or the query parameters. So always converting the Url to lowercase is not going to result in a successful crawl for those sites.

Instead of always converting the Url to lowercase, just do it for the UrlHash calculation.

Issue with Author is required

{OriginalFormat}: An unhandled exception has occurred while executing the request.
Application version: 1.0.0.0
AspNetCoreEnvironment: Development
Assembly: RssFeeder.Mvc, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
CategoryName: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware
Client IP address: 0.0.0.0
ConnectionId: 0HLPBV57MUOK0
DeveloperMode: true
Device type: PC
Event count: 1.00
Event time: 8/28/2019 4:53:48 PM
EventId: 1
EventName: UnhandledException
Exception type: System.ArgumentException
Failed method: RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
FormattedMessage: An unhandled exception has occurred while executing the request.
Operation ID: 38a85840ed1eab45aced74b4244077cf
Operation name: GET Rss/Get [id]
Operation parent ID: |38a85840ed1eab45aced74b4244077cf.d1307d15_
Outer exception assembly: Microsoft.SyndicationFeed.ReaderWriter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
Outer exception message: Author is required
Outer exception method: Microsoft.SyndicationFeed.Atom.AtomFormatter.CreateContent
Outer exception type: System.ArgumentException
Problem ID: System.ArgumentException at RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
RequestId: 0HLPBV57MUOK0:00000001
RequestPath: /api/rss/drudge-report
Role instance: office-dell
SDK version: il:2.10.0-41092
Session ID: Pl4v9
Severity level: Error
User ID: LVaVP

Stack Trace
System.ArgumentException: Author is required
at Microsoft.SyndicationFeed.Atom.AtomFormatter.CreateContent
at Microsoft.SyndicationFeed.Atom.AtomFormatter.Format
at Microsoft.SyndicationFeed.XmlFeedWriter.Write
at RssFeeder.Mvc.Controllers.RssController+d__6.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at System.Runtime.CompilerServices.TaskAwaiter1.GetResult at RssFeeder.Mvc.Controllers.RssController+<Get>d__5.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.TaskAwaiter1.GetResult
at Microsoft.AspNetCore.Mvc.Internal.ActionMethodExecutor+TaskOfIActionResultExecutor+d__0.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at System.Threading.Tasks.ValueTask`1.get_Result
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__12.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__10.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next
at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker+d__13.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__23.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Rethrow
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker.Next
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__18.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Mvc.Internal.ResourceInvoker+d__16.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Builder.RouterMiddleware+d__4.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware+d__6.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware+d__7.MoveNext
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware+d__7.MoveNext

Issue with rssfeedermvc GET /sitemap.xml

_MS.ProcessedByMetricExtractors: (Name:'Requests', Ver:'1.1')
Account Id:
Application version: 1.0.0.0
AspNetCoreEnvironment: Production
Auth user Id:
Browser version:
City: Ashburn
Client IP address: 0.0.0.0
Cloud role instance: RD00155D8A493E
Cloud role name: rssfeedermvc
Country or region: United States
Device model:
Device type: PC
Event time: 2/23/2020, 10:15:58 AM
Operating system:
Operation Id: 278437dba44d7044a56c83dc5c6915d8
Operation name: GET /sitemap.xml
Parent Id: 278437dba44d7044a56c83dc5c6915d8
Request Id: 4a9a6a2b49cde648
Request name: GET /sitemap.xml
Request source:
Request URL: https://rssfeedermvc.azurewebsites.net/sitemap.xml
Response code: 404
Response time: 1.01 ms
Sample Rate: 100
SDK version: aspnet5c:2.8.2
Session Id:
Source of synthetic traffic:
State or province: Virginia
Successful request: false
Telemetry type: request
User Id:

View this instance in the Azure portal<\a>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.