neuecc/etwstream

Logs are event streams. EtwStream provides In-Process and Out-of-Process ObservableEventListener. Everything can compose and output to anywhere by Reactive Extensions.

I'm Yoshifumi Kawai, CEO/CTO of Cysharp, Inc.

I have published a lot of C# OSS.
Currently, I continue to publish OSS at github.com/Cysharp, so please look there for the latest production OSS.
I strive to provide state-of-the-art libraries that are always mindful of performance and modern APIs.
As the original author of MessagePack for C#, I also continue to maintain it in the MessagePack-CSharp Organization.

If you want to support me, please see GitHub Sponsors

This is currently my primary focus for C# OSS.

Serializer

• MessagePack-CSharp
• MemoryPack

Framework

• MagicOnion
• ConsoleAppFramework

For GameEngine

• R3
• UniTask

Toolkit

• ZLogger
• MasterMemory

etwstream's Issues

Separate In-Process Logger

Remote ETW Logging

ETW does not support listen from remote machine.

In remote machine add agent(Out-of-process EtwStreamService or In-Process)
.LogToRemote()

In local machine listen serialized event.
.FromRemote()

Technology

Not connect directly, I choice Redis Pub/Sub for remote relay server.
http://redis.io/topics/pubsub

How do you use this with powershell?

Hello,

Can you provide additional samples how you can use this library inside powershell to consume events.

Support self-describing event on In-Proces EventSource

sorry, in-process support is very poor currently.

Write Directory if path does not exsits at FileSink

Add ObservableEventListener.FromRegisteredTraceEvent

use TraceEventSource.Registered

FromEventSource overload not passing all arguments

I noticed this as I was getting log events I did not want:

public static IObservable<EventWrittenEventArgs> FromEventSource(string eventSourceName, EventLevel level = EventLevel.Verbose, EventKeywords matchAnyKeyword = EventKeywords.None, IDictionary<string, string> arguments = null)
        {
            if (eventSourceName == null) throw new ArgumentNullException("eventSourceName");

            foreach (var item in EventSource.GetSources())
            {
                if (item.Name.Equals(eventSourceName, StringComparison.Ordinal))
                {
                    return FromEventSource(item, level);    <== should pass matchAnyKeyword  and arguments
                }
            }
            . . . 
        }

Check traceEvent manifest cache

If schema updated, renew the cache?

Getting error about mismatched version of System.Reactive when running console application

PS C:\Users\Administrator> C:\Users\Administrator\Desktop\EtwStream.Service_1.3.3\EtwStream.Service.exe
Configuration Result:
[Success] Name EtwStream.Service
[Success] DisplayName EtwStream Service
[Success] Description EtwStream Out-of-Process Service, trace event and output to anywhere.
[Success] ServiceName EtwStream.Service
Topshelf v3.3.154.0, .NET Framework v4.0.30319.42000
Topshelf.Hosts.ConsoleRunHost Error: 0 : An exception occurred, System.AggregateException: One or more errors occurred.
---> System.TypeInitializationException: The type initializer for 'System.Reactive.Linq.Observable' threw an exception.
---> System.TypeInitializationException: The type initializer for 'System.Reactive.Linq.QueryServices' threw an exceptio
n. ---> System.TypeInitializationException: The type initializer for 'System.Reactive.PlatformServices.PlatformEnlighten
mentProvider' threw an exception. ---> System.IO.FileLoadException: Could not load file or assembly 'System.Reactive.Pla
tformServices, Version=3.0.0.0, Culture=neutral, PublicKeyToken=94bc3704cddfc263' or one of its dependencies. The locate
d assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
   at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOn
ly, StackCrawlMarkHandle stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName, ObjectHandleOnStack type)
   at System.RuntimeTypeHandle.GetTypeByName(String name, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOn
ly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean loadTypeFromPartialName)
   at System.RuntimeType.GetType(String typeName, Boolean throwOnError, Boolean ignoreCase, Boolean reflectionOnly, Stac
kCrawlMark& stackMark)
   at System.Type.GetType(String typeName, Boolean throwOnError)
   at System.Reactive.PlatformServices.PlatformEnlightenmentProvider.CreatePlatformProvider()
   at System.Reactive.PlatformServices.PlatformEnlightenmentProvider..cctor()
   --- End of inner exception stack trace ---
   at System.Reactive.PlatformServices.PlatformEnlightenmentProvider.get_Current()

Write ReadMe, How to use sinks

Out-of-Process Service with Roslyn C# Scripting

xml/json configuration(such as SLAB's SemanticLogging-svc.xml) is legacy style.
It can’t use Rx's power of ObservableListener.
I choice csx - Roslyn C# Scripting as config and running environment with https://www.nuget.org/packages/Microsoft.CodeAnalysis.Scripting/

proposal of API Design

// configuration.csx
var subscription = ObservableEventListener.FromTraceEvent("xxx")
// .GroupBy() // grouping by Rx
// .Where() // filter by Rx
.LogToFile(/* option */);

// return and start of scripting.
EtwStreamService.CompleteConfiguration(subscription);

Migrate to System.Reactive

Hi,

Have been playing around with EtwStream and it is looking really good. Much easier to use than some other alternatives. It would be good to migrate from Rx-Main to the System.Reactive.* nuget packages if possible.

Cheers,

Isaac

BigQuery Sink

use new google SDK
https://github.com/GoogleCloudPlatform/gcloud-dotnet

Expose PID for the trace events

Given:

ObservableEventListener.FromTraceEvent(WellKnownEventSources.TplEventSource)
.DumpWithColor(withProcessName: true, withProviderName: true);

Outputs:

So if I have multiple instances of the same process name running, I will have to use the overload (Func<TraceEvent, String>) to get the PID. It will be more convenient if PID was added as its own column just before the ProcesName.