3f/conari

🧬 One-touch unmanaged memory, runtime dynamic use of the unmanaged native C/C++ in .NET world, related P/Invoke features, and …

	More ...

*update every hour

[ ☕ ]

conari's Issues

Support of .NET libraries

It probably should be useful features for work without domains and their common problems with unloading assemblies.

So main idea (noticed here 3F/DllExport#9) to provide unified work with managed and unmanaged code via lightweight bridge.

using(var l = new ConariL("managed_net_lib.dll")) {
    // any convenient work via Bridge & dynamic API layer
}
// fully unloaded !

Basic example is a https://github.com/3F/vsSolutionBuildEvent

where Provider in the role of the binder between API & main core:

But do not forget about ~ access security and partially-trusted code, etc.

I don't think about fully compatible way, but the any useful bridge, like in example above, this is what we need...

Failed loading '...': Check used architecture or existence of file.[Error: 193]

Failed loading '...': Check used architecture or existence of file.[Error: 193]

Details

Error 193 (0xC1)

ERROR_BAD_EXE_FORMAT

    193 (0xC1)
    %1 is not a valid Win32 application.

How to fix

Probably you need to check Target Platform of your project.

The Any CPU (?) will give a 64bit pointers for x64 system and 32bit pointers for x32 system (except for anycpu32bitpreferred when your application runs in 32-bit mode on systems that support both 64-bit and 32-bit).

So how about to check library, that you want to load ?

The Conari supports x32 & x64 architecture, leaving the choice to you. Thus, to select compatible for library that's should be loaded for your project:

• Properties - Build - Platform target:

LoadLibrary/Ex vs Conari. Same memory regions between processes and threads

I had the opportunity to think briefly about this official OS "feature" in the context of the Conari project. Because we must understand this is not really a bug. But it can produce the bugs in user space.

Problems such as #12 (comment) are possible not only in multithreading but even between third processes.

Because Windows will prevent new loading and return the same handle as for the first loaded module due to used reference count for each trying to load the same module (dll or exe).

However, actual new loading and its new handle is possible when reference count is less than 1. Through Conari this means each decrementing when disposing is processed on implemented ConariL object.

That is, each new instance will increase total reference count by +1 and each disposing will decrease it by -1. Just think about this as for the new/delete rule in C++. WinAPI's Load/Free functions follows this rule too.

And this is what I'm talking about. Conari will not fix this "feature" but it will provide at least some option to configure this behavior.

I just think this is a bad idea to isolate this for all cases.

Anyway, I've already tested draft of the new feature for that "feature". Thus, it was already planned together with 1.4 release. Next week. Comment if you have questions.

Work with Native C/C++ structures without declaration

I want a more flexible using of native C/C++ complex types (like a struct), without additional declarations.
it would be nice to avoid some re-declaration of the same equivalents from C++ to C#

we can also skip some declaration if this required only inside unmanaged code, i.e.:

IntPtr codec; // we will store pointer to AVCodec struct
IntPtr context = IntPtr.Zero; // we will store pointer to AVCodecContext struct

l.bind<Action>("avcodec_register_all")();
codec   = l.bind<Func<int, IntPtr>>("avcodec_find_encoder")(AV_CODEC_ID_MP3);
context = l.bind<Func<IntPtr, IntPtr>>("avcodec_alloc_context3")(codec); // pass allocated AVCodec* to avcodec_alloc_context3
...

but if we need to work with the context above, we also should declare this type (see AVCodecContext in avcodec.h) and finally marshal it:

AVCodecContext context = (AVCodecContext)Marshal.PtrToStructure(ptr, typeof(AVCodecContext));

even if it all will be encapsulated by layer of upper level... just not so cool :)

However, we cannot provide this automatically, because final data does not have any markers of data etc.
Just byte-sequence, because the main idea it's headers, for example:

struct Spec
{
    int a;
    int b;
    Spec2* m;
};

~0x0572b018
-----------
[0]  | 0x05     <<< 4 bytes of integer type, value is 5 from a of struct Spec
[1]  | 0x00
[2]  | 0x00
[3]  | 0x00   ^
[4]  | 0x07     <<< 4 bytes of integer type, value is 7 from b of struct Spec
[5]  | 0x00
[6]  | 0x00
[7]  | 0x00   ^
[8]  | 0xd8     <<< 4 bytes of integer type, value is a pointer to struct Spec2
[9]  | 0xb1
[10] | 0x72
[11] | 0x05   ^
[12] | 0xfd
[13] | 0xfd
[14] | 0xfd
[15] | 0xfd
...

so, how about to define it automatically by size of types ? We can't detect what types are present in this sequences, but for work with complex native types, we can simply like a:

var c = get(context, int, int, long) as AVCodecContext;

c["sample_rate"]  = 44100;
c["channels"]     = 2;
c["bit_rate"]     = 64000;
...
ret = l.bind<FuncRef4<AVCodecContext, AVPacket, AVFrame, int, int>>("avcodec_encode_audio2")(c, pkt, frame, ref output);

the order-sensitive is similar for Marshal.PtrToStructure - not so good for both :)

need to think...

Failed loading '...': Check used architecture or existence of file.[Error: 126]

Describe the bug
I have a net5 wpf x32 program that references a dynamic link library where a function opens a dialog box with the current program as its owner, the first call without any problems, but when the dynamic library is re-instantiated later and the call throws an exception like the title
What code is involved

Expected behavior

Screenshots

Unmanaged EmitCalli and netstandard2.0

In #12 I already mentioned about the main problem for netstandard2.0. And this is what I'm talking about:

An unmanaged EmitCalli is available only with netcoreapp2.1+ https://github.com/dotnet/corefx/issues/9800

But System.Reflection.Emit.ILGeneration is just metadata while mscorlib/src/System/Reflection/Emit/DynamicILGenerator implements this since dotnet/coreclr#16546

See System.Private.CoreLib.dll

More probably we can try inject this logic at runtime o_O Or does exist something related? Cecil is extra cost for just single EmitCalli <_<

logo

just noticed :) maybe this variant will be better:

continued thoughts ... like:

sketch ==>

nyaaaa :)

anyway, not important of course, just drawing a some logo
and yes, need to redraw ugly strong borders of image and very dark spiral

Support of exported variables / constants

3F/DllExport#24

Something like:

LIBAPI DWORD  ADDR_SPEC = 0x00001CE8;

l.RVA.variables<UInt32>("ADDR_SPEC")
l.RVAvar.ADDR_SPEC
...

anyway, do not use l.DLR node

Does it support for .net5 sdk?

File Not Found

my code is just

MessageBox.Show(typeof(ConariL).FullName);

.Net Target: 4.5

ExVar.getVar returning wrong value

Hello,

I have a C# function that is getting invoked from a C++ executable. In this function, I am attempting to do the literal equivalent of

int number;
std::cin >> number;

Obviously I could do something like number = int.Parse(Console.ReadLine()), but in this case I really need to do exactly what C++ is doing (and besides, what fun would that be?). Leveraging a decompiler to get the exact mangled names, I came up with this code:

            int number= 0;
            using (var l = new ConariL(@"c:\windows\system32\MSVCP140.dll"))
            {                
                var cin = l.ExVar.getVar<IntPtr>("?cin@std@@3V?$basic_istream@DU?$char_traits@D@std@@@1@A");
                
                l.bind<FuncOut2<IntPtr, int, IntPtr>>("??5?$basic_istream@DU?$char_traits@D@std@@@std@@QEAAAEAV01@AEAH@Z").Invoke(cin, out number);
            }

The function binding works correctly, but it throws an access violation inside of the function because the address of cin is incorrect. The intptr returned by getVar is 0x00007ffe2a0cc110, which the Visual Studio debugger tells me is the address of std::basic_istream<char,std::char_traits<char>>:: vbtable', not of std::basic_istream<char,std::char_traits<char>> std::cin. Indeed, if I put std::cout << &(std::cin); in the C++ app, it outputs 00007FFE2A0F7200, which Visual Studio agrees is correct.

If I force 00007FFE2A0F7200 into my cin variable instead of getting it with getVar, the function executes correctly, prompting for user input and storing it in number.

The type or namespace name 'DotNet' does not exist

error CS0234: The type or namespace name 'DotNet' does not exist in the namespace 'net.r_eg' (are you missing an assembly reference?)

using net.r_eg.DotNet.System.Reflection.Emit
namespace not found. some module is missing from the Conari-master package

Speed

For planned v1.3 I implemented new caching of the binding in core provider

test via snet tool (part of regXwild)

x32 library:

Unicode: true
iterations(10000) x average(4)

.... regXwild via Conari (Lambda) - ESS version: ~61ms
.... regXwild via Conari (DLR)- ESS version: ~222ms
.... regXwild via Conari (Lambda) - EXT version: ~58ms
.... regXwild via Conari (DLR) - EXT version: ~218ms

....

x64 library:

.NET version via Conari engine

Unicode: true
iterations(10000) x average(4)

.... regXwild via Conari (Lambda) - ESS version: ~43ms
.... regXwild via Conari (DLR)- ESS version: ~245ms
.... regXwild via Conari (Lambda) - EXT version: ~37ms
.... regXwild via Conari (DLR) - EXT version: ~237ms

#####

The regex engine is much more slower (~30-90sec for 10000 iterations), please wait...


Compiled: true
iterations(10000) x average(4)

.... .NET Regex engine: ~37611ms
.... .NET Regex engine(only as ^match$ like a simple '=='): ~2ms

-------

Compiled: false
iterations(10000) x average(4)

.... .NET Regex engine: ~31034ms
.... .NET Regex engine(only as ^match$ like a simple '=='): < 1ms

Done.

looks good !

old result:

( <= v1.2)	+icase [x32]	+icase [x64]
regXwild via Conari (Lambda) - ESS	~1032ms	~1418ms
regXwild via Conari (DLR) - ESS	~1238ms	~1609ms
regXwild via Conari (Lambda) - EXT	~1117ms	~1457ms
regXwild via Conari (DLR) - EXT	~1246ms	~1601ms

but I'm not sure about that binding to specific values are correct for all cases. I added new unit-tests and this also looks good, so ...

Currently I implemented this via 0x29 opcode (Calli), and the all arguments should be pushed onto the stack before each calling.
Thus the cached TDyn should be valid for custom values via std. stack

for(int i = 0; i < mParams.Length; ++i) {
    il.Emit(OpCodes.Ldarg, i);
}

Use IProvider.Cache if not, and report about this here.

How to properly use the "NativeStruct" from current Conari Version?

In your video: https://www.youtube.com/watch?v=QXMj9-8XJnY

you used the obsolete-way with UnmanagedStructure, now you suggest using the NativeStruct way, can u maybe give me a short example of how to make use of those properly, to be able to call it from C++/Delphi?

Because, when i compile the following code:

I get the error you can see in the "Error-dialogbox" below in the screenshot.

How to adapt to that new change you have done for Conari?

Much thanks!

The BreakPionts Not Work

Help me!The BreakPionts Not Work!

This my code:
using (var conariL = new ConariL(@".\x64\libcurl-x64.dll"))
{
var curl = conariL.DLR.curl_easy_init();
conariL.DLR.curl_easy_setopt(curl, 10002, "http://www.baidu.com");
var res = conariL.DLR.curl_easy_perform(curl);
}

Aliases for exported-functions and variables

hmm, just thought that is a good additional variant for: Undecorate functions from C++ compilers #3

Moreover, it will be useful for any other cases like for DllImport

[PrefixL]procName -> [PrefixR]alias1
                  -> [PrefixR]alias2

where PrefixL/R it's a main IProvider.Prefix if used
etc.

TODO

Incorrect boolean values from unmanaged code

found problem from https://github.com/3F/regXwild

REGXWILD_API bool searchEss(const tstring& data, const tstring& filter, bool ignoreCase)
{
    core::ESS::AlgorithmEss alg;

    return alg.search(data, filter, ignoreCase);
}

always returns true:

l.DLR.searchEss<bool>(data, filter, false);

seems need to help understand that we want get 1byte of integer type -_-

I will look it tomorrow

IUnknown. COM Interface support

Currently all works fine, but I need more flexible things like here #2

How we can play with current versions:

// the working half-declaration of original interface:
[Guid("23170F69-40C1-278A-0000-000600600000")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IInArchive
{
      void a(); void b();
 
0x02: uint GetNumberOfArchiveProperties();                             // <-- uint GetNumberOfItems();
                                                                            |
      void c(); void d(); void e(); void f(); void g();                     |
                                                                            |
0x08: uint GetNumberOfItems(); //uint GetNumberOfArchiveProperties();  // <-/  moreover, it can be like MySPecial001() for calling from our side.
}
...

IInArchive ar;
// DLR for STDAPI CreateObject(const GUID *clsid, const GUID *iid, void **outObject)
l.DLR.CreateObject<int>(ref cid, ref iid, out ar); 
ar.GetNumberOfItems() <--> GetNumberOfArchiveProperties()  // ok

How about of Inheritance:

In COM it does not mean code reuse, it means only that contract associated with an interface is inherited.

The coreclr could have better support for all IUnknown interfaces like a COW. I mean we already have contract for all this, why not to wrap this via optional allocation of new required records and remap pointers for new slot. But anyway, it requires any modifications of this, so... my time -_-

[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
[Guid("23170F69-40C1-278A-0000-000300010000")]
public interface ISequentialInStream
{
    void Read(...);
}
 
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
[Guid("23170F69-40C1-278A-0000-000300030000")]
public interface IInStream: ISequentialInStream
{
    void m(...);    // only as offset of position like above. The Read() from base we still can't use
    void Seek(...); // ok 
}

just to think

Undecorate functions from C++ compilers

the develop branch is already contains DLR features, but it's not possible for exported functions from C++ compiler.

i.e.:

#define LIBAPI_CPP  __declspec(dllexport) // will decorated like: ?getSeven@API@UnLib@Conari@r_eg@net@@YAGXZ
                                          // e.g.: unsigned short net::r_eg::Conari::UnLib::API::getSeven(void)
#define LIBAPI  extern "C" __declspec(dllexport) // will undecorated like from C compiler: getSeven()

thus we can't use this like l->?getSeven@API@UnLib@Conari@r_eg@net@@YAGXZ() even if it's possible, it is... omg :)

Currently, we can also use a standard bind methods, like:

l.bind<Func<ushort>>("?getD_Seven@API@UnLib@Conari@r_eg@net@@YAGXZ")();

// or

l.bind(Dynamic.GetMethodInfo(typeof(ushort)), "?getD_Seven@API@UnLib@Conari@r_eg@net@@YAGXZ")
                 .dynamic
                 .Invoke(null, new object[0]);

but it still not useful at all :)

For C++ compiler we can simply tell: export it, as it would be for C compiler i.e.

• extern "C" __declspec(dllexport)

but of course, this should be from third party code.. so, what to do:

The main problem: the same functions cannot be same from different compilers :(

TODO: just to find already existing solution for mangling names

Then we should provide support for example:

If user wants to call this: net::r_eg::Conari::UnLib::API::getSeven(void)

l.Mangling  = true; // switch of decoration
l.Prefix    = "net::r_eg::Conari::UnLib::API::"; //for all other functions

ushort val  = dll.getSeven<ushort>();  // to call unsigned short getD_Seven(void)
// before calling we should get final name: ?getD_Seven@API@UnLib@Conari@r_eg@net@@YAGXZ
// then simply call

Decoration for a C functions (and C++ if used C linkage):

• MSDN. Format of a C decorated name

In a 64-bit environment, functions are not decorated.

-	-
__cdecl	Underscore character (_) is prefixed to names, except when __cdecl functions that use C linkage are exported.
__stdcall	Leading underscore (_) and a trailing at sign (@) followed by the number of bytes in the parameter list in decimal
__fastcall	Leading and trailing at signs (@) followed by a decimal number representing the number of bytes in the parameter list
__vectorcall	Two trailing at signs (@@) followed by a decimal number of bytes in the parameter list

Support [Guide to add DllExport attribute to my project]

I can't add the DllExport attribute to my project, Could you please provide a guide to achieve this?
Thanks a lot.