How to use google/protobuf/any.proto

[timostamm]

How to use google/protobuf/any.proto

[SyedAsimAliSE]

it seems it need path normalization , i already tried to replace paths, but it says module not found .

import { Any } from "./google/protobuf/any";

Managed to compile it with :

import { Any } from "google-protobuf/google/protobuf/any_pb.js";

but don't know if it works or not because i am unable to go past this :

i am trying to migrate from grpc-js to protobuf-ts in my NestJS project, using nodejs 14.15.1 with tsconfigs set to ES2020.

after reading the documentation installed Twirp transport

and added the following call to the server.

                const transport = new TwirpFetchTransport({
			baseUrl: "0.0.0.0:50052"
		});

		const client = new ProcessTypeWritesServiceClient(transport);

		const payload:CreateProcessTypeInput_GRPC = {
			processTypeStatus:input.processTypeStatus
		}

		const {response} = await client.createProcessTypeGRPC(payload);
		console.log("Response! " + response)

but getting :

TypeError: globalThis.Headers is not a constructor at TwirpFetchTransport.unary 

thanks

[timostamm]

But when i am running the generator it is generating the import like :

import { Any } from "./google\\protobuf\\any";

Fixed in v1.0.11

[timostamm]

That looks interesting :)

Do you happen to be on Windows?

[SyedAsimAliSE]

yep i am using windows 10

[timostamm]

Ok. I'm pretty sure that this "importPath" has windows path separators (\) that have to be swapped.

protobuf-ts/packages/plugin-framework/src/typescript-import-manager.ts

Lines 79 to 82 in 7d0855f

	const importPath = createRelativeImportPath(
	this.source.getSourceFile().fileName,
	symbolReg.file.getFilename()
	);

But I have to setup a test env to confirm and fix. You'll have to wait a few days...

In the meantime, I can think of two workarounds:

1. run protoc in the linux subsystem (WSL)
2. replace paths in generated code with search and replace

[timostamm]

import { Any } from "./google/protobuf/any";

That should work. I guess the file is not there?

Use protoc argument '--ts_opt generate_dependencies':

• "generate_dependencies"
  By default, only the PROTO_FILES passed as input to protoc are generated,
  not the files they import. Set this option to generate code for dependencies
  too.

Are you running the twirp transport in nodejs?

This transport requires the fetch API. globalThis.fetch, globalThis.Headers and globalThis.AbortController are expected. The transport is tested with the polyfill packages node-fetch and abort-controller.

[SyedAsimAliSE]

You are right i missed the flag --ts_opt generate_dependencies i got the any file.

yes i am running it in nodejs, also installed node-fetch and abort-controller both of them but i have no idea what to do next with them :D because it still says TypeError: globalThis.Headers is not a constructor at TwirpFetchTransport.unary

[timostamm]

Have a look here:

protobuf-ts/packages/twirp-transport/clientcompat/client.ts

Lines 12 to 16 in 7d0855f

	import {default as fetch, Headers} from "node-fetch";
	// fetch polyfill via https://github.com/node-fetch/node-fetch
	globalThis.fetch = fetch;
	globalThis.Headers = Headers;

And here:

protobuf-ts/packages/test-generated/spec/generated-client-style-call.spec.ts

Lines 3 to 6 in 7d0855f

	import {AbortController} from "abort-controller";
	import {Int32Value, StringValue} from "../ts-out/google/protobuf/wrappers";
	globalThis.AbortController = AbortController; // AbortController polyfill via https://github.com/mysticatea/abort-controller

[SyedAsimAliSE]

i tried everything but it seems to fail on me every time. if i don't set Headers then it is always giving :

TypeError: globalThis.Headers is not a constructor

if uncomment it ,getting the following type errors. did google for it allot but i guess i am the only one getting these :(

error TS2322: Type 'typeof Headers' is not assignable to type '{ new (init?: HeadersInit): Headers; prototype: Headers; }'.
  The types of 'prototype.forEach' are incompatible between these types.
    Type '(callback: (value: string, name: string) => void) => void' is not assignable to type '(callbackfn: (value: string, key: string, parent: Headers) => void, thisArg?: any) => void'.
      Types of parameters 'callback' and 'callbackfn' are incompatible.

37 globalThis.Headers = Headers;
   ~~~~~~~~~~~~~~~~~~

is there any other way i can send calls to the server ?

[timostamm]

The code snippets in the comments above are part of the CI workflow.
They run on every single commit to this repository, in node js.
They also run every single time I make a release.

Take a step back and get Headers working as expected, see examples here: https://developer.mozilla.org/en-US/docs/Web/API/Headers

The fetch API is nice and node-fetch is an excellent implementation.
Try making some simple requests, then add some custom headers.

Once you have that working, the TypeError is either gone - or you should be confident enough to just do globalThis.Headers = Headers as unknown as any; This will suppress the TypeError. But if you have some import error or broken node-fetch version, it will not help you at all. So get fetch and Headers running first.

[SyedAsimAliSE]

found out that the problem is not with the node-fetch lib.

The globalThis.Headers = Headers; is taking me to the nestjs Header const declaration

export declare const Headers: (property?: string) => ParameterDecorator;

and the error is always taking me to this

globalThis.fetch(url, {
            --
            headers: createTwirpRequestHeader(new globalThis.Headers(), !!opt.sendJson, opt.meta),
            --
            --
        })

now need to find out how do i switch this global for the protobuf-ts.

[SyedAsimAliSE]

Alright the problem with globalThis.Headers = Headers; is really not a problem but Intellij IDE isn't picking up the TS Typings, the project is working in VSCode perfectly fine.

[timostamm]

I'm using jetbrains IDEs as well. Some times the TypeScript service shows invalid errors. Helps to restart it in this case.

But your issue might have been something else. Glad you solved it.

[SyedAsimAliSE]

Thank you for taking your time :)

For anyone encountering the globalThis.Headers = Headers; problem in Intellij IDE, can solve it like this:

[hamming]

// I am using Any type in one of my proto file 

import "google/protobuf/any.proto";

But when i am running the generator it is generating the import like :

import { Any } from "./google\\protobuf\\any";

1. npm install node-fetch

2. npm install abort-controller

3. and --ts_opt generate_dependencies in your cmd