opiproject/opi-api

Open Programmable Infrastructure API and Behavioral Model

opi-api's Issues

Is `OUTPUT_ONLY` a correct annotation for `name` field ?

          Is `OUTPUT_ONLY` a correct annotation here?

This is a hard one... According to the annotation description:

including the field in a message in a request does nothing (the server must clear out any value in this field and must not throw an error as a result of the presence of a value in this field on input). Similarly, services must ignore the presence of output only fields in update field masks.

Based on that, a user cannot specify a name, since the server must ignore it. But this resource is used in Update call and name is used to find that resource internally...

This might be a resolution: 2 weeks ago a PR from an aip team member was created where it is said that name field must not be annotated.

Originally posted by @artek-koltun in #317 (comment)

code generator protoc-gen-grpc-gateway hasn't been updated to support optional fields in proto3

see grpc-ecosystem/grpc-gateway#1278
see namely/docker-protoc#361

An exception occurs when I use namely/protoc-all:1.51_2 with the parameter: --with-gateway to convert a proto file to a REST service:

... is a proto3 file that contains optional fields, but code generator protoc-gen-grpc-gateway hasn't been updated to support optional fields in proto3. Please ask the owner of this code generator to support proto3 optional.--grpc-gateway_out:
make: *** [Makefile:9: all] Error 1

Would like to update the protoc version in the namely/protoc-all mirror to support this.

front end additions from PR #169

I suggest if you can please create new PR @mkalderon with both additions:

• split CQ and SQ
• add max_queue_depth

Originally posted by @glimchb in #169 (comment)

add or remove auto-gen files

see #139 vs #141

fix `google.api.http` URLs with proper hierarchy

see #186 (comment)
see #186 (comment)

we might want to prefix the path with nvme, virtio, etc. i.e. instead of implicit assumptions we can keep the nvme path to /v1/nvme/subsystems, /v1/nvme/controllers, etc.
or i.e. /v1/nvme/frontend/subsystems or /v1/nvme/fe/subsystems vs. /v1/nvme/subsystems

also relationship between subsystems and controllers something like: post: "/v1/subsystems/{subsystem}/controllers"

[storage] : Change APIs according to `Resource-oriented design` https://google.aip.dev/121

After #171 is done we need a separate PR to fix all storage according to https://google.aip.dev/121

• https://google.aip.dev/131 - Get
• https://google.aip.dev/132 - List
• https://google.aip.dev/133 - Create
• https://google.aip.dev/134 - Update
• https://google.aip.dev/135 - Delete

and https://google.aip.dev/127 - HTTP and gRPC Transcoding

Network: Expand LAN Service specifics in Network Readme

consider `VolumeStats` to become `int64` instead of `int32`

see

opi-api/storage/v1alpha1/opicommon.proto

Line 41 in ee306f8

int32 read_bytes_count = 1;

comment from @llabordehpe on #220

security: please separate reference implementation VS generic APIs

looking at this section of the readme file : https://github.com/opiproject/opi-api/blob/main/security/README.md#architecture

I do think we need to separate reference implementation VS generic APIs...
vici on this page is just an example...
everybody else are free to use their own implementation instead of strongswan or vici...
see the picture from https://github.com/opiproject/opi-poc/blob/main/storage/OPI-storage-SPDK-bridge.png

also storage in this repo doesn't talk about specific implementations storage readme

Originally posted by @glimchb in #102 (comment)

storage: move from `ID` to `UUID` everywhere in proto

also return UUID where // Intentionally empty. is

[storage] fix `string stats`

All objects have a string object for stats
let's fix that

ci: enable linter for proto

in make we can use

docker run -v $PWD:/defs namely/protoc-all --lint -d proto -l go -o ./proto/  --go-source-relative

that produces

frontend.proto:43:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:44:12: 'NQN' - Use underscore_separated_names for field names.
frontend.proto:45:12: 'SerialNumber' - Use underscore_separated_names for field names.
frontend.proto:46:12: 'ModelNumber' - Use underscore_separated_names for field names.
frontend.proto:47:11: 'MaxNs' - Use underscore_separated_names for field names.
frontend.proto:51:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:52:12: 'Name' - Use underscore_separated_names for field names.
frontend.proto:53:12: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:54:12: 'PCIeID' - Use underscore_separated_names for field names.
frontend.proto:55:11: 'MaxIOQPs' - Use underscore_separated_names for field names.
frontend.proto:56:11: 'MaxNs' - Use underscore_separated_names for field names.
frontend.proto:60:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:61:12: 'Name' - Use underscore_separated_names for field names.
frontend.proto:62:12: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:63:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:64:11: 'NsId' - Use underscore_separated_names for field names.
frontend.proto:65:12: 'Bdev' - Use underscore_separated_names for field names.
frontend.proto:66:11: 'BlockSize' - Use underscore_separated_names for field names.
frontend.proto:67:11: 'NumBlocks' - Use underscore_separated_names for field names.
frontend.proto:68:12: 'NGUID' - Use underscore_separated_names for field names.
frontend.proto:69:12: 'EUI64' - Use underscore_separated_names for field names.
frontend.proto:70:12: 'UUID' - Use underscore_separated_names for field names.
frontend.proto:71:12: 'Multipath' - Use underscore_separated_names for field names.
frontend.proto:72:12: 'Authentication' - Use underscore_separated_names for field names.
frontend.proto:76:19: 'Subsystem' - Use underscore_separated_names for field names.
frontend.proto:84:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:92:19: 'Subsystem' - Use underscore_separated_names for field names.
frontend.proto:104:28: 'Subsystem' - Use underscore_separated_names for field names.
frontend.proto:108:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:112:19: 'Subsystem' - Use underscore_separated_names for field names.
frontend.proto:116:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:120:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:121:12: 'Stats' - Use underscore_separated_names for field names.
frontend.proto:125:20: 'Controller' - Use underscore_separated_names for field names.
frontend.proto:133:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:134:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:142:20: 'Controller' - Use underscore_separated_names for field names.
frontend.proto:150:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:154:29: 'Controller' - Use underscore_separated_names for field names.
frontend.proto:158:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:159:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:163:20: 'Controller' - Use underscore_separated_names for field names.
frontend.proto:167:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:168:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:172:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:173:12: 'Stats' - Use underscore_separated_names for field names.
frontend.proto:177:19: 'Namespace' - Use underscore_separated_names for field names.
frontend.proto:185:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:186:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:187:11: 'NamespaceId' - Use underscore_separated_names for field names.
frontend.proto:195:19: 'Namespace' - Use underscore_separated_names for field names.
frontend.proto:203:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:204:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:208:28: 'Namespace' - Use underscore_separated_names for field names.
frontend.proto:212:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:213:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:214:11: 'NamespaceId' - Use underscore_separated_names for field names.
frontend.proto:218:19: 'Namespace' - Use underscore_separated_names for field names.
frontend.proto:222:11: 'SubsystemId' - Use underscore_separated_names for field names.
frontend.proto:223:11: 'ControllerId' - Use underscore_separated_names for field names.
frontend.proto:224:11: 'NamespaceId' - Use underscore_separated_names for field names.
frontend.proto:228:11: 'Id' - Use underscore_separated_names for field names.
frontend.proto:229:12: 'Stats' - Use underscore_separated_names for field names.
backend.proto:25:11: 'Id' - Use underscore_separated_names for field names.
backend.proto:62:26: 'Controller' - Use underscore_separated_names for field names.
backend.proto:70:11: 'Id' - Use underscore_separated_names for field names.
backend.proto:78:11: 'Id' - Use underscore_separated_names for field names.
backend.proto:86:11: 'Id' - Use underscore_separated_names for field names.
backend.proto:90:35: 'Controller' - Use underscore_separated_names for field names.
backend.proto:94:11: 'Id' - Use underscore_separated_names for field names.
backend.proto:98:26: 'Controller' - Use underscore_separated_names for field names.
backend.proto:102:11: 'Id' - Use underscore_separated_names for field names.
backend.proto:106:11: 'Id' - Use underscore_separated_names for field names.
backend.proto:107:12: 'Stats' - Use underscore_separated_names for field names.
--lint_out: protoc-gen-lint: Plugin failed with status code 74.

explore `Batch methods` https://google.aip.dev/233

https://google.aip.dev/233

enhance `VolumeStats` object with new fields

Consider add:

• Flush commands count in addition to Read/Write/Unmap
• Admin commands count in addition to Read/Write/Unmap
• Error count to every Read/Write/Unmap/Flush/Admin Ops
• Async event count
• change from int32 to int64 (comment from @llabordehpe on #220)

@benlwalker @llabordehpe @mkalderon @jainvipin @GottliebNoam @orendu WDYT ?

Some of them applicable to FrontEnd only sand not BackEnd volumes, I think...

see https://github.com/opiproject/opi-marvell-bridge/blob/main/spdk.go#L287 - MrvlNvmGetCtrlrStatsResult
see https://github.com/opiproject/opi-nvidia-bridge/blob/main/spdk.go#L115 - NvdaControllerNvmeStatsResult

align fields in `NvmeSubsystemSpec`, `NvmeControllerSpec` and `NvmeNamespaceSpec` to reflect the correct NVMe SPEC definition

few examples by @jkogutINTC here https://github.com/opiproject/opi-api/pull/133/files#r987781806

Storage: add API for performing the NVMe discovery

No API is available under the Host facing API for performing the NVMe discover command. We need the NVMe discover command to find the target NQN, which is required in the NVMe connect command.

storage: create folder structure for versioning i.e. v1

example:
proto/v1/frontend/nvme.proto vs proto/v1/frontend/virtio-blk.proto

created per comment in #118 (review)

Add require attribute to new `name` fields

          approved

will open an issue to add require attribute like this:

    string name = 1 [
        (google.api.field_behavior) = REQUIRED,
        (google.api.resource_reference).type = "opi_api.storage.v1/AioVolume"
    ];

Originally posted by @glimchb in #315 (review)

storage: add `File` and `Object` to storage APIs

we focusing on block, but file and object are also important... S3, NFS, virtio-fs,...

See for example ceph cluster provides all 3 categories:
https://insujang.github.io/assets/images/200830/ceph-overview2.png
https://res.cloudinary.com/canonical/image/fetch/f_auto,q_auto,fl_sanitize,c_fill,w_1440/https://ubuntu.com/wp-content/uploads/127b/cephDiagramm.png

update slack channel link

I am unable to join the slack channel through the current link(https://opi-project.slack.com/archives/C0344KMEAKB). It requires an @amd.com email address.
Can we change the Slack channel link on the README to an invite link, like the one of SPDK(https://spdk.io/community/). So everyone else can join the channel, or is there an issue with my operation?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v37.140.8

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• chore(deps): update docker/setup-buildx-action digest to edfb0fe

Detected dependencies

dockerfile

doc/redhat-bluefield/provision/ocp/images/Dockerfile.ovn

github-actions

.github/workflows/buf.yml

• actions/checkout v4
• bufbuild/buf-setup-action v1
• bufbuild/buf-lint-action v1

.github/workflows/commitlint.yml

.github/workflows/inventory-skip.yml

.github/workflows/inventory.yml

• actions/checkout v4
• docker/setup-qemu-action v3
• docker/setup-buildx-action 7703e82fbced3d0c9eec08dff4429c023a5fd9a9
• actions/upload-artifact v4

.github/workflows/linters.yml

• actions/checkout v4
• actions/checkout v4

.github/workflows/network-skip.yml

.github/workflows/network.yml

• actions/checkout v4
• docker/setup-qemu-action v3
• docker/setup-buildx-action 7703e82fbced3d0c9eec08dff4429c023a5fd9a9
• actions/upload-artifact v4

.github/workflows/security-skip.yml

.github/workflows/security.yml

• actions/checkout v4
• docker/setup-qemu-action v3
• docker/setup-buildx-action 7703e82fbced3d0c9eec08dff4429c023a5fd9a9
• actions/upload-artifact v4

.github/workflows/storage-skip.yml

.github/workflows/storage.yml

• actions/checkout v4
• docker/setup-qemu-action v3
• docker/setup-buildx-action 7703e82fbced3d0c9eec08dff4429c023a5fd9a9
• actions/upload-artifact v4

pre-commit

.pre-commit-config.yaml

• pre-commit/pre-commit-hooks v4.5.0
• renovatebot/pre-commit-hooks 37.131.0
• alessandrojcm/commitlint-pre-commit-hook v9.11.0

regex

inventory/Makefile

• namely/protoc-all 1.51_2
• namely/protoc-all 1.51_2
• namely/protoc-all 1.51_2
• namely/protoc-all 1.51_2

security/Makefile

• namely/protoc-all 1.51_2
• namely/protoc-all 1.51_2
• namely/protoc-all 1.51_2

inventory/Makefile

• pseudomuto/protoc-gen-doc 1.5.1

network/cloud/Makefile

• pseudomuto/protoc-gen-doc 1.5.1

network/evpn-gw/Makefile

• pseudomuto/protoc-gen-doc 1.5.1

network/k8s/Makefile

• pseudomuto/protoc-gen-doc 1.5.1

network/opinetcommon/Makefile

• pseudomuto/protoc-gen-doc 1.5.1

network/telco/Makefile

• pseudomuto/protoc-gen-doc 1.5.1

security/Makefile

• pseudomuto/protoc-gen-doc 1.5.1
• pseudomuto/protoc-gen-doc 1.5.1

storage/Makefile

• pseudomuto/protoc-gen-doc 1.5.1

inventory/Makefile

• ghcr.io/docker-multiarch/google-api-linter 1.63.1
• ghcr.io/docker-multiarch/google-api-linter 1.63.1

security/Makefile

• ghcr.io/docker-multiarch/google-api-linter 1.63.1
• ghcr.io/docker-multiarch/google-api-linter 1.63.1

---

• Check this box to trigger a request for Renovate to run again on this repository

renovate: handle makefile protoc version

see

opi-api/.github/renovate.json

Lines 9 to 16 in 48b273e

	"regexManagers": [
	{
	"fileMatch": ["^Makefile$"],
	"matchStrings": ["namely/protoc-all:(?<currentValue>.*?)"],
	"datasourceTemplate": "docker",
	"depNameTemplate": "docker.io/namely/protoc-all",
	}
	]

Fixing Google AIP api linter

docker run --user=$(id -u):$(id -g) --rm -v "opiproject/opi-api/storage"/v1alpha1/:/out -w /out --entrypoint=sh ghcr.io/docker-multiarch/google-api-linter:1.36.0 -c "api-linter /out/*.proto --output-format summary"
+--------------------------------------+------------------+----------------+
|                 RULE                 | TOTAL VIOLATIONS | VIOLATED FILES |
+--------------------------------------+------------------+----------------+
| core::0191::file-layout              |                1 |              1 |
| core::0140::reserved-words           |                1 |              1 |
| core::0191::java-outer-classname     |                1 |              1 |
| core::0142::time-field-type          |                2 |              1 |
| core::0203::optional                 |                2 |              1 |
| core::0126::unspecified              |                3 |              1 |
| core::0141::count-suffix             |                4 |              4 |
| core::0123::resource-annotation      |                4 |              4 |
| core::0192::only-leading-comments    |                6 |              6 |
| core::0136::http-method              |               12 |              3 |
| core::0136::http-body                |               12 |              3 |
| core::0127::resource-name-extraction |               12 |              3 |
| core::0141::forbidden-types          |               14 |              2 |
| core::0127::http-annotation          |               24 |              6 |
| core::0136::http-uri-suffix          |               30 |              3 |
| core::0192::has-comments             |              282 |              6 |
+--------------------------------------+------------------+----------------+
Linted 13 proto files

Add OPI Security firewall API based on flow-offload

Add a flow-based offload API to integrate with firewalls for offloading flows onto DPUs and IPUs, similar to the api defined in the sessionOffload API of the sessionOffload project.

think of a better name for `message Crypto`

see #208
@benlwalker

Should we call it Crypto or CryptoVolume? Without the word "Volume" I think it leaves people room to misinterpret how this object really works. It's not the global crypto service, it's a specific volume (that may forward to another volume).

@jainvipin

we could call it Encryption (vs. crypto which is more associated with  the currency, which may not confuse storage audience, but nevertheless). Crypto is also generic name for the domain not indicating encryption for transit traffic; will also comment on your PR 

OPI Security API Spec and Protobuf

Tracking an initial cut at the OPI Security API. This will at first be IPsec tunnels.

[storage]: enable `BUF` linter and compiler

see https://buf.build/docs/lint/usage#5.-docker

and 0729d7f :

jobs:
  lint-protos:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: bufbuild/buf-setup-action@v1
      - uses: bufbuild/buf-lint-action@v1

need API to get DPU `hostnqn`

in CSI, for example, we need DPU host NQN before DPU can establish remote backend connectivity to the storage targets

this is needed so we can allow those hosts to connect to the target.
assuming allow_any is not an option
see example nvmf_subsystem_add_host here https://spdk.io/doc/jsonrpc.html#rpc_nvmf_subsystem_add_host

so we need new API that can either return us the generated by DPU hostnqn or a new API to set hostnqn to the DPU

Network: Define Goals and Objectives for agreement

Define the goals and objectives of the API for the network functionality

discuss `firmware_revision`

from slack:

Vipin Jain
1 hour ago
can we also close on the firmware version?

Ben Walker
1 hour ago
sure - can you clarify what information you're trying to convey through the NVMe firmware version?

Vipin Jain
1 hour ago
the version of the control/datapath being run for NVMe, which is different from that of the device's firmware. In our case customer changes this information based on their own P4 and so they want us to specify what version is being run

Ben Walker
1 hour ago
does this need to be discovered via NVMe commands by the host?

Vipin Jain
1 hour ago
well nvme driver does report the firmware revision, so that's what needs to be shown there

Ben Walker
1 hour ago
but the customers need to see it through the NVMe firmware version?

Ben Walker
1 hour ago
they can't query the device config some other way?

Vipin Jain
1 hour ago
but firmware version is exactly for that reason, why use some other config or sideband way?

Ben Walker
1 hour ago
because firmware version is not really clearly defined for the case where parts of the firmware are being dynamically loaded/modified

Ben Walker
1 hour ago
for instance, if you present a subsystem with two namespaces, can each namespace have different P4? Certainly in our product they can be programmed very differently.

Ben Walker
1 hour ago
and what firmware version would we report in that case?

Vipin Jain
1 hour ago
to me, the dynamic notion is more flexible and allows modularity of individual personas of an xPU

Vipin Jain
1 hour ago
a subsystem having a firmware_version is what seems usefule for now, I am not sure about the granularity beyond that

Ben Walker
1 hour ago
but if you have two namespaces that use different data paths, what firmware version do you report?

Ben Walker
1 hour ago
if they're in the same subsystem

Vipin Jain
1 hour ago
a subsystem uses one logic of P4 code for everything on the xPU - at least knowing how P4 modularity works, it is perhaps too far for anyone to have a different P4 program for different namespaces

Ben Walker
1 hour ago
your subsystem does

Ben Walker
1 hour ago
let's say ours, or someones, doesn't

Ben Walker
1 hour ago
certainly ours is much more programmable than that

Vipin Jain
1 hour ago
ok, I will put this field in vendor extensions then

Vipin Jain
1 hour ago
and remove it from the PR

Vipin Jain
1 hour ago
I think if we have it this way, everyone can report the generic/same version and I can do it differently

Ben Walker
1 hour ago
we'll support things like an iSCSI-backed namespace and an NVMe-oF/RDMA backed namespace in one subsystem

Ben Walker
1 hour ago
we'll support custom data services written by the user that we don't even know about or can enumerate

Ben Walker
1 hour ago
it's too complex to possibly capture in a single firmware version

Ben Walker
1 hour ago
so I agree with you - OPI should not have this and you can do it in the vendor extension

Vipin Jain
1 hour ago
ok

Vipin Jain
1 hour ago
pushed the change, thanks!

Ben Walker
1 hour ago
lgtm, approved

add `google.api.field_behavior` optional/mandatory

see https://linter.aip.dev/203/optional and https://google.aip.dev/203
example:

diff --git a/storage/v1alpha1/frontend_nvme_pcie.proto b/storage/v1alpha1/frontend_nvme_pcie.proto
index 610239e..303b684 100755
--- a/storage/v1alpha1/frontend_nvme_pcie.proto
+++ b/storage/v1alpha1/frontend_nvme_pcie.proto
@@ -15,8 +15,10 @@ import "object_key.proto";
 import "uuid.proto";
 import "google/protobuf/empty.proto";
 import "google/api/annotations.proto";
+import "google/api/field_behavior.proto";

 // Front End (host-facing) APIs. Mostly used for NVMe/PCIe emulation and host presentation.
+
 service FrontendNvmeService {
     rpc NVMeSubsystemCreate (NVMeSubsystemCreateRequest) returns (NVMeSubsystem) {
         option (google.api.http) = {
@@ -151,7 +153,7 @@ message NVMeController {

     // maximum number of host completion queues allowed.
     // If not set, the xPU will provide a default.
-    int32 max_ncq = 6;
+    int32 max_ncq = 6 [(google.api.field_behavior) = OPTIONAL];

     // maximum number of submission queue entries per submission queue, as a power of 2.
     // default value as per spec is 6
@@ -195,7 +197,7 @@ message NVMeNamespace {

     // 64bit Extended unique identifier for the namespace
     // mandatory if guid is not specified
-    int64 eui64 = 8;
+    int64 eui64 = 8 [(google.api.field_behavior) = OPTIONAL];

     // Globally unique identifier for the namespace
     common.v1.Uuid uuid = 9;
(END)

storage: add `capabilities` to spec

functions that return supported features and protocols and capabilities

consider `volume_services_id` instead of `volume_id` on #182

see #182 (comment)

proto file define the same http annotation with different rpc method

when i use grpc-gateway to proxy the storage gRPC service, i found the same http route in the .gw.pb files.

for example: storage/v1alpha1/frontend_nvme_pcie.proto

rpc ListNvmeSubsystems   (ListNvmeSubsystemsRequest)   returns (ListNvmeSubsystemsResponse)   {
      option (google.api.http) = {
          get: "/v1/{parent=subsystems}"
      };
      option (google.api.method_signature) = "parent";
  }

and

rpc ListNvmeControllers   (ListNvmeControllersRequest)   returns (ListNvmeControllersResponse)   {
      option (google.api.http) = {
          get: "/v1/{parent=subsystems}"
      };
      option (google.api.method_signature) = "parent";
}

above two different rpc method: ListNvmeSubsystems、ListNvmeControllers has the same http anotation url.

[storage]: Front-end Enable / Disable Namespace Proto-Bufs

@jainvipin I saw you suggested adding this...

storage: split `frontend.proto` into files

This is host emulation options

• nvme-pcie.proto
• virtio-blk.proto
• virtio-scsi.proto
• virtio-fs.proto

contribute `openoffload` back to `att/sessionOffload`

https://github.com/opiproject/opi-api/blob/main/security/proto/openoffload.proto
is enhanced copy of
https://github.com/att/sessionOffload/blob/master/protos/openoffload.proto

we can consider contributing changes back to https://github.com/att/sessionOffload

@stefanchulski fyi

storage: split `backend.proto` into files

• nvme-pcie.proto (for local drives)
• nvme-tcp.proto (rdma and tcp should be together, both for ipv4 and ipv6)
• iscsi.proto
• null.proto (for debug)

storage: structure protobuf definition in even more folders

proto/frontend
proto/middlend
proto/backend

Network: Expand use case(s) for networking

1. Define the requirements for each use case
2. Address the usage of the OVS DB, Openflow, OpenConfig, P4, etc.
3. Map the use case with the various APIs that need to be considered to support the use of OVS, SONiC, VPP, SONiC in the networking functionality that can reside in the xPU
4. Include operations such as switching, routing, firewall in the use cases and API
5. Include the scope for ports, links, vnets, etc.

Use more appropriate names for AioController and NullDebug

@benlwalker suggested renaming AioController and NullDebug proto structures to AioDisk/AioDevice and NullDisk/NullDevice.
Maybe AioVolume and NullVolume can be also considered...

Consider VolumeStats renaming

VolumeStats message is used for example in NvmeSubsystem which can contain more than one volume. To avoid confusion, we should select a more appropriate name.

I'd propose IoStats as an alternative

make: remove old `.pb.go`

for example https://github.com/opiproject/opi-api/blob/main/storage/Makefile
remove old https://github.com/opiproject/opi-api/tree/main/storage/proto before generating new ones...

storage: fix objects handles (user or server) see `User-specified IDs`

In the current OPI storage API, the handles are int64 "id"s, and are "mostly" assigned by the gRPC client (except for NVMeSubsystemCreate).

In the proposed PR #133 handles are keys defined to be 128bits opaque values?

To be most flexible, shouldn't the handles be opaque strings returned by the gRPC server ?

we also don't conform to Google's AIP (ex. Create functions should return the created object and not a CreateResponse).

test

test

Update child_id and ike_id to be uint64 values

In the strongSwan code, these are u_int values, which translates to unsigned char. We will assume we're running on 64-bit systems and translate these to uint64 values here.

Per the vici code, these are uint's and not strings. You can see this here and here.

See opiproject/opi-poc#336 for the resulting implementation.

[storage]: Front-end Attach / Detach Namespace Proto-Bufs

Need to add support for attaching / detaching controllers from a namespace.
We can take this. Just raising as issue in-case someone has prior thoughts on this ?
@glimchb

security: does OPI for `ipsec` need to have SW vs Offloaded solution definitions/Boolean in proto ?

AdminState redeclared in this block

opinetcommon/v1alpha1/networktypes.proto:283:enum AdminState {
opinetcommon/v1alpha1/openconfig_interfaces.proto:259:enum AdminState {

[storage] Object Spec and Status

https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/