ocsf / ocsf-server Goto Github PK

View Code? Open in Web Editor NEW

34.0 34.0 25.0 2.71 MB

OCSF Schema WEB Server

License: Apache License 2.0

Elixir 80.21% HTML 11.63% CSS 1.59% JavaScript 5.86% Shell 0.29% Dockerfile 0.41% Batchfile 0.01%

phoenix

ocsf-server's People

Contributors

Stargazers

Watchers

ocsf-server's Issues

Add deprecated annotation

As the schema involves, we need to have a formal way to deprecate existing classes, objects, and attributes.

To deprecate an existing class, object, or attribute, you can use the @deprecated annotation to mark a schema entity as deprecated. The @deprecated means that the entity has been superseded and may cease to exist in the future. The annotation emits a warning in the schema server, explaining that its usage is discouraged. In order to deprecate a schema entity, a proposed alternative must already exist.

The '@deprecated' annotation is a JSON object with the following fields:

message describes the proposed alternative.
since describes when the entity was deprecated.

Notes

The deprecated entities can be only removed in major releases. This means deprecated entities in OCSF schema v1.x shall only be removed in OCSF schema v2.x.
The deprecation must also be documented in CHANGELOG.

Example

To mark an object as deprecated, you can use the following snippet:

{
  "@deprecated": {
    "message": "Use the new [Bar](https://schema.ocsf.io/objects/bar) object.",
    "since": "v1.0.0"
  },
  "name": "foo",
  "caption: "The Foo Object",
  "description": "...",
  // ...
}

This will generate a warning message when the Foo object is used in the schema:

The foo object has been deprecated. Use the new [Bar](https://schema.ocsf.io/objects/bar) object.

Custom data types not converted to JSON Schema

When defining a custom data type like below it is not converted to its underlying type when generating JSON schema, this renders the schema invalid.

Invalid JSON schema type: managed_state_t. Path '$defs.device.properties.managed_state.type'

 "managed_state_t": {
        "caption": "Managed State",
        "description": "Device managed states. One of <code>Unsupported/code>, <code>Secured</code>, <code>Unsecured</code>, or <code>Unknown</code>.",
        "values": [
          "Unsupported",
          "Secured",
          "Unsecured",
          "Unknown"
        ],
        "type": "string_t",
        "type_name": "String"
      }

metadata.profiles = STRING passes validation

If I set metadata.profiles to a string:

"metadata": {
   "profiles": "tanium"
}

The API validation allows it as valid, but no profiles are applied.

As metadata.profiles is an array of strings, it should fail if the value is a single string.

Validation of schema using API is incorrect

I've been attempting to validate my "api_activity" event object using the OCSF swagger API and it keeps reporting that "api" is an "undefined attribute name". Removing it reports no error (though API is a required object). I think there is a bug in the validation endpoint. Steps to reproduce:

Go to https://schema.ocsf.io/doc/index.html
Go to "/sample/classes/{name}" and enter "api_activity" in the "Class name" text box. Click "Execute"
Copy result
Go to "/api/validate" and copy the result and click execute

This, for me, resulted in the sample schema failing validation.

Add a filter recommended attributes in the class, object, profile view

Currently, the sever offers a drop down filter for optional attributes, it will be great to have a filter for recommended attributes as well. Something similar to -

Date/Time profile captions generation

When applying the Date/Time profile to an object the captions for the datetime_t fields are not derived from the original timestamp_t attribute.

Add a functioning 'Deprecated' filter

Add the capability for the user to toggle on/off deprecated attributes in their view, similar to this

Random Profile Generation

Supplying an empty profile in the Class sample generation request creates random profiles -(url)

"reputation":{"profiles":["tomorrow conclusion lawyers","talks linking views"],"raw_score":71,"score":"Exercise Caution","score_id":6},"subnet_uid":"b548080c-3ebb-11ed-9bb3-0242ac110002","svc_name":"da wonder wyoming","uid":"b5480d16-3ebb-11ed-9311-0242ac110002","vpc_uid":"b548107c-3ebb-11ed-a6f7-0242ac110002"},"duration":12,"end_time":1664321143954142,"message":"school wonderful influence","metadata":{"correlation_uid":"b547b4c4-3ebb-11ed-b3fc-0242ac110002","labels":

https://schema.ocsf.io/sample/classes/network_activity?profiles=

UI: Behavior of Extension Checkbox

The extensions checkbox seems to only add/remove objects from the server's view. It appears that the checkbox has no impact on the dictionary and other pages.

Also, it seems like there is still some grey area for adopters when it comes to when to use profiles vs extensions. I added an issue to the docs repo to track this here.

Extension ID Ordering & Expansion capability for sidebar (extensions & profiles)

The Extension IDs are out of sequence, most likely because of alphabetical sorting by name. Show the extension id on mouse hover.

Also, as Extensions and Profiles grow, so will the list on the sidebar. To make a slight improvement to the web ui, add a triangle to expand/collapse extensions and profiles like so:

> Extensions (collapsed)
-------------------------
> Profiles (collapsed)
-------------------------

> Extensions (expanded)
   Linux
   Windows
   Macwhatever
-------------------------
> Profiles (expanded)
  Cloud
  Next Profile
  Yadda Yadda
-------------------------

Test Issue

Testing 123

Samples have duplicate keys

When getting samples from the OCSF site there are multiple records which have duplicate keys.

Examples:

Sample	Duplicates
https://schema.ocsf.io/sample/classes/dev/cloud_api	type_name
https://schema.ocsf.io/sample/classes/security_finding	type_name, class_name
https://schema.ocsf.io/sample/classes/network_activity	type_name, class_name

It looks like these are auto-generated with data. Happy to attempt a fix if you can point me in the right direction.

Profiles page

Add a new page to show the profiles and the attributes that they defined.

timestamp_t not creating datetime counterpart in object

Introducing a custom timestamp_t attribute in extension creates the counterpart in the dictionary, but not in the object.

Steps to reproduce:

Create new timestamp_t attribute in an extension dictionary, add it as field in object.
Run server and navigate to object
Toggle date/time profile, datetime_t attribute does not appear

Redesign Profile Implementation

Background: A few things broke in our testing pipelines when working with the Network Activity class schema, it turns out the class now includes 5 profiles (“cloud”, “domain_security”, “host”, “malware”, “user”), which basically add the attributes as defined in the profile definitions to the rendered schema by default. These definitions then define requirements for the attributes within. For example, this is how the malware profile looks like.
With this example in mind, now if you query for a network_activity class schema in the browser, you’ll get a JSON schema that includes all the attributes defined in the profile, which includes additional required fields that you don’t really require to normalize a network activity log. With the malware profile, it adds disposion_id & disposion as required fields (side note - there appears to be a spelling mistake in this attribute, assuming it is disposition_id)
Thus, even if, I as a user do NOT want malware profile for my network_activity class schema, I am forced with the requirement to populate/map disposion_id, else my data will technically be non-compliant with OCSF, which doesn’t make sense.
Additionally, there is a disconnect between browser view and the actually rendered JSON schema, the JSON seems to by default include all profile attributes regardless of what you choose in the browser via the checkboxes. So the checkboxes give false sense of control to the user.

In summary, profiles appear to have become a compulsive factor instead allowing for originally intended freedom.

From my understanding, it boils down to how profiles are defined in the event_class (as includes in the JSON definitions of the event_class)

I think the way it should be implemented is,
You shouldn’t define profiles in the JSON class definitions at all, instead the profile selection should an api parameter where you can explicitly mention which profiles you want for any desired event_class. The same can be translated in the browser in the form of checkboxes.

If no profiles are mentioned the user must be presented with the vanilla definition of the event_class both in the browser & in the actual JSON schema. Profiles should only be added (i.e. the attributes defined in the profiles), when explicitly selected in the api/browser.

Update Objects page description

The desc should be:
The OCSF objects. An object is a complex data type, which is a collection of other attributes. Some objects represent entities or artifacts, but not all.

Add UI column to specify base-event/inherited attribute vs. net-new attribute

As discussed in the July 18th, 2023 OCSF zoom: This data is already specified when hovering over an attribute in the UI, but I think it warrants its own column for more convenient reading. This would be particularly helpful for consumers who are actively developing with OCSF (i.e. new developers or those who are more comfortable with visual representations). Again, the data is already there -- just need to reformat it/present it in a more stable way.

Because this info may not be necessary for passive readers of the schema, it may be worth having this field as a toggle on-and-off.

Rename event_uid/name as type_uid/name

The problem with event_uid is -- it gives a notion that the identifier is uniquely identifying an event (a unique occurrence of an activity as recorded by a log, think Cloudtrail event), thats going by what an Incident responder would think, at least based on my own experience.

However, the way we currently have event_uid it is simply saying this is a unique type of an event within OCSF which is akin to an event code/type.

JSON Schema export

It would be nice to be able to export the whole JSON schema via the Schema server for internal usage.

Some other nice features would be adding a switch to tell the server how deep you want it to go when exporting some of the circular nested fields

Profiles: show cross references to classes and objects

It would be nice to show at the bottom all of the classes and objects that have registered the profile, like we cross reference other things.

We could do it like objects, where we have the references at the bottom, or we could do it like the dictionary, where we have a column in the profiles page

Improve filtering droplist UX

The "n items selected" droplist next to the search box that exists on many screens. This is a filtering mechanism that has confused users in many cases. We want to make these settings more explicit.

Ideas:

Move the filtering options to the left hand gutter so it is always visible.
Change the HTML so the upper part of the page is always visible.

Keep mobile viewing in mind. The left hand gutter, for instance, doesn't show up at all on mobile browsers (at least not on iPhone). Keeping a portion a page always visible could make mobile viewing more difficult with a large percentage of the visible view locked in place.

https://schema.ocsf.io/api/base_event does not return "requirement" key for every attribute.

It seems that https://schema.ocsf.io/api/base_event does not return the requirement property for

unmapped
count
start_time

I don't see why these would not have a corresponding requirement property, so raising this here. Thanks!

EDIT: Just discovered there are also a couple of others missing this property beyond the ones mentioned above.

Redirect http traffic on schema.ocf.io

It doesn't look like we have enabled redirection of http traffic to https on schema.ocsf.io

Currently, if you navigate to http://schema.ocsf.io, it doesn't respond (good that it doesn't accept port 80 connections), but we should be redirecting such to 443 instead, for a seamless experience.

API to convert real data to OCSF schema format

I explored different APIs in OCSF-server and found that it does not have any API for converting existing data to schema format.
To convert a data to this schema, one would require two things:

Mapping of their keys to ocsf schema
actual data

If it's not in your plan, or not having bandwidth. I can build it.
Or, your plan is to create this automation separately from ocsf-server?

Any thoughts?

Sort Classes by uid in the Categories Landing page

Currently, it appears the classes are sorted alphabetically, leading to a view similar to -

It will be a more consistent visual experience if the classes are sorted by uid

Limit the recursive object generation

Add a limit on the depth of recursive objects generation.

Filter objects on class and object graphs

Use the user selected extensions, profiles, and the pulldown filter options to limit the objects shown on the class and object graph pages.

Please document how to use with multiple versions

We would like to run the OCSF server internally supporting multiple versions in a similar way https://schema.ocsf.io/1.0.0/ is deployed (version picker letting the user to select a version). I can't find how to configure OCSF server to do it, can you please document it?

Filter profiles by extension

Currently all profiles are shown, including profiles defined in extensions, regardless whether or not the extensions is check.

Expected behavior, when an extension is checked show its profiles, and if the extension is unchecked then hide its profiles.

Set "Access-Control-Allow-Headers" for validate endpoint

Follow-up from #16

In order to use the /api/validate endpoint from browser context, the
Access-Control-Allow-Headers should be set to allow for the Content-Type header to be submitted by the client. The request works without a content-type set but fails if Content-Type: application/json is passed explicitly.

Run the server docker image

docker run -it --rm --volume /path/to/ocsf-schema:/app/schema -p 8080:8080 -p 8443:8443 ocsf-server

Change the path/to to your local OCSF schema directory (use an absolute path). Note, the -p 8443:8443 parameter enables HTTPS with a self-signed SSL certificate.

The repo of schema doesn't contain folders and files like /path/to/ocsf-schema:/app/schema

The "source" of object attributes are not accurate

The "source" of object attributes are not accurate: it always shows the top level class, not where the attribute was added or last updated (modified). This is different from classes, where this information is showing correctly. Internally, this is in the dictionary attribute's _source field.

When this is fixed, we can show this information in the same manner as class attribute on the "Attribute Dictionary" page, as well as the "Objects" and specific object pages.

Related to issue #63.

Extension leakage from attribute dictionary.

The Compliance event class which is a dev extension and part of the Policy category, also a dev extension, shows up in the cross reference of the CVSS Scores, a core object, even when the Development extension is not selected. This may also occur from the attribute dictionary although I didn't find an instance of it. The browser should not leak extension classes (nor objects) when not selected.

Add a new page to show profile's attributes

Currently there is no a simple way to see the attributes defined in the profiles. Add a new page to show the profile's attributes. The page should look similar to the object page.

Swagger.json Validation Issue

When attempting to validate the Swagger.json file using the online Swagger Parser tool at https://apitools.dev/swagger-parser/online/, the validation process fails, indicating the validation log.

Swagger schema validation failed.
  #/info must NOT have additional properties
  #/info must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/schema must be object
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1enrich/post/parameters/2 must have required property 'type'
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1enrich/post/parameters/2 must have required property '$ref'
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2/schema must be object
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2 must have required property 'type'
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2 must have required property '$ref'
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0/schema must be object
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0 must have required property 'type'
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0 must have required property '$ref'
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf


Error: Swagger schema validation failed.
  #/info must NOT have additional properties
  #/info must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/schema must be object
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1enrich/post/parameters/2 must have required property 'type'
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1enrich/post/parameters/2 must have required property '$ref'
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2/schema must be object
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2 must have required property 'type'
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2 must have required property '$ref'
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0/schema must be object
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0 must have required property 'type'
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0 must have required property '$ref'
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf

    at o (https://apitools.dev/swagger-parser/online/js/bundle.min.js:1:95564)
    at https://apitools.dev/swagger-parser/online/js/bundle.min.js:29:45042

SyntaxError: Swagger schema validation failed.
  #/info must NOT have additional properties
  #/info must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/schema must be object
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1enrich/post/parameters/2 must have required property 'type'
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1enrich/post/parameters/2 must have required property '$ref'
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1enrich/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2/schema must be object
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2/in must be equal to one of the allowed values
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2 must have required property 'type'
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1translate/post/parameters/2 must have required property '$ref'
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must NOT have additional properties
  #/paths/~1api~1translate/post/parameters/2 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0/schema must be object
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0/in must be equal to one of the allowed values
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0 must have required property 'type'
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf
  #/paths/~1api~1validate/post/parameters/0 must have required property '$ref'
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must NOT have additional properties
  #/paths/~1api~1validate/post/parameters/0 must match exactly one schema in oneOf

    at Function.o [as syntax] (https://apitools.dev/swagger-parser/online/js/bundle.min.js:1:95564)
    at validateSchema (https://apitools.dev/swagger-parser/online/js/bundle.min.js:1:6322)
    at SwaggerParser.validate (https://apitools.dev/swagger-parser/online/js/bundle.min.js:1:3391)

Attribute is not shown unless an unrelated profile is selected

The situation is a bit complex so I have created a reproducer here.

I have an extension which has an object "my_object"
The object is used to define an attribute in dictionary.json
The attribute is used in parent_object. The parent object is not used anywhere, but it does not influence the issue.
If I look at the parent_object in OCSF server, I see the my_object attribute as expected
Now I add the my_object attribute to a profile
If I apply the profile on an event like this the OCSF server starts to show the field in parent_object only if I select the Test Profile. This is unexpected the Parent Object should not be influenced by the profile in any way.

Surprisingly enough, the same situation is in OCSF core with "device" that is used in "logger" object and in the "host" profile, but it works as expected there 🤷‍♂️

JSON Schema Generator automatically including all profiles

When generating the JSON Schema for an inventory_info event, it appears that profiles are added even if they aren't included as a query parameter.

This causes the cloud profile to be added and makes the cloud object a required field. This ultimately invalidates the schema for an inventory_info event that does not originate in the cloud.

See -> curl -X GET "https://schema.ocsf.io/schema/classes/inventory_info" -H "accept: application/json"

Show additional properties of the attributes

The schema can be extended by adding extra properties of the attributes. For example, some of attributes in the dictionary already have a default property, which defines a default value of the attribute if it is missing in the event data. For example:

{
  "count": {
    "caption": "Count",
    "default": 1,
    "description": "The number of times that events in the same logical group occurred during the event <strong>Start Time</strong> to <strong>End Time</strong> period.",
    "type": "integer_t",
    "type_name": "Integer"
  }
}

The schema browser should show those properties.

Support more ways to define observables

This issue proposes 2 changes:

Add two new ways to define observables.
~~Add a new observable type registry.~~ Instead we are adding duplicate observable type_id detection to the ocsf-server and ocsf-validator.

New ways to define observables

Currently observables can be defined on types in the attribute dictionary and in object definitions. We want to add the capability to also define these observables on attributes in new ways:

Define observable on dictionary attributes.
Define observable on class and object attributes.
Define observables in class definitions by attribute paths. This allows class-specific definitions for attributes that could be nested in a sub-object or sub-array in a new section using a path-based syntax. For definitions that navigate to or through an array, the definition applies to each element of the array.

The class path-based definition will look like this (or something equivalent):

{
    "class": "foo",
    .
    .
    .
    "observables" {
        {
            "device.file.path": 42
        }
    }
}

With these additions, it is possible that observable definitions could overlap. In cases where a specific observable matches more than one type_id, the most general one should be used. The following list ranks from most to least specific:

Observable by attribute's type: a scalar type (defined in types section of attribute dictionary) or object type (defined in the object's definition). Note that attribute scalar types and object types cannot collide.
Observable by object (for attributes who's type is an object).
Observable by dictionary attribute (defined in the attributes section of the attribute dictionary).
Observable by class or object attribute (defined for in a class or object's attributes property for specific attributes).
Observable by class-specific path (defined in class definition).

Observable registry

We also want to add a new observable type registry. A new observable-types.json will be supported to unify these definitions. Extensions can extend these by placing an observable-types.json in the extension directory, as with attributes extensions using an extension-specific dictionary.json. The OCSF Server (and any future tools that read the metaschema) will log problems like missing missing type_id values and colliding type_id values. The OCSF Server (and future tools) may optionally stop these sort of errors, though there should always be a way to continue on in case there metaschema is in a temporarily inconsistent state.

This idea doesn't help avoid observable type_id collisions. As mentioned above, we will instead detect collisions as a validation.

Schema validation

When validating the schema, profiles == null is equal to no profiles.

Proposal: improve 404 page or redirect -dev versions after release

I've been working with colleagues on mapping detection findings, which was a part of the 1.1.0 release. Before release, this version was listed under 1.1.0-dev, and so our discussions often involved linking to the OCSF server with a version reference like so:

https://schema.ocsf.io/1.1.0-dev/classes/detection_finding?extensions=

As this was the only way to see the new detection finding class. Now that 1.1.0 is released, the link above just goes to a basic 404 page.

I wonder if there's a better experience to provide here? Ideas:

Simple: improve the basic 404 page so there are at least links back to OCSF schema proper.
More complicated: on release of a -dev version, would it make sense if /1.1.0-dev redirected to /1.1.0?

Remove object_type from the attribute definitions

Defining a new attribute of an object type requires two statements, type and object_type:

{
  "name": "cat", 
  "caption": "Cat",
  "type": "object_t", 
  "object_type": "cat"
}

The definition could be simplified by removing the object_type and using only the type statement:

{
  "name": "cat", 
  "caption": "Cat",
  "type": "cat"
}

Missing requirement property in the JSON schema

When the OCSF server renders a JSON schema for an event class, it does NOT add a requirement property if the underlying json definition of the schema doesn't explicitly add a requirement.

Note that, while defining the schema json files, missing requirement implies it is optional, but that is not translated to the JSON schema that the server consequently renders.

Example object - Metadata object definition

Example json schema -

      "modified_time": {
        "type": "timestamp_t",
        "description": "The time when the event was last modified or enriched.",
        "caption": "Modified Time",
        "_source": "metadata",
        "type_name": "Timestamp"
      }

Support user-defined data types

Using extension to add new data types (already supported) or to extend existing data types (add contains or data examples).

For example:

The schema defines country attribute as a string (the country code, as defined by the ISO 3166-1 standard -- Alpha-2 code). However, some users would like to use a stricter definition, such as a list of country codes. This could be achieved by defining a new type, country_code_t, which is a subtype of string_t, and lists the valid values.

  "country_code_t": {
    "caption": "Country Code",
    "description": "Defined by the ISO 3166-1 standard -- Alpha-2 code. For example: <code>AA</code>.",
    "type": "string_t",
    "values": ["AA", "AB", "AC", "AD", "AE", "AF", "AG", "AH", "AI", "AJ", "AK", "..."]
  },

Then, the data new type can be used to overwrite the dictionary definition of the country attribute:

{
  "country": {
    "type": "country_code_t"
  }
}

Clicking "JSON Schema" from "Base Event" page leads to 404

Steps to reproduce:

Click on "Base Event" from navigation. This leads to the special /base_event URL.
Click on "JSON Schema".

This results in a 404. Note that navigating from /class/base_event works. This URL can be navigated to from a few location, including from the class list on the "Classes" page.

Cause

This is happening because the JSON Schema JavaScript in priv/static/js/app.js is navigating to /schema/base_event, which doesn't exist. When navigating from a /classes/<class_name> URL, the JavaScript function leads to the working /schema/classes/base_event URL.

Possible fixes

There are (at least) two ways to fix:

Fix the JavaScript function to handle coming from the /base_event URL.
Remove the special /base_event URL entirely from the OCSF Server since it is equivalent to /classes/base_event. This is a larger change, but could result in a overall cleaner solution.

Dictionary cross-references are not shown for attributes that are overridden from Base Event

As an example, status status_id status_detail are overridden in Detection Finding but when showing them in the browser, only Base Event is cross-referenced.

Switch schema.ocsf.io SSL certificate to Let's Encrypt by August 1, 2024

The current *.ocsf.io certificate used by the OCSF server at schema.ocsf.io expires on August 1, 2024. We should convert this to a Let's Encrypt certificate, along setting up the ACME protocol to automatically update the certificate.

This work should be started roughly 3 months before the certificate expires to give plenty of time for the conversion, so around May 1, 2024.

Add a link to the API doc page

Simplify schema usage from client-side JS

Similar to schema.org (eg. https://schema.org/version/latest/schemaorg-current-https.jsonld), the OCSF Schema Server should serve the appropriate CORS header to simplify client-side usage.

Container profile is incorrectly shown on most class detail pages

Most class detail pages are showing the Container profile when the profile is not, in fact, applicable. With the main ocsf-schema, 62 out of 64 classes include the container profile with only base_event and incident_finding not included. The Container profile associated with no classes (per its profile detail page) and is the only profile that is associated with objects.

What might be happening is that class detail "profiles" section is including classes that reference the objects that in turn use the profile, whereas the profile details page is only showing the direct references in the class and object definitions.

ocsf / ocsf-server Goto Github PK

ocsf-server's People

Contributors

Stargazers

Watchers

Forkers

ocsf-server's Issues

Notes

Example

Support more ways to define observables

New ways to define observables

Observable registry

For example:

Steps to reproduce:

Cause

Possible fixes

Recommend Projects

Recommend Topics

Recommend Org