Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 29 additions & 0 deletions architecture/gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -821,6 +821,35 @@ startup and disables export. Export is best-effort — the SDK logs runtime
failures, and a failed batch is dropped rather than retried. Buffered spans
flush after the server loop exits so `SIGTERM` does not drop in-flight traces.

### OCSF file collection

The gateway owns one optional native OCSF JSONL output, configured through
`openshell.gateway.ocsf_log`. An in-process bounded queue captures
gateway-produced OCSF records independently of diagnostic filtering and sandbox
log subscriptions, including events without a sandbox association. It preserves
event identity and does not translate records into destination-specific schemas.

Console formatting and sandbox log subscriptions also read the structured event:
the console renders shorthand, and the log bus routes by the affected container's
UID. Gateway-wide events have no sandbox subscription. These consumers remain
available without enabling JSONL output. Event origin identifies the producer
(`Gateway` or `Supervisor`), independently of the affected sandbox.

Tracing callbacks queue serialized records with count and byte bounds; overflow
drops incoming records. A dedicated filesystem worker owns appends, recovery of
incomplete tails, daily rename-based rotation, and retention. Failed writes are
not replayed, and subsequent records are discarded during bounded reopen backoff.
Metrics expose writer failures and known or uncertain losses outside the output.
Tail recovery reports discarded bytes, not an invented missing-record count.

Each replica requires its own file. An external shipper owns rotation discovery,
checkpoints, remote retries, credentials, and SIEM or OTLP conversion. Retention
must exceed expected shipper outages to avoid pruning unread segments. Concurrent
writers and external copy-truncate rotation are unsupported. Output failures do
not block sandbox execution. Shutdown allows five seconds to drain; an outstanding
OS write cannot be cancelled and remains uncertain. Flush is not fsync, and neither
the queue nor append results promise durable acceptance or audit completeness.

### Package-managed gateway registry

The CLI reads its active-gateway and per-gateway metadata from
Expand Down
3 changes: 3 additions & 0 deletions architecture/sandbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -457,6 +457,9 @@ Security-relevant sandbox behavior uses OCSF structured events; internal
diagnostics use ordinary tracing.
The OCSF device describes the sandbox environment, with type ID Other and type
label `Sandbox`; its operating system is a separate attribute.
Gateway-origin events describe the gateway process's OS, independently of any
associated sandbox. Linux supervisor events retain Linux as their device OS,
including when a native macOS or Windows gateway manages the sandbox.

## Policy Proposals

Expand Down
27 changes: 25 additions & 2 deletions crates/openshell-ocsf/src/builders/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,19 @@ use crate::enums::StatusId;
use crate::events::base_event::BaseEventData;
use crate::objects::{Container, Device, Endpoint, Image, Metadata, Product};

/// Which `OpenShell` component produced an event.
#[derive(Debug, Clone, PartialEq, Eq, Default)]
pub enum EventOrigin {
/// The supervisor process enforcing sandbox policy.
#[default]
Supervisor,
/// The gateway process, which has no sandbox or container of its own.
Gateway {
/// Operator-assigned gateway name (`[openshell.gateway] name`).
name: String,
},
}

/// Immutable context created once at sandbox startup.
///
/// Passed to every event builder to populate shared OCSF fields
Expand All @@ -195,15 +208,21 @@ pub struct EventContext {
pub proxy_ip: IpAddr,
/// Proxy listen port.
pub proxy_port: u16,
/// Which component is emitting.
pub origin: EventOrigin,
}

impl EventContext {
/// Build the OCSF `Metadata` object for any event.
#[must_use]
pub fn metadata(&self, profiles: &[&str]) -> Metadata {
let product = match self.origin {
EventOrigin::Supervisor => Product::openshell_sandbox(&self.product_version),
EventOrigin::Gateway { .. } => Product::openshell_gateway(&self.product_version),
};
Metadata {
version: OCSF_VERSION.to_string(),
product: Product::openshell_sandbox(&self.product_version),
product,
profiles: profiles.iter().map(|s| (*s).to_string()).collect(),
uid: Some(uuid::Uuid::new_v4().to_string()),
log_source: None,
Expand All @@ -228,7 +247,10 @@ impl EventContext {
/// Build the OCSF `Device` object.
#[must_use]
pub fn device(&self) -> Device {
Device::linux(&self.hostname)
match &self.origin {
EventOrigin::Supervisor => Device::linux(&self.hostname),
EventOrigin::Gateway { name } => Device::gateway(&self.hostname, name),
}
}

/// Build the `proxy_endpoint` object for the Network Proxy profile.
Expand Down Expand Up @@ -268,6 +290,7 @@ pub(crate) fn test_sandbox_context() -> EventContext {
product_version: "0.1.0".to_string(),
proxy_ip: "10.42.0.1".parse().unwrap(),
proxy_port: 3128,
origin: EventOrigin::Supervisor,
}
}

Expand Down
3 changes: 2 additions & 1 deletion crates/openshell-ocsf/src/ctx.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
//! not been set (e.g. unit tests that exercise builders without booting the
//! sandbox).

use crate::EventContext;
use crate::{EventContext, EventOrigin};
use std::sync::{LazyLock, OnceLock};

static OCSF_CTX: OnceLock<EventContext> = OnceLock::new();
Expand All @@ -21,6 +21,7 @@ static OCSF_CTX_FALLBACK: LazyLock<EventContext> = LazyLock::new(|| EventContext
product_version: env!("CARGO_PKG_VERSION").to_string(),
proxy_ip: std::net::IpAddr::from([127, 0, 0, 1]),
proxy_port: 3128,
origin: EventOrigin::Supervisor,
});

/// Initialise the process-wide OCSF sandbox context.
Expand Down
10 changes: 9 additions & 1 deletion crates/openshell-ocsf/src/enums/device_type.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ use serde_repr::{Deserialize_repr, Serialize_repr};
pub enum DeviceTypeId {
/// 0 — Unknown
Unknown = 0,
/// 1 — Server
Server = 1,
/// 99 — Other
Other = 99,
}
Expand All @@ -30,6 +32,7 @@ impl std::fmt::Display for DeviceTypeId {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str(match self {
Self::Unknown => "Unknown",
Self::Server => "Server",
Self::Other => "Other",
})
}
Expand All @@ -43,6 +46,7 @@ mod tests {
fn device_type_display_uses_schema_labels() {
for (device_type, expected) in [
(DeviceTypeId::Unknown, "Unknown"),
(DeviceTypeId::Server, "Server"),
(DeviceTypeId::Other, "Other"),
] {
assert_eq!(device_type.to_string(), expected);
Expand All @@ -52,7 +56,11 @@ mod tests {

#[test]
fn device_type_json_roundtrip() {
for (device_type, expected) in [(DeviceTypeId::Unknown, 0), (DeviceTypeId::Other, 99)] {
for (device_type, expected) in [
(DeviceTypeId::Unknown, 0),
(DeviceTypeId::Server, 1),
(DeviceTypeId::Other, 99),
] {
let json = serde_json::to_value(device_type).unwrap();
assert_eq!(json, serde_json::json!(expected));
let decoded: DeviceTypeId = serde_json::from_value(json).unwrap();
Expand Down
4 changes: 2 additions & 2 deletions crates/openshell-ocsf/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ pub use objects::{
// --- Builders ---
pub use builders::{
ApiActivityBuilder, AppLifecycleBuilder, BaseEventBuilder, ConfigStateChangeBuilder,
DetectionFindingBuilder, EventContext, HttpActivityBuilder, NetworkActivityBuilder,
ProcessActivityBuilder, SshActivityBuilder,
DetectionFindingBuilder, EventContext, EventOrigin, HttpActivityBuilder,
NetworkActivityBuilder, ProcessActivityBuilder, SshActivityBuilder,
};

// --- Tracing layers ---
Expand Down
41 changes: 41 additions & 0 deletions crates/openshell-ocsf/src/objects/device.rs
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,28 @@ impl Device {
}),
}
}

/// Create the device for a gateway replica.
#[must_use]
pub fn gateway(hostname: &str, name: &str) -> Self {
Self {
hostname: hostname.to_string(),
type_id: DeviceTypeId::Server,
type_label: DeviceTypeId::Server.to_string(),
name: Some(name.to_string()),
// Keep the replica identity opaque rather than encoding multiple fields in the UID.
uid: Some(hostname.to_string()),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could device.uid include the gateway installation identity as well as the replica hostname? Hostnames such as openshell-gateway-0 can repeat across namespaces or clusters, so a SIEM aggregating multiple installations may merge distinct gateways under one device UID. For example, deriving the UID from both name and hostname—or omitting it when global uniqueness cannot be guaranteed—would better preserve the field’s uniqueness contract.

os: Some(OsInfo {
name: match std::env::consts::OS {
"linux" => "Linux",
"windows" => "Windows",
"macos" => "macOS",
other => other,
}
.to_string(),
}),
}
}
}

#[cfg(test)]
Expand Down Expand Up @@ -85,4 +107,23 @@ mod tests {
assert_eq!(decoded, device);
assert_eq!(serde_json::to_value(&decoded).unwrap(), json);
}

#[test]
fn gateway_device_does_not_inherit_the_sandbox_type() {
let json = serde_json::to_value(Device::gateway("gateway-0", "production")).unwrap();

assert_eq!(json["type_id"], 1);
assert_eq!(json["type"], "Server");
}

#[test]
fn gateway_replicas_have_distinct_device_uids() {
let first = Device::gateway("openshell-gateway-0", "production");
let second = Device::gateway("openshell-gateway-1", "production");

assert_ne!(
first.uid, second.uid,
"gateway replicas must have distinct OCSF device UIDs"
);
}
}
10 changes: 10 additions & 0 deletions crates/openshell-ocsf/src/objects/metadata.rs
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,16 @@ impl Product {
version: Some(version.to_string()),
}
}

/// Create the `OpenShell` Gateway product, for control-plane events.
#[must_use]
pub fn openshell_gateway(version: &str) -> Self {
Self {
name: "OpenShell Gateway".to_string(),
vendor_name: "OpenShell".to_string(),
version: Some(version.to_string()),
}
}
}

#[cfg(test)]
Expand Down
41 changes: 40 additions & 1 deletion crates/openshell-ocsf/tests/event_identity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@

use std::net::{IpAddr, Ipv4Addr};

use openshell_ocsf::{ActivityId, EventContext, NetworkActivityBuilder, OcsfEvent, SeverityId};
use openshell_ocsf::{
ActivityId, EventContext, EventOrigin, NetworkActivityBuilder, OcsfEvent, SeverityId,
};

fn sandbox_ctx(container_image: &str) -> EventContext {
EventContext {
Expand All @@ -16,6 +18,22 @@ fn sandbox_ctx(container_image: &str) -> EventContext {
product_version: "0.42.1".to_string(),
proxy_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),
proxy_port: 8888,
origin: EventOrigin::Supervisor,
}
}

fn gateway_ctx(sandbox_id: &str, sandbox_name: &str) -> EventContext {
EventContext {
sandbox_id: sandbox_id.to_string(),
sandbox_name: sandbox_name.to_string(),
container_image: String::new(),
hostname: "openshell-gateway-0".to_string(),
product_version: "0.42.1".to_string(),
proxy_ip: IpAddr::V4(Ipv4Addr::LOCALHOST),
proxy_port: 0,
origin: EventOrigin::Gateway {
name: "production-us-west".to_string(),
},
}
}

Expand Down Expand Up @@ -56,6 +74,27 @@ fn the_sandbox_id_is_carried_by_the_container() {
assert_eq!(json["container"]["name"], "agent-01");
}

#[test]
fn a_gateway_event_about_a_sandbox_still_names_that_container() {
let json = event(&gateway_ctx("sb-7", "agent-07"))
.to_json()
.expect("serializes");

assert_eq!(json["container"]["uid"], "sb-7");
assert_eq!(json["container"]["name"], "agent-07");
assert_eq!(json["metadata"]["product"]["name"], "OpenShell Gateway");
}

#[test]
fn a_gateway_event_about_no_sandbox_omits_the_container() {
let json = event(&gateway_ctx("", "")).to_json().expect("serializes");

assert!(
json.get("container").is_none(),
"an event with no sandbox association has no container: {json}"
);
}

#[test]
fn a_container_without_an_image_omits_the_image() {
let json = event(&sandbox_ctx("")).to_json().expect("serializes");
Expand Down
Loading
Loading