- Module: a running component (e.g., camera, mapping, nav).
- Stream: a unidirectional flow of messages owned by a module (one broadcaster → many receivers).
- Topic: the name/identifier used by a transport or pubsub backend.
- Message: payload carried on a stream (often
dimos.msgs.*, but can be bytes / images / pointclouds / etc.).
What the transport layer guarantees (and what it doesn’t)
Modules don’t know or care how data moves. They just:- emit messages (broadcast)
- subscribe to messages (receive)
- Some are best-effort (e.g., UDP multicast / LCM): loss can happen.
- Some can be reliable (e.g., TCP-backed, Redis, some DDS configs) but may add latency/backpressure.
Choosing a backend
For most users, the important choice is betweenlcm, zenoh, and shared memory overrides:
lcm: current legacy default on most platforms. Fast and simple, but UDP multicast is best-effort.zenoh: network transport with reliable delivery semantics and the same typed message model throughLCMEncoderMixin.- shared memory (
pSHMTransport, etc.): best for large local streams on a single machine.
zenoh there. Other platforms default to lcm.
Zenoh quickstart
Zenoh ships with DimOS by default (eclipse-zenoh is a base dependency), so there is nothing extra to install.
Default global stream transport (only applies when you do not pass --transport or set DIMOS_TRANSPORT):
| Situation | Default |
|---|---|
| macOS | zenoh |
| Any other platform | lcm |
- CLI:
dimos --transport=zenoh ...ordimos --transport=lcm ...(see CLI for precedence with.envand blueprints). - Environment:
DIMOS_TRANSPORT=zenohorDIMOS_TRANSPORT=lcm.
lcm until you opt in):
Benchmarks
Quick view on performance of our pubsub backends:skip
Abstraction layers
Using transports with blueprints
See Blueprints for the blueprint API. Fromunitree/go2/blueprints/smart/unitree_go2.py.
Example: rebind a few streams from the default LCMTransport to ROSTransport (defined at transport.py) so you can visualize in rviz2.
skip
Using transports with modules
Each stream on a module can use a different transport. Set.transport on the stream before starting modules.
The runnable example below uses a tiny synthetic image publisher instead of CameraModule so it works without a webcam and in CI; the wiring is the same as with a real camera.
ansi=false
Inspecting LCM traffic (CLI)
lcmspy shows topic frequency/bandwidth stats:
dimos topic echo /topic listens on typed channels like /topic#pkg.Msg and decodes automatically:
skip
Implementing a transport
At the stream layer, a transport is implemented by subclassingTransport (see core/stream.py) and implementing:
broadcast(...)subscribe(...)
Transport.__init__ args can be anything meaningful for your backend:
(ip, port)- a shared-memory segment name
- a filesystem path
- a Redis channel
Encoding helpers
Many of our message types providelcm_encode / lcm_decode for compact, language-agnostic binary encoding (often faster than pickle). For details, see LCM.
PubSub transports
Even though transport can be anything (TCP connection, unix socket) for now all our transport backends implement thePubSub interface.
publish(topic, message)subscribe(topic, callback) -> unsubscribe
LCM (UDP multicast)
LCM is UDP multicast. It’s very fast on a robot LAN, but it’s best-effort (packets can drop). For local emission it autoconfigures system in a way in which it’s more robust and faster then other more common protocols like ROS, DDSZenoh
Zenoh provides network pubsub without relying on UDP multicast for the user-facing stream transport. In DimOS it carries the same typed messages by encoding them withLCMEncoderMixin, so existing dimos.msgs.* types still work.
Use Zenoh when:
- you want a transport that behaves better than UDP multicast on macOS
- you are replaying large or high-rate data and want a more reliable network path
- you want to keep the DimOS typed stream model while changing the transport backend
ZenohTransport and pZenohTransport. Install, defaults, and CLI versus environment overrides are in the Zenoh quickstart above.
Performance note: zenoh’s session-to-session path (modules in different processes, the common case) benchmarks faster than LCM for small messages and for >=2MiB ones. Delivery within one shared session (co-located modules in one worker) is its slow path for 256KiB-1MiB messages (a few GiB/s); pin shared memory transports for heavy co-located streams. The benchmark has both cases (Zenoh = shared session, ZenohPeers = separate sessions).
The Rerun bridge also follows the global transport. When transport=zenoh, the bridge listens on Zenoh and on LCM for TF data.
Per-topic QoS
Zenoh publisher QoS lives on the ZenohTopic object (see zenohpubsub.py):
skip
default_zenoh_qos in transport_factory.py):
- RPC topics and the agent channels (
human_input,agent,agent_idle): reliable, block under congestion (never drop). Image/PointCloud2streams: best-effort, drop under congestion (latest wins).- Everything else: zenoh defaults (reliable, drop under congestion).
transport=zenoh.
Shared memory (IPC)
Shared memory is highest performance, but only works on the same machine.DDS Transport
For network communication, DDS uses the Data Distribution Service (DDS) protocol:skip session=dds_demo ansi=false
A minimal transport: Memory
The simplest toy backend is Memory (single process). Start from there when implementing a new pubsub backend.
pubsub/impl/memory.py for the complete source.
Encode/decode mixins
Transports often need to serialize messages before sending and deserialize after receiving.PubSubEncoderMixin at pubsub/encoders.py provides a clean way to add encoding/decoding to any pubsub implementation.
Available mixins
| Mixin | Encoding | Use case |
|---|---|---|
PickleEncoderMixin | Python pickle | Any Python object, Python-only |
LCMEncoderMixin | LCM binary | Cross-language (C/C++/Python/Go/…) |
JpegEncoderMixin | JPEG compressed | Image data, reduces bandwidth |
LCMEncoderMixin is especially useful: you can use LCM message definitions with any transport (not just UDP multicast). See LCM for details.
Creating a custom mixin
session=jsonencoder no-result
session=jsonencoder no-result
session=jsonencoder no-result
Testing and benchmarks
Spec tests
Seepubsub/test_spec.py for the grid tests your new backend should pass.
Benchmarks
Add your backend to benchmarks to compare in context:skip
Available transports
| Transport | Use case | Cross-process | Network | Notes |
|---|---|---|---|---|
Memory | Testing only, single process | No | No | Minimal reference impl |
SharedMemory | Multi-process on same machine | Yes | No | Highest throughput (IPC) |
LCM | Robot LAN broadcast (UDP multicast) | Yes | Yes | Best-effort; can drop packets on LAN |
Zenoh | Reliable network stream transport | Yes | Yes | Recommended on macOS for heavy replay |
Redis | Network pubsub via Redis server | Yes | Yes | Central broker; adds hop |
ROS | ROS 2 topic communication | Yes | Yes | Integrates with RViz/ROS tools |
DDS | Cyclone DDS without ROS (WIP) | Yes | Yes | WIP |
