Crate rusteron_archive

rusteron-archive

rusteron-archive is a module within the rusteron project that provides functionality for interacting with Aeron’s archive system in a Rust environment. This module builds on rusteron-client, adding support for recording, managing, and replaying archived streams.

---

Sponsored by GSR

Rusteron is proudly sponsored and maintained by GSR, a global leader in algorithmic trading and market making in digital assets.

It powers mission-critical infrastructure in GSR’s real-time trading stack and is now developed under the official GSR GitHub organization as part of our commitment to open-source excellence and community collaboration.

We welcome contributions, feedback, and discussions. If you’re interested in integrating or contributing, please open an issue or reach out directly.

---

Overview

The rusteron-archive module enables Rust developers to leverage Aeron’s archive functionality, including recording and replaying messages with minimal friction.

For MacOS users, the easiest way to get started is by using the static library with precompiled C dependencies. This avoids the need for cmake or Java:

rusteron-archive = { version = "0.1", features = ["static", "precompile"] }

---

Installation

Add rusteron-archive to your Cargo.toml depending on your setup:

# Dynamic linking (default)
rusteron-archive = "0.1"

# Static linking
rusteron-archive = { version = "0.1", features = ["static"] }

# Static linking with precompiled C libraries (best for Mac users, no Java/cmake needed)
rusteron-archive = { version = "0.1", features = ["static", "precompile"] }

When using the default dynamic configuration, you must ensure Aeron C libraries are available at runtime. The static option embeds them automatically into the binary.

---

Development

To simplify development, we use just, a command runner similar to make.

To view all available commands, run just in the command line.

If you don’t have just installed, install it with: cargo install just

---

Features

• Stream Recording – Record Aeron streams for replay or archival.
• Replay Handling – Replay previously recorded messages.
• Publication/Subscription – Publish to and subscribe from Aeron channels.
• Callbacks – Receive events such as new publications, subscriptions, and errors.
• Automatic Resource Management (via new() only) – Constructors automatically call *_init and clean up with *_close or *_destroy when dropped.
• String Handling – new() and setter methods accept &CStr; getter methods return &str.

---

General Patterns

Cloneable Wrappers

All wrapper types in rusteron-archive implement Clone and share the same underlying Aeron C resource. For shallow copies of raw structs, use .clone_struct().

Mutable and Immutable APIs

Most methods use &self, allowing mutation without full ownership transfer.

Resource Management Caveats

Automatic cleanup applies only to new() constructors. Other methods (e.g. set_aeron()) require manual lifetime and validity tracking to prevent resource misuse.

Manual Handler Management

Handlers must be passed into C bindings using Handlers::leak(...) and explicitly cleaned up using release() when no longer needed.

For short-lived operations such as polling, closures can be used directly:

subscription.poll_once(|msg, header| {
    println!("msg={:?}, header={:?}", msg, header)
});

---

Handlers and Callbacks

There are two primary patterns for defining callbacks:

1. Trait-Based Handlers (Recommended)

The preferred and most efficient approach is to define a trait and implement it for a struct:

use rusteron_archive::*;

pub trait AeronErrorHandlerCallback {
    fn handle_aeron_error_handler(&mut self, errcode: ::std::os::raw::c_int, message: &str);
}

pub struct AeronErrorHandlerLogger;

impl AeronErrorHandlerCallback for AeronErrorHandlerLogger {
    fn handle_aeron_error_handler(&mut self, errcode: ::std::os::raw::c_int, message: &str) {
        eprintln!("Error {}: {}", errcode, message);
    }
}

You then wrap the implementation in a handler using Handlers::leak(...).

2. Wrapping Callbacks with Handler

Regardless of approach, callbacks must be wrapped in a Handler to interact with Aeron’s C bindings. Use Handlers::leak(...) to pass it into the system, and call release() once cleanup is needed.

---

Handler Convenience Methods

You can pass None if a handler isn’t required, but dealing with typed Options can be awkward. rusteron-archive offers helpers like:

pub fn no_error_handler_handler() -> Option<&'static Handler<AeronErrorHandlerLogger>> {
    None::<&Handler<AeronErrorHandlerLogger>>
}

These helpers return None with the correct generic type to reduce boilerplate.

---

Error Handling with Aeron C Bindings

Operations in rusteron-archive return Result<i32, AeronCError>, using idiomatic Rust error types.

AeronErrorType Enum

Variant	Description
NullOrNotConnected	Null value or unconnected
ClientErrorDriverTimeout	Driver timed out
ClientErrorClientTimeout	Client timed out
ClientErrorConductorServiceTimeout	Conductor service timeout
ClientErrorBufferFull	Buffer full
PublicationBackPressured	Publication is back-pressured
PublicationAdminAction	Admin action in progress
PublicationClosed	Publication has closed
PublicationMaxPositionExceeded	Max position exceeded
PublicationError	Generic publication error
TimedOut	Timeout occurred
Unknown(i32)	Unrecognized error code

The AeronCError struct exposes these enums alongside descriptive messages.

---

Safety Considerations

1. Aeron Lifetime – The AeronArchive depends on an external Aeron instance. Ensure Aeron outlives all references to the archive.
2. Unsafe Bindings – The module interfaces directly with Aeron’s C API. Improper resource handling can cause undefined behavior.
3. Manual Cleanup – Handlers and other leaked objects must be manually cleaned up using .release().
4. Thread Safety – Use care when accessing Aeron objects across threads. Synchronize access appropriately.

---

Typical Workflow

1. Initialize client and archive contexts.
2. Start Recording a specific channel and stream.
3. Publish Messages to the stream.
4. Stop Recording once complete.
5. Locate the Recording using archive queries.
6. Replay Setup: Configure replay target/channel.
7. Subscribe and Receive replayed messages.

---

Benchmarks

For latency and throughput benchmarks, refer to BENCHMARKS.md.

---

Contributing

Contributions are more than welcome! Please:

• Submit bug reports, ideas, or improvements via GitHub Issues
• Propose changes via pull requests
• Read our CONTRIBUTING.md

We’re especially looking for help with:

• API design reviews
• Safety and idiomatic improvements
• Dockerized and deployment examples

---

License

Licensed under either MIT License or Apache License 2.0 at your option.

---

Acknowledgments

Special thanks to:

• @mimran1980, a core low-latency developer at GSR and the original creator of Rusteron - your work made this possible!
• @bspeice for the original libaeron-sys
• The Aeron community for open protocol excellence

Features

• static: When enabled, this feature statically links the Aeron C code. By default, the library uses dynamic linking to the Aeron C libraries.
• backtrace - When enabled will log a backtrace for each AeronCError
• extra-logging - When enabled will log when resource is created and destroyed. useful if your seeing a segfault due to a resource being closed
• precompile - When enabled will use precompiled c code instead of requiring cmake and java to me installed

Modules

bindings

testing

Structs

Aeron

AeronAgentStartFuncLogger

AeronArchive

AeronArchiveAsyncConnect

AeronArchiveContext

AeronArchiveControlResponsePoller

AeronArchiveCredentialsChallengeSupplierFuncLogger

AeronArchiveCredentialsEncodedCredentialsSupplierFuncLogger

AeronArchiveCredentialsFreeFuncLogger

AeronArchiveDelegatingInvokerFuncLogger

AeronArchiveEncodedCredentials

AeronArchiveProxy

AeronArchiveRecordingDescriptor

Struct containing the details of a recording

AeronArchiveRecordingDescriptorConsumerFuncLogger

AeronArchiveRecordingDescriptorPoller

AeronArchiveRecordingSignal

Struct containing the details of a recording signal.

AeronArchiveRecordingSignalConsumerFuncLogger

AeronArchiveRecordingSubscriptionDescriptor

Struct containing the details of a recording subscription

AeronArchiveRecordingSubscriptionDescriptorConsumerFuncLogger

AeronArchiveRecordingSubscriptionDescriptorPoller

AeronArchiveReplayMerge

AeronArchiveReplayParams

Struct containing the available replay parameters.

AeronArchiveReplicationParams

Struct containing the available replication parameters.

AeronAsyncAddCounter

AeronAsyncAddExclusivePublication

AeronAsyncAddPublication

AeronAsyncAddSubscription

AeronAsyncDestination

AeronAsyncDestinationById

AeronAvailableCounterLogger

AeronAvailableCounterPair

AeronAvailableImageLogger

AeronBlockHandlerLogger

AeronBufferClaim

Structure used to hold information for a try_claim function call.

AeronCError

Represents an Aeron-specific error with a code and an optional message.

AeronClientRegisteringResource

AeronCloseClientLogger

AeronCloseClientPair

AeronCnc

AeronCncConstants

AeronCncMetadata

AeronContext

AeronControlledFragmentAssembler

AeronControlledFragmentHandlerLogger

AeronCounter

AeronCounterConstants

Configuration for a counter that does not change during it’s lifetime.

AeronCounterMetadataDescriptor

AeronCounterValueDescriptor

AeronCountersReader

AeronCountersReaderBuffers

AeronCountersReaderForeachCounterFuncLogger

AeronDataHeader

AeronDataHeaderAsLongs

AeronError

AeronErrorHandlerLogger

AeronErrorLogReaderFuncLogger

AeronErrorLogger

AeronExclusivePublication

AeronFragmentAssembler

AeronFragmentClosureAssembler

AeronFragmentHandlerLogger

AeronFrameHeader

AeronHeader

AeronHeaderValues

AeronHeaderValuesFrame

AeronIdleStrategyFuncLogger

AeronImage

AeronImageConstants

Configuration for an image that does not change during it’s lifetime.

AeronImageControlledFragmentAssembler

AeronImageFragmentAssembler

AeronIovec

AeronIpcChannelParams

AeronLogBuffer

AeronLogbufferMetadata

AeronLossReporter

AeronLossReporterEntry

AeronLossReporterReadEntryFuncLogger

AeronMappedBuffer

AeronMappedFile

AeronMappedRawLog

AeronNakHeader

AeronNewPublicationLogger

AeronNewSubscriptionLogger

AeronNotificationLogger

AeronOptionHeader

AeronPerThreadError

AeronPublication

AeronPublicationConstants

Configuration for a publication that does not change during it’s lifetime.

AeronPublicationErrorFrameHandlerLogger

AeronPublicationErrorValues

AeronReservedValueSupplierLogger

AeronResolutionHeader

AeronResolutionHeaderIpv4

AeronResolutionHeaderIpv6

AeronResponseSetupHeader

AeronRttmHeader

AeronSetupHeader

AeronStatusMessageHeader

AeronStatusMessageOptionalHeader

AeronStrToPtrHashMap

AeronStrToPtrHashMapKey

AeronSubscription

AeronSubscriptionConstants

AeronUdpChannelParams

AeronUnavailableCounterLogger

AeronUnavailableCounterPair

AeronUnavailableImageLogger

AeronUri

AeronUriParam

AeronUriParams

AeronUriParseCallbackLogger

AeronUriStringBuilder

AtomicWideCounterBindgenTy1

ChannelUri

Represents the Aeron URI parser and handler.

FnMutMessageHandler

Handler

Handler

Handlers

Utility method for setting empty handlers

ManagedCResource

A custom struct for managing C resources with automatic cleanup.

NoOpAeronIdleStrategyFunc

PthreadCondS

PthreadInternalList

PthreadMutexS

RecordingPos

Enums

AeronErrorType

AeronSystemCounterType

CResource

ControlMode

Enum for control modes.

Media

Enum for media types.

Constants

AERON_DIR_PROP_NAME

AERON_IPC_MEDIA

AERON_UDP_MEDIA

DRIVER_TIMEOUT_MS_DEFAULT

SOURCE_LOCATION_LOCAL

SOURCE_LOCATION_REMOTE

SPY_PREFIX

TAG_PREFIX

Statics

AERON_IPC_STREAM

Traits

AeronAgentStartFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronArchiveCredentialsChallengeSupplierFuncCallback

Callback to return encoded credentials given a specific encoded challenge.

AeronArchiveCredentialsEncodedCredentialsSupplierFuncCallback

Callback to return encoded credentials.

AeronArchiveCredentialsFreeFuncCallback

Callback to return encoded credentials so they may be reused or freed.

AeronArchiveDelegatingInvokerFuncCallback

Callback to allow execution of a delegating invoker to be run.

AeronArchiveRecordingDescriptorConsumerFuncCallback

Callback to return recording descriptors.

AeronArchiveRecordingSignalConsumerFuncCallback

Callback to return recording signals.

AeronArchiveRecordingSubscriptionDescriptorConsumerFuncCallback

Callback to return recording subscription descriptors.

AeronAvailableCounterCallback

Function called by aeron_client_t to deliver notifications that a counter has been added to the driver.

AeronAvailableImageCallback

Function called by aeron_client_t to deliver notifications that an aeron_image_t was added.

AeronBlockHandlerCallback

Callback for handling a block of messages being read from a log.

AeronCloseClientCallback

Function called by aeron_client_t to deliver notifications that the client is closing.

AeronControlledFragmentHandlerCallback

Callback for handling fragments of data being read from a log.

AeronCountersReaderForeachCounterFuncCallback

Function called by aeron_counters_reader_foreach_counter for each counter in the aeron_counters_reader_t.

AeronErrorHandlerCallback

The error handler to be called when an error occurs.

AeronErrorLogReaderFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronFragmentHandlerCallback

Callback for handling fragments of data being read from a log.

AeronIdleStrategyFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronLossReporterReadEntryFuncCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

AeronNewPublicationCallback

Function called by aeron_client_t to deliver notification that the media driver has added an aeron_publication_t or aeron_exclusive_publication_t successfully.

AeronNewSubscriptionCallback

Function called by aeron_client_t to deliver notification that the media driver has added an aeron_subscription_t successfully.

AeronNotificationCallback

Generalised notification callback.

AeronPublicationErrorFrameHandlerCallback

The error frame handler to be called when the driver notifies the client about an error frame being received. The data passed to this callback will only be valid for the lifetime of the callback. The user should use aeron_publication_error_values_copy if they require the data to live longer than that.

AeronReservedValueSupplierCallback

Function called when filling in the reserved value field of a message.

AeronUnavailableCounterCallback

Function called by aeron_client_t to deliver notifications that a counter has been removed from the driver.

AeronUnavailableImageCallback

Function called by aeron_client_t to deliver notifications that an aeron_image_t has been removed from use and should not be used any longer.

AeronUriParseCallbackCallback

(note you must copy any arguments that you use afterwards even those with static lifetimes)

IntoCString

Functions

find_unused_udp_port

is_udp_port_available

Type Aliases

SourceLocation