Crate kernel

Scarlet Kernel

Scarlet is an operating system kernel written in Rust that implements a transparent ABI conversion layer for executing binaries across different operating systems and architectures. The kernel provides a universal container runtime environment with strong isolation capabilities and comprehensive filesystem support.

Multi-ABI Execution System

The core innovation of Scarlet is its ability to run binaries from different operating systems transparently within the same runtime environment:

ABI Module Architecture

• Modular ABI Implementation: Each ABI module implements its own complete syscall interface using shared kernel APIs, rather than translating between syscalls
• Binary Detection: Automatic identification of binary format and target ABI through ELF header analysis and magic number detection
• Shared Kernel Resources: All ABIs operate on common kernel objects (VFS, memory, devices) ensuring consistent behavior and efficient resource utilization
• Native Implementation: Each ABI provides full syscall implementation using underlying kernel abstractions, enabling complete OS compatibility

Supported ABIs

• Scarlet Native ABI: Direct kernel interface with optimal performance, featuring:

  • Handle-based resource management with capability-based security
  • Modern VFS operations with namespace isolation
  • Advanced IPC mechanisms including pipes and shared memory
  • Container-native filesystem operations

• Linux Compatibility ABI (in development): Full POSIX syscall implementation

• xv6 Compatibility ABI (in development): Educational OS syscall implementation

Container Runtime Environment

Scarlet provides enterprise-grade containerization features:

Filesystem Isolation

• Mount Namespace Isolation: Per-task filesystem namespaces enabling complete isolation
• Bind Mount Operations: Selective resource sharing between containers
• Overlay Filesystem: Copy-on-write semantics with whiteout support for efficient layering
• Device File Management: Controlled access to hardware through DevFS integration

Resource Management

• Handle-Based Security: Capability-based access control with fine-grained permissions
• Memory Isolation: Per-task memory spaces with controlled sharing mechanisms
• Task Lifecycle Management: Complete process management with environment variable support
• IPC Mechanisms: Pipes, shared memory, and other inter-process communication primitives

Virtual File System v2

Scarlet implements a modern VFS architecture designed for container environments:

Core Architecture

• VfsEntry: Path hierarchy cache providing fast O(1) path resolution with automatic cleanup
• VfsNode: Abstract file entity interface with metadata access and clean downcasting
• FileSystemOperations: Unified driver API consolidating all filesystem operations
• Mount Tree Management: Hierarchical mount point management with O(log n) resolution

Filesystem Drivers

• TmpFS: High-performance memory-based filesystem with configurable size limits

• CpioFS: Read-only CPIO archive filesystem optimized for initramfs and embedded data

• OverlayFS: Advanced union filesystem with copy-up semantics and whiteout support

• DevFS: Device file system providing controlled hardware access

• Memory Safety: Prevention of use-after-free, double-free, and data races at compile time:

  • The type system ensures resources are not used after being freed
  • Mutable references are exclusive, preventing data races
  • Lifetimes ensure references do not outlive the data they point to

• Trait-based Abstractions: Common interfaces for device drivers and subsystems enabling modularity:

  • The BlockDevice trait defines operations for block-based storage
  • The SerialDevice trait provides a common interface for UART and console devices
  • The FileSystem trait provides unified filesystem operations for VFS v2 integration

Boot Process

Scarlet follows a structured initialization sequence:

1. Early Architecture Init: CPU feature detection, interrupt vector setup
2. FDT Parsing: Hardware discovery from Flattened Device Tree
3. Memory Subsystem: Heap allocator initialization, virtual memory setup
4. Device Discovery: Platform device enumeration and driver binding
5. Interrupt Setup: CLINT/PLIC initialization for timer and external interrupts
6. VFS Initialization: Mount root filesystem, initialize global VFS manager
7. Task System: Scheduler setup, initial task creation
8. User Space Transition: Load initial programs and switch to user mode

Each stage validates successful completion before proceeding, with detailed logging available through the early console interface.

System Integration

Core Subsystems

• Task Management: Complete process lifecycle with environment variables and IPC
• Memory Management: Virtual memory with per-task address spaces and shared regions
• Device Framework: Unified device interface supporting block, character, and platform devices
• Interrupt Handling: Event-driven architecture with proper context switching
• Handle System: Capability-based resource access with fine-grained permissions

ABI Module Integration

Each ABI module integrates with the kernel through standardized interfaces:

• Binary Loading: ELF loader with format detection and validation
• Syscall Dispatch: Per-ABI syscall tables with transparent routing
• Resource Management: Shared kernel object access through common APIs
• Environment Setup: ABI-specific process initialization and cleanup
• Mount Operations: mount(), umount(), pivot_root() for dynamic filesystem management
• Process Management: execve(), fork(), wait(), exit() with proper cleanup
• IPC Operations: Pipe creation, communication, and resource sharing

Architecture Support

Currently implemented for RISC-V 64-bit architecture with comprehensive hardware support:

• Interrupt Handling: Complete trap frame management with timer and external interrupts
• Memory Management: Virtual memory with page tables and memory protection
• SBI Interface: Supervisor Binary Interface for firmware communication
• Instruction Abstractions: RISC-V specific optimizations with compressed instruction support

Rust Language Features

Scarlet leverages Rust’s advanced features for safe and efficient kernel development:

Memory Safety

• Zero-cost Abstractions: High-level constructs compile to efficient machine code
• Ownership System: Automatic memory management without garbage collection overhead
• Lifetime Validation: Compile-time prevention of use-after-free and dangling pointer errors
• Borrowing Rules: Exclusive mutable access prevents data races at compile time
• No Buffer Overflows: Array bounds checking and safe pointer arithmetic

Type System Features

• Trait-based Design: Generic programming with zero-cost abstractions for device drivers
• Pattern Matching: Exhaustive matching prevents unhandled error cases
• Option/Result Types: Explicit error handling without exceptions or null pointer errors
• Custom Test Framework: #[test_case] attribute for no-std kernel testing
• Const Generics: Compile-time array sizing and type-level programming

No-std Environment

• Embedded-first Design: No standard library dependency for minimal kernel footprint
• Custom Allocators: Direct control over memory allocation strategies
• Inline Assembly: Direct hardware access when needed with type safety
• Custom Panic Handler: Controlled kernel panic behavior for debugging
• Boot-time Initialization: Static initialization and controlled startup sequence

Development Framework

Testing Infrastructure

Scarlet provides a comprehensive testing framework designed for kernel development:

#[test_case]
fn test_vfs_operations() {
    // Kernel unit tests run in privileged mode
    let vfs = VfsManager::new();
    // ... test implementation
}

• Custom Test Runner: #[test_case] attribute for kernel-specific testing
• No-std Testing: Tests run directly in kernel mode without standard library
• Integration Tests: Full subsystem testing including multi-ABI scenarios
• Hardware-in-the-Loop: Testing on real hardware and QEMU emulation
• Performance Benchmarks: Kernel performance measurement and regression testing

Debugging Support

• Early Console: Serial output available from early boot stages
• Panic Handler: Detailed panic information with stack traces
• GDB Integration: Full debugging support through QEMU’s GDB stub
• Memory Debugging: Allocation tracking and leak detection
• Tracing: Event tracing for performance analysis and debugging

Build System Integration

The kernel integrates with cargo-make for streamlined development:

• cargo make build: Full kernel build with user programs
• cargo make test: Run all kernel tests
• cargo make debug: Launch kernel with GDB support
• cargo make run: Quick development cycle execution

Entry Points

The kernel provides multiple entry points for different scenarios:

• start_kernel(): Main bootstrap processor initialization
• start_ap(): Application processor startup for multicore systems
• test_main(): Test framework entry point when built with testing enabled

Module Organization

Core kernel modules provide focused functionality:

• abi/: Multi-ABI implementation modules (Scarlet Native, Linux, xv6)
• arch/: Architecture-specific code (currently RISC-V 64-bit)
• drivers/: Hardware device drivers (UART, block devices, VirtIO)
• fs/: Filesystem implementations and VFS v2 core
• task/: Task management, scheduling, and process lifecycle
• mem/: Memory management, allocators, and virtual memory
• syscall/: System call dispatch and implementation
• object/: Kernel object system with handle management
• interrupt/: Interrupt handling and controller support

Note: Currently, Scarlet Native ABI is fully implemented. Linux and xv6 ABI support are under development and will be available in future releases.

Modules

abi

ABI module.

arch

Architecture-specific code for Scarlet kernel

device

Device module.

drivers

Device drivers module.

earlycon

Early console for generic architecture.

environment

executor

Transparent Executor Module

fs

Virtual File System (VFS) Module - Version 2 Architecture

initcall

Initcall System

interrupt

Interrupt management system

ipc

Inter-Process Communication (IPC) module

library

Library module for the kernel.

mem

Memory management module.

object

Kernel object management system

sched

Scheduler module.

sync

Synchronization primitives module

syscall

System call interface module.

task

Task module.

time

Time utilities for the kernel

timer

Kernel timer module.

traits

vm

Virtual memory module.

Macros

defer

Macro to defer execution of a block of code. This macro allows you to specify a block of code that will be executed when the current scope is exited. It is similar to the defer function but provides a more concise syntax.

driver_initcall

A macro used to register driver initialization functions to be called during the system boot process.

early_initcall

early_print

early_println

late_initcall

print

println

register_abi

Functions

panic 🔒

A panic handler is required in Rust, this is probably the most basic one possible

start_ap

start_kernel