Abstract: A distributed artificial intelligence inference system comprising a hierarchical multi-tier architecture with device, edge, metro, and data center tiers. A workload distribution controller computes a composite routing score for each inference task based on computational complexity, latency requirement, privacy classification, data volume, device resource state, and network conditions, and selects a target processing tier accordingly. A privacy classification engine assigns sensitivity levels to data elements, constraining eligible processing tiers. Multiple inference pipelines are co-located on shared edge nodes and exchange data via local shared memory. A model-architecture-specific state compression engine compresses session state using methods selected based on whether the inference pipeline employs a state-space model, transformer, or convolutional neural network architecture.