EDGE-BASED LLM INFERENCE FOR SMART CITIES

multi-modal reasoning, and autonomous decision support, making them compelling candidates for smart city management systems. However, the multi-billion parameter scale of frontier LLMs renders cloud-exclusive inference architectures inadequate for latency-sensitive smart city services including real-time emergency dispatch reasoning, adaptive traffic signal control, and citizen query resolution, where round-trip cloud latency and data sovereignty constraints together prohibit centralized processing. This paper proposes the Edge-Based LLM Inference Framework for Smart Cities (ELIS-C) that enables low-latency, privacy-preserving, context-aware LLM inference across a hierarchical edge-fog-cloud continuum through three principal innovations. First, a speculative decoding pipeline paired with a city-domain-fine-tuned small language model (SLM) draft network accelerates token generation at edge nodes by a factor of 3.1x compared to standard autoregressive decoding while maintaining output equivalence with the full target LLM. Second, an Adaptive Model Partitioning Engine (AMPE) dynamically splits transformer layers between edge nodes and fog servers based on real-time resource telemetry, query complexity classification, and data sensitivity tags, minimizing end-to-end time-to-first token (TTFT) latency. Third, a Retrieval-Augmented Generation (RAG) module grounded in a continuously updated smart city knowledge graph incorporating live sensor feeds, municipal databases, and real-time event streams ensures factually grounded LLM responses without cloud transmission of raw citizen data. Security provisions include prompt injection detection, differential privacy for context retrieval, and hardware-attested model integrity. Experimental evaluation across three smart city LLM application domains demonstrate a 52% reduction in median TTFT latency, a 61% reduction in cloud backhaul bandwidth, a 94.3% response factual accuracy, and full compliance with GDPR data residency requirements, significantly outperforming cloud-only and static edge partition baselines.