What prompted that? A structured approach to prompt inversion

Prompt inversion, also known as Reverse Prompt Engineering, aims to reconstruct the original user prompt from the output of Large Language Models (LLMs). This task is increasingly recognized as crucial for AI transparency, interpretability, and security, enabling better understanding and auditing of model behavior. However, despite growing interest, reverse prompt engineering remains a nascent field, with relatively few approaches addressing the challenges of generating semantically meaningful and syntactically valid prompts, especially in black-box scenarios where internal model information is inaccessible. To address this gap, this work proposes a novel multi-stage pipeline that combines prompt-type classification, syntactic pattern extraction, sentiment analysis, and constraint-based prompt reconstruction. By integrating these components, the framework guides a LLM to produce faithful and interpretable reconstructions of user prompts using only the output responses. This approach leverages both semantic cues and grammatical structures to improve the plausibility and alignment of the reconstructed prompts with the original inputs. Experimental evaluation on the Alpaca-GPT4 dataset demonstrates high performance, achieving average cosine similarities of 0.78 for prompts and 0.73 for responses when comparing originals and reconstructions. Ablation studies validate the contribution of each pipeline component, while qualitative analysis sheds light on the strengths and current limitations of the method. Overall, the results suggest that combining semantic and syntactic constraints can significantly enhance prompt inversion in practical black-box settings.