Physics, Information and Inference: High-Dimensional Models under Structured Dependencies