Background Fatty acid binding protein 5 (FABP5) is implicated in hepatocellular carcinoma (HCC) progression and represents a potential therapeutic target.Methods We integrated machine learning-based virtual screening, molecular docking and molecular dynamics simulations to identify natural compounds with high binding affinity to FABP5. Candidate compounds were further validated by in-vitro assays in HCC cell lines, including proliferation, migration/invasion, apoptosis/ferroptosis-related readouts, and mechanistic validation.Results The optimized models enabled efficient screening of natural products and prioritized gamma-linolenic acid (GLA) as a top candidate FABP5 inhibitor. Docking and simulations supported stable binding and key residue interactions. Experimentally, GLA inhibited HCC cell proliferation and aggressiveness and promoted cell death-related pathways consistent with anti-tumor activity.Conclusion Our deep learning-guided workflow identified gamma-linolenic acid as a natural FABP5 inhibitor and supports its potential as a lead compound for HCC therapy.