Machine learning models are increasingly used to predict ship fuel consumption from high-frequency operational data. However, the strong temporal autocorrelation of shipboard sensor measurements raises concerns regarding the validity of commonly adopted evaluation practices. Many existing studies rely on random data partitioning and cross-validation schemes that can introduce information leakage, resulting in overly optimistic performance estimates. This study examined the impact of the validation strategy on fuel consumption prediction using one-minute operational data collected from a bulk carrier over approximately 16 months. Gradient-boosted decision tree models (XGBoost and CatBoost) and deep learning architectures were evaluated using three validation schemes: random train-test splitting, blocked chronological hold-out, and rolling-window temporal evaluation. The results demonstrate that random splitting substantially inflates the predictive performance, with coefficients of determination exceeding 0.99, whereas temporally consistent validation reveals significantly reduced accuracy and performance degradation across unseen operating periods. Under realistic temporal testing, gradient-boosted models exhibit greater robustness than deep learning models, which exhibit higher sensitivity to distributional shifts. These findings highlight the critical importance of temporally aware validation for obtaining credible generalisation estimates in ship fuel consumption modelling.