Carbon-Aware Reinforcement Learning for Cost-Optimal Energy Management of Hydrogen-Electric Heavy-Duty Trucks
DOI:
https://doi.org/10.70088/svdegw68Keywords:
Hydrogen-electric heavy-duty truck, Carbon pricing, Reinforcement learning, Energy management system, Soft Actor-Critic, Low-carbon transportationAbstract
The decarbonization of heavy-duty road transportation is a critical component of global carbon neutrality strategies, particularly under the increasing adoption of carbon credit pricing mechanisms such as emissions trading systems. Hydrogen-electric hybrid heavy-duty trucks (H2-E HDTs) offer a promising pathway toward low-carbon freight transport; however, their operational cost-effectiveness strongly depends on coordinated energy management decisions that jointly account for energy prices and carbon emission costs. In this paper, we propose a Carbon-aware Reinforcement Learning based Cost-optimal Energy Management System (CRL-CEMS) for hydrogen-electric heavy-duty trucks operating under carbon pricing environments. The proposed framework explicitly integrates dynamic carbon credit prices into the reinforcement learning decision process, enabling adaptive optimization of hydrogen and electricity usage with respect to both energy economics and carbon economics. A real-time carbon emission estimation model is developed to map hydrogen and electricity consumption into monetary carbon costs based on upstream carbon intensities. The energy management problem is formulated as a continuous control task and solved using the Soft Actor-Critic algorithm, where the agent determines the optimal power split between the fuel cell and battery systems. Extensive simulation experiments are conducted under multiple driving cycles and carbon price scenarios. Simulation results show that CRL-CEMS reduces the total operating cost by 7.2% and lowers carbon emissions by 12.3% compared with a carbon-unaware reinforcement learning baseline, while maintaining stable battery operation. These findings demonstrate that carbon-aware reinforcement learning effectively balances economic and environmental objectives, providing a practical approach for low-carbon, cost-optimal hydrogen-electric heavy-duty truck operation and supporting intelligent fleet energy management under dynamic carbon pricing scenarios.References
Y. Gao and M. Ehsani, “Parametric design of the traction motor and energy storage for series hybrid off-road and military vehicles,” IEEE Transactions on Power Electronics, vol. 21, no. 3, pp. 749–755, 2006.
A. Sciarretta, M. Back, and L. Guzzella, “Optimal control of parallel hybrid electric vehicles,” IEEE Transactions on Control Systems Technology, vol. 12, no. 3, pp. 352–363, 2004.
H. Borhan, A. Vahidi, A. M. Phillips, et al., “MPC-based energy management of a power-split hybrid electric vehicle,” IEEE Transactions on Control Systems Technology, vol. 20, no. 3, pp. 593–603, 2011.
X. Lü, Y. Wu, J. Lian, et al., “Energy management of hybrid electric vehicles: A review of energy optimization of fuel cell hybrid power system based on genetic algorithm,” Energy Conversion and Management, vol. 205, p. 112474, 2020.
T. Liu, X. Hu, S. E. Li, et al., “Reinforcement learning optimized look-ahead energy management of a parallel hybrid electric vehicle,” IEEE/ASME Transactions on Mechatronics, vol. 22, no. 4, pp. 1497–1507, 2017.
G. Du, Y. Zou, X. Zhang, et al., “Deep reinforcement learning based energy management for a hybrid electric vehicle,” Energy, vol. 201, p. 117591, 2020.
T. Haarnoja, A. Zhou, P. Abbeel, et al., “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in International Conference on Machine Learning, PMLR, 2018, pp. 1861–1870.
C. Zheng, D. Zhang, Y. Xiao, et al., “Reinforcement learning-based energy management strategies of fuel cell hybrid vehicles with multi-objective control,” Journal of Power Sources, vol. 543, p. 231841, 2022.
Z. E. Liu, Y. Li, Q. Zhou, et al., “Deep reinforcement learning-based energy management for heavy duty HEV considering discrete-continuous hybrid action space,” IEEE Transactions on Transportation Electrification, vol. 10, no. 4, pp. 9864–9876, 2024.
F. Creutzig, P. Jochem, O. Y. Edelenbosch, et al., “Transport: A roadblock to climate change mitigation?,” Science, vol. 350, no. 6263, pp. 911–912, 2015.
Q. S. Khalid et al., “An emission-capacitated vehicle routing model for sustainable urban waste collection using hybrid guided local search,” Scientific Reports, 2026.
L. Zhang et al., “Research on the orderly charging and discharging mechanism of electric vehicles considering travel characteristics and carbon quota,” IEEE Transactions on Transportation Electrification, vol. 10, no. 2, pp. 3012–3027, 2023.
P. Fan, H. Li, and M. Hu, “Profit-Oriented Production and Pricing Optimization for Manufacturing Enterprises Using Proximal Policy Optimization,” Economics and Management Innovation, vol. 3, no. 2, pp. 8–17, 2026.
Z. Zhong and Y. Huang, “CRE-CNN-LSTM-XAI: An Explainable Deep Learning Framework for Risk Assessment of Real Estate Collateralized Financial Products,” Journal of Computer Science and Frontier Technologies, vol. 3, no. 1, pp. 127–140, 2026.
X. Y. He and K. C. Liu, “FRAdRec: A Federated Real-Time Advertising Recommendation Framework Based on User Behavior Modeling and Cloud-Native Data Infrastructure,” Journal of Computer Science and Frontier Technologies, vol. 3, no. 2, pp. 102–117, 2026.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jiawei Li, Yue Luo, Yunting Ling (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
