Helicopter aerial refueling refers to the process of refueling a helicopter in mid-flight with the aid of a tanker aircraft. This maneuver is particularly challenging due to 1) complex aerodynamic interactions between the helicopter, the tanker, and the refueling hose-drogue system, 2) high pilot workload, 3) strict safety constraints, and 4) the contact-critical nature of the operation. To address these challenges, we propose a novel autonomous control methodology that combines model-based control with data-driven approaches such as reinforcement learning (RL). Given that aerial refueling is a safety-critical operation, it is essential to guarantee stability and performance for this hybrid control system. This talk will explain how Lyapunov stability and ultimate boundedness can be leveraged to establish rigorous stability and performance guarantees while accounting for controller parameters and uncertainties in the drogue dynamics.