The motivations behind 5G networks include seamless handovers, higher data rates, lower latencies of about one milli second, and enhanced coverage compared to 4G networks. To achieve these goals, network densification has been implemented to cope with increasing capacity demands. Networks with ultra-densification have large numbers of heterogeneous small cell deployments such as femto-cell s, relays and microcells which complicate mobility management, resulting in unnecessary, frequent, and ping-pong handovers as UEs move within the network. To address these challenges, state of the art approaches using fuzzy logic, adaptive neuro-networks or their combination have been proposed. However, these approaches majorly address the QoS issues, ignoring the security aspect of handovers. In this paper, a handover protocol that incorporates both security and QoS in the handover process is proposed. The simulation results showed that this protocol reduced handover latency, packet losses, number of executed handovers and ping pong rate by 56.1%, 38.8 %, 74.6% and 24.1% respectively. In additi on, the developed protocol yielded a 27.1% increase in the handover success rate, and a 27.3% reduction in handover fail ure rate. This protocol was also shown to be robust against de-synchronization and session hijacking attacks.