Abstract: The present disclosure relates to a 5G communication system or a 6G communication system for supporting higher data rates beyond a 4G communication system such as long term evolution (LTE). The present disclosure provides a system and method for security protocol implemented in a communication network. The method performed by a first network entity includes determining based on a HUB being in a trusted area or in a non-trusted area, whether one or more network functions (NFs) are configured to perform security operations based on a generation of a message by a corresponding one or more NFs.