An Azure service that provides a platform-managed, scalable, and highly available application delivery controller as a service.
The only thing that worked so far, is just trick the Application Gateway by creating another public IP address and assign it to Frontend IP configuration that way it forces Azure to release the default application load balancer and eventually delete the stale IP address.
There should be a better way to handle this!