On the Configure (shared) Alerts page, there is an extra column provided which shows whether an alert is configured for webhook notification. You can select rows and click “Add Webhook” to activate the alert for webhooks, or click “Remove Webhook” to deactivate the alert for webhooks.
If you Add a Webhook, and the alert is not activated, it will be automatically activated. Removing the webhook, does not automatically deactivate an alert, however.