This post-mortem deals with a recent outage on the Porter platform on the 2nd of October 2024, which rendered the platform inaccessible for users. This incident did not affect user workloads, and only affected our users' ability to access the Porter dashboard as well as run builds.
In line with our SOC 2 compliance policy, we planned for pushing out some new HTTP security headers for our platform, in order to boost our general security posture. At approximately 11:59am UTC, a few changes were pushed out to the dashboard, adding support for content security policy, permissions policy and access control headers. Once the change was pushed out, a degradation in site accessibility was immediately noted, and an incident was declared at 12:02pm UTC.
While it was clear that the new headers were breaking user access, it wasn’t immediately clear which header was the cause. We attempted to tweak our content security policy and access control headers and realised that this needed more time to understand which combination of header values worked best for the platform. Since user access was affected, we elected to simply revert all the changes. This revert was issued at 12:03pm UTC.
Once the revert was issued, we were able to see the platform come back online, and were able to access the dashboard by 12:04pm UTC. We decided to wait a few minutes and ensure the platform was fully back up, before declaring this incident as resolved.