13th December 2021
Improved Health Checks and Networking
We have been working hard on building tooling to alleviate the limitations of Kubernetes health-checks, Kubernetes endpoint slices and Istio. This increases the reliability during blue green deployments, free service continuous deployment and correctness of health-check routing for start-up, readiness and liveness probes.
Northflank has developed a Kubernetes admission controller to enable graceful termination of user workloads by forcing DNS propagation before service termination - allowing in-progress requests to complete and preventing new requests to soon-to-be terminated containers, solely routing to healthy containers.
Northflank’s service mesh will auto-retry three times before failing a request to maintain service uptime during redeployments. We have identified a number of factors that could interfere with request routing:
- Avoid transient failures - a client should not perceive a failed request if a service was momentarily unavailable.
- Expect race conditions occurring in Kubernetes state vs DNS configuration - this often could be just a few milliseconds.
- Workload stability - load-balance retries across available workload instances.
Further UX improvements have been made:
- Improved real-time health check propagation via Prometheus metrics into the Northflank UI
- When configuring an HTTP liveness probe a readiness probe will automatically be generated
- New health checks descriptions:
- Readiness Probe - ensures a new container is ready to serve traffic before terminating an old container and handling new requests
- Liveness Probe - will restart containers that are failing checks
- Startup Probe - will delay readiness and liveness probes until checks are passed
Other features & fixes
- Introduced a new instance selector that is now applied across the whole UI
- Added a new secret group type
Build & Environment
which is fetched both during build and deployment phases - Fixed an issue where very large repositories would take a long time to build
- Fixed an issue where an empty project dashboard would render strangely on Safari
- Fixed an issue where API endpoint
assign service to subdomain
would error - Fixed an issue where full DNS entry on Networking would not display service name as slug
- Fixed an issue where
heroku/buildpacks:20
with PHP and Laravel would not run due to an error with permissions - Improved changing ports and custom domains RBAC
- Fixed rendering of the team member role selector
- Fixed rendering custom CMD override if quotes were used in the command