Skip to content

tags: - cloud-computing - kubernetes - devops - internal-tools - atlassian title: Reflection on 8 Years at Atlassian: Building Self‑Service Load Balancing Platform original_url: https://www.youtube.com/watch?v=55pTFVoclvE

Reflection on 8 Years at Atlassian: Building Self‑Service Load Balancing Platform

Introduction
The author recounts an eight‑year journey at Atlassian, focusing on the technical challenges of creating a self‑service provisioning platform for internal developers. The narrative covers the hiring process, the design and implementation of an open service broker, the use of Envoy as a dynamic proxy, and the broader organizational impact of centralizing load balancing and related operational concerns.

Interview Experience and Hiring Process

  • The interview began with a HackerRank coding quiz, which the author aced with full marks.
  • The first technical round involved a white‑paper discussion on Cloudflare custom domains, followed by questions on microservices, containers, and architectural concepts.
  • A troubleshooting exercise simulated a real‑world denial‑of‑service incident, testing the ability to gather information and reason under pressure.
  • The values interview included a forward‑looking question: what would the hiring manager need to see in 12 months to consider the hire successful? The answer referenced building a self‑service load‑balancing framework for Atlassian’s internal developers.

Building the Open Service Broker

  • The initial task was to create an open service broker that could provision resources (e.g., databases, load balancers) via an API.
  • The author first experimented with a Python library (connection) that generated API handlers from an OpenAPI specification, then migrated to Flask and finally to FastAPI for the production implementation.
  • A worker process consumed provisioning requests from an SQS queue, performed asynchronous tasks (DNS record creation, CloudFront distribution setup, API calls), and recorded status in a DynamoDB‑backed database.
  • Clients polled the API for completion status; the worker updated the database, and the web layer returned success or error responses.

Proxy Infrastructure and Envoy Management

  • To replace costly enterprise load balancers, the team adopted Envoy as an open‑source, cloud‑native proxy.
  • A dedicated management server, called Sovereign, was built using FastAPI. It rendered dynamic Envoy configuration from templates and contextual data sourced from a database, an S3 bucket, and other services.
  • Proxies were provisioned via AWS CloudFormation templates that defined VPCs, subnets, security groups, auto‑scaling groups, and AMIs.
  • An AMI was generated with HashiCorp Packer and SaltStack, installing and hardening Envoy, logging agents, security tools, and observability components.
  • The resulting long‑lived infrastructure allowed developers to request load‑balancing resources through the broker, with the management server dynamically updating Envoy configurations based on the latest state.

Migration and Centralized Load Balancing

  • After establishing the foundation, the team migrated core Atlassian products (Jira, Confluence, Bitbucket, Status Page, etc.) onto the new edge infrastructure.
  • This shift enforced a policy where services could only be publicly exposed through the centralized load‑balancing layer, improving security and consistency.
  • The platform’s dynamic configuration enabled per‑service routing, header manipulation, redirects, and direct responses, handling the complexity of multi‑tenant traffic patterns.

Non‑Technical Learnings

  • Diplomacy and conflict resolution: Exposure to diverse manager and teammate styles fostered stronger communication, persuasion, and mentorship skills.
  • Maintenance mindset: Over time, the author emphasized proactive monitoring, on‑call readiness, and the importance of clear documentation to aid new team members.
  • Code churn and coupling: Recognizing areas of high churn signaled emerging complexity; early refactoring and modular design helped mitigate long‑term maintenance burdens.
  • Mentorship challenges: While the author excelled at explaining concepts, balancing guidance with autonomy proved difficult, especially without personal mentorship experience.

Conclusion

The eight‑year journey at Atlassian illustrates how a blend of thoughtful architecture, robust tooling (FastAPI, Envoy, CloudFormation, Packer), and strong non‑technical practices can transform internal development workflows. By centralizing load balancing and provisioning, the team delivered scalable, secure services to a wide range of products while navigating the human complexities of mentorship, conflict, and evolving codebases. The experience underscores the value of building reusable platforms that empower developers, reduce operational overhead, and create a foundation for future innovation.