Senior Site Reliability Engineer, Observability

About UsChainlink Labs is one of the primary contributing developers of Chainlink, the industry-standard oracle platform bringing the capital markets onchain and powering the majority of decentralized finance. The Chainlink stack provides the essential data, interoperability, compliance, and privacy standards needed to power advanced blockchain use cases for institutional tokenized assets, Decentralized Finance (DeFi), payments, stablecoins, and more. Many of the world’s largest financial services institutions have also adopted Chainlink’s standards and infrastructure, including Swift, Euroclear, Mastercard, Fidelity International, UBS, ANZ, Aave, GMX, Lido, and many others.Chainlink Labs is a world-class team of over 600 developers, researchers, and capital markets experts, and has ranked among Fortune's Best Workplaces in Technology, Fortune's Best Medium Workplace, and the Top 100 Global Most Loved Workplaces. Learn more at chain.link or chainlinklabs.com.The Observability Team enables Chainlink development and empowers engineers to continue building and supporting crucial products and services that have a profound impact in the blockchain industry. Reliability is vital to the success of our company. As a Senior SRE, you will help us accelerate and enable other engineering teams by increasing self-service and decreasing cognitive load.This job would be perfect for someone who has a strong DevOps mentality, is passionate about building and maintaining a mature GitOps environment, and has experience focusing on observability. The entire engineering team is expanding, and you would have plenty of opportunities to build, learn, and grow.We all have different backgrounds and are determined to help you succeed no matter where you are or who you are. If you think you would do a great job at Chainlink, we are looking forward to speaking with you, even if you don't match 100% of the job requirements: those describe people we've usually had a great time working with, but they're not a tick-box exercise.Your ImpactBuild and orchestrate Modern OTEL-based Observability PlatformSupport multiple telemetry types, like metrics, logs and traces.Define and support modern governance in observability and problems at scale.Ensure reliability, security, and performance exceed our defined SLAsWork with engineers from across the company to help troubleshoot issues, deploy new products and services, and increase velocity while decreasing cognitive loadLead the design and deployment of monitoring/observability services to detect and alert the team of needed action.Ingest, aggregate, transform, and utilize data from a multitude of sources in our real time data pipeline.Oversee the availability, performance, and supportability of our observability infrastructure.Create processes around alert response operations and support the team to ensure the reliable delivery of oracle data.Make recommendations to ensure sufficient metrics are collected to create alerts with every new feature release.Champion reliability and security by taking the time to do your work right the first timeRequirements7+ years of relevant professional experience. You probably have worked on a devops, infrastructure, SRE, and/or platform team beforeAbility to develop software outside of the scope of typical infrastructure requirements and configurationsExperience programming in C, C++, Java, Python, Go, Perl, or RubyExpert knowledge in all aspects of designing, developing, and managing large real-time systemsExperience with monitoring and logging. You know how to export metrics using Prometheus, have built a Grafana dashboard or two, and have experience with a centralized logging solution like an ELK Stack, Splunk or Grafana Stack.Experience with distributed systems and container orchestration. You have maintained or even built Kubernetes clusters before and feel comfortable deploying completely new services on themStrong communication skills. You can give and receive constructive feedback, and you do not shy away from planning meetings and code reviewsDesired QualificationsExcitement for blockchain, Web 3.0, and similar decentralized technologies.Experience running any infrastructure in the blockchain/web3 spaceAbility to scale systems sustainably through mechanisms like automation, and evolve systems by pushing for changes that improve reliability and velocityExperience working remotely in a distributed teamA strong desire to grow and challenge yourself. We would expect you to constantly find ways to improve and automate services to reduce toilSome of the tools and services we use daily or almost daily are:AWS; Terraform/Terragrunt; Kubernetes, Calico and ArgoCD; Prometheus and Grafana; GitHub Actions; PackerWe expect you to be comfortable with most of those tools and very proficient in several of them.All roles with Chainlink Labs are global and remote-based. Unless otherwise stated, we ask that you try to overlap some working hours with Eastern Standard Time (EST).We carefully review all applications and aim to provide a response to every candidate within two weeks after the job posting closes. The closing date is listed on the job advert, so we encourage you to take the time to thoughtfully prepare your application. We want to fully consider your experience and skills, and you will hear from us regarding the status of your application shortly after the closing date.Commitment to Equal OpportunityChainlink Labs is an equal opportunity employer. All qualified applicants will receive equal consideration for employment in compliance with applicable laws, regulations, or ordinances. If you need assistance or accommodation due to a disability or special need when applying for a role or in our recruitment process, please contact us via this form.Global Data Privacy Notice for Job Candidates and ApplicantsInformation collected and processed as part of your Chainlink Labs Careers profile, and any job applications you choose to submit is subject to our Privacy Policy. By submitting your application, you are agreeing to our use and processing of your data as required.