Airbyte secures petabytes of data with Infisical
Airbyte is the leading open-source data integration platform with over 7,000 customers using it to operate their data every day. They partnered with Infisical to secure and automate secret management across their infrastructure.
North America
101-200 employees
$181.2M raised from Accel, Benchmark, and Coatue.
2 petabytes of data synced per month by Airbyte
Enhanced security posture across Airbyte's infrastructure
Significant engineering efficiency boost due to self-serve secret management approach
Features Highlighted
Infisical has automated and secured many of the manual secret management processes at Airbyte. Switching from an in-house solution to Infisical allowed us to boost our security while increasing engineering efficiency. Charles Giardina, VP of Engineering, Airbyte
Looking to improve your secret management processes? Talk to an expert
Challenge
Considering the exceptionally large volume of data that Airbyte processes for its customers (over 2 petabytes per month), security is a very important consideration for everyone internally. Achieving highest levels of security without compromising developer efficiency is not easy, so the engineering team at Airbyte reached out to Infisical citing the following challenges:Reducing security risks: Airbyte team must ensure that its security infrastructure exceeds industry standards. This includes maintaining tight access controls, having all-encompassing audit logs, embracing "Security Shift Left", and more. Switching from an internal tool: In the early days, Airbyte has created an internal secrets management tool around GCP Secret Manager. This tool lasted the company long enough but eventually could not keep up with the pace of growth and expanding needs of the engineering team. Automating manual secret management workflows: With a large engineering team, Airbyte needs to keep developers productive which includes providing self-serve secret management workflows, facilitating developer on/off-boarding, etc. Managing secrets across complex infrastructure: Given complex kubernetes-based infrastructure, previous in-house solution did not provide enough automations and integrations with external services. Having to build them out internally would be very time-consuming and distract the team from their main priorities. Secret sprawl: As infrastructure complexity increases and Airbyte undertakes new engineering projects, secret sprawl becomes a real problem (both from the security and efficiency standpoint). The team wanted to achieve single source of truth for their secrets across all of their infrastructure and engineering teams.
Solution
To solve the above problems, Airbyte partnered with Infisical to centralize its secrets management workflows and establish a single source of truth for infrastructure credentials. Some of the functionality that the team at Airbyte is using includes:Web Dashboard: Infisical's UI enabled Airbyte engineers to effortlessly manage secrets in different projects and environments in a self-serve way according to the predefined permissions. Infisical CLI: When developing locally, Airbyte engineers can use Infisical CLI to inject secrets into their local application environments – completely removing the need for .env files and reducing security risks that stem from having secrets on local machines. Kubernetes Operator: With Airbyte relying heavily on Kubernetes for managing its infrastructure, the team was able to utilize Infisical's Kubernetes Operator to easily propagate secrets to the right containers and automatically redeploy their applications. GCP & GitHub Integrations: Using Infisical's native integrations, Airbyte team is able to establish a single source of truth for their secrets across many platforms and tools they are using. GCP Native Authentication: Given GCP is the main cloud platform used by Airbyte, GCP Native Authentication by Infisical allowed the team to get rid of the secret-zero problem and simplify many secret management workflows.
Build vs Buy
Like many companies, Airbyte considered building instead of buying, and understandably so. With a highly technical and incredibly strong engineering team, there’s no doubt they have the skills. But the cost of building, combined with the cost of ownership of such critical software made neither financial nor practical sense.One approach that the team considered was to build required functionality on top of GCP Secret Manager, but that still required too much engineering time (e.g., building access requests, change approvals, integrations, CLI) and would potentially create security risks (due to sensitivity of secret management infrastructure).
Results
With the help of Infisical, Airbyte was able to increase operational efficiency and significantly enhance their security posture through centralized secrets management. On the other hand, achieving single source of truth for secrets enabled Airbyte's developers to move faster and be more productive.
Security Shift Left
Using Infisical to provide self-serve secret management workflows (depending on developers' permissions), Airbyte was able to shift developers into acting more securely, following responsible coding practices, and sharing secrets via encrypted channels. Infisical's main thesis is that "company can get the most secure product out there, but if it's not easy enough for their developers to use, they will find ways around it – causing unexpected security issues." And considering that, Infisical was able to deliver a truly simple and all-encompassing secret management solution for Airbyte's engineering team.
Starting with Infisical is simple, fast, and free.
PRODUCT
Secret Management
Secret Scanning
Share Secret
Pricing
Security
USE CASES
Infisical AgentKubernetesDynamic SecretsTerraformAnsibleJenkinsDockerAWS ECSGitLabGitHubSDKsDEVELOPERS
DocumentationChangelogAPI ReferenceStatusFeedback & RequestsCommunity SlackHow to contributeRESOURCES
Blog
Infisical vs Vault
Careers
Hiring
Forum
Open Source Friends
Customers
Company Handbook
Trust Center
LEGAL
Terms of Service
Privacy Policy
Subprocessors
Service Level Agreement
CONTACT
Team Email
Sales
Support