[{"data":1,"prerenderedAt":1088},["ShallowReactive",2],{"/blog/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass":3,"navigation-en-us":301,"banner-en-us":729,"footer-en-us":739,"blog-post-authors-en-us-Igor Drozdov":984,"blog-related-posts-en-us-why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass":999,"blog-promotions-en-us":1025,"next-steps-en-us":1078},{"id":4,"title":5,"authors":6,"body":8,"category":280,"date":281,"description":282,"extension":283,"externalUrl":284,"faq":284,"featured":285,"heroImage":286,"meta":287,"navigation":288,"path":289,"seo":290,"slug":294,"stem":295,"tags":296,"template":299,"updatedDate":284,"__hash__":300},"blogPosts/en-us/blog/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass.md","Why we implemented our own SSHD solution",[7],"Igor Drozdov",{"type":9,"value":10,"toc":268},"minimark",[11,22,27,52,90,95,102,106,112,116,119,134,138,141,247,251,254],[12,13,14,15,21],"p",{},"The story of why we moved to our own SSHD is an interesting one. GitLab provides ",[16,17,20],"a",{"href":18,"rel":19},"https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/doc/features.md",[],"a number of features"," that execute via an SSH connection. The most popular one is Git-over-SSH, which enables communicating with a Git server via SSH. Historically, we implemented the features through a combination of OpenSSH Server and a separate component, a binary called GitLab Shell. GitLab Shell processes every connection established by OpenSSH Server to communicate data back and forth between the SSH client and the Git server. The solution was battle-tested, and relied on a trusted component such as OpenSSH. Here's why we decided to implement our own SSHD.",[23,24,26],"h2",{"id":25},"community-contribution","Community contribution",[12,28,29,34,35,40,41,46,47,51],{},[16,30,33],{"href":31,"rel":32},"https://handbook.gitlab.com/handbook/company/mission/#mission",[],"Everyone can contribute at GitLab","! A ",[16,36,39],{"href":37,"rel":38},"https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/394",[],"community contribution"," from ",[16,42,45],{"href":43,"rel":44},"https://gitlab.com/lorenz",[],"@lorenz",", ",[48,49,50],"code",{},"gitlab-sshd",", was suggested as a lightweight alternative to our existing setup. A self-contained binary with minimal external dependencies would be beneficial for containerized deployments. A GitLab-supported replacement also opened up new opportunities:",[53,54,55,65,72,75,81,87],"ul",{},[56,57,58,59,64],"li",{},"PROXY protocol support to enable ",[16,60,63],{"href":61,"rel":62},"https://gitlab.com/gitlab-org/gitlab/-/issues/271673",[],"Group IP address restriction via SSH",": Group IP address restriction didn’t work for the Gitlab Shell + OpenSSH solution, because OpenSSH didn't provide PROXY protocol support. As a result, Gitlab Shell couldn't see the real users’ IP addresses. We had to either use a patched version of OpenSSH, or implement our own solution to support the PROXY protocol. Then, with our own solution, we could enable PROXY protocol, receive the real IP addresses of users, and provide Group IP address restriction functionality.",[56,66,67,68,71],{},"Kubernetes compatibility with graceful shutdown, liveness, and readiness probes: With OpenSSH, we had no control over established connections. When Kubernetes pods were rotated, all the ongoing connections were immediately dropped, and thus could interrupt long-running ",[48,69,70],{},"git clone"," operations. With a dedicated server, the connections now become manageable and can be shut down gracefully: the server listens for an interrupting signal and, when the signal is received, stops accepting new connections and waits for a grace period before shutting down completely. This grace period gives ongoing connections an opportunity to finish.",[56,73,74],{},"Prometheus metrics and profiling became possible: In our previous approach, Gitlab Shell was just a binary that created a process that lived as long as the SSH connection lived. This approach didn’t provide a straightforward way to run a metrics server. With a dedicated server, we can now collect metrics and implement detailed logging for monitoring and debugging purposes.",[56,76,77,78,80],{},"Performance and resource usage is significantly lower in some scenarios: Lightweight goroutines are cheaper than spawning a separate process for every SSH connection. Spawning separate processes performed better in basic cases, but our real-world scenarios didn't demonstrate a drastic performance improvement. However, with ",[48,79,50],{}," it became possible to introduce a Go profiler to surface performance problems, which was a significant improvement from an operating perspective.",[56,82,83,84,86],{},"Reduced attack surface by using only a restricted set of SSH implementation features: With the previous approach, we allowed establishing an SSH connection to the OpenSSH server, but restricted it to a specific feature set. With ",[48,85,50],{},", any unpredictable call to an OpenSSH feature that we don’t support is no longer possible, dramatically reducing the attack surface.",[56,88,89],{},"Simplified architecture is now easier to understand:",[91,92,94],"h3",{"id":93},"the-previous-architecture","The previous architecture",[12,96,97],{},[98,99],"img",{"alt":100,"src":101},"Old architecture","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398129/blog/Content%20Images/create-source-code/gitlab-sshd/old-architecture.png",[91,103,105],{"id":104},"our-current-architecture","Our current architecture",[12,107,108],{},[98,109],{"alt":110,"src":111},"New architecture","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398129/blog/Content%20Images/create-source-code/gitlab-sshd/new-architecture.png",[23,113,115],{"id":114},"risks-and-challenges","Risks and challenges",[12,117,118],{},"However, changing a critical component that is broadly used, and is responsible for security, carries tremendous risks. We experienced both challenges and risks:",[53,120,121,128],{},[56,122,123,127],{},[124,125,126],"strong",{},"Security perspective",": An SSH server is a critical component; it establishes a secure connection between a user and a server, and allows them to communicate privately. Any failures could open security holes, permitting anything from authentication bypass to remote code execution. To mitigate the risks, we performed multiple rounds of security reviews – before the development (by examining the used components), during the development, and after the working versions of the code were deployed to staging environments.",[56,129,130,133],{},[124,131,132],{},"Operational perspective",": The component is broadly used. Any failures would affect a vast number of users. To mitigate the risks, we rolled the changes out gradually to 1%, 5%, etc. of the traffic, and rolled back if a problem was encountered. After 8 attempts, we had the server successfully running for 100% of traffic!",[23,135,137],{"id":136},"problems-we-encountered","Problems we encountered",[12,139,140],{},"A component with a scope this broad could have a wide range of problems. We encountered, and resolved, the following problems:",[53,142,143,152,234],{},[56,144,145,148,149,151],{},[124,146,147],{},"Incompatibility with other in-progress features:"," Our first ",[48,150,50],{}," deployment consumed huge amounts of memory. It interacted negatively with another feature under development at the same time. We must always keep the interaction with other components in mind when introducing a general component.",[56,153,154,161,162,167,168],{},[124,155,156,157,160],{},"Limited feature set in the ",[48,158,159],{},"golang.org/x/crypto"," library:"," This library establishes SSH connection, and has limited support for algorithms and features available in OpenSSH. We created ",[16,163,166],{"href":164,"rel":165},"https://gitlab.com/gitlab-org/golang-crypto",[],"our own fork"," to provide the missing features:\n",[53,169,170,190,206],{},[56,171,172,173,176,177,183,184,189],{},"The OpenSSH client has deprecated SHA-1 based signatures in host certificates in version 8.2 because of safety reasons; however, backward compatibility is provided by the ",[48,174,175],{},"server-sig-algs"," extension, but ",[16,178,181],{"href":179,"rel":180},"https://pkg.go.dev/golang.org/x/crypto",[],[48,182,159],{}," didn't support it. We ",[16,185,188],{"href":186,"rel":187},"https://gitlab.com/gitlab-org/golang-crypto/-/merge_requests/1",[],"started supporting"," this extension.",[56,191,192,193,196,197,200,201,205],{},"Some MACs and key exchange algorithms are unsupported: ",[48,194,195],{},"hmac-sha2-512"," and ",[48,198,199],{},"hmac-sha2-256"," are the most noticeable. We ",[16,202,188],{"href":203,"rel":204},"https://gitlab.com/gitlab-org/golang-crypto/-/merge_requests/4",[]," these algorithms.",[56,207,208,209,196,212,215,216,219,220,223,224,227,228,233],{},"Buggy SSH clients, such as ",[48,210,211],{},"gpg-agent v2.2.4",[48,213,214],{},"OpenSSH v7.6"," shipped in ",[48,217,218],{},"Ubuntu 18.04",", might send ",[48,221,222],{},"ssh-rsa-512"," as the public key algorithm but actually include a ",[48,225,226],{},"rsa-sha"," signature. We had to ",[16,229,232],{"href":230,"rel":231},"https://gitlab.com/gitlab-org/golang-crypto/-/merge_requests/9",[],"relax the RSA signature check"," to resolve this issue.",[56,235,236,239,240,196,243,246],{},[124,237,238],{},"Re-implementing options available in OpenSSH:"," Familiar OpenSSH options like ",[48,241,242],{},"LoginGraceTime",[48,244,245],{},"ClientAliveInterval"," were unavailable, so we implemented multiple alternatives to preserve the features we needed.",[23,248,250],{"id":249},"lessons-learned","Lessons learned",[12,252,253],{},"Unfortunately, issues became visible on production environment, thanks both to the load and the variety of possible OpenSSH configurations. Even though we caught some bugs on our staging environments, predicting all types of problems was almost impossible. However, these actions helped us resolve the issues:",[53,255,256,262],{},[56,257,258,261],{},[124,259,260],{},"Incremental rollouts:"," The rollout plan proved to be extremely effective. It enabled us to iterate without disrupting service to most users.",[56,263,264,267],{},[124,265,266],{},"Seeking multiple perspectives:"," We sought a diverse set of opinions from a variety of groups, such as Security, Infrastructure, Quality and Scalability. It helped us to evaluate the project from multiple perspectives, mitigate the risks, and prevent the majority of issues from happening.",{"title":269,"searchDepth":270,"depth":270,"links":271},"",2,[272,277,278,279],{"id":25,"depth":270,"text":26,"children":273},[274,276],{"id":93,"depth":275,"text":94},3,{"id":104,"depth":275,"text":105},{"id":114,"depth":270,"text":115},{"id":136,"depth":270,"text":137},{"id":249,"depth":270,"text":250},"engineering","2022-08-17","Until recently we used OpenSSH Server to handle SSH connections to provide SSH-related features, but we ultimately decided to implement our own SSHD solution. Learn more!","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749669673/Blog/Hero%20Images/engineering.png",{},true,"/en-us/blog/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass",{"title":5,"description":282,"ogTitle":5,"ogDescription":282,"noIndex":285,"ogImage":286,"ogUrl":291,"ogSiteName":292,"ogType":293,"canonicalUrls":291},"https://about.gitlab.com/blog/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass","https://about.gitlab.com","article","why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass","en-us/blog/why-we-have-implemented-our-own-sshd-solution-on-gitlab-sass",[297,298],"production","security","BlogPost","Be9xqHvzCPRC-2_fyRmcaIR_PwPq5fgjB373TmjxqoU",{"logo":302,"freeTrial":307,"sales":312,"login":317,"items":322,"search":649,"minimal":680,"duo":699,"switchNav":708,"pricingDeployment":719},{"config":303},{"href":304,"dataGaName":305,"dataGaLocation":306},"/","gitlab logo","header",{"text":308,"config":309},"Get free trial",{"href":310,"dataGaName":311,"dataGaLocation":306},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":313,"config":314},"Request a demo",{"href":315,"dataGaName":316,"dataGaLocation":306},"/sales/?contact-topic=request-demo","sales",{"text":318,"config":319},"Sign in",{"href":320,"dataGaName":321,"dataGaLocation":306},"https://gitlab.com/users/sign_in/","sign in",[323,352,452,457,571,627],{"text":324,"config":325,"menu":327},"Platform",{"dataNavLevelOne":326},"platform",{"type":328,"columns":329},"cards",[330,336,344],{"title":324,"description":331,"link":332},"The intelligent orchestration platform for DevSecOps",{"text":333,"config":334},"Explore our Platform",{"href":335,"dataGaName":326,"dataGaLocation":306},"/platform/",{"title":337,"description":338,"link":339},"GitLab Duo Agent Platform","Agentic AI for the entire software lifecycle",{"text":340,"config":341},"Meet GitLab Duo",{"href":342,"dataGaName":343,"dataGaLocation":306},"/gitlab-duo-agent-platform/","gitlab duo agent platform",{"title":345,"description":346,"link":347},"Why GitLab","See the top reasons enterprises choose GitLab",{"text":348,"config":349},"Learn more",{"href":350,"dataGaName":351,"dataGaLocation":306},"/why-gitlab/","why gitlab",{"text":353,"left":288,"config":354,"menu":356},"Product",{"dataNavLevelOne":355},"solutions",{"type":357,"link":358,"columns":362,"feature":431},"lists",{"text":359,"config":360},"View all Solutions",{"href":361,"dataGaName":355,"dataGaLocation":306},"/solutions/",[363,387,410],{"title":364,"description":365,"link":366,"items":371},"Automation","CI/CD and automation to accelerate deployment",{"config":367},{"icon":368,"href":369,"dataGaName":370,"dataGaLocation":306},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[372,376,379,383],{"text":373,"config":374},"CI/CD",{"href":375,"dataGaLocation":306,"dataGaName":373},"/solutions/continuous-integration/",{"text":337,"config":377},{"href":342,"dataGaLocation":306,"dataGaName":378},"gitlab duo agent platform - product menu",{"text":380,"config":381},"Source Code Management",{"href":382,"dataGaLocation":306,"dataGaName":380},"/solutions/source-code-management/",{"text":384,"config":385},"Automated Software Delivery",{"href":369,"dataGaLocation":306,"dataGaName":386},"Automated software delivery",{"title":388,"description":389,"link":390,"items":395},"Security","Deliver code faster without compromising security",{"config":391},{"href":392,"dataGaName":393,"dataGaLocation":306,"icon":394},"/solutions/application-security-testing/","security and compliance","ShieldCheckLight",[396,400,405],{"text":397,"config":398},"Application Security Testing",{"href":392,"dataGaName":399,"dataGaLocation":306},"Application security testing",{"text":401,"config":402},"Software Supply Chain Security",{"href":403,"dataGaLocation":306,"dataGaName":404},"/solutions/supply-chain/","Software supply chain security",{"text":406,"config":407},"Software Compliance",{"href":408,"dataGaName":409,"dataGaLocation":306},"/solutions/software-compliance/","software compliance",{"title":411,"link":412,"items":417},"Measurement",{"config":413},{"icon":414,"href":415,"dataGaName":416,"dataGaLocation":306},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[418,422,426],{"text":419,"config":420},"Visibility & Measurement",{"href":415,"dataGaLocation":306,"dataGaName":421},"Visibility and Measurement",{"text":423,"config":424},"Value Stream Management",{"href":425,"dataGaLocation":306,"dataGaName":423},"/solutions/value-stream-management/",{"text":427,"config":428},"Analytics & Insights",{"href":429,"dataGaLocation":306,"dataGaName":430},"/solutions/analytics-and-insights/","Analytics and insights",{"title":432,"type":357,"items":433},"GitLab for",[434,440,446],{"text":435,"config":436},"Enterprise",{"icon":437,"href":438,"dataGaLocation":306,"dataGaName":439},"Building","/enterprise/","enterprise",{"text":441,"config":442},"Small Business",{"icon":443,"href":444,"dataGaLocation":306,"dataGaName":445},"Work","/small-business/","small business",{"text":447,"config":448},"Public Sector",{"icon":449,"href":450,"dataGaLocation":306,"dataGaName":451},"Organization","/solutions/public-sector/","public sector",{"text":453,"config":454},"Pricing",{"href":455,"dataGaName":456,"dataGaLocation":306,"dataNavLevelOne":456},"/pricing/","pricing",{"text":458,"config":459,"menu":461},"Resources",{"dataNavLevelOne":460},"resources",{"type":357,"link":462,"columns":466,"feature":560},{"text":463,"config":464},"View all resources",{"href":465,"dataGaName":460,"dataGaLocation":306},"/resources/",[467,500,527],{"title":468,"items":469},"Getting started",[470,475,480,485,490,495],{"text":471,"config":472},"Install",{"href":473,"dataGaName":474,"dataGaLocation":306},"/install/","install",{"text":476,"config":477},"Quick start guides",{"href":478,"dataGaName":479,"dataGaLocation":306},"/get-started/","quick setup checklists",{"text":481,"config":482},"Learn",{"href":483,"dataGaLocation":306,"dataGaName":484},"https://university.gitlab.com/","learn",{"text":486,"config":487},"Product documentation",{"href":488,"dataGaName":489,"dataGaLocation":306},"https://docs.gitlab.com/","product documentation",{"text":491,"config":492},"Best practice videos",{"href":493,"dataGaName":494,"dataGaLocation":306},"/getting-started-videos/","best practice videos",{"text":496,"config":497},"Integrations",{"href":498,"dataGaName":499,"dataGaLocation":306},"/integrations/","integrations",{"title":501,"items":502},"Discover",[503,508,513,518,522],{"text":504,"config":505},"Customer success stories",{"href":506,"dataGaName":507,"dataGaLocation":306},"/customers/","customer success stories",{"text":509,"config":510},"Blog",{"href":511,"dataGaName":512,"dataGaLocation":306},"/blog/","blog",{"text":514,"config":515},"Demo Hub",{"href":516,"dataGaName":517,"dataGaLocation":306},"/demo-hub/","demo hub",{"text":519,"config":520},"The Source",{"href":521,"dataGaName":512,"dataGaLocation":306},"/the-source/",{"text":523,"config":524},"Remote",{"href":525,"dataGaName":526,"dataGaLocation":306},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"title":528,"items":529},"Connect",[530,535,540,545,550,555],{"text":531,"config":532},"GitLab Services",{"href":533,"dataGaName":534,"dataGaLocation":306},"/services/","services",{"text":536,"config":537},"Contribute",{"href":538,"dataGaName":539,"dataGaLocation":306},"https://contributors.gitlab.com","contribute",{"text":541,"config":542},"Community",{"href":543,"dataGaName":544,"dataGaLocation":306},"/community/","community",{"text":546,"config":547},"Forum",{"href":548,"dataGaName":549,"dataGaLocation":306},"https://forum.gitlab.com/","forum",{"text":551,"config":552},"Events",{"href":553,"dataGaName":554,"dataGaLocation":306},"/events/","events",{"text":556,"config":557},"Partners",{"href":558,"dataGaName":559,"dataGaLocation":306},"/partners/","partners",{"config":561,"title":564,"text":565,"link":566},{"background":562,"textColor":563},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1777322348/qpq8yrgn8knii57omj0c.png')","#000","What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":567,"config":568},"Read the latest",{"href":569,"dataGaName":570,"dataGaLocation":306},"/whats-new/","whats new",{"text":572,"config":573,"menu":575},"Company",{"dataNavLevelOne":574},"company",{"type":357,"columns":576},[577],{"items":578},[579,584,590,592,597,602,607,612,617,622],{"text":580,"config":581},"About",{"href":582,"dataGaName":583,"dataGaLocation":306},"/company/","about",{"text":585,"config":586,"footerGa":589},"Jobs",{"href":587,"dataGaName":588,"dataGaLocation":306},"/jobs/","jobs",{"dataGaName":588},{"text":551,"config":591},{"href":553,"dataGaName":554,"dataGaLocation":306},{"text":593,"config":594},"Leadership",{"href":595,"dataGaName":596,"dataGaLocation":306},"/company/team/e-group/","leadership",{"text":598,"config":599},"Handbook",{"href":600,"dataGaName":601,"dataGaLocation":306},"https://handbook.gitlab.com/","handbook",{"text":603,"config":604},"Investor relations",{"href":605,"dataGaName":606,"dataGaLocation":306},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":608,"config":609},"Trust Center",{"href":610,"dataGaName":611,"dataGaLocation":306},"/security/","trust center",{"text":613,"config":614},"AI Transparency Center",{"href":615,"dataGaName":616,"dataGaLocation":306},"/ai-transparency-center/","ai transparency center",{"text":618,"config":619},"Newsletter",{"href":620,"dataGaName":621,"dataGaLocation":306},"/company/contact/#contact-forms","newsletter",{"text":623,"config":624},"Press",{"href":625,"dataGaName":626,"dataGaLocation":306},"/press/","press",{"text":628,"config":629,"menu":630},"Contact us",{"dataNavLevelOne":574},{"type":357,"columns":631},[632],{"items":633},[634,639,644],{"text":635,"config":636},"Talk to sales",{"href":637,"dataGaName":638,"dataGaLocation":306},"/sales/","talk to sales",{"text":640,"config":641},"Support portal",{"href":642,"dataGaName":643,"dataGaLocation":306},"https://support.gitlab.com/hc/en-us","support portal",{"text":645,"config":646},"Customer portal",{"href":647,"dataGaName":648,"dataGaLocation":306},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":650,"login":651,"suggestions":658},"Close",{"text":652,"link":653},"To search repositories and projects, login to",{"text":654,"config":655},"gitlab.com",{"href":320,"dataGaName":656,"dataGaLocation":657},"search login","search",{"text":659,"default":660},"Suggestions",[661,663,667,669,673,677],{"text":337,"config":662},{"href":342,"dataGaName":337,"dataGaLocation":657},{"text":664,"config":665},"Code Suggestions (AI)",{"href":666,"dataGaName":664,"dataGaLocation":657},"/solutions/code-suggestions/",{"text":373,"config":668},{"href":375,"dataGaName":373,"dataGaLocation":657},{"text":670,"config":671},"GitLab on AWS",{"href":672,"dataGaName":670,"dataGaLocation":657},"/partners/technology-partners/aws/",{"text":674,"config":675},"GitLab on Google Cloud",{"href":676,"dataGaName":674,"dataGaLocation":657},"/partners/technology-partners/google-cloud-platform/",{"text":678,"config":679},"Why GitLab?",{"href":350,"dataGaName":678,"dataGaLocation":657},{"freeTrial":681,"mobileIcon":686,"desktopIcon":691,"secondaryButton":694},{"text":682,"config":683},"Start free trial",{"href":684,"dataGaName":311,"dataGaLocation":685},"https://gitlab.com/-/trials/new/","nav",{"altText":687,"config":688},"Gitlab Icon",{"src":689,"dataGaName":690,"dataGaLocation":685},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":687,"config":692},{"src":693,"dataGaName":690,"dataGaLocation":685},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":695,"config":696},"Get Started",{"href":697,"dataGaName":698,"dataGaLocation":685},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":700,"mobileIcon":704,"desktopIcon":706},{"text":701,"config":702},"Learn more about GitLab Duo",{"href":342,"dataGaName":703,"dataGaLocation":685},"gitlab duo",{"altText":687,"config":705},{"src":689,"dataGaName":690,"dataGaLocation":685},{"altText":687,"config":707},{"src":693,"dataGaName":690,"dataGaLocation":685},{"button":709,"mobileIcon":714,"desktopIcon":716},{"text":710,"config":711},"/switch",{"href":712,"dataGaName":713,"dataGaLocation":685},"#contact","switch",{"altText":687,"config":715},{"src":689,"dataGaName":690,"dataGaLocation":685},{"altText":687,"config":717},{"src":718,"dataGaName":690,"dataGaLocation":685},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":720,"mobileIcon":725,"desktopIcon":727},{"text":721,"config":722},"Back to pricing",{"href":455,"dataGaName":723,"dataGaLocation":685,"icon":724},"back to pricing","GoBack",{"altText":687,"config":726},{"src":689,"dataGaName":690,"dataGaLocation":685},{"altText":687,"config":728},{"src":693,"dataGaName":690,"dataGaLocation":685},{"title":730,"titleMobile":731,"button":732,"config":737},"Duo Agent Platform delivers 400% ROI, per new Forrester Consulting study.","400% ROI: Forrester TEI for GitLab Duo",{"text":348,"config":733},{"href":734,"dataGaName":735,"dataGaLocation":736},"https://about.gitlab.com/blog/gitlab-duo-agent-platform-delivers-400-percent-roi/","forrester-tei-dap-banner","global-banner",{"layout":738,"disabled":285},"release",{"data":740},{"text":741,"source":742,"edit":748,"contribute":753,"config":758,"items":763,"minimal":973},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":743,"config":744},"View page source",{"href":745,"dataGaName":746,"dataGaLocation":747},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":749,"config":750},"Edit this page",{"href":751,"dataGaName":752,"dataGaLocation":747},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":754,"config":755},"Please contribute",{"href":756,"dataGaName":757,"dataGaLocation":747},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":759,"facebook":760,"youtube":761,"linkedin":762},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[764,811,865,909,941],{"title":453,"links":765,"subMenu":780},[766,770,775],{"text":767,"config":768},"View plans",{"href":455,"dataGaName":769,"dataGaLocation":747},"view plans",{"text":771,"config":772},"Why Premium?",{"href":773,"dataGaName":774,"dataGaLocation":747},"/pricing/premium/","why premium",{"text":776,"config":777},"Why Ultimate?",{"href":778,"dataGaName":779,"dataGaLocation":747},"/pricing/ultimate/","why ultimate",[781],{"title":782,"links":783},"Contact Us",[784,787,789,791,796,801,806],{"text":785,"config":786},"Contact sales",{"href":637,"dataGaName":316,"dataGaLocation":747},{"text":640,"config":788},{"href":642,"dataGaName":643,"dataGaLocation":747},{"text":645,"config":790},{"href":647,"dataGaName":648,"dataGaLocation":747},{"text":792,"config":793},"Status",{"href":794,"dataGaName":795,"dataGaLocation":747},"https://status.gitlab.com/","status",{"text":797,"config":798},"Terms of use",{"href":799,"dataGaName":800,"dataGaLocation":747},"/terms/","terms of use",{"text":802,"config":803},"Privacy statement",{"href":804,"dataGaName":805,"dataGaLocation":747},"/privacy/","privacy statement",{"text":807,"config":808},"Cookie preferences",{"dataGaName":809,"dataGaLocation":747,"id":810,"isOneTrustButton":288},"cookie preferences","ot-sdk-btn",{"title":353,"links":812,"subMenu":821},[813,817],{"text":814,"config":815},"DevSecOps platform",{"href":335,"dataGaName":816,"dataGaLocation":747},"devsecops platform",{"text":818,"config":819},"AI-Assisted Development",{"href":342,"dataGaName":820,"dataGaLocation":747},"ai-assisted development",[822],{"title":823,"links":824},"Topics",[825,830,835,840,845,850,855,860],{"text":826,"config":827},"CICD",{"href":828,"dataGaName":829,"dataGaLocation":747},"/topics/ci-cd/","cicd",{"text":831,"config":832},"GitOps",{"href":833,"dataGaName":834,"dataGaLocation":747},"/topics/gitops/","gitops",{"text":836,"config":837},"DevOps",{"href":838,"dataGaName":839,"dataGaLocation":747},"/topics/devops/","devops",{"text":841,"config":842},"Version Control",{"href":843,"dataGaName":844,"dataGaLocation":747},"/topics/version-control/","version control",{"text":846,"config":847},"DevSecOps",{"href":848,"dataGaName":849,"dataGaLocation":747},"/topics/devsecops/","devsecops",{"text":851,"config":852},"Cloud Native",{"href":853,"dataGaName":854,"dataGaLocation":747},"/topics/cloud-native/","cloud native",{"text":856,"config":857},"AI for Coding",{"href":858,"dataGaName":859,"dataGaLocation":747},"/topics/devops/ai-for-coding/","ai for coding",{"text":861,"config":862},"Agentic AI",{"href":863,"dataGaName":864,"dataGaLocation":747},"/topics/agentic-ai/","agentic ai",{"title":866,"links":867},"Solutions",[868,870,872,877,881,884,888,891,893,896,899,904],{"text":397,"config":869},{"href":392,"dataGaName":397,"dataGaLocation":747},{"text":386,"config":871},{"href":369,"dataGaName":370,"dataGaLocation":747},{"text":873,"config":874},"Agile development",{"href":875,"dataGaName":876,"dataGaLocation":747},"/solutions/agile-delivery/","agile delivery",{"text":878,"config":879},"SCM",{"href":382,"dataGaName":880,"dataGaLocation":747},"source code management",{"text":826,"config":882},{"href":375,"dataGaName":883,"dataGaLocation":747},"continuous integration & delivery",{"text":885,"config":886},"Value stream management",{"href":425,"dataGaName":887,"dataGaLocation":747},"value stream management",{"text":831,"config":889},{"href":890,"dataGaName":834,"dataGaLocation":747},"/solutions/gitops/",{"text":435,"config":892},{"href":438,"dataGaName":439,"dataGaLocation":747},{"text":894,"config":895},"Small business",{"href":444,"dataGaName":445,"dataGaLocation":747},{"text":897,"config":898},"Public sector",{"href":450,"dataGaName":451,"dataGaLocation":747},{"text":900,"config":901},"Education",{"href":902,"dataGaName":903,"dataGaLocation":747},"/solutions/education/","education",{"text":905,"config":906},"Financial services",{"href":907,"dataGaName":908,"dataGaLocation":747},"/solutions/finance/","financial services",{"title":458,"links":910},[911,913,915,917,920,922,925,927,929,931,933,935,937,939],{"text":471,"config":912},{"href":473,"dataGaName":474,"dataGaLocation":747},{"text":476,"config":914},{"href":478,"dataGaName":479,"dataGaLocation":747},{"text":481,"config":916},{"href":483,"dataGaName":484,"dataGaLocation":747},{"text":486,"config":918},{"href":488,"dataGaName":919,"dataGaLocation":747},"docs",{"text":509,"config":921},{"href":511,"dataGaName":512,"dataGaLocation":747},{"text":923,"config":924},"What's new",{"href":569,"dataGaName":570,"dataGaLocation":747},{"text":504,"config":926},{"href":506,"dataGaName":507,"dataGaLocation":747},{"text":523,"config":928},{"href":525,"dataGaName":526,"dataGaLocation":747},{"text":531,"config":930},{"href":533,"dataGaName":534,"dataGaLocation":747},{"text":536,"config":932},{"href":538,"dataGaName":539,"dataGaLocation":747},{"text":541,"config":934},{"href":543,"dataGaName":544,"dataGaLocation":747},{"text":546,"config":936},{"href":548,"dataGaName":549,"dataGaLocation":747},{"text":551,"config":938},{"href":553,"dataGaName":554,"dataGaLocation":747},{"text":556,"config":940},{"href":558,"dataGaName":559,"dataGaLocation":747},{"title":572,"links":942},[943,945,947,949,951,953,957,962,964,966,968],{"text":580,"config":944},{"href":582,"dataGaName":574,"dataGaLocation":747},{"text":585,"config":946},{"href":587,"dataGaName":588,"dataGaLocation":747},{"text":593,"config":948},{"href":595,"dataGaName":596,"dataGaLocation":747},{"text":598,"config":950},{"href":600,"dataGaName":601,"dataGaLocation":747},{"text":603,"config":952},{"href":605,"dataGaName":606,"dataGaLocation":747},{"text":954,"config":955},"Sustainability",{"href":956,"dataGaName":954,"dataGaLocation":747},"/sustainability/",{"text":958,"config":959},"Diversity, inclusion and belonging (DIB)",{"href":960,"dataGaName":961,"dataGaLocation":747},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":608,"config":963},{"href":610,"dataGaName":611,"dataGaLocation":747},{"text":618,"config":965},{"href":620,"dataGaName":621,"dataGaLocation":747},{"text":623,"config":967},{"href":625,"dataGaName":626,"dataGaLocation":747},{"text":969,"config":970},"Modern Slavery Transparency Statement",{"href":971,"dataGaName":972,"dataGaLocation":747},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":974},[975,978,981],{"text":976,"config":977},"Terms",{"href":799,"dataGaName":800,"dataGaLocation":747},{"text":979,"config":980},"Cookies",{"dataGaName":809,"dataGaLocation":747,"id":810,"isOneTrustButton":288},{"text":982,"config":983},"Privacy",{"href":804,"dataGaName":805,"dataGaLocation":747},[985],{"id":986,"title":7,"body":284,"config":987,"content":989,"description":284,"extension":993,"meta":994,"navigation":288,"path":995,"seo":996,"stem":997,"__hash__":998},"blogAuthors/en-us/blog/authors/igor-drozdov.yml",{"template":988},"BlogAuthor",{"name":7,"config":990},{"headshot":991,"ctfId":992},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749672455/Blog/Author%20Headshots/igor.png","igordrozdov","yml",{},"/en-us/blog/authors/igor-drozdov",{},"en-us/blog/authors/igor-drozdov","GbcuC9UkaMFqZ3erj7ATpsWd1EAKvl8xALQR97C1zv4",[1000,1009,1017],{"title":1001,"description":1002,"heroImage":1003,"category":280,"date":1004,"authors":1005,"slug":1008,"externalUrl":284},"Confidential AI for GitLab Self-Hosted","Give developers AI coding agents in GitLab Duo without source code leaving a hardware-encrypted boundary — no GPUs needed.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1773866173/vte9qh8rriznvyclhkes.png","2026-08-06",[1006,1007],"Mathias Ewald","Martin Paloncy, Edgeless Systems","confidential-ai-for-gitlab-self-hosted",{"title":1010,"description":1011,"heroImage":1012,"category":280,"date":1013,"authors":1014,"slug":1016,"externalUrl":284},"Green DevOps: Why carbon measurement belongs in your CI/CD pipeline","CI/CD pipelines have a hidden carbon cost. Here's why measuring it matters, and how you can get started with Eco CI and Carmen in GitLab.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1765809212/noh0mdfn9o94ry9ykura.png","2026-07-09",[1015],"Lysanne Pinto","green-devops-carbon-measurement-cicd-pipeline",{"title":1018,"description":1019,"heroImage":1020,"category":280,"date":1021,"authors":1022,"slug":1024,"externalUrl":284},"How to build CI/CD observability at scale","This practical guide to GitLab pipeline analytics helps self-managed users gain operational insights using Prometheus and Grafana.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-04-28",[1023],"Paul Meresanu","how-to-build-ci-cd-observability-at-scale",{"promotions":1026},[1027,1041,1053,1064],{"id":1028,"categories":1029,"header":1031,"text":1032,"button":1033,"image":1038},"ai-modernization",[1030],"ai","Is AI achieving its promise at scale?","Quiz will take 5 minutes or less",{"text":1034,"config":1035},"Get your AI maturity score",{"href":1036,"dataGaName":1037,"dataGaLocation":512},"/assessments/ai-modernization-assessment/","modernization assessment",{"config":1039},{"src":1040},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/qix0m7kwnd8x2fh1zq49.png",{"id":1042,"categories":1043,"header":1045,"text":1032,"button":1046,"image":1050},"devops-modernization",[1044,849],"product","Are you just managing tools or shipping innovation?",{"text":1047,"config":1048},"Get your DevOps maturity score",{"href":1049,"dataGaName":1037,"dataGaLocation":512},"/assessments/devops-modernization-assessment/",{"config":1051},{"src":1052},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138785/eg818fmakweyuznttgid.png",{"id":1054,"categories":1055,"header":1056,"text":1032,"button":1057,"image":1061},"security-modernization",[298],"Are you trading speed for security?",{"text":1058,"config":1059},"Get your security maturity score",{"href":1060,"dataGaName":1037,"dataGaLocation":512},"/assessments/security-modernization-assessment/",{"config":1062},{"src":1063},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/p4pbqd9nnjejg5ds6mdk.png",{"id":1065,"paths":1066,"header":1069,"text":1070,"button":1071,"image":1076},"github-azure-migration",[1067,1068],"migration-from-azure-devops-to-gitlab","integrating-azure-devops-scm-and-gitlab","Is your team ready for GitHub's Azure move?","GitHub is already rebuilding around Azure. Find out what it means for you.",{"text":1072,"config":1073},"See how GitLab compares to GitHub",{"href":1074,"dataGaName":1075,"dataGaLocation":512},"/compare/gitlab-vs-github/github-azure-migration/","github azure migration",{"config":1077},{"src":1052},{"header":1079,"blurb":1080,"button":1081,"secondaryButton":1086},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":1082,"config":1083},"Get your free trial",{"href":1084,"dataGaName":311,"dataGaLocation":1085},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":785,"config":1087},{"href":637,"dataGaName":316,"dataGaLocation":1085},1786803754746]