[{"data":1,"prerenderedAt":1064},["ShallowReactive",2],{"/blog/upgrading-database-os":3,"navigation-en-us":278,"banner-en-us":705,"footer-en-us":715,"blog-post-authors-en-us-David Smith":959,"blog-related-posts-en-us-upgrading-database-os":974,"blog-promotions-en-us":1000,"next-steps-en-us":1054},{"id":4,"title":5,"authors":6,"body":8,"category":256,"date":257,"description":258,"extension":259,"externalUrl":260,"faq":260,"featured":261,"heroImage":262,"meta":263,"navigation":264,"path":265,"seo":266,"slug":270,"stem":271,"tags":272,"template":276,"updatedDate":260,"__hash__":277},"blogPosts/en-us/blog/upgrading-database-os.md","We are upgrading the operating system on our Postgres database clusters",[7],"David Smith",{"type":9,"value":10,"toc":237},"minimark",[11,26,35,40,54,57,68,72,75,79,88,99,102,106,109,116,119,129,132,135,139,142,146,151,164,168,171,175,188,192,200,204,207,211,214,218,221],[12,13,14,15,20,21,25],"p",{},"Continuing on the theme of ",[16,17,19],"a",{"href":18},"/blog/path-to-decomposing-gitlab-database-part1/","improving the performance and reliability of GitLab.com",", we have another step we will be taking for our clusters of Postgres database nodes. These nodes have been running on Ubuntu 16.04 with extended security maintenance patches and it is now time to get them to a more current version. Usually, this kind of upgrade is a behind-the-scenes event, but there is an underlying technicality that will require us to take a maintenance window to do the upgrade (more on that ",[16,22,24],{"href":23},"#the-challenge","below",").",[12,27,28,29,34],{},"We have been preparing for and ",[16,30,33],{"href":31,"rel":32},"https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7577",[],"practicing this upgrade"," and are now ready to schedule the window to do this work for GitLab.com.",[36,37,39],"h2",{"id":38},"when-will-the-os-upgrade-take-place-and-what-does-this-mean-for-users-of-gitlabcom","When will the OS upgrade take place and what does this mean for users of GitLab.com?",[12,41,42,43,47,48,53],{},"This change is planned to take place on 2022-09-03 (Saturday) between 11:00 UTC and 14:00 UTC. The implementation of this change is anticipated to include a ",[44,45,46],"strong",{},"service downtime of up to 180 minutes"," (see ",[16,49,52],{"href":50,"rel":51},"https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7543",[],"reference issue","). During this time you will experience complete service disruption of GitLab.com.",[12,55,56],{},"We are taking downtime to ensure that the application works as expected following the OS upgrade and to minimize the risk of any data integrity issues.",[58,59,60],"blockquote",{},[12,61,62,63,67],{},"Join us at ",[16,64,66],{"href":65},"/events/","GitLab Commit 2022"," and connect with the ideas, technologies, and people that are driving DevOps and digital transformation.",[36,69,71],{"id":70},"background","Background",[12,73,74],{},"GitLab.com's database architecture uses two Patroni/Postgres database clusters: main and CI. We recently did functional decomposition and now the CI Cluster stores the data generated by CI GitLab features. Each Patroni cluster has primary and multiple read-only replicas. For each of the Patroni clusters, the Postgres database size is ~18 TB running on Ubuntu 16.04. During the scheduled change window, we will be switching over to our newly built Ubuntu 20.04 clusters.",[36,76,78],{"id":77},"the-challenge","The challenge",[12,80,81,82,87],{},"Ubuntu 18.10 introduced an updated version of glibc (2.28), which includes a ",[16,83,86],{"href":84,"rel":85},"https://wiki.postgresql.org/wiki/Locale_data_changes",[],"major update to locale data"," and causes Postgres indexes created with earlier versions of glibc to be corrupted. Because we are upgrading to Ubuntu 20.04, our indexes are affected by this. Therefore, during the downtime window scheduled for this work, we need to detect potentially corrupt indexes and have them reindexed before we enable production traffic again. We currently have the following types and the approximate number of indexes:",[89,90,96],"pre",{"className":91,"code":93,"language":94,"meta":95},[92],"language-text","\n Index Type | # of Indexes\n------------+--------------\n btree      |         4079\n gin        |          101\n gist       |            3\n hash       |            1\n\n","text","",[97,98,93],"code",{"__ignoreMap":95},[12,100,101],{},"As you can appreciate, given the sheer number (and size) of these indexes, it would take far too long to reindex every single index during the scheduled downtime window, so we need to streamline the process.",[36,103,105],{"id":104},"options-to-upgrade-to-ubuntu-2004-safely","Options to upgrade to Ubuntu 20.04 safely",[12,107,108],{},"There are a number of ways to deal with the problem of potentially corrupt indexes:",[12,110,111,112,115],{},"a. Reindex ",[44,113,114],{},"all"," indexes during the scheduled downtime window",[12,117,118],{},"b. Transport data to target 20.04 clusters in a logical (not binary) way, including:",[120,121,122,126],"ul",{},[123,124,125],"li",{},"Backups/upgrades using pg_dump",[123,127,128],{},"Logical replication",[12,130,131],{},"c. Use streaming replication from 16.04 to 20.04 and during the downtime window, break replication and promote the 20.04 clusters followed by reindexing of potentially corrupt indexes",[12,133,134],{},"It might be feasible for a small to a medium-size Postgres implementation to use options a or b; however, at the GitLab.com scale, it would require a much larger downtime window and our aim is to reduce the impact to our customers as much as possible.",[36,136,138],{"id":137},"high-level-approach-for-the-os-upgrade","High-level approach for the OS upgrade",[12,140,141],{},"To perform an OS upgrade on our Patroni clusters, we use Postgres streaming replication to replicate data from our current Ubuntu 16.04 clusters to the brand new Ubuntu 20.04 standby Patroni clusters. During the scheduled downtime window, we will stop all traffic to the current 16.04 clusters, promote the 20.04 clusters by making them Primary and demote the Ubuntu 16.04 clusters by reconfiguring to act as Standby while replicating from the new 20.04 primaries. We will then reindex all the identified potentially corrupt indexes, and update DNS to point the application to the new 20.04 Patroni clusters before opening traffic to the public.",[36,143,145],{"id":144},"identifying-potentially-corrupt-indexes-and-our-approach-to-handling-the-reindexing-for-different-types-of-indexes","Identifying potentially corrupt indexes and our approach to handling the reindexing for different types of indexes",[147,148,150],"h3",{"id":149},"b-tree","B-Tree",[12,152,153,154,157,158,163],{},"We use ",[97,155,156],{},"bt_index_parent_check"," ",[16,159,162],{"href":160,"rel":161},"https://www.postgresql.org/docs/12/amcheck.html",[],"amcheck function"," to identify potentially corrupt indexes and we will reindex them during the downtime window.",[147,165,167],{"id":166},"gist-and-hash","GiST and Hash",[12,169,170],{},"Since we do not have many GiST and Hash indexes, and reindexing them is a relatively quick operation, we will reindex them all during the downtime window.",[147,172,174],{"id":173},"gin","GIN",[12,176,177,178,181,182,187],{},"Currently, the production version of amcheck is limited to detecting potential corruption in B-Tree indexes only. Our GIN indexes are reasonably sized and it would require a significant amount of time to reindex them during the scheduled downtime window, which is not feasible as we cannot have the site unavailable to our customers for that long. We have collaborated closely with our database team to produce a list of business-critical GIN indexes to be reindexed ",[44,179,180],{},"during"," the downtime window, and any other GIN indexes will be reindexed immediately after we open up traffic to the public using the ",[16,183,186],{"href":184,"rel":185},"https://www.postgresql.org/docs/current/sql-reindex.html#SQL-REINDEX-CONCURRENTLY",[],"CONCURRENTLY"," option. Using this option means it will take longer to reindex, but it allows normal operations to continue while the indexes are being rebuilt.",[36,189,191],{"id":190},"performance-improvements","Performance improvements",[12,193,194,195,199],{},"We started looking into options to improve the performance of the reindexing (see ",[16,196,52],{"href":197,"rel":198},"https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/15559#note_940517257",[],"). There are a couple of areas where we needed to improve performance.",[147,201,203],{"id":202},"identify-potentially-corrupt-b-tree-indexes-quickly","Identify potentially corrupt B-Tree indexes quickly",[12,205,206],{},"When we first started using the amcheck to identify potentially corrupt indexes, it was single threaded so it was taking just under five days to run the amcheck script to identify potentially corrupt indexes on production data. After a few iterations, our amcheck script now runs a separate background worker process for each index, so we essentially get a performance improvement of about 96 times when we use a 96 CPU core VM to run amcheck. The performance is limited by the time it takes to run amcheck on the largest index. The script is customizable to skip or include a specific set of tables/indexes, and we can decide the number of parallel worker processes to use based on the number of CPU cores available on the VM we use to run amcheck. Now with the improved speed, we can run the amcheck script on a copy of production data a day or two before the scheduled OS upgrade downtime window.",[147,208,210],{"id":209},"improve-reindexing-speed-to-reduce-the-downtime","Improve reindexing speed to reduce the downtime",[12,212,213],{},"Our initial test to reindex was performed sequentially with the default Postgres parameters. We have tested reindexing with different Postgres parameters and parallelized the reindex process. We are now able to perform our reindexing in less than half the time it used to take to reindex.",[36,215,217],{"id":216},"reading-material","Reading material",[12,219,220],{},"For more information, please see the following links:",[120,222,223,230],{},[123,224,225],{},[16,226,229],{"href":227,"rel":228},"https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/637",[],"Ubuntu 20.04 Upgrade Epic",[123,231,232],{},[16,233,236],{"href":234,"rel":235},"https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/15384#note_867281334",[],"Research on the types of indexes and steps to identify corruption",{"title":95,"searchDepth":238,"depth":238,"links":239},2,[240,241,242,243,244,245,251,255],{"id":38,"depth":238,"text":39},{"id":70,"depth":238,"text":71},{"id":77,"depth":238,"text":78},{"id":104,"depth":238,"text":105},{"id":137,"depth":238,"text":138},{"id":144,"depth":238,"text":145,"children":246},[247,249,250],{"id":149,"depth":248,"text":150},3,{"id":166,"depth":248,"text":167},{"id":173,"depth":248,"text":174},{"id":190,"depth":238,"text":191,"children":252},[253,254],{"id":202,"depth":248,"text":203},{"id":209,"depth":248,"text":210},{"id":216,"depth":238,"text":217},"engineering","2022-08-12","Learn when these upgrades will happen and how they will help boost performance and reliability on GitLab.com.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749669673/Blog/Hero%20Images/engineering.png",{},true,"/en-us/blog/upgrading-database-os",{"title":5,"description":258,"ogTitle":5,"ogDescription":258,"noIndex":261,"ogImage":262,"ogUrl":267,"ogSiteName":268,"ogType":269,"canonicalUrls":267},"https://about.gitlab.com/blog/upgrading-database-os","https://about.gitlab.com","article","upgrading-database-os","en-us/blog/upgrading-database-os",[273,274,275],"news","performance","DevOps","BlogPost","wzkr9hFUX-FEri-UthDjDx4XcBv1QHfMmC1MEqM7ljM",{"logo":279,"freeTrial":284,"sales":289,"login":294,"items":299,"search":625,"minimal":656,"duo":675,"switchNav":684,"pricingDeployment":695},{"config":280},{"href":281,"dataGaName":282,"dataGaLocation":283},"/","gitlab logo","header",{"text":285,"config":286},"Get free trial",{"href":287,"dataGaName":288,"dataGaLocation":283},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":290,"config":291},"Request a demo",{"href":292,"dataGaName":293,"dataGaLocation":283},"/sales/?contact-topic=request-demo","sales",{"text":295,"config":296},"Sign in",{"href":297,"dataGaName":298,"dataGaLocation":283},"https://gitlab.com/users/sign_in/","sign in",[300,329,429,434,547,603],{"text":301,"config":302,"menu":304},"Platform",{"dataNavLevelOne":303},"platform",{"type":305,"columns":306},"cards",[307,313,321],{"title":301,"description":308,"link":309},"The intelligent orchestration platform for DevSecOps",{"text":310,"config":311},"Explore our Platform",{"href":312,"dataGaName":303,"dataGaLocation":283},"/platform/",{"title":314,"description":315,"link":316},"GitLab Duo Agent Platform","Agentic AI for the entire software lifecycle",{"text":317,"config":318},"Meet GitLab Duo",{"href":319,"dataGaName":320,"dataGaLocation":283},"/gitlab-duo-agent-platform/","gitlab duo agent platform",{"title":322,"description":323,"link":324},"Why GitLab","See the top reasons enterprises choose GitLab",{"text":325,"config":326},"Learn more",{"href":327,"dataGaName":328,"dataGaLocation":283},"/why-gitlab/","why gitlab",{"text":330,"left":264,"config":331,"menu":333},"Product",{"dataNavLevelOne":332},"solutions",{"type":334,"link":335,"columns":339,"feature":408},"lists",{"text":336,"config":337},"View all Solutions",{"href":338,"dataGaName":332,"dataGaLocation":283},"/solutions/",[340,364,387],{"title":341,"description":342,"link":343,"items":348},"Automation","CI/CD and automation to accelerate deployment",{"config":344},{"icon":345,"href":346,"dataGaName":347,"dataGaLocation":283},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[349,353,356,360],{"text":350,"config":351},"CI/CD",{"href":352,"dataGaLocation":283,"dataGaName":350},"/solutions/continuous-integration/",{"text":314,"config":354},{"href":319,"dataGaLocation":283,"dataGaName":355},"gitlab duo agent platform - product menu",{"text":357,"config":358},"Source Code Management",{"href":359,"dataGaLocation":283,"dataGaName":357},"/solutions/source-code-management/",{"text":361,"config":362},"Automated Software Delivery",{"href":346,"dataGaLocation":283,"dataGaName":363},"Automated software delivery",{"title":365,"description":366,"link":367,"items":372},"Security","Deliver code faster without compromising security",{"config":368},{"href":369,"dataGaName":370,"dataGaLocation":283,"icon":371},"/solutions/application-security-testing/","security and compliance","ShieldCheckLight",[373,377,382],{"text":374,"config":375},"Application Security Testing",{"href":369,"dataGaName":376,"dataGaLocation":283},"Application security testing",{"text":378,"config":379},"Software Supply Chain Security",{"href":380,"dataGaLocation":283,"dataGaName":381},"/solutions/supply-chain/","Software supply chain security",{"text":383,"config":384},"Software Compliance",{"href":385,"dataGaName":386,"dataGaLocation":283},"/solutions/software-compliance/","software compliance",{"title":388,"link":389,"items":394},"Measurement",{"config":390},{"icon":391,"href":392,"dataGaName":393,"dataGaLocation":283},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[395,399,403],{"text":396,"config":397},"Visibility & Measurement",{"href":392,"dataGaLocation":283,"dataGaName":398},"Visibility and Measurement",{"text":400,"config":401},"Value Stream Management",{"href":402,"dataGaLocation":283,"dataGaName":400},"/solutions/value-stream-management/",{"text":404,"config":405},"Analytics & Insights",{"href":406,"dataGaLocation":283,"dataGaName":407},"/solutions/analytics-and-insights/","Analytics and insights",{"title":409,"type":334,"items":410},"GitLab for",[411,417,423],{"text":412,"config":413},"Enterprise",{"icon":414,"href":415,"dataGaLocation":283,"dataGaName":416},"Building","/enterprise/","enterprise",{"text":418,"config":419},"Small Business",{"icon":420,"href":421,"dataGaLocation":283,"dataGaName":422},"Work","/small-business/","small business",{"text":424,"config":425},"Public Sector",{"icon":426,"href":427,"dataGaLocation":283,"dataGaName":428},"Organization","/solutions/public-sector/","public sector",{"text":430,"config":431},"Pricing",{"href":432,"dataGaName":433,"dataGaLocation":283,"dataNavLevelOne":433},"/pricing/","pricing",{"text":435,"config":436,"menu":438},"Resources",{"dataNavLevelOne":437},"resources",{"type":334,"link":439,"columns":443,"feature":536},{"text":440,"config":441},"View all resources",{"href":442,"dataGaName":437,"dataGaLocation":283},"/resources/",[444,477,504],{"title":445,"items":446},"Getting started",[447,452,457,462,467,472],{"text":448,"config":449},"Install",{"href":450,"dataGaName":451,"dataGaLocation":283},"/install/","install",{"text":453,"config":454},"Quick start guides",{"href":455,"dataGaName":456,"dataGaLocation":283},"/get-started/","quick setup checklists",{"text":458,"config":459},"Learn",{"href":460,"dataGaLocation":283,"dataGaName":461},"https://university.gitlab.com/","learn",{"text":463,"config":464},"Product documentation",{"href":465,"dataGaName":466,"dataGaLocation":283},"https://docs.gitlab.com/","product documentation",{"text":468,"config":469},"Best practice videos",{"href":470,"dataGaName":471,"dataGaLocation":283},"/getting-started-videos/","best practice videos",{"text":473,"config":474},"Integrations",{"href":475,"dataGaName":476,"dataGaLocation":283},"/integrations/","integrations",{"title":478,"items":479},"Discover",[480,485,490,495,499],{"text":481,"config":482},"Customer success stories",{"href":483,"dataGaName":484,"dataGaLocation":283},"/customers/","customer success stories",{"text":486,"config":487},"Blog",{"href":488,"dataGaName":489,"dataGaLocation":283},"/blog/","blog",{"text":491,"config":492},"Demo Hub",{"href":493,"dataGaName":494,"dataGaLocation":283},"/demo-hub/","demo hub",{"text":496,"config":497},"The Source",{"href":498,"dataGaName":489,"dataGaLocation":283},"/the-source/",{"text":500,"config":501},"Remote",{"href":502,"dataGaName":503,"dataGaLocation":283},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"title":505,"items":506},"Connect",[507,512,517,522,527,531],{"text":508,"config":509},"GitLab Services",{"href":510,"dataGaName":511,"dataGaLocation":283},"/services/","services",{"text":513,"config":514},"Contribute",{"href":515,"dataGaName":516,"dataGaLocation":283},"https://contributors.gitlab.com","contribute",{"text":518,"config":519},"Community",{"href":520,"dataGaName":521,"dataGaLocation":283},"/community/","community",{"text":523,"config":524},"Forum",{"href":525,"dataGaName":526,"dataGaLocation":283},"https://forum.gitlab.com/","forum",{"text":528,"config":529},"Events",{"href":65,"dataGaName":530,"dataGaLocation":283},"events",{"text":532,"config":533},"Partners",{"href":534,"dataGaName":535,"dataGaLocation":283},"/partners/","partners",{"config":537,"title":540,"text":541,"link":542},{"background":538,"textColor":539},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1777322348/qpq8yrgn8knii57omj0c.png')","#000","What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":543,"config":544},"Read the latest",{"href":545,"dataGaName":546,"dataGaLocation":283},"/whats-new/","whats new",{"text":548,"config":549,"menu":551},"Company",{"dataNavLevelOne":550},"company",{"type":334,"columns":552},[553],{"items":554},[555,560,566,568,573,578,583,588,593,598],{"text":556,"config":557},"About",{"href":558,"dataGaName":559,"dataGaLocation":283},"/company/","about",{"text":561,"config":562,"footerGa":565},"Jobs",{"href":563,"dataGaName":564,"dataGaLocation":283},"/jobs/","jobs",{"dataGaName":564},{"text":528,"config":567},{"href":65,"dataGaName":530,"dataGaLocation":283},{"text":569,"config":570},"Leadership",{"href":571,"dataGaName":572,"dataGaLocation":283},"/company/team/e-group/","leadership",{"text":574,"config":575},"Handbook",{"href":576,"dataGaName":577,"dataGaLocation":283},"https://handbook.gitlab.com/","handbook",{"text":579,"config":580},"Investor relations",{"href":581,"dataGaName":582,"dataGaLocation":283},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":584,"config":585},"Trust Center",{"href":586,"dataGaName":587,"dataGaLocation":283},"/security/","trust center",{"text":589,"config":590},"AI Transparency Center",{"href":591,"dataGaName":592,"dataGaLocation":283},"/ai-transparency-center/","ai transparency center",{"text":594,"config":595},"Newsletter",{"href":596,"dataGaName":597,"dataGaLocation":283},"/company/contact/#contact-forms","newsletter",{"text":599,"config":600},"Press",{"href":601,"dataGaName":602,"dataGaLocation":283},"/press/","press",{"text":604,"config":605,"menu":606},"Contact us",{"dataNavLevelOne":550},{"type":334,"columns":607},[608],{"items":609},[610,615,620],{"text":611,"config":612},"Talk to sales",{"href":613,"dataGaName":614,"dataGaLocation":283},"/sales/","talk to sales",{"text":616,"config":617},"Support portal",{"href":618,"dataGaName":619,"dataGaLocation":283},"https://support.gitlab.com/hc/en-us","support portal",{"text":621,"config":622},"Customer portal",{"href":623,"dataGaName":624,"dataGaLocation":283},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":626,"login":627,"suggestions":634},"Close",{"text":628,"link":629},"To search repositories and projects, login to",{"text":630,"config":631},"gitlab.com",{"href":297,"dataGaName":632,"dataGaLocation":633},"search login","search",{"text":635,"default":636},"Suggestions",[637,639,643,645,649,653],{"text":314,"config":638},{"href":319,"dataGaName":314,"dataGaLocation":633},{"text":640,"config":641},"Code Suggestions (AI)",{"href":642,"dataGaName":640,"dataGaLocation":633},"/solutions/code-suggestions/",{"text":350,"config":644},{"href":352,"dataGaName":350,"dataGaLocation":633},{"text":646,"config":647},"GitLab on AWS",{"href":648,"dataGaName":646,"dataGaLocation":633},"/partners/technology-partners/aws/",{"text":650,"config":651},"GitLab on Google Cloud",{"href":652,"dataGaName":650,"dataGaLocation":633},"/partners/technology-partners/google-cloud-platform/",{"text":654,"config":655},"Why GitLab?",{"href":327,"dataGaName":654,"dataGaLocation":633},{"freeTrial":657,"mobileIcon":662,"desktopIcon":667,"secondaryButton":670},{"text":658,"config":659},"Start free trial",{"href":660,"dataGaName":288,"dataGaLocation":661},"https://gitlab.com/-/trials/new/","nav",{"altText":663,"config":664},"Gitlab Icon",{"src":665,"dataGaName":666,"dataGaLocation":661},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":663,"config":668},{"src":669,"dataGaName":666,"dataGaLocation":661},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":671,"config":672},"Get Started",{"href":673,"dataGaName":674,"dataGaLocation":661},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":676,"mobileIcon":680,"desktopIcon":682},{"text":677,"config":678},"Learn more about GitLab Duo",{"href":319,"dataGaName":679,"dataGaLocation":661},"gitlab duo",{"altText":663,"config":681},{"src":665,"dataGaName":666,"dataGaLocation":661},{"altText":663,"config":683},{"src":669,"dataGaName":666,"dataGaLocation":661},{"button":685,"mobileIcon":690,"desktopIcon":692},{"text":686,"config":687},"/switch",{"href":688,"dataGaName":689,"dataGaLocation":661},"#contact","switch",{"altText":663,"config":691},{"src":665,"dataGaName":666,"dataGaLocation":661},{"altText":663,"config":693},{"src":694,"dataGaName":666,"dataGaLocation":661},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":696,"mobileIcon":701,"desktopIcon":703},{"text":697,"config":698},"Back to pricing",{"href":432,"dataGaName":699,"dataGaLocation":661,"icon":700},"back to pricing","GoBack",{"altText":663,"config":702},{"src":665,"dataGaName":666,"dataGaLocation":661},{"altText":663,"config":704},{"src":669,"dataGaName":666,"dataGaLocation":661},{"title":706,"titleMobile":707,"button":708,"config":713},"Duo Agent Platform delivers 400% ROI, per new Forrester Consulting study.","400% ROI: Forrester TEI for GitLab Duo",{"text":325,"config":709},{"href":710,"dataGaName":711,"dataGaLocation":712},"https://about.gitlab.com/blog/gitlab-duo-agent-platform-delivers-400-percent-roi/","forrester-tei-dap-banner","global-banner",{"layout":714,"disabled":261},"release",{"data":716},{"text":717,"source":718,"edit":724,"contribute":729,"config":734,"items":739,"minimal":948},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":719,"config":720},"View page source",{"href":721,"dataGaName":722,"dataGaLocation":723},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":725,"config":726},"Edit this page",{"href":727,"dataGaName":728,"dataGaLocation":723},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":730,"config":731},"Please contribute",{"href":732,"dataGaName":733,"dataGaLocation":723},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":735,"facebook":736,"youtube":737,"linkedin":738},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[740,787,840,884,916],{"title":430,"links":741,"subMenu":756},[742,746,751],{"text":743,"config":744},"View plans",{"href":432,"dataGaName":745,"dataGaLocation":723},"view plans",{"text":747,"config":748},"Why Premium?",{"href":749,"dataGaName":750,"dataGaLocation":723},"/pricing/premium/","why premium",{"text":752,"config":753},"Why Ultimate?",{"href":754,"dataGaName":755,"dataGaLocation":723},"/pricing/ultimate/","why ultimate",[757],{"title":758,"links":759},"Contact Us",[760,763,765,767,772,777,782],{"text":761,"config":762},"Contact sales",{"href":613,"dataGaName":293,"dataGaLocation":723},{"text":616,"config":764},{"href":618,"dataGaName":619,"dataGaLocation":723},{"text":621,"config":766},{"href":623,"dataGaName":624,"dataGaLocation":723},{"text":768,"config":769},"Status",{"href":770,"dataGaName":771,"dataGaLocation":723},"https://status.gitlab.com/","status",{"text":773,"config":774},"Terms of use",{"href":775,"dataGaName":776,"dataGaLocation":723},"/terms/","terms of use",{"text":778,"config":779},"Privacy statement",{"href":780,"dataGaName":781,"dataGaLocation":723},"/privacy/","privacy statement",{"text":783,"config":784},"Cookie preferences",{"dataGaName":785,"dataGaLocation":723,"id":786,"isOneTrustButton":264},"cookie preferences","ot-sdk-btn",{"title":330,"links":788,"subMenu":797},[789,793],{"text":790,"config":791},"DevSecOps platform",{"href":312,"dataGaName":792,"dataGaLocation":723},"devsecops platform",{"text":794,"config":795},"AI-Assisted Development",{"href":319,"dataGaName":796,"dataGaLocation":723},"ai-assisted development",[798],{"title":799,"links":800},"Topics",[801,806,811,815,820,825,830,835],{"text":802,"config":803},"CICD",{"href":804,"dataGaName":805,"dataGaLocation":723},"/topics/ci-cd/","cicd",{"text":807,"config":808},"GitOps",{"href":809,"dataGaName":810,"dataGaLocation":723},"/topics/gitops/","gitops",{"text":275,"config":812},{"href":813,"dataGaName":814,"dataGaLocation":723},"/topics/devops/","devops",{"text":816,"config":817},"Version Control",{"href":818,"dataGaName":819,"dataGaLocation":723},"/topics/version-control/","version control",{"text":821,"config":822},"DevSecOps",{"href":823,"dataGaName":824,"dataGaLocation":723},"/topics/devsecops/","devsecops",{"text":826,"config":827},"Cloud Native",{"href":828,"dataGaName":829,"dataGaLocation":723},"/topics/cloud-native/","cloud native",{"text":831,"config":832},"AI for Coding",{"href":833,"dataGaName":834,"dataGaLocation":723},"/topics/devops/ai-for-coding/","ai for coding",{"text":836,"config":837},"Agentic AI",{"href":838,"dataGaName":839,"dataGaLocation":723},"/topics/agentic-ai/","agentic ai",{"title":841,"links":842},"Solutions",[843,845,847,852,856,859,863,866,868,871,874,879],{"text":374,"config":844},{"href":369,"dataGaName":374,"dataGaLocation":723},{"text":363,"config":846},{"href":346,"dataGaName":347,"dataGaLocation":723},{"text":848,"config":849},"Agile development",{"href":850,"dataGaName":851,"dataGaLocation":723},"/solutions/agile-delivery/","agile delivery",{"text":853,"config":854},"SCM",{"href":359,"dataGaName":855,"dataGaLocation":723},"source code management",{"text":802,"config":857},{"href":352,"dataGaName":858,"dataGaLocation":723},"continuous integration & delivery",{"text":860,"config":861},"Value stream management",{"href":402,"dataGaName":862,"dataGaLocation":723},"value stream management",{"text":807,"config":864},{"href":865,"dataGaName":810,"dataGaLocation":723},"/solutions/gitops/",{"text":412,"config":867},{"href":415,"dataGaName":416,"dataGaLocation":723},{"text":869,"config":870},"Small business",{"href":421,"dataGaName":422,"dataGaLocation":723},{"text":872,"config":873},"Public sector",{"href":427,"dataGaName":428,"dataGaLocation":723},{"text":875,"config":876},"Education",{"href":877,"dataGaName":878,"dataGaLocation":723},"/solutions/education/","education",{"text":880,"config":881},"Financial services",{"href":882,"dataGaName":883,"dataGaLocation":723},"/solutions/finance/","financial services",{"title":435,"links":885},[886,888,890,892,895,897,900,902,904,906,908,910,912,914],{"text":448,"config":887},{"href":450,"dataGaName":451,"dataGaLocation":723},{"text":453,"config":889},{"href":455,"dataGaName":456,"dataGaLocation":723},{"text":458,"config":891},{"href":460,"dataGaName":461,"dataGaLocation":723},{"text":463,"config":893},{"href":465,"dataGaName":894,"dataGaLocation":723},"docs",{"text":486,"config":896},{"href":488,"dataGaName":489,"dataGaLocation":723},{"text":898,"config":899},"What's new",{"href":545,"dataGaName":546,"dataGaLocation":723},{"text":481,"config":901},{"href":483,"dataGaName":484,"dataGaLocation":723},{"text":500,"config":903},{"href":502,"dataGaName":503,"dataGaLocation":723},{"text":508,"config":905},{"href":510,"dataGaName":511,"dataGaLocation":723},{"text":513,"config":907},{"href":515,"dataGaName":516,"dataGaLocation":723},{"text":518,"config":909},{"href":520,"dataGaName":521,"dataGaLocation":723},{"text":523,"config":911},{"href":525,"dataGaName":526,"dataGaLocation":723},{"text":528,"config":913},{"href":65,"dataGaName":530,"dataGaLocation":723},{"text":532,"config":915},{"href":534,"dataGaName":535,"dataGaLocation":723},{"title":548,"links":917},[918,920,922,924,926,928,932,937,939,941,943],{"text":556,"config":919},{"href":558,"dataGaName":550,"dataGaLocation":723},{"text":561,"config":921},{"href":563,"dataGaName":564,"dataGaLocation":723},{"text":569,"config":923},{"href":571,"dataGaName":572,"dataGaLocation":723},{"text":574,"config":925},{"href":576,"dataGaName":577,"dataGaLocation":723},{"text":579,"config":927},{"href":581,"dataGaName":582,"dataGaLocation":723},{"text":929,"config":930},"Sustainability",{"href":931,"dataGaName":929,"dataGaLocation":723},"/sustainability/",{"text":933,"config":934},"Diversity, inclusion and belonging (DIB)",{"href":935,"dataGaName":936,"dataGaLocation":723},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":584,"config":938},{"href":586,"dataGaName":587,"dataGaLocation":723},{"text":594,"config":940},{"href":596,"dataGaName":597,"dataGaLocation":723},{"text":599,"config":942},{"href":601,"dataGaName":602,"dataGaLocation":723},{"text":944,"config":945},"Modern Slavery Transparency Statement",{"href":946,"dataGaName":947,"dataGaLocation":723},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":949},[950,953,956],{"text":951,"config":952},"Terms",{"href":775,"dataGaName":776,"dataGaLocation":723},{"text":954,"config":955},"Cookies",{"dataGaName":785,"dataGaLocation":723,"id":786,"isOneTrustButton":264},{"text":957,"config":958},"Privacy",{"href":780,"dataGaName":781,"dataGaLocation":723},[960],{"id":961,"title":7,"body":260,"config":962,"content":964,"description":260,"extension":968,"meta":969,"navigation":264,"path":970,"seo":971,"stem":972,"__hash__":973},"blogAuthors/en-us/blog/authors/david-smith.yml",{"template":963},"BlogAuthor",{"name":7,"config":965},{"headshot":966,"ctfId":967},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749671135/Blog/Author%20Headshots/dawsmith-headshot.jpg","dawsmith","yml",{},"/en-us/blog/authors/david-smith",{},"en-us/blog/authors/david-smith","XzEXAOpoYpoefqOm9h7rAHs6sqroy1rGgZroiP5L8iQ",[975,984,992],{"title":976,"description":977,"heroImage":978,"category":256,"date":979,"authors":980,"slug":983,"externalUrl":260},"Confidential AI for GitLab Self-Hosted","Give developers AI coding agents in GitLab Duo without source code leaving a hardware-encrypted boundary — no GPUs needed.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1773866173/vte9qh8rriznvyclhkes.png","2026-08-06",[981,982],"Mathias Ewald","Martin Paloncy, Edgeless Systems","confidential-ai-for-gitlab-self-hosted",{"title":985,"description":986,"heroImage":987,"category":256,"date":988,"authors":989,"slug":991,"externalUrl":260},"Green DevOps: Why carbon measurement belongs in your CI/CD pipeline","CI/CD pipelines have a hidden carbon cost. Here's why measuring it matters, and how you can get started with Eco CI and Carmen in GitLab.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1765809212/noh0mdfn9o94ry9ykura.png","2026-07-09",[990],"Lysanne Pinto","green-devops-carbon-measurement-cicd-pipeline",{"title":993,"description":994,"heroImage":995,"category":256,"date":996,"authors":997,"slug":999,"externalUrl":260},"How to build CI/CD observability at scale","This practical guide to GitLab pipeline analytics helps self-managed users gain operational insights using Prometheus and Grafana.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-04-28",[998],"Paul Meresanu","how-to-build-ci-cd-observability-at-scale",{"promotions":1001},[1002,1016,1028,1040],{"id":1003,"categories":1004,"header":1006,"text":1007,"button":1008,"image":1013},"ai-modernization",[1005],"ai","Is AI achieving its promise at scale?","Quiz will take 5 minutes or less",{"text":1009,"config":1010},"Get your AI maturity score",{"href":1011,"dataGaName":1012,"dataGaLocation":489},"/assessments/ai-modernization-assessment/","modernization assessment",{"config":1014},{"src":1015},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/qix0m7kwnd8x2fh1zq49.png",{"id":1017,"categories":1018,"header":1020,"text":1007,"button":1021,"image":1025},"devops-modernization",[1019,824],"product","Are you just managing tools or shipping innovation?",{"text":1022,"config":1023},"Get your DevOps maturity score",{"href":1024,"dataGaName":1012,"dataGaLocation":489},"/assessments/devops-modernization-assessment/",{"config":1026},{"src":1027},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138785/eg818fmakweyuznttgid.png",{"id":1029,"categories":1030,"header":1032,"text":1007,"button":1033,"image":1037},"security-modernization",[1031],"security","Are you trading speed for security?",{"text":1034,"config":1035},"Get your security maturity score",{"href":1036,"dataGaName":1012,"dataGaLocation":489},"/assessments/security-modernization-assessment/",{"config":1038},{"src":1039},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/p4pbqd9nnjejg5ds6mdk.png",{"id":1041,"paths":1042,"header":1045,"text":1046,"button":1047,"image":1052},"github-azure-migration",[1043,1044],"migration-from-azure-devops-to-gitlab","integrating-azure-devops-scm-and-gitlab","Is your team ready for GitHub's Azure move?","GitHub is already rebuilding around Azure. Find out what it means for you.",{"text":1048,"config":1049},"See how GitLab compares to GitHub",{"href":1050,"dataGaName":1051,"dataGaLocation":489},"/compare/gitlab-vs-github/github-azure-migration/","github azure migration",{"config":1053},{"src":1027},{"header":1055,"blurb":1056,"button":1057,"secondaryButton":1062},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":1058,"config":1059},"Get your free trial",{"href":1060,"dataGaName":288,"dataGaLocation":1061},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":761,"config":1063},{"href":613,"dataGaName":293,"dataGaLocation":1061},1786803756096]