[{"data":1,"prerenderedAt":916},["ShallowReactive",2],{"/blog/gitlab-dot-com-outage-on-2015-09-01":3,"navigation-en-us":131,"banner-en-us":558,"footer-en-us":568,"blog-post-authors-en-us-Jacob Vosmaer":813,"blog-related-posts-en-us-gitlab-dot-com-outage-on-2015-09-01":828,"blog-promotions-en-us":852,"next-steps-en-us":906},{"id":4,"title":5,"authors":6,"body":8,"category":109,"date":110,"description":111,"extension":112,"externalUrl":113,"faq":113,"featured":114,"heroImage":115,"meta":116,"navigation":121,"path":122,"seo":123,"slug":127,"stem":128,"tags":113,"template":129,"updatedDate":113,"__hash__":130},"blogPosts/en-us/blog/gitlab-dot-com-outage-on-2015-09-01.md","GitLab.com outage on 2015-09-01",[7],"Jacob Vosmaer",{"type":9,"value":10,"toc":104},"minimark",[11,15,23,33,47,54,60,63,72,77,84,87,90,93,97],[12,13,14],"p",{},"This morning GitLab.com was offline for one hour while we were\ninvestigating what seemed to be a filesystem corruption issue.  With\nthis blog post we want to tell you more about what was going on and\nwhat we discovered.",[12,16,17,18,22],{},"This morning around 7:00 UTC the NFS server that holds the Git\nrepositories hosted on GitLab.com became very slow. This made\nGitLab.com also very slow. While investigating this issue on the\nNFS server, we saw the following error messages in the output of\n",[19,20,21],"code",{},"dmesg",":",[24,25,31],"pre",{"className":26,"code":28,"language":29,"meta":30},[27],"language-text","[870355.932072] EXT4-fs (dm-0): error count since last fsck: 2\n[870355.932076] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[870355.932079] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[956863.452065] EXT4-fs (dm-0): error count since last fsck: 2\n[956863.452070] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[956863.452072] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[1043370.972077] EXT4-fs (dm-0): error count since last fsck: 2\n[1043370.972081] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[1043370.972084] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[1129878.492074] EXT4-fs (dm-0): error count since last fsck: 2\n[1129878.492108] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[1129878.492111] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[1216386.012085] EXT4-fs (dm-0): error count since last fsck: 2\n[1216386.012135] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[1216386.012139] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[1302893.532065] EXT4-fs (dm-0): error count since last fsck: 2\n","text","",[19,32,28],{"__ignoreMap":30},[12,34,35,36,39,40,46],{},"This made us worry that something was really wrong with the filesystem\nthat holds all GitLab.com data, and we decided that we should take\nthe filesystem (and hence GitLab.com) offline to run ",[19,37,38],{},"fsck"," on it.\nThis was at ",[41,42,45],"a",{"href":43,"rel":44},"https://twitter.com/gitlabstatus/status/638621030060290048",[],"7:54\nUTC",".\nWith fsck under way we started thinking about other, faster ways\nto bring GitLab.com back online. We considered restoring a backup,\nbut that would have set back the clock 6-7 hours (we make backups\nonce a day) and because the backups use block device snapshots,\nthere was a good chance that whatever file system corruption we\nwere seeing would be present in the backup too.",[12,48,49,50,53],{},"At some point we went back to the dmesg output to try and better\nunderstand the errors. What was it really saying? What were those\ntime stamps at the start of each line? It turns out dmesg can show\nhuman-readable timestamps if you pass the ",[19,51,52],{},"-T"," option. Now things\nsuddenly looked very different:",[24,55,58],{"className":56,"code":57,"language":29,"meta":30},[27],"[Wed Aug 26 14:52:55 2015] EXT4-fs (dm-0): error count since last fsck: 2\n[Wed Aug 26 14:52:55 2015] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[Wed Aug 26 14:52:55 2015] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[Thu Aug 27 14:54:43 2015] EXT4-fs (dm-0): error count since last fsck: 2\n[Thu Aug 27 14:54:43 2015] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[Thu Aug 27 14:54:43 2015] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[Fri Aug 28 14:56:31 2015] EXT4-fs (dm-0): error count since last fsck: 2\n[Fri Aug 28 14:56:31 2015] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[Fri Aug 28 14:56:31 2015] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[Sat Aug 29 14:58:18 2015] EXT4-fs (dm-0): error count since last fsck: 2\n[Sat Aug 29 14:58:18 2015] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[Sat Aug 29 14:58:18 2015] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[Sun Aug 30 15:00:06 2015] EXT4-fs (dm-0): error count since last fsck: 2\n[Sun Aug 30 15:00:06 2015] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[Sun Aug 30 15:00:06 2015] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[Mon Aug 31 15:01:53 2015] EXT4-fs (dm-0): error count since last fsck: 2\n[Mon Aug 31 15:01:53 2015] EXT4-fs (dm-0): initial error at time 1439995223: ext4_mb_generate_buddy:756\n[Mon Aug 31 15:01:53 2015] EXT4-fs (dm-0): last error at time 1439995223: ext4_mb_generate_buddy:756\n[Tue Sep  1 07:53:11 2015] nfsd: last server has exited, flushing export cache\n",[19,59,57],{"__ignoreMap":30},[12,61,62],{},"What we saw now was that we were getting the same error once a day,\ngoing back to the first day we mounted this filesystem on the server\n(after our migration from Germany to the US). The scary error message\nthat made us assume the worst and led us to take emergency measures\n(taking GitLab.com offline for fsck) turned out to have been repeating\nfor weeks, like clockwork, around 15:00 UTC every day. Moreover,\nwe saw no new errors appearing, just a regular repeat of the same\nerror. Although the message signals a problem that we should not\nignore, looking at it and understanding the pattern it no longer\nlooked like a likely cause for the load issues (NFFS server trouble)\nthat started all of this.",[12,64,65,66,71],{},"At this point we decided that the best course of action was to abort\nthe filesystem check and bring GitLab.com back online. This was\naround ",[41,67,70],{"href":68,"rel":69},"https://twitter.com/gitlabstatus/status/638635500274909184",[],"8:52\nUTC",".\nAfter seeing more 502 errors than usual for a few minutes, GitLab.com\nwas operating normally again.",[73,74,76],"h2",{"id":75},"next-steps","Next steps",[12,78,79,80,83],{},"We are in the process of moving all GitLab.com data off of the ext4\nfilesystem that scared us with its two errors. This is because three\nyears after we created it, we are outgrowing its built-in maximum\nsize of 16 TB. We will run ",[19,81,82],{},"git fsck"," (Git's built-in consistency\ncheck) on each repository once we are on the new filesystem.",[12,85,86],{},"We are still in the dark about the cause of the NFS slowdowns. We\nsee no spikes of any kind of web requests around the slowdowns. The\nbackend server only shows the ext4 errors mentioned above, which\ndo not coincide with the NFS trouble, and no NFS error messages.\nThe NFS clients show kernel messages about processes hanging during\nfilesystem operations, which really only tells us that here is an\nNFS problem.",[12,88,89],{},"Last week, on 2015-08-27, we had a similar problem on GitLab.com,\nwhich we responded to by doubling the number of servers that handle\nuser traffic. That helped that time but it does not seem to have helped with this problem.",[12,91,92],{},"We will keep looking for potential causes and keep monitoring the\nsituation.",[73,94,96],{"id":95},"lessons","Lessons",[12,98,99,100,103],{},"As an operations team we still struggle with giving enough status\nupdates during a crisis. All people who were on this problem were\nfocused on the GitLab.com servers, and nobody was talking to the\nGitLab.com users, leading to an hour of radio silence from\n",[19,101,102],{},"@gitlabstatus"," during the incident. We need to improve on this,\nbut like debugging NFS server meltdowns, it is a hard problem.",{"title":30,"searchDepth":105,"depth":105,"links":106},2,[107,108],{"id":75,"depth":105,"text":76},{"id":95,"depth":105,"text":96},"company","2015-09-01","This morning GitLab.com was offline for one hour while we were investigating what seemed to be a filesystem corruption issue.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749663397/Blog/Hero%20Images/logoforblogpost.jpg",{"excerpt":117},{"type":9,"value":118},[119],[12,120,14],{},true,"/en-us/blog/gitlab-dot-com-outage-on-2015-09-01",{"title":5,"description":111,"ogTitle":5,"ogDescription":111,"noIndex":114,"ogImage":115,"ogUrl":124,"ogSiteName":125,"ogType":126,"canonicalUrls":124},"https://about.gitlab.com/blog/gitlab-dot-com-outage-on-2015-09-01","https://about.gitlab.com","article","gitlab-dot-com-outage-on-2015-09-01","en-us/blog/gitlab-dot-com-outage-on-2015-09-01","BlogPost","HMyLPwk9DVhMfP3Xax0MzikL2I6YAHzbXwhQZEzWWGc",{"logo":132,"freeTrial":137,"sales":142,"login":147,"items":152,"search":478,"minimal":509,"duo":528,"switchNav":537,"pricingDeployment":548},{"config":133},{"href":134,"dataGaName":135,"dataGaLocation":136},"/","gitlab logo","header",{"text":138,"config":139},"Get free trial",{"href":140,"dataGaName":141,"dataGaLocation":136},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":143,"config":144},"Request a demo",{"href":145,"dataGaName":146,"dataGaLocation":136},"/sales/?contact-topic=request-demo","sales",{"text":148,"config":149},"Sign in",{"href":150,"dataGaName":151,"dataGaLocation":136},"https://gitlab.com/users/sign_in/","sign in",[153,182,282,287,401,456],{"text":154,"config":155,"menu":157},"Platform",{"dataNavLevelOne":156},"platform",{"type":158,"columns":159},"cards",[160,166,174],{"title":154,"description":161,"link":162},"The intelligent orchestration platform for DevSecOps",{"text":163,"config":164},"Explore our Platform",{"href":165,"dataGaName":156,"dataGaLocation":136},"/platform/",{"title":167,"description":168,"link":169},"GitLab Duo Agent Platform","Agentic AI for the entire software lifecycle",{"text":170,"config":171},"Meet GitLab Duo",{"href":172,"dataGaName":173,"dataGaLocation":136},"/gitlab-duo-agent-platform/","gitlab duo agent platform",{"title":175,"description":176,"link":177},"Why GitLab","See the top reasons enterprises choose GitLab",{"text":178,"config":179},"Learn more",{"href":180,"dataGaName":181,"dataGaLocation":136},"/why-gitlab/","why gitlab",{"text":183,"left":121,"config":184,"menu":186},"Product",{"dataNavLevelOne":185},"solutions",{"type":187,"link":188,"columns":192,"feature":261},"lists",{"text":189,"config":190},"View all Solutions",{"href":191,"dataGaName":185,"dataGaLocation":136},"/solutions/",[193,217,240],{"title":194,"description":195,"link":196,"items":201},"Automation","CI/CD and automation to accelerate deployment",{"config":197},{"icon":198,"href":199,"dataGaName":200,"dataGaLocation":136},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[202,206,209,213],{"text":203,"config":204},"CI/CD",{"href":205,"dataGaLocation":136,"dataGaName":203},"/solutions/continuous-integration/",{"text":167,"config":207},{"href":172,"dataGaLocation":136,"dataGaName":208},"gitlab duo agent platform - product menu",{"text":210,"config":211},"Source Code Management",{"href":212,"dataGaLocation":136,"dataGaName":210},"/solutions/source-code-management/",{"text":214,"config":215},"Automated Software Delivery",{"href":199,"dataGaLocation":136,"dataGaName":216},"Automated software delivery",{"title":218,"description":219,"link":220,"items":225},"Security","Deliver code faster without compromising security",{"config":221},{"href":222,"dataGaName":223,"dataGaLocation":136,"icon":224},"/solutions/application-security-testing/","security and compliance","ShieldCheckLight",[226,230,235],{"text":227,"config":228},"Application Security Testing",{"href":222,"dataGaName":229,"dataGaLocation":136},"Application security testing",{"text":231,"config":232},"Software Supply Chain Security",{"href":233,"dataGaLocation":136,"dataGaName":234},"/solutions/supply-chain/","Software supply chain security",{"text":236,"config":237},"Software Compliance",{"href":238,"dataGaName":239,"dataGaLocation":136},"/solutions/software-compliance/","software compliance",{"title":241,"link":242,"items":247},"Measurement",{"config":243},{"icon":244,"href":245,"dataGaName":246,"dataGaLocation":136},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[248,252,256],{"text":249,"config":250},"Visibility & Measurement",{"href":245,"dataGaLocation":136,"dataGaName":251},"Visibility and Measurement",{"text":253,"config":254},"Value Stream Management",{"href":255,"dataGaLocation":136,"dataGaName":253},"/solutions/value-stream-management/",{"text":257,"config":258},"Analytics & Insights",{"href":259,"dataGaLocation":136,"dataGaName":260},"/solutions/analytics-and-insights/","Analytics and insights",{"title":262,"type":187,"items":263},"GitLab for",[264,270,276],{"text":265,"config":266},"Enterprise",{"icon":267,"href":268,"dataGaLocation":136,"dataGaName":269},"Building","/enterprise/","enterprise",{"text":271,"config":272},"Small Business",{"icon":273,"href":274,"dataGaLocation":136,"dataGaName":275},"Work","/small-business/","small business",{"text":277,"config":278},"Public Sector",{"icon":279,"href":280,"dataGaLocation":136,"dataGaName":281},"Organization","/solutions/public-sector/","public sector",{"text":283,"config":284},"Pricing",{"href":285,"dataGaName":286,"dataGaLocation":136,"dataNavLevelOne":286},"/pricing/","pricing",{"text":288,"config":289,"menu":291},"Resources",{"dataNavLevelOne":290},"resources",{"type":187,"link":292,"columns":296,"feature":390},{"text":293,"config":294},"View all resources",{"href":295,"dataGaName":290,"dataGaLocation":136},"/resources/",[297,330,357],{"title":298,"items":299},"Getting started",[300,305,310,315,320,325],{"text":301,"config":302},"Install",{"href":303,"dataGaName":304,"dataGaLocation":136},"/install/","install",{"text":306,"config":307},"Quick start guides",{"href":308,"dataGaName":309,"dataGaLocation":136},"/get-started/","quick setup checklists",{"text":311,"config":312},"Learn",{"href":313,"dataGaLocation":136,"dataGaName":314},"https://university.gitlab.com/","learn",{"text":316,"config":317},"Product documentation",{"href":318,"dataGaName":319,"dataGaLocation":136},"https://docs.gitlab.com/","product documentation",{"text":321,"config":322},"Best practice videos",{"href":323,"dataGaName":324,"dataGaLocation":136},"/getting-started-videos/","best practice videos",{"text":326,"config":327},"Integrations",{"href":328,"dataGaName":329,"dataGaLocation":136},"/integrations/","integrations",{"title":331,"items":332},"Discover",[333,338,343,348,352],{"text":334,"config":335},"Customer success stories",{"href":336,"dataGaName":337,"dataGaLocation":136},"/customers/","customer success stories",{"text":339,"config":340},"Blog",{"href":341,"dataGaName":342,"dataGaLocation":136},"/blog/","blog",{"text":344,"config":345},"Demo Hub",{"href":346,"dataGaName":347,"dataGaLocation":136},"/demo-hub/","demo hub",{"text":349,"config":350},"The Source",{"href":351,"dataGaName":342,"dataGaLocation":136},"/the-source/",{"text":353,"config":354},"Remote",{"href":355,"dataGaName":356,"dataGaLocation":136},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"title":358,"items":359},"Connect",[360,365,370,375,380,385],{"text":361,"config":362},"GitLab Services",{"href":363,"dataGaName":364,"dataGaLocation":136},"/services/","services",{"text":366,"config":367},"Contribute",{"href":368,"dataGaName":369,"dataGaLocation":136},"https://contributors.gitlab.com","contribute",{"text":371,"config":372},"Community",{"href":373,"dataGaName":374,"dataGaLocation":136},"/community/","community",{"text":376,"config":377},"Forum",{"href":378,"dataGaName":379,"dataGaLocation":136},"https://forum.gitlab.com/","forum",{"text":381,"config":382},"Events",{"href":383,"dataGaName":384,"dataGaLocation":136},"/events/","events",{"text":386,"config":387},"Partners",{"href":388,"dataGaName":389,"dataGaLocation":136},"/partners/","partners",{"config":391,"title":394,"text":395,"link":396},{"background":392,"textColor":393},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1777322348/qpq8yrgn8knii57omj0c.png')","#000","What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":397,"config":398},"Read the latest",{"href":399,"dataGaName":400,"dataGaLocation":136},"/whats-new/","whats new",{"text":402,"config":403,"menu":404},"Company",{"dataNavLevelOne":109},{"type":187,"columns":405},[406],{"items":407},[408,413,419,421,426,431,436,441,446,451],{"text":409,"config":410},"About",{"href":411,"dataGaName":412,"dataGaLocation":136},"/company/","about",{"text":414,"config":415,"footerGa":418},"Jobs",{"href":416,"dataGaName":417,"dataGaLocation":136},"/jobs/","jobs",{"dataGaName":417},{"text":381,"config":420},{"href":383,"dataGaName":384,"dataGaLocation":136},{"text":422,"config":423},"Leadership",{"href":424,"dataGaName":425,"dataGaLocation":136},"/company/team/e-group/","leadership",{"text":427,"config":428},"Handbook",{"href":429,"dataGaName":430,"dataGaLocation":136},"https://handbook.gitlab.com/","handbook",{"text":432,"config":433},"Investor relations",{"href":434,"dataGaName":435,"dataGaLocation":136},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":437,"config":438},"Trust Center",{"href":439,"dataGaName":440,"dataGaLocation":136},"/security/","trust center",{"text":442,"config":443},"AI Transparency Center",{"href":444,"dataGaName":445,"dataGaLocation":136},"/ai-transparency-center/","ai transparency center",{"text":447,"config":448},"Newsletter",{"href":449,"dataGaName":450,"dataGaLocation":136},"/company/contact/#contact-forms","newsletter",{"text":452,"config":453},"Press",{"href":454,"dataGaName":455,"dataGaLocation":136},"/press/","press",{"text":457,"config":458,"menu":459},"Contact us",{"dataNavLevelOne":109},{"type":187,"columns":460},[461],{"items":462},[463,468,473],{"text":464,"config":465},"Talk to sales",{"href":466,"dataGaName":467,"dataGaLocation":136},"/sales/","talk to sales",{"text":469,"config":470},"Support portal",{"href":471,"dataGaName":472,"dataGaLocation":136},"https://support.gitlab.com/hc/en-us","support portal",{"text":474,"config":475},"Customer portal",{"href":476,"dataGaName":477,"dataGaLocation":136},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":479,"login":480,"suggestions":487},"Close",{"text":481,"link":482},"To search repositories and projects, login to",{"text":483,"config":484},"gitlab.com",{"href":150,"dataGaName":485,"dataGaLocation":486},"search login","search",{"text":488,"default":489},"Suggestions",[490,492,496,498,502,506],{"text":167,"config":491},{"href":172,"dataGaName":167,"dataGaLocation":486},{"text":493,"config":494},"Code Suggestions (AI)",{"href":495,"dataGaName":493,"dataGaLocation":486},"/solutions/code-suggestions/",{"text":203,"config":497},{"href":205,"dataGaName":203,"dataGaLocation":486},{"text":499,"config":500},"GitLab on AWS",{"href":501,"dataGaName":499,"dataGaLocation":486},"/partners/technology-partners/aws/",{"text":503,"config":504},"GitLab on Google Cloud",{"href":505,"dataGaName":503,"dataGaLocation":486},"/partners/technology-partners/google-cloud-platform/",{"text":507,"config":508},"Why GitLab?",{"href":180,"dataGaName":507,"dataGaLocation":486},{"freeTrial":510,"mobileIcon":515,"desktopIcon":520,"secondaryButton":523},{"text":511,"config":512},"Start free trial",{"href":513,"dataGaName":141,"dataGaLocation":514},"https://gitlab.com/-/trials/new/","nav",{"altText":516,"config":517},"Gitlab Icon",{"src":518,"dataGaName":519,"dataGaLocation":514},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":516,"config":521},{"src":522,"dataGaName":519,"dataGaLocation":514},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":524,"config":525},"Get Started",{"href":526,"dataGaName":527,"dataGaLocation":514},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":529,"mobileIcon":533,"desktopIcon":535},{"text":530,"config":531},"Learn more about GitLab Duo",{"href":172,"dataGaName":532,"dataGaLocation":514},"gitlab duo",{"altText":516,"config":534},{"src":518,"dataGaName":519,"dataGaLocation":514},{"altText":516,"config":536},{"src":522,"dataGaName":519,"dataGaLocation":514},{"button":538,"mobileIcon":543,"desktopIcon":545},{"text":539,"config":540},"/switch",{"href":541,"dataGaName":542,"dataGaLocation":514},"#contact","switch",{"altText":516,"config":544},{"src":518,"dataGaName":519,"dataGaLocation":514},{"altText":516,"config":546},{"src":547,"dataGaName":519,"dataGaLocation":514},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":549,"mobileIcon":554,"desktopIcon":556},{"text":550,"config":551},"Back to pricing",{"href":285,"dataGaName":552,"dataGaLocation":514,"icon":553},"back to pricing","GoBack",{"altText":516,"config":555},{"src":518,"dataGaName":519,"dataGaLocation":514},{"altText":516,"config":557},{"src":522,"dataGaName":519,"dataGaLocation":514},{"title":559,"titleMobile":560,"button":561,"config":566},"Duo Agent Platform delivers 400% ROI, per new Forrester Consulting study.","400% ROI: Forrester TEI for GitLab Duo",{"text":178,"config":562},{"href":563,"dataGaName":564,"dataGaLocation":565},"https://about.gitlab.com/blog/gitlab-duo-agent-platform-delivers-400-percent-roi/","forrester-tei-dap-banner","global-banner",{"layout":567,"disabled":114},"release",{"data":569},{"text":570,"source":571,"edit":577,"contribute":582,"config":587,"items":592,"minimal":802},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":572,"config":573},"View page source",{"href":574,"dataGaName":575,"dataGaLocation":576},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":578,"config":579},"Edit this page",{"href":580,"dataGaName":581,"dataGaLocation":576},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":583,"config":584},"Please contribute",{"href":585,"dataGaName":586,"dataGaLocation":576},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":588,"facebook":589,"youtube":590,"linkedin":591},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[593,640,694,738,770],{"title":283,"links":594,"subMenu":609},[595,599,604],{"text":596,"config":597},"View plans",{"href":285,"dataGaName":598,"dataGaLocation":576},"view plans",{"text":600,"config":601},"Why Premium?",{"href":602,"dataGaName":603,"dataGaLocation":576},"/pricing/premium/","why premium",{"text":605,"config":606},"Why Ultimate?",{"href":607,"dataGaName":608,"dataGaLocation":576},"/pricing/ultimate/","why ultimate",[610],{"title":611,"links":612},"Contact Us",[613,616,618,620,625,630,635],{"text":614,"config":615},"Contact sales",{"href":466,"dataGaName":146,"dataGaLocation":576},{"text":469,"config":617},{"href":471,"dataGaName":472,"dataGaLocation":576},{"text":474,"config":619},{"href":476,"dataGaName":477,"dataGaLocation":576},{"text":621,"config":622},"Status",{"href":623,"dataGaName":624,"dataGaLocation":576},"https://status.gitlab.com/","status",{"text":626,"config":627},"Terms of use",{"href":628,"dataGaName":629,"dataGaLocation":576},"/terms/","terms of use",{"text":631,"config":632},"Privacy statement",{"href":633,"dataGaName":634,"dataGaLocation":576},"/privacy/","privacy statement",{"text":636,"config":637},"Cookie preferences",{"dataGaName":638,"dataGaLocation":576,"id":639,"isOneTrustButton":121},"cookie preferences","ot-sdk-btn",{"title":183,"links":641,"subMenu":650},[642,646],{"text":643,"config":644},"DevSecOps platform",{"href":165,"dataGaName":645,"dataGaLocation":576},"devsecops platform",{"text":647,"config":648},"AI-Assisted Development",{"href":172,"dataGaName":649,"dataGaLocation":576},"ai-assisted development",[651],{"title":652,"links":653},"Topics",[654,659,664,669,674,679,684,689],{"text":655,"config":656},"CICD",{"href":657,"dataGaName":658,"dataGaLocation":576},"/topics/ci-cd/","cicd",{"text":660,"config":661},"GitOps",{"href":662,"dataGaName":663,"dataGaLocation":576},"/topics/gitops/","gitops",{"text":665,"config":666},"DevOps",{"href":667,"dataGaName":668,"dataGaLocation":576},"/topics/devops/","devops",{"text":670,"config":671},"Version Control",{"href":672,"dataGaName":673,"dataGaLocation":576},"/topics/version-control/","version control",{"text":675,"config":676},"DevSecOps",{"href":677,"dataGaName":678,"dataGaLocation":576},"/topics/devsecops/","devsecops",{"text":680,"config":681},"Cloud Native",{"href":682,"dataGaName":683,"dataGaLocation":576},"/topics/cloud-native/","cloud native",{"text":685,"config":686},"AI for Coding",{"href":687,"dataGaName":688,"dataGaLocation":576},"/topics/devops/ai-for-coding/","ai for coding",{"text":690,"config":691},"Agentic AI",{"href":692,"dataGaName":693,"dataGaLocation":576},"/topics/agentic-ai/","agentic ai",{"title":695,"links":696},"Solutions",[697,699,701,706,710,713,717,720,722,725,728,733],{"text":227,"config":698},{"href":222,"dataGaName":227,"dataGaLocation":576},{"text":216,"config":700},{"href":199,"dataGaName":200,"dataGaLocation":576},{"text":702,"config":703},"Agile development",{"href":704,"dataGaName":705,"dataGaLocation":576},"/solutions/agile-delivery/","agile delivery",{"text":707,"config":708},"SCM",{"href":212,"dataGaName":709,"dataGaLocation":576},"source code management",{"text":655,"config":711},{"href":205,"dataGaName":712,"dataGaLocation":576},"continuous integration & delivery",{"text":714,"config":715},"Value stream management",{"href":255,"dataGaName":716,"dataGaLocation":576},"value stream management",{"text":660,"config":718},{"href":719,"dataGaName":663,"dataGaLocation":576},"/solutions/gitops/",{"text":265,"config":721},{"href":268,"dataGaName":269,"dataGaLocation":576},{"text":723,"config":724},"Small business",{"href":274,"dataGaName":275,"dataGaLocation":576},{"text":726,"config":727},"Public sector",{"href":280,"dataGaName":281,"dataGaLocation":576},{"text":729,"config":730},"Education",{"href":731,"dataGaName":732,"dataGaLocation":576},"/solutions/education/","education",{"text":734,"config":735},"Financial services",{"href":736,"dataGaName":737,"dataGaLocation":576},"/solutions/finance/","financial services",{"title":288,"links":739},[740,742,744,746,749,751,754,756,758,760,762,764,766,768],{"text":301,"config":741},{"href":303,"dataGaName":304,"dataGaLocation":576},{"text":306,"config":743},{"href":308,"dataGaName":309,"dataGaLocation":576},{"text":311,"config":745},{"href":313,"dataGaName":314,"dataGaLocation":576},{"text":316,"config":747},{"href":318,"dataGaName":748,"dataGaLocation":576},"docs",{"text":339,"config":750},{"href":341,"dataGaName":342,"dataGaLocation":576},{"text":752,"config":753},"What's new",{"href":399,"dataGaName":400,"dataGaLocation":576},{"text":334,"config":755},{"href":336,"dataGaName":337,"dataGaLocation":576},{"text":353,"config":757},{"href":355,"dataGaName":356,"dataGaLocation":576},{"text":361,"config":759},{"href":363,"dataGaName":364,"dataGaLocation":576},{"text":366,"config":761},{"href":368,"dataGaName":369,"dataGaLocation":576},{"text":371,"config":763},{"href":373,"dataGaName":374,"dataGaLocation":576},{"text":376,"config":765},{"href":378,"dataGaName":379,"dataGaLocation":576},{"text":381,"config":767},{"href":383,"dataGaName":384,"dataGaLocation":576},{"text":386,"config":769},{"href":388,"dataGaName":389,"dataGaLocation":576},{"title":402,"links":771},[772,774,776,778,780,782,786,791,793,795,797],{"text":409,"config":773},{"href":411,"dataGaName":109,"dataGaLocation":576},{"text":414,"config":775},{"href":416,"dataGaName":417,"dataGaLocation":576},{"text":422,"config":777},{"href":424,"dataGaName":425,"dataGaLocation":576},{"text":427,"config":779},{"href":429,"dataGaName":430,"dataGaLocation":576},{"text":432,"config":781},{"href":434,"dataGaName":435,"dataGaLocation":576},{"text":783,"config":784},"Sustainability",{"href":785,"dataGaName":783,"dataGaLocation":576},"/sustainability/",{"text":787,"config":788},"Diversity, inclusion and belonging (DIB)",{"href":789,"dataGaName":790,"dataGaLocation":576},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":437,"config":792},{"href":439,"dataGaName":440,"dataGaLocation":576},{"text":447,"config":794},{"href":449,"dataGaName":450,"dataGaLocation":576},{"text":452,"config":796},{"href":454,"dataGaName":455,"dataGaLocation":576},{"text":798,"config":799},"Modern Slavery Transparency Statement",{"href":800,"dataGaName":801,"dataGaLocation":576},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":803},[804,807,810],{"text":805,"config":806},"Terms",{"href":628,"dataGaName":629,"dataGaLocation":576},{"text":808,"config":809},"Cookies",{"dataGaName":638,"dataGaLocation":576,"id":639,"isOneTrustButton":121},{"text":811,"config":812},"Privacy",{"href":633,"dataGaName":634,"dataGaLocation":576},[814],{"id":815,"title":7,"body":113,"config":816,"content":818,"description":113,"extension":822,"meta":823,"navigation":121,"path":824,"seo":825,"stem":826,"__hash__":827},"blogAuthors/en-us/blog/authors/jacob-vosmaer.yml",{"template":817},"BlogAuthor",{"name":7,"config":819},{"headshot":820,"ctfId":821},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749659488/Blog/Author%20Headshots/gitlab-logo-extra-whitespace.png","Jacob-Vosmaer","yml",{},"/en-us/blog/authors/jacob-vosmaer",{},"en-us/blog/authors/jacob-vosmaer","8CR6ShwBsKxqGM4AAocQnzaLp61rqybzr4XSir9Y3Ag",[829,837,844],{"title":830,"description":831,"heroImage":832,"category":109,"date":833,"authors":834,"slug":836,"externalUrl":113},"Why GitLab signed the Open Weights and American AI Leadership letter","GitLab supports customer choice, data privacy, and AI model neutrality, empowering teams to run secure, cost-effective, local solutions.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1785333264/ww4mlighwlvvzgnmbe6w.png","2026-07-29",[835],"Bill Staples","open-weight-model-letter",{"title":838,"description":839,"heroImage":840,"category":109,"date":841,"authors":842,"slug":843,"externalUrl":113},"GitLab Act 2","A letter to our customers and our investors.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1776362275/ozbwn9tk0dditpnfddlz.png","2026-05-11",[835],"gitlab-act-2",{"title":845,"description":846,"heroImage":847,"category":109,"date":848,"authors":849,"slug":851,"externalUrl":113},"GitLab named a 2026 Omdia Universe Leader","Omdia's 2026 report on AI-assisted software development ranked 19 vendors. Here is what GitLab's top scores mean for engineering teams.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-04-13",[850],"Rebecca Carter","gitlab-named-a-2026-omdia-universe-leader",{"promotions":853},[854,868,880,892],{"id":855,"categories":856,"header":858,"text":859,"button":860,"image":865},"ai-modernization",[857],"ai","Is AI achieving its promise at scale?","Quiz will take 5 minutes or less",{"text":861,"config":862},"Get your AI maturity score",{"href":863,"dataGaName":864,"dataGaLocation":342},"/assessments/ai-modernization-assessment/","modernization assessment",{"config":866},{"src":867},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/qix0m7kwnd8x2fh1zq49.png",{"id":869,"categories":870,"header":872,"text":859,"button":873,"image":877},"devops-modernization",[871,678],"product","Are you just managing tools or shipping innovation?",{"text":874,"config":875},"Get your DevOps maturity score",{"href":876,"dataGaName":864,"dataGaLocation":342},"/assessments/devops-modernization-assessment/",{"config":878},{"src":879},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138785/eg818fmakweyuznttgid.png",{"id":881,"categories":882,"header":884,"text":859,"button":885,"image":889},"security-modernization",[883],"security","Are you trading speed for security?",{"text":886,"config":887},"Get your security maturity score",{"href":888,"dataGaName":864,"dataGaLocation":342},"/assessments/security-modernization-assessment/",{"config":890},{"src":891},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/p4pbqd9nnjejg5ds6mdk.png",{"id":893,"paths":894,"header":897,"text":898,"button":899,"image":904},"github-azure-migration",[895,896],"migration-from-azure-devops-to-gitlab","integrating-azure-devops-scm-and-gitlab","Is your team ready for GitHub's Azure move?","GitHub is already rebuilding around Azure. Find out what it means for you.",{"text":900,"config":901},"See how GitLab compares to GitHub",{"href":902,"dataGaName":903,"dataGaLocation":342},"/compare/gitlab-vs-github/github-azure-migration/","github azure migration",{"config":905},{"src":879},{"header":907,"blurb":908,"button":909,"secondaryButton":914},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":910,"config":911},"Get your free trial",{"href":912,"dataGaName":141,"dataGaLocation":913},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":614,"config":915},{"href":466,"dataGaName":146,"dataGaLocation":913},1786803767839]