[{"data":1,"prerenderedAt":1223},["ShallowReactive",2],{"/blog/gitlab-dot-com-database-incident":3,"navigation-en-us":435,"banner-en-us":863,"footer-en-us":873,"blog-post-authors-en-us-GitLab":1118,"blog-related-posts-en-us-gitlab-dot-com-database-incident":1133,"blog-promotions-en-us":1159,"next-steps-en-us":1213},{"id":4,"title":5,"authors":6,"body":8,"category":404,"date":405,"description":406,"extension":407,"externalUrl":408,"faq":408,"featured":409,"heroImage":410,"meta":411,"navigation":253,"path":426,"seo":427,"slug":431,"stem":432,"tags":408,"template":433,"updatedDate":408,"__hash__":434},"blogPosts/en-us/blog/gitlab-dot-com-database-incident.md","GitLab.com database incident",[7],"GitLab",{"type":9,"value":10,"toc":391},"minimark",[11,20,23,32,35,40,49,54,57,64,67,72,77,90,94,97,102,107,110,196,200,217,223,226,256,260,306,310,313,331,376,379,384],[12,13,14,15],"p",{},"Update: please see ",[16,17,19],"a",{"href":18},"/blog/postmortem-of-database-outage-of-january-31/","our postmortem for this incident",[12,21,22],{},"Yesterday we had a serious incident with one of our databases. We lost six hours of database data (issues, merge requests, users, comments, snippets, etc.) for GitLab.com. Git/wiki repositories and self-managed installations were not affected. Losing production data is unacceptable and in a few days we'll publish a post on why this happened and a list of measures we will implement to prevent it happening again.",[12,24,25],{},[26,27,28],"em",{},[29,30,31],"strong",{},"Update 6:14pm UTC: GitLab.com is back online",[12,33,34],{},"As of time of writing, we’re restoring data from a six-hour-old backup of our database. This means that any data between 5:20pm UTC and 11:25pm UTC from the database (projects, issues, merge requests, users, comments, snippets, etc.) is lost by the time GitLab.com is live again.",[12,36,37],{},[29,38,39],{},"Git data (repositories and wikis) and self-managed instances of GitLab are not affected.",[12,41,42,43,48],{},"Read below for a brief summary of the events. You’re also welcome to view ",[16,44,47],{"href":45,"rel":46},"https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCxIABGiryG7_z_6jHdVik/pub",[],"our active postmortem doc",".",[50,51,53],"h2",{"id":52},"first-incident","First incident",[12,55,56],{},"At 2017/01/31 6pm UTC, we detected that spammers were hammering the database by creating snippets, making it unstable. We then started troubleshooting to understand what the problem was and how to fight it.",[12,58,59],{},[60,61],"img",{"alt":62,"src":63},"","https://about.gitlab.comdb_incident/snippets.png",[12,65,66],{},"At 2017/01/31 9pm UTC, this escalated, causing a lockup on writes on the database, which caused some downtime.",[12,68,69],{},[60,70],{"alt":62,"src":71},"https://about.gitlab.comdb_incident/locks.png",[73,74,76],"h3",{"id":75},"actions-taken","Actions taken",[78,79,80,84,87],"ul",{},[81,82,83],"li",{},"We blocked the spammers based on IP address",[81,85,86],{},"We removed a user for using a repository as some form of CDN, resulting in 47 000 IPs signing in using the same account (causing high DB load)",[81,88,89],{},"We removed users for spamming (by creating snippets)",[50,91,93],{"id":92},"second-incident","Second incident",[12,95,96],{},"At 2017/01/31 10pm UTC, we got paged because DB Replication lagged too far behind, effectively stopping. This happened because there was a spike in writes that were not processed ontime by the secondary database.",[12,98,99],{},[60,100],{"alt":62,"src":101},"https://about.gitlab.comdb_incident/used.png",[12,103,104],{},[60,105],{"alt":62,"src":106},"https://about.gitlab.comdb_incident/rep_lag.png",[73,108,76],{"id":109},"actions-taken-1",[78,111,112,120,130,147,163,166,184,189],{},[81,113,114,115,119],{},"Attempt to fix ",[116,117,118],"code",{},"db2",", it’s lagging behind by about 4 GB at this point",[81,121,122,125,126,129],{},[116,123,124],{},"db2.cluster"," refuses to replicate, ",[116,127,128],{},"/var/opt/gitlab/postgresql/data"," is wiped to ensure a clean replication",[81,131,132,134,135,138,139,142,143,146],{},[116,133,124],{}," refuses to connect to ",[116,136,137],{},"db1",", complaining about ",[116,140,141],{},"max_wal_senders"," being too low. This setting is used to limit the number of ",[116,144,145],{},"WAL (= replication)"," clients",[81,148,149,152,153,155,156,159,160,162],{},[26,150,151],{},"Team-member-1"," adjusts ",[116,154,141],{}," to ",[116,157,158],{},"32"," on ",[116,161,137],{},", restarts PostgreSQL",[81,164,165],{},"PostgreSQL complains about too many semaphores being open, refusing to start",[81,167,168,152,170,155,173,176,177,180,181,183],{},[26,169,151],{},[116,171,172],{},"max_connections",[116,174,175],{},"2000"," from ",[116,178,179],{},"8000",", PostgreSQL starts again (despite ",[116,182,179],{}," having been used for almost a year)",[81,185,186,188],{},[116,187,124],{}," still refuses to replicate, though it no longer complains about connections; instead it just hangs there not doing anything",[81,190,191,192,195],{},"At this point frustration begins to kick in. Earlier this night ",[26,193,194],{},"team-member-1"," explicitly mentioned he was going to sign off as it was getting late (23:00 or so local time), but didn’t due to the replication problems popping up all of a sudden.",[50,197,199],{"id":198},"third-incident","Third incident",[12,201,202,203,205,206,209,210,213,214,48],{},"At 2017/01/31 11pm-ish UTC, ",[26,204,194],{}," thinks that perhaps ",[116,207,208],{},"pg_basebackup"," is refusing to work due to the PostgreSQL data directory being present (despite being empty), decides to remove the directory. After a second or two he notices he ran it on ",[116,211,212],{},"db1.cluster.gitlab.com",", instead of ",[116,215,216],{},"db2.cluster.gitlab.com",[12,218,219,220,222],{},"At 2017/01/31 11:27pm UTC, ",[26,221,194],{}," - terminates the removal, but it’s too late. Of around 300 GB only about 4.5 GB is left.",[12,224,225],{},"We had to bring GitLab.com down and shared this information on Twitter:",[227,228,231,250],"div",{"className":229},[230],"center",[232,233,237,245,246],"blockquote",{"className":234,"dataLang":236},[235],"twitter-tweet","en",[12,238,240,241,244],{"lang":236,"dir":239},"ltr","We are performing emergency database maintenance, ",[16,242,243],{"href":243},"https://t.co/r11UmmDLDE"," will be taken offline","— GitLab.com Status (@gitlabstatus) ",[16,247,249],{"href":248},"https://twitter.com/gitlabstatus/status/826572933304827904","January 31, 2017",[251,252],"script",{"async":253,"src":254,"charSet":255},true,"//platform.twitter.com/widgets.js","utf-8",[50,257,259],{"id":258},"problems-encountered","Problems encountered",[78,261,262,268,278,288,291,294,297,300,303],{},[81,263,264,265,267],{},"LVM snapshots are by default only taken once every 24 hours. ",[26,266,151],{}," happened to run one manually about six hours prior to the outage because he was working in load balancing for the database.",[81,269,270,271,273,274,277],{},"Regular backups seem to also only be taken once per 24 hours, though ",[26,272,194],{}," has not yet been able to figure out where they are stored. According to ",[26,275,276],{},"team-member-2"," these don’t appear to be working, producing files only a few bytes in size.",[81,279,280,283,284,287],{},[26,281,282],{},"Team-member-3",": It looks like ",[116,285,286],{},"pg_dump"," may be failing because PostgreSQL 9.2 binaries are being run instead of 9.6 binaries. This happens because omnibus only uses Pg 9.6 if data/PG_VERSION is set to 9.6, but on workers this file does not exist. As a result it defaults to 9.2, failing silently. No SQL dumps were made as a result. Fog gem may have cleaned out older backups.",[81,289,290],{},"Disk snapshots in Azure are enabled for the NFS server, but not for the DB servers.",[81,292,293],{},"The synchronisation process removes webhooks once it has synchronised data to staging. Unless we can pull these from a regular backup from the past 24 hours they will be lost",[81,295,296],{},"The replication procedure is super fragile, prone to error, relies on a handful of random shell scripts, and is badly documented",[81,298,299],{},"Our backups to S3 apparently don’t work either: the bucket is empty",[81,301,302],{},"So in other words, out of five backup/replication techniques deployed none are working reliably or set up in the first place. We ended up restoring a six-hour-old backup.",[81,304,305],{},"pg_basebackup will silently wait for a master to initiate the replication progress, according to another production engineer this can take up to 10 minutes. This can lead to one thinking the process is stuck somehow. Running the process using “strace” provided no useful information about what might be going on.",[50,307,309],{"id":308},"recovery","Recovery",[12,311,312],{},"We’re working on recovering right now by using a backup of the database from a staging database.",[227,314,316,329],{"className":315},[230],[232,317,319,245,325],{"className":318,"dataLang":236},[235],[12,320,321,322],{"lang":236,"dir":239},"We accidentally deleted production data and might have to restore from backup. Google Doc with live notes ",[16,323,324],{"href":324},"https://t.co/EVRbHzYlk8",[16,326,328],{"href":327},"https://twitter.com/gitlabstatus/status/826591961444384768","February 1, 2017",[251,330],{"async":253,"src":254,"charSet":255},[78,332,333,340,347,356,366],{},[81,334,335,336,339],{},"2017/02/01 00:36 - Backup ",[116,337,338],{},"db1.staging.gitlab.com"," data",[81,341,342,343,159,345],{},"2017/02/01 00:55 - Mount ",[116,344,338],{},[116,346,212],{},[81,348,349,350,353,354],{},"Copy data from staging ",[116,351,352],{},"/var/opt/gitlab/postgresql/data/"," to production ",[116,355,352],{},[81,357,358,359,362,363],{},"2017/02/01 01:05 - ",[116,360,361],{},"nfs-share01"," server commandeered as temp storage place in ",[116,364,365],{},"/var/opt/gitlab/db-meltdown",[81,367,368,369,372,373],{},"2017/02/01 01:18 - Copy of remaining production data, including ",[116,370,371],{},"pg_xlog"," tar’ed up as ",[116,374,375],{},"20170131-db-meltodwn-backup.tar.gz",[12,377,378],{},"Below a graph showing the time of deletion and subsequent copying in of data.",[12,380,381],{},[60,382],{"alt":62,"src":383},"https://about.gitlab.comdb_incident/delete.png",[12,385,386],{},[16,387,390],{"href":388,"rel":389},"https://twitter.com/i/moments/826818668948549632",[],"Also, we'd like to thank everyone for the amazing support we've received on Twitter and elsewhere through #hugops",{"title":62,"searchDepth":392,"depth":392,"links":393},2,[394,398,401,402,403],{"id":52,"depth":392,"text":53,"children":395},[396],{"id":75,"depth":397,"text":76},3,{"id":92,"depth":392,"text":93,"children":399},[400],{"id":109,"depth":397,"text":76},{"id":198,"depth":392,"text":199},{"id":258,"depth":392,"text":259},{"id":308,"depth":392,"text":309},"engineering","2017-02-01","Yesterday we had a serious incident with one of our databases. We lost six hours of database data (issues, merge requests, users, comments, snippets, etc.) for GitLab.com.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749663397/Blog/Hero%20Images/logoforblogpost.jpg",{"excerpt":412},{"type":9,"value":413},[414,418,420],[12,415,14,416],{},[16,417,19],{"href":18},[12,419,22],{},[12,421,422],{},[26,423,424],{},[29,425,31],{},"/en-us/blog/gitlab-dot-com-database-incident",{"title":5,"description":406,"ogTitle":5,"ogDescription":406,"noIndex":409,"ogImage":410,"ogUrl":428,"ogSiteName":429,"ogType":430,"canonicalUrls":428},"https://about.gitlab.com/blog/gitlab-dot-com-database-incident","https://about.gitlab.com","article","gitlab-dot-com-database-incident","en-us/blog/gitlab-dot-com-database-incident","BlogPost","avrRNG2GBnu8YfRkS2SGXXb98tlNP2NFNyXmQ2KjQFE",{"logo":436,"freeTrial":441,"sales":446,"login":451,"items":456,"search":783,"minimal":814,"duo":833,"switchNav":842,"pricingDeployment":853},{"config":437},{"href":438,"dataGaName":439,"dataGaLocation":440},"/","gitlab logo","header",{"text":442,"config":443},"Get free trial",{"href":444,"dataGaName":445,"dataGaLocation":440},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":447,"config":448},"Request a demo",{"href":449,"dataGaName":450,"dataGaLocation":440},"/sales/?contact-topic=request-demo","sales",{"text":452,"config":453},"Sign in",{"href":454,"dataGaName":455,"dataGaLocation":440},"https://gitlab.com/users/sign_in/","sign in",[457,486,586,591,705,761],{"text":458,"config":459,"menu":461},"Platform",{"dataNavLevelOne":460},"platform",{"type":462,"columns":463},"cards",[464,470,478],{"title":458,"description":465,"link":466},"The intelligent orchestration platform for DevSecOps",{"text":467,"config":468},"Explore our Platform",{"href":469,"dataGaName":460,"dataGaLocation":440},"/platform/",{"title":471,"description":472,"link":473},"GitLab Duo Agent Platform","Agentic AI for the entire software lifecycle",{"text":474,"config":475},"Meet GitLab Duo",{"href":476,"dataGaName":477,"dataGaLocation":440},"/gitlab-duo-agent-platform/","gitlab duo agent platform",{"title":479,"description":480,"link":481},"Why GitLab","See the top reasons enterprises choose GitLab",{"text":482,"config":483},"Learn more",{"href":484,"dataGaName":485,"dataGaLocation":440},"/why-gitlab/","why gitlab",{"text":487,"left":253,"config":488,"menu":490},"Product",{"dataNavLevelOne":489},"solutions",{"type":491,"link":492,"columns":496,"feature":565},"lists",{"text":493,"config":494},"View all Solutions",{"href":495,"dataGaName":489,"dataGaLocation":440},"/solutions/",[497,521,544],{"title":498,"description":499,"link":500,"items":505},"Automation","CI/CD and automation to accelerate deployment",{"config":501},{"icon":502,"href":503,"dataGaName":504,"dataGaLocation":440},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[506,510,513,517],{"text":507,"config":508},"CI/CD",{"href":509,"dataGaLocation":440,"dataGaName":507},"/solutions/continuous-integration/",{"text":471,"config":511},{"href":476,"dataGaLocation":440,"dataGaName":512},"gitlab duo agent platform - product menu",{"text":514,"config":515},"Source Code Management",{"href":516,"dataGaLocation":440,"dataGaName":514},"/solutions/source-code-management/",{"text":518,"config":519},"Automated Software Delivery",{"href":503,"dataGaLocation":440,"dataGaName":520},"Automated software delivery",{"title":522,"description":523,"link":524,"items":529},"Security","Deliver code faster without compromising security",{"config":525},{"href":526,"dataGaName":527,"dataGaLocation":440,"icon":528},"/solutions/application-security-testing/","security and compliance","ShieldCheckLight",[530,534,539],{"text":531,"config":532},"Application Security Testing",{"href":526,"dataGaName":533,"dataGaLocation":440},"Application security testing",{"text":535,"config":536},"Software Supply Chain Security",{"href":537,"dataGaLocation":440,"dataGaName":538},"/solutions/supply-chain/","Software supply chain security",{"text":540,"config":541},"Software Compliance",{"href":542,"dataGaName":543,"dataGaLocation":440},"/solutions/software-compliance/","software compliance",{"title":545,"link":546,"items":551},"Measurement",{"config":547},{"icon":548,"href":549,"dataGaName":550,"dataGaLocation":440},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[552,556,560],{"text":553,"config":554},"Visibility & Measurement",{"href":549,"dataGaLocation":440,"dataGaName":555},"Visibility and Measurement",{"text":557,"config":558},"Value Stream Management",{"href":559,"dataGaLocation":440,"dataGaName":557},"/solutions/value-stream-management/",{"text":561,"config":562},"Analytics & Insights",{"href":563,"dataGaLocation":440,"dataGaName":564},"/solutions/analytics-and-insights/","Analytics and insights",{"title":566,"type":491,"items":567},"GitLab for",[568,574,580],{"text":569,"config":570},"Enterprise",{"icon":571,"href":572,"dataGaLocation":440,"dataGaName":573},"Building","/enterprise/","enterprise",{"text":575,"config":576},"Small Business",{"icon":577,"href":578,"dataGaLocation":440,"dataGaName":579},"Work","/small-business/","small business",{"text":581,"config":582},"Public Sector",{"icon":583,"href":584,"dataGaLocation":440,"dataGaName":585},"Organization","/solutions/public-sector/","public sector",{"text":587,"config":588},"Pricing",{"href":589,"dataGaName":590,"dataGaLocation":440,"dataNavLevelOne":590},"/pricing/","pricing",{"text":592,"config":593,"menu":595},"Resources",{"dataNavLevelOne":594},"resources",{"type":491,"link":596,"columns":600,"feature":694},{"text":597,"config":598},"View all resources",{"href":599,"dataGaName":594,"dataGaLocation":440},"/resources/",[601,634,661],{"title":602,"items":603},"Getting started",[604,609,614,619,624,629],{"text":605,"config":606},"Install",{"href":607,"dataGaName":608,"dataGaLocation":440},"/install/","install",{"text":610,"config":611},"Quick start guides",{"href":612,"dataGaName":613,"dataGaLocation":440},"/get-started/","quick setup checklists",{"text":615,"config":616},"Learn",{"href":617,"dataGaLocation":440,"dataGaName":618},"https://university.gitlab.com/","learn",{"text":620,"config":621},"Product documentation",{"href":622,"dataGaName":623,"dataGaLocation":440},"https://docs.gitlab.com/","product documentation",{"text":625,"config":626},"Best practice videos",{"href":627,"dataGaName":628,"dataGaLocation":440},"/getting-started-videos/","best practice videos",{"text":630,"config":631},"Integrations",{"href":632,"dataGaName":633,"dataGaLocation":440},"/integrations/","integrations",{"title":635,"items":636},"Discover",[637,642,647,652,656],{"text":638,"config":639},"Customer success stories",{"href":640,"dataGaName":641,"dataGaLocation":440},"/customers/","customer success stories",{"text":643,"config":644},"Blog",{"href":645,"dataGaName":646,"dataGaLocation":440},"/blog/","blog",{"text":648,"config":649},"Demo Hub",{"href":650,"dataGaName":651,"dataGaLocation":440},"/demo-hub/","demo hub",{"text":653,"config":654},"The Source",{"href":655,"dataGaName":646,"dataGaLocation":440},"/the-source/",{"text":657,"config":658},"Remote",{"href":659,"dataGaName":660,"dataGaLocation":440},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"title":662,"items":663},"Connect",[664,669,674,679,684,689],{"text":665,"config":666},"GitLab Services",{"href":667,"dataGaName":668,"dataGaLocation":440},"/services/","services",{"text":670,"config":671},"Contribute",{"href":672,"dataGaName":673,"dataGaLocation":440},"https://contributors.gitlab.com","contribute",{"text":675,"config":676},"Community",{"href":677,"dataGaName":678,"dataGaLocation":440},"/community/","community",{"text":680,"config":681},"Forum",{"href":682,"dataGaName":683,"dataGaLocation":440},"https://forum.gitlab.com/","forum",{"text":685,"config":686},"Events",{"href":687,"dataGaName":688,"dataGaLocation":440},"/events/","events",{"text":690,"config":691},"Partners",{"href":692,"dataGaName":693,"dataGaLocation":440},"/partners/","partners",{"config":695,"title":698,"text":699,"link":700},{"background":696,"textColor":697},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1777322348/qpq8yrgn8knii57omj0c.png')","#000","What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":701,"config":702},"Read the latest",{"href":703,"dataGaName":704,"dataGaLocation":440},"/whats-new/","whats new",{"text":706,"config":707,"menu":709},"Company",{"dataNavLevelOne":708},"company",{"type":491,"columns":710},[711],{"items":712},[713,718,724,726,731,736,741,746,751,756],{"text":714,"config":715},"About",{"href":716,"dataGaName":717,"dataGaLocation":440},"/company/","about",{"text":719,"config":720,"footerGa":723},"Jobs",{"href":721,"dataGaName":722,"dataGaLocation":440},"/jobs/","jobs",{"dataGaName":722},{"text":685,"config":725},{"href":687,"dataGaName":688,"dataGaLocation":440},{"text":727,"config":728},"Leadership",{"href":729,"dataGaName":730,"dataGaLocation":440},"/company/team/e-group/","leadership",{"text":732,"config":733},"Handbook",{"href":734,"dataGaName":735,"dataGaLocation":440},"https://handbook.gitlab.com/","handbook",{"text":737,"config":738},"Investor relations",{"href":739,"dataGaName":740,"dataGaLocation":440},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":742,"config":743},"Trust Center",{"href":744,"dataGaName":745,"dataGaLocation":440},"/security/","trust center",{"text":747,"config":748},"AI Transparency Center",{"href":749,"dataGaName":750,"dataGaLocation":440},"/ai-transparency-center/","ai transparency center",{"text":752,"config":753},"Newsletter",{"href":754,"dataGaName":755,"dataGaLocation":440},"/company/contact/#contact-forms","newsletter",{"text":757,"config":758},"Press",{"href":759,"dataGaName":760,"dataGaLocation":440},"/press/","press",{"text":762,"config":763,"menu":764},"Contact us",{"dataNavLevelOne":708},{"type":491,"columns":765},[766],{"items":767},[768,773,778],{"text":769,"config":770},"Talk to sales",{"href":771,"dataGaName":772,"dataGaLocation":440},"/sales/","talk to sales",{"text":774,"config":775},"Support portal",{"href":776,"dataGaName":777,"dataGaLocation":440},"https://support.gitlab.com/hc/en-us","support portal",{"text":779,"config":780},"Customer portal",{"href":781,"dataGaName":782,"dataGaLocation":440},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":784,"login":785,"suggestions":792},"Close",{"text":786,"link":787},"To search repositories and projects, login to",{"text":788,"config":789},"gitlab.com",{"href":454,"dataGaName":790,"dataGaLocation":791},"search login","search",{"text":793,"default":794},"Suggestions",[795,797,801,803,807,811],{"text":471,"config":796},{"href":476,"dataGaName":471,"dataGaLocation":791},{"text":798,"config":799},"Code Suggestions (AI)",{"href":800,"dataGaName":798,"dataGaLocation":791},"/solutions/code-suggestions/",{"text":507,"config":802},{"href":509,"dataGaName":507,"dataGaLocation":791},{"text":804,"config":805},"GitLab on AWS",{"href":806,"dataGaName":804,"dataGaLocation":791},"/partners/technology-partners/aws/",{"text":808,"config":809},"GitLab on Google Cloud",{"href":810,"dataGaName":808,"dataGaLocation":791},"/partners/technology-partners/google-cloud-platform/",{"text":812,"config":813},"Why GitLab?",{"href":484,"dataGaName":812,"dataGaLocation":791},{"freeTrial":815,"mobileIcon":820,"desktopIcon":825,"secondaryButton":828},{"text":816,"config":817},"Start free trial",{"href":818,"dataGaName":445,"dataGaLocation":819},"https://gitlab.com/-/trials/new/","nav",{"altText":821,"config":822},"Gitlab Icon",{"src":823,"dataGaName":824,"dataGaLocation":819},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":821,"config":826},{"src":827,"dataGaName":824,"dataGaLocation":819},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":829,"config":830},"Get Started",{"href":831,"dataGaName":832,"dataGaLocation":819},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":834,"mobileIcon":838,"desktopIcon":840},{"text":835,"config":836},"Learn more about GitLab Duo",{"href":476,"dataGaName":837,"dataGaLocation":819},"gitlab duo",{"altText":821,"config":839},{"src":823,"dataGaName":824,"dataGaLocation":819},{"altText":821,"config":841},{"src":827,"dataGaName":824,"dataGaLocation":819},{"button":843,"mobileIcon":848,"desktopIcon":850},{"text":844,"config":845},"/switch",{"href":846,"dataGaName":847,"dataGaLocation":819},"#contact","switch",{"altText":821,"config":849},{"src":823,"dataGaName":824,"dataGaLocation":819},{"altText":821,"config":851},{"src":852,"dataGaName":824,"dataGaLocation":819},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":854,"mobileIcon":859,"desktopIcon":861},{"text":855,"config":856},"Back to pricing",{"href":589,"dataGaName":857,"dataGaLocation":819,"icon":858},"back to pricing","GoBack",{"altText":821,"config":860},{"src":823,"dataGaName":824,"dataGaLocation":819},{"altText":821,"config":862},{"src":827,"dataGaName":824,"dataGaLocation":819},{"title":864,"titleMobile":865,"button":866,"config":871},"Duo Agent Platform delivers 400% ROI, per new Forrester Consulting study.","400% ROI: Forrester TEI for GitLab Duo",{"text":482,"config":867},{"href":868,"dataGaName":869,"dataGaLocation":870},"https://about.gitlab.com/blog/gitlab-duo-agent-platform-delivers-400-percent-roi/","forrester-tei-dap-banner","global-banner",{"layout":872,"disabled":409},"release",{"data":874},{"text":875,"source":876,"edit":882,"contribute":887,"config":892,"items":897,"minimal":1107},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":877,"config":878},"View page source",{"href":879,"dataGaName":880,"dataGaLocation":881},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":883,"config":884},"Edit this page",{"href":885,"dataGaName":886,"dataGaLocation":881},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":888,"config":889},"Please contribute",{"href":890,"dataGaName":891,"dataGaLocation":881},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":893,"facebook":894,"youtube":895,"linkedin":896},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[898,945,999,1043,1075],{"title":587,"links":899,"subMenu":914},[900,904,909],{"text":901,"config":902},"View plans",{"href":589,"dataGaName":903,"dataGaLocation":881},"view plans",{"text":905,"config":906},"Why Premium?",{"href":907,"dataGaName":908,"dataGaLocation":881},"/pricing/premium/","why premium",{"text":910,"config":911},"Why Ultimate?",{"href":912,"dataGaName":913,"dataGaLocation":881},"/pricing/ultimate/","why ultimate",[915],{"title":916,"links":917},"Contact Us",[918,921,923,925,930,935,940],{"text":919,"config":920},"Contact sales",{"href":771,"dataGaName":450,"dataGaLocation":881},{"text":774,"config":922},{"href":776,"dataGaName":777,"dataGaLocation":881},{"text":779,"config":924},{"href":781,"dataGaName":782,"dataGaLocation":881},{"text":926,"config":927},"Status",{"href":928,"dataGaName":929,"dataGaLocation":881},"https://status.gitlab.com/","status",{"text":931,"config":932},"Terms of use",{"href":933,"dataGaName":934,"dataGaLocation":881},"/terms/","terms of use",{"text":936,"config":937},"Privacy statement",{"href":938,"dataGaName":939,"dataGaLocation":881},"/privacy/","privacy statement",{"text":941,"config":942},"Cookie preferences",{"dataGaName":943,"dataGaLocation":881,"id":944,"isOneTrustButton":253},"cookie preferences","ot-sdk-btn",{"title":487,"links":946,"subMenu":955},[947,951],{"text":948,"config":949},"DevSecOps platform",{"href":469,"dataGaName":950,"dataGaLocation":881},"devsecops platform",{"text":952,"config":953},"AI-Assisted Development",{"href":476,"dataGaName":954,"dataGaLocation":881},"ai-assisted development",[956],{"title":957,"links":958},"Topics",[959,964,969,974,979,984,989,994],{"text":960,"config":961},"CICD",{"href":962,"dataGaName":963,"dataGaLocation":881},"/topics/ci-cd/","cicd",{"text":965,"config":966},"GitOps",{"href":967,"dataGaName":968,"dataGaLocation":881},"/topics/gitops/","gitops",{"text":970,"config":971},"DevOps",{"href":972,"dataGaName":973,"dataGaLocation":881},"/topics/devops/","devops",{"text":975,"config":976},"Version Control",{"href":977,"dataGaName":978,"dataGaLocation":881},"/topics/version-control/","version control",{"text":980,"config":981},"DevSecOps",{"href":982,"dataGaName":983,"dataGaLocation":881},"/topics/devsecops/","devsecops",{"text":985,"config":986},"Cloud Native",{"href":987,"dataGaName":988,"dataGaLocation":881},"/topics/cloud-native/","cloud native",{"text":990,"config":991},"AI for Coding",{"href":992,"dataGaName":993,"dataGaLocation":881},"/topics/devops/ai-for-coding/","ai for coding",{"text":995,"config":996},"Agentic AI",{"href":997,"dataGaName":998,"dataGaLocation":881},"/topics/agentic-ai/","agentic ai",{"title":1000,"links":1001},"Solutions",[1002,1004,1006,1011,1015,1018,1022,1025,1027,1030,1033,1038],{"text":531,"config":1003},{"href":526,"dataGaName":531,"dataGaLocation":881},{"text":520,"config":1005},{"href":503,"dataGaName":504,"dataGaLocation":881},{"text":1007,"config":1008},"Agile development",{"href":1009,"dataGaName":1010,"dataGaLocation":881},"/solutions/agile-delivery/","agile delivery",{"text":1012,"config":1013},"SCM",{"href":516,"dataGaName":1014,"dataGaLocation":881},"source code management",{"text":960,"config":1016},{"href":509,"dataGaName":1017,"dataGaLocation":881},"continuous integration & delivery",{"text":1019,"config":1020},"Value stream management",{"href":559,"dataGaName":1021,"dataGaLocation":881},"value stream management",{"text":965,"config":1023},{"href":1024,"dataGaName":968,"dataGaLocation":881},"/solutions/gitops/",{"text":569,"config":1026},{"href":572,"dataGaName":573,"dataGaLocation":881},{"text":1028,"config":1029},"Small business",{"href":578,"dataGaName":579,"dataGaLocation":881},{"text":1031,"config":1032},"Public sector",{"href":584,"dataGaName":585,"dataGaLocation":881},{"text":1034,"config":1035},"Education",{"href":1036,"dataGaName":1037,"dataGaLocation":881},"/solutions/education/","education",{"text":1039,"config":1040},"Financial services",{"href":1041,"dataGaName":1042,"dataGaLocation":881},"/solutions/finance/","financial services",{"title":592,"links":1044},[1045,1047,1049,1051,1054,1056,1059,1061,1063,1065,1067,1069,1071,1073],{"text":605,"config":1046},{"href":607,"dataGaName":608,"dataGaLocation":881},{"text":610,"config":1048},{"href":612,"dataGaName":613,"dataGaLocation":881},{"text":615,"config":1050},{"href":617,"dataGaName":618,"dataGaLocation":881},{"text":620,"config":1052},{"href":622,"dataGaName":1053,"dataGaLocation":881},"docs",{"text":643,"config":1055},{"href":645,"dataGaName":646,"dataGaLocation":881},{"text":1057,"config":1058},"What's new",{"href":703,"dataGaName":704,"dataGaLocation":881},{"text":638,"config":1060},{"href":640,"dataGaName":641,"dataGaLocation":881},{"text":657,"config":1062},{"href":659,"dataGaName":660,"dataGaLocation":881},{"text":665,"config":1064},{"href":667,"dataGaName":668,"dataGaLocation":881},{"text":670,"config":1066},{"href":672,"dataGaName":673,"dataGaLocation":881},{"text":675,"config":1068},{"href":677,"dataGaName":678,"dataGaLocation":881},{"text":680,"config":1070},{"href":682,"dataGaName":683,"dataGaLocation":881},{"text":685,"config":1072},{"href":687,"dataGaName":688,"dataGaLocation":881},{"text":690,"config":1074},{"href":692,"dataGaName":693,"dataGaLocation":881},{"title":706,"links":1076},[1077,1079,1081,1083,1085,1087,1091,1096,1098,1100,1102],{"text":714,"config":1078},{"href":716,"dataGaName":708,"dataGaLocation":881},{"text":719,"config":1080},{"href":721,"dataGaName":722,"dataGaLocation":881},{"text":727,"config":1082},{"href":729,"dataGaName":730,"dataGaLocation":881},{"text":732,"config":1084},{"href":734,"dataGaName":735,"dataGaLocation":881},{"text":737,"config":1086},{"href":739,"dataGaName":740,"dataGaLocation":881},{"text":1088,"config":1089},"Sustainability",{"href":1090,"dataGaName":1088,"dataGaLocation":881},"/sustainability/",{"text":1092,"config":1093},"Diversity, inclusion and belonging (DIB)",{"href":1094,"dataGaName":1095,"dataGaLocation":881},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":742,"config":1097},{"href":744,"dataGaName":745,"dataGaLocation":881},{"text":752,"config":1099},{"href":754,"dataGaName":755,"dataGaLocation":881},{"text":757,"config":1101},{"href":759,"dataGaName":760,"dataGaLocation":881},{"text":1103,"config":1104},"Modern Slavery Transparency Statement",{"href":1105,"dataGaName":1106,"dataGaLocation":881},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":1108},[1109,1112,1115],{"text":1110,"config":1111},"Terms",{"href":933,"dataGaName":934,"dataGaLocation":881},{"text":1113,"config":1114},"Cookies",{"dataGaName":943,"dataGaLocation":881,"id":944,"isOneTrustButton":253},{"text":1116,"config":1117},"Privacy",{"href":938,"dataGaName":939,"dataGaLocation":881},[1119],{"id":1120,"title":1121,"body":408,"config":1122,"content":1124,"description":408,"extension":1127,"meta":1128,"navigation":253,"path":1129,"seo":1130,"stem":1131,"__hash__":1132},"blogAuthors/en-us/blog/authors/gitlab.yml","Gitlab",{"template":1123},"BlogAuthor",{"name":7,"config":1125},{"headshot":1126,"ctfId":7},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749659488/Blog/Author%20Headshots/gitlab-logo-extra-whitespace.png","yml",{},"/en-us/blog/authors/gitlab",{},"en-us/blog/authors/gitlab","XCBKIcPoCs6zi2oHG7o-bAp52Jhaw8_zGhIJ2jNrEjU",[1134,1143,1151],{"title":1135,"description":1136,"heroImage":1137,"category":404,"date":1138,"authors":1139,"slug":1142,"externalUrl":408},"Confidential AI for GitLab Self-Hosted","Give developers AI coding agents in GitLab Duo without source code leaving a hardware-encrypted boundary — no GPUs needed.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1773866173/vte9qh8rriznvyclhkes.png","2026-08-06",[1140,1141],"Mathias Ewald","Martin Paloncy, Edgeless Systems","confidential-ai-for-gitlab-self-hosted",{"title":1144,"description":1145,"heroImage":1146,"category":404,"date":1147,"authors":1148,"slug":1150,"externalUrl":408},"Green DevOps: Why carbon measurement belongs in your CI/CD pipeline","CI/CD pipelines have a hidden carbon cost. Here's why measuring it matters, and how you can get started with Eco CI and Carmen in GitLab.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1765809212/noh0mdfn9o94ry9ykura.png","2026-07-09",[1149],"Lysanne Pinto","green-devops-carbon-measurement-cicd-pipeline",{"title":1152,"description":1153,"heroImage":1154,"category":404,"date":1155,"authors":1156,"slug":1158,"externalUrl":408},"How to build CI/CD observability at scale","This practical guide to GitLab pipeline analytics helps self-managed users gain operational insights using Prometheus and Grafana.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-04-28",[1157],"Paul Meresanu","how-to-build-ci-cd-observability-at-scale",{"promotions":1160},[1161,1175,1187,1199],{"id":1162,"categories":1163,"header":1165,"text":1166,"button":1167,"image":1172},"ai-modernization",[1164],"ai","Is AI achieving its promise at scale?","Quiz will take 5 minutes or less",{"text":1168,"config":1169},"Get your AI maturity score",{"href":1170,"dataGaName":1171,"dataGaLocation":646},"/assessments/ai-modernization-assessment/","modernization assessment",{"config":1173},{"src":1174},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/qix0m7kwnd8x2fh1zq49.png",{"id":1176,"categories":1177,"header":1179,"text":1166,"button":1180,"image":1184},"devops-modernization",[1178,983],"product","Are you just managing tools or shipping innovation?",{"text":1181,"config":1182},"Get your DevOps maturity score",{"href":1183,"dataGaName":1171,"dataGaLocation":646},"/assessments/devops-modernization-assessment/",{"config":1185},{"src":1186},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138785/eg818fmakweyuznttgid.png",{"id":1188,"categories":1189,"header":1191,"text":1166,"button":1192,"image":1196},"security-modernization",[1190],"security","Are you trading speed for security?",{"text":1193,"config":1194},"Get your security maturity score",{"href":1195,"dataGaName":1171,"dataGaLocation":646},"/assessments/security-modernization-assessment/",{"config":1197},{"src":1198},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/p4pbqd9nnjejg5ds6mdk.png",{"id":1200,"paths":1201,"header":1204,"text":1205,"button":1206,"image":1211},"github-azure-migration",[1202,1203],"migration-from-azure-devops-to-gitlab","integrating-azure-devops-scm-and-gitlab","Is your team ready for GitHub's Azure move?","GitHub is already rebuilding around Azure. Find out what it means for you.",{"text":1207,"config":1208},"See how GitLab compares to GitHub",{"href":1209,"dataGaName":1210,"dataGaLocation":646},"/compare/gitlab-vs-github/github-azure-migration/","github azure migration",{"config":1212},{"src":1186},{"header":1214,"blurb":1215,"button":1216,"secondaryButton":1221},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":1217,"config":1218},"Get your free trial",{"href":1219,"dataGaName":445,"dataGaLocation":1220},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":919,"config":1222},{"href":771,"dataGaName":450,"dataGaLocation":1220},1786803749959]