[{"data":1,"prerenderedAt":1371},["ShallowReactive",2],{"/blog/scaling-repository-maintenance":3,"navigation-en-us":583,"banner-en-us":1011,"footer-en-us":1021,"blog-post-authors-en-us-Patrick Steinhardt":1266,"blog-related-posts-en-us-scaling-repository-maintenance":1281,"blog-promotions-en-us":1307,"next-steps-en-us":1361},{"id":4,"title":5,"authors":6,"body":8,"category":561,"date":562,"description":563,"extension":564,"externalUrl":565,"faq":565,"featured":566,"heroImage":567,"meta":568,"navigation":569,"path":570,"seo":571,"slug":575,"stem":576,"tags":577,"template":581,"updatedDate":565,"__hash__":582},"blogPosts/en-us/blog/scaling-repository-maintenance.md","Future-proofing Git repository maintenance",[7],"Patrick Steinhardt",{"type":9,"value":10,"toc":545},"minimark",[11,22,27,35,63,70,73,81,84,88,91,94,102,105,108,147,150,167,171,174,191,197,204,216,219,224,239,242,251,255,261,268,272,280,289,293,299,303,314,321,324,355,358,366,369,373,379,387,390,417,420,445,448,454,457,468,471,475,492,502,506,509,518,542],[12,13,14,15,21],"p",{},"Users get the most from ",[16,17,20],"a",{"href":18,"rel":19},"https://docs.gitlab.com/administration/gitaly/",[],"Gitaly",", the service responsible for the storage and maintenance of all Git repositories in GitLab, when traffic hitting it is efficiently handled. Therefore, we must ensure our Git repositories remain in a well-optimized state. When it comes to Git monorepositories, this maintenance can be a complex task that can cause a lot of overhead by itself because repository housekeeping becomes more expensive the larger the repositories get. This blog post explains in depth what we have done over the past few GitLab releases to rework our approach to repository housekeeping for better scaling and to maintain an optimized state to deliver the best peformance for our users.",[23,24,26],"h2",{"id":25},"the-challenge-with-git-monorepository-maintenance","The challenge with Git monorepository maintenance",[12,28,29,30,34],{},"To ensure that Git repositories remain performant, Git regularly runs a set of\nmaintenance tasks. On the client side, this usually happens by automatically\nrunning ",[31,32,33],"code",{},"git-gc(1)"," periodically, which:",[36,37,38,46,53,56],"ul",{},[39,40,41,42,45],"li",{},"Compresses revisions into a ",[31,43,44],{},"packed-refs"," file.",[39,47,48,49,52],{},"Compresses objects into ",[31,50,51],{},"packfiles",".",[39,54,55],{},"Prunes objects that aren't reachable by any of the revisions and that have\nnot been used for a while.",[39,57,58,59,62],{},"Generates and updates data structures like ",[31,60,61],{},"commit-graphs"," that help to speed\nup queries against the Git repository.",[12,64,65,66,69],{},"Git periodically runs ",[31,67,68],{},"git gc --auto"," automatically in the background, which\nanalyzes your repository and only performs maintenance tasks if required.",[12,71,72],{},"At GitLab, we can't use this infrastructure because it does not give us enough\ncontrol over which maintenance tasks are executed at what point in time.\nFurthermore, it does not give us full control over exactly which data\nstructures we opt in to. Instead, we have implemented our own maintenance\nstrategies that are specific to how GitLab works and catered to our specific\nneeds. Unfortunately, the way GitLab implemented repository maintenance has\nbeen limiting us for quite a while by now.",[36,74,75,78],{},[39,76,77],{},"It is unsuitable for large monorepositories.",[39,79,80],{},"It does not give us the ability to easily iterate on the employed maintenance\nstrategy.",[12,82,83],{},"This post explains our previous maintenance strategy and its problems as well as\nhow we revamped the architecture to allow us to iterate faster and more\nefficiently maintain repositories.",[23,85,87],{"id":86},"our-previous-repository-maintenance-strategy","Our previous repository maintenance strategy",[12,89,90],{},"In the early days of GitLab, most of the application ran on a single server.\nOn this single server, GitLab directly accessed Git repositories. For various\nreasons, this architecture limited us, so we created the standalone Gitaly\nserver that provides a gRPC API to access Git repositories.",[12,92,93],{},"To migrate to exclusively accessing Git repository data using Gitaly we:",[36,95,96,99],{},[39,97,98],{},"Migrated all the logic that was previously contained in the Rails\napplication to Gitaly.",[39,100,101],{},"Created Gitaly RPCs and updated Rails to not execute the logic directly, but\ninstead invoke the newly-implemented RPC.",[12,103,104],{},"While this was the easiest way to tackle the huge task back then, the end\nresult was that there were still quite a few areas in the Rails codebase that\nrelied on knowing how the Git repositories were stored on disk.",[12,106,107],{},"One such area was repository maintenance. In an ideal world, the Rails server\nwould not need to know about the on-disk state of a Git repository. Instead,\nthe Rails server would only care about the data it wants to get out of the\nrepository or commit to it. Because of the Gitaly migration path we took,\nthe Rails application was still responsible for executing fine-grained\nrepository maintenance by calling certain RPCs:",[36,109,110,116,126,135,141],{},[39,111,112,115],{},[31,113,114],{},"Cleanup"," to delete stale, temporary files that have accumulated",[39,117,118,121,122,125],{},[31,119,120],{},"RepackIncremental"," and ",[31,123,124],{},"RepackFull"," to either pack all loose objects into a\nnew packfile or alternatively to repack all packfiles into a single one",[39,127,128,131,132,134],{},[31,129,130],{},"PackRefs"," to compress all references into a single ",[31,133,44],{}," file",[39,136,137,140],{},[31,138,139],{},"WriteCommitGraph"," to update the commit-graph",[39,142,143,146],{},[31,144,145],{},"GarbageCollect"," to perform various different tasks",[12,148,149],{},"These low-level details of repository maintenance were being managed by the\nclient. But because clients didn't have any information on the on-disk state of\nthe repository, they could not even determine which of these maintenance tasks\nhad to be executed in the first place. Instead, we had a very simple heuristic:\nEvery few pushes, we ran one of the above RPCs to perform one of the maintenance\ntasks. While this heuristic worked, it wasn't great for the following reasons:",[36,151,152,155,158,161,164],{},[39,153,154],{},"Repositories can be modified without using pushes at all. So if users only\nuse the Web IDE to commit to repositories, they may not get repacked at all.",[39,156,157],{},"Because repository maintenance is controlled by the client, Gitaly can't\nassume a specific repository state.",[39,159,160],{},"The threshold for executing housekeeping tasks is set globally across all\nprojects rather than on a per-project basis. Consequently, no matter\nwhether you have a tiny repository or a huge monorepository, we would use the\nsame intervals for executing maintenance tasks. As you may imagine though,\ndoing a full repack of a Git repository that is only a few dozen megabytes in\nsize is a few orders of magnitudes faster than repacking a monorepository\nthat is multiple gigabytes in size.",[39,162,163],{},"Specific types of Git repositories hosted by Gitaly need special care and we\nrequired Gitaly clients to know about these.",[39,165,166],{},"Repository maintenance was inefficient overall. Clients do not know about the\non-disk state of repositories. Consequently, they had no choice except to\nrepeatedly ask Gitaly to optimize specific data structures without knowing\nwhether this was required in the first place.",[23,168,170],{"id":169},"heuristical-maintenance-strategy","Heuristical maintenance strategy",[12,172,173],{},"It was clear that we needed to change the strategy we used for repository\nmaintenance. Most importantly, we wanted to:",[36,175,176,179,182,185,188],{},[39,177,178],{},"Make Gitaly the single source of truth for how we maintain repositories.\nClients should not need to worry about low-level specifics, and Gitaly should\nbe able to easily iterate on the strategy.",[39,180,181],{},"Make the default maintenance strategy work for repositories of all sizes.",[39,183,184],{},"Make the maintenance strategy work for repositories of all types. A client\nshould not need to worry about which maintenance tasks must be executed for\nwhat repository type.",[39,186,187],{},"Avoid optimizing data structures that already are in an optimal state.",[39,189,190],{},"Improve visibility into the optimizations we perform.",[12,192,193,194,196],{},"As mentioned in the introduction, Git periodically runs ",[31,195,68],{},". This\ncommand inspects the repository's state and performs optimizations only when it\nfinds that the repository is in a sufficiently bad state to warrant the cost.\nWhile using this command directly in the context of Gitaly does not give us\nenough flexibility, it did serve as the inspiration for our new architecture.",[12,198,199,200,203],{},"Instead of providing fine-grained RPCs to maintain various parts of a Git\nrepository, we now only provide a single RPC ",[31,201,202],{},"OptimizeRepository"," that works as\na black-box to the caller. This RPC call:",[205,206,207,210,213],"ol",{},[39,208,209],{},"Cleans up stale data in the repository if there is any.",[39,211,212],{},"Analyzes the on-disk state of the repository.",[39,214,215],{},"Depending on this on-disk state, performs only these maintenance tasks that\nare deemed to be necessary.",[12,217,218],{},"Because we can analyze and use the on-disk state of the repository, we can be\nfar more intelligent about repository maintenance compared to the previous\nstrategy where we optimized some bits of the repository every few pushes.",[220,221,223],"h3",{"id":222},"packing-objects","Packing objects",[12,225,226,227,229,230,232,233,236,237,52],{},"In the old-style repository maintenance, the client would call either\n",[31,228,120],{}," or ",[31,231,124],{},". This would either: Pack all loose objects into a new ",[31,234,235],{},"packfile"," or repack all objects into a single ",[31,238,235],{},[12,240,241],{},"By default, we would perform a full repack every five repacks. While this may be\na good default for small repositories, it gets prohibitively expensive for huge\nmonorepositories where a full repack may easily take several minutes.",[12,243,244,245,247,248,250],{},"The new heuristical maintenance strategy instead scales the allowed number of\n",[31,246,51],{}," by the total size of all combined ",[31,249,51],{},". As a result, the\nlarger the repository becomes, the less frequently we perform a full repack.",[220,252,254],{"id":253},"pruning-objects","Pruning objects",[12,256,257,258,260],{},"In the past, clients would periodically call ",[31,259,145],{},". In addition to\nrepacking objects, this RPC would also prune any objects that are unreachable\nand that haven't been accessed for a specific grace period.",[12,262,263,264,267],{},"The new heuristical maintenance strategy scans through all loose objects that\nexist in the repository. If the number of loose objects that have a modification\ntime older than two weeks exceeds a certain threshold, it spawns the\n",[31,265,266],{},"git prune"," command to prune these objects.",[220,269,271],{"id":270},"packing-references","Packing references",[12,273,274,275,277,278,45],{},"In the past, clients would call ",[31,276,130],{}," to repack references into the\n",[31,279,44],{},[12,281,282,283,285,286,288],{},"Because the time to compress references scales with the size of the\n",[31,284,44],{}," file, the new heuristical maintenance strategy takes into account\nboth the size of the ",[31,287,44],{}," file and the number of loose references that\nexist in the repository. If a ratio between these two figures is exceeded, we\ncompress the loose references.",[220,290,292],{"id":291},"auxiliary-data-structures","Auxiliary data structures",[12,294,295,296,298],{},"There are auxiliary data structures like ",[31,297,61],{}," that are used by Git\nto speed up various queries. With the new heuristical maintenance strategy,\nGitaly now automatically updates these as required, either when they are\ndeemed to be out-of-date, or when they are missing altogether.",[220,300,302],{"id":301},"heuristical-maintenance-strategy-rollout","Heuristical maintenance strategy rollout",[12,304,305,306,313],{},"We rolled out this new heuristical maintenance strategy to GitLab.com in March 2022. Initially, we only rolled it out for\n",[16,307,310],{"href":308,"rel":309},"https://gitlab.com/gitlab-org/gitlab",[],[31,311,312],{},"gitlab-org/gitlab",", which is a\nrepository where maintenance performed particularly poorly in the past. You can\nsee the impact of the rollout in the following graph:",[12,315,316],{},[317,318],"img",{"alt":319,"src":320},"Latency of OptimizeRepository for gitlab-org/gitlab","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398956/blog/Content%20Images/repo-housekeeping-gitlab-org-gitlab-latency.png",[12,322,323],{},"In this graph, you can see that:",[205,325,326,336,339,352],{},[39,327,328,329,331,332,121,334,52],{},"Until March 19, we used the legacy fine-grained RPC calls. We spent most\nof the time in ",[31,330,124],{},", followed by ",[31,333,120],{},[31,335,145],{},[39,337,338],{},"Because March 19 and 20 occurred on a weekend, nothing much happens with\nhousekeeping.",[39,340,341,342,344,345,347,348,351],{},"Early on March 21 we switched ",[31,343,312],{}," to use heuristical\nhousekeeping using ",[31,346,202],{},". Initially, there didn't seem to be\nmuch of an improvement. There wasn't much difference in how much time we\nspent maintaining this repository compared to the past.",[349,350],"br",{},"However, this was caused by an inefficient heuristic. Instead of only pruning\nobjects when there were stale ones, we always pruned objects when we saw that\nthere were too many loose objects.",[39,353,354],{},"We deployed a fix for this bug on March 22, which led to a significant drop in\ntime spent optimizing this repository compared to before.",[12,356,357],{},"This demonstrated two things:",[36,359,360,363],{},[39,361,362],{},"We're easily able to iterate on the heuristics that we have in Gitaly.",[39,364,365],{},"Using the heuristics saves a lot of compute time as we don't unnecessarily\noptimize anymore.",[12,367,368],{},"We have subsequently rolled this out to all of GitLab.com, starting on March\n29, 2022, with similar improvements. With this change, we more than halved the CPU\nload when performing repository optimizations.",[23,370,372],{"id":371},"observability","Observability",[12,374,375,376,378],{},"While it is great that ",[31,377,202],{}," has managed to save us a lot of\ncompute power, one goal was to improve visibility into repository housekeeping.\nMore specifically, we wanted to:",[36,380,381,384],{},[39,382,383],{},"Gain visibility on the global level to see what optimizations are performed\nacross all of our repositories.",[39,385,386],{},"Gain visibility on the repository level to know what state a specific\nrepository is in.",[12,388,389],{},"In order to improve global visibility, we expose a set of Prometheus metrics that\nallow us to observe important details about our repository maintenance. The\nfollowing graphs show the optimizations performed in a 30-minute window of our\nproduction systems on GitLab.com.",[36,391,392,401,409],{},[39,393,394,395,397],{},"The optimizations, which are being performed in general.",[349,396],{},[317,398],{"alt":399,"src":400},"Repository optimization metrics for GitLab.com","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398960/blog/Content%20Images/repo-housekeeping-metrics-optimizations.png",[39,402,403,404,406],{},"The average latency it takes to perform each of these optimizations.",[349,405],{},[317,407],{"alt":399,"src":408},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398959/blog/Content%20Images/repo-housekeeping-metrics-latencies.png",[39,410,411,412,414],{},"What kind of stale data we are cleaning up.",[349,413],{},[317,415],{"alt":399,"src":416},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398958/blog/Content%20Images/repo-housekeeping-metrics-cleanups.png",[12,418,419],{},"To improve visibility into the state each repository is in we have started to\nlog structured data that includes all the relevant bits. A subset of the\ninformation it exposes is:",[36,421,422,425,431,434,439],{},[39,423,424],{},"The number of loose objects and their sizes.",[39,426,427,428,430],{},"The number of ",[31,429,51],{}," and their combined size.",[39,432,433],{},"The number of loose references.",[39,435,436,437,45],{},"The size of the ",[31,438,44],{},[39,440,441,442,444],{},"Information about ",[31,443,61],{},", bitmaps and other auxiliary data\nstructures.",[12,446,447],{},"This information is also exposed through Prometheus metrics:",[12,449,450],{},[317,451],{"alt":452,"src":453},"Repository state metrics for GitLab.com","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398960/blog/Content%20Images/repo-state-metrics.png",[12,455,456],{},"These graphs expose important metrics of the on-disk state of our repositories:",[36,458,459,462,465],{},[39,460,461],{},"The top panel shows which data structures exist.",[39,463,464],{},"The heatmaps on the left show how large specific data structures are.",[39,466,467],{},"The heatmaps on the right show how many of these data structures we have.",[12,469,470],{},"Combining both the global and per-repository information allows us to easily\nobserve how repository maintenance behaves during normal operations. But more\nimportantly, it gives us meaningful data when rolling out new features that\nchange the way repositories are maintained.",[23,472,474],{"id":473},"manually-enabling-heuristical-housekeeping","Manually enabling heuristical housekeeping",[12,476,477,478,481,482,487,488,491],{},"While the heuristical housekeeping is enabled by default starting with GitLab\n15.9, it has already been introduced with GitLab 14.10. If you want to use the\nnew housekeeping strategy before upgrading to 15.9, you can opt in by\nsetting the ",[31,479,480],{},"optimized_housekeeping"," ",[16,483,486],{"href":484,"rel":485},"https://docs.gitlab.com/administration/feature_flags/#how-to-enable-and-disable-features-behind-flags",[],"feature flag",".\nYou can do so via the ",[31,489,490],{},"gitlab-rails"," console:",[493,494,500],"pre",{"className":495,"code":497,"language":498,"meta":499},[496],"language-text","Feature.enable(:optimized_housekeeping)\n","text","",[31,501,497],{"__ignoreMap":499},[23,503,505],{"id":504},"future-improvements","Future improvements",[12,507,508],{},"While the new heuristical optimization strategy has been successfully\nbattle-tested for a while now for GitLab.com, at the time of writing this\nblog post, it still wasn't enabled by default for self-deployed installations.\nThis has finally changed with GitLab 15.8, where we have default-enabled the new\nheuristical maintenance strategy.",[12,510,511,512,517],{},"We are not done yet, though. Now that Gitaly is the only source of truth for how\nrepositories are optimized, we are tracking improvements to our maintenance\nstrategy in ",[16,513,516],{"href":514,"rel":515},"https://gitlab.com/groups/gitlab-org/-/epics/7443",[],"epic 7443",":",[36,519,520,528,536],{},[39,521,522,527],{},[16,523,526],{"href":524,"rel":525},"https://git-scm.com/docs/multi-pack-index",[],"Multi-pack indices"," and geometric\nrepacking will help us to further reduce the time spent repacking objects.",[39,529,530,535],{},[16,531,534],{"href":532,"rel":533},"https://git-scm.com/docs/cruft-packs",[],"Cruft packs"," will help us to further\nreduce the time spent pruning objects and reduce the overall size of\nunreachable objects.",[39,537,538,539,541],{},"Gitaly will automatically run housekeeping tasks when receiving mutating RPC\ncalls so that clients don't have to call ",[31,540,202],{}," at all anymore.",[12,543,544],{},"So stay tuned!",{"title":499,"searchDepth":546,"depth":546,"links":547},2,[548,549,550,558,559,560],{"id":25,"depth":546,"text":26},{"id":86,"depth":546,"text":87},{"id":169,"depth":546,"text":170,"children":551},[552,554,555,556,557],{"id":222,"depth":553,"text":223},3,{"id":253,"depth":553,"text":254},{"id":270,"depth":553,"text":271},{"id":291,"depth":553,"text":292},{"id":301,"depth":553,"text":302},{"id":371,"depth":546,"text":372},{"id":473,"depth":546,"text":474},{"id":504,"depth":546,"text":505},"engineering","2023-03-20","Learn how we revamped our architecture for faster iteration and more efficiently maintained repositories.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749677736/Blog/Hero%20Images/Git.png",{},true,"/en-us/blog/scaling-repository-maintenance",{"title":5,"description":563,"ogTitle":5,"ogDescription":563,"noIndex":566,"ogImage":567,"ogUrl":572,"ogSiteName":573,"ogType":574,"canonicalUrls":572},"https://about.gitlab.com/blog/scaling-repository-maintenance","https://about.gitlab.com","article","scaling-repository-maintenance","en-us/blog/scaling-repository-maintenance",[578,579,580],"git","production","performance","BlogPost","ViYYvH5T6GFknx50D3Lcw6q6b6j_X7nVn7VD9nBQqdg",{"logo":584,"freeTrial":589,"sales":594,"login":599,"items":604,"search":931,"minimal":962,"duo":981,"switchNav":990,"pricingDeployment":1001},{"config":585},{"href":586,"dataGaName":587,"dataGaLocation":588},"/","gitlab logo","header",{"text":590,"config":591},"Get free trial",{"href":592,"dataGaName":593,"dataGaLocation":588},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":595,"config":596},"Request a demo",{"href":597,"dataGaName":598,"dataGaLocation":588},"/sales/?contact-topic=request-demo","sales",{"text":600,"config":601},"Sign in",{"href":602,"dataGaName":603,"dataGaLocation":588},"https://gitlab.com/users/sign_in/","sign in",[605,634,734,739,853,909],{"text":606,"config":607,"menu":609},"Platform",{"dataNavLevelOne":608},"platform",{"type":610,"columns":611},"cards",[612,618,626],{"title":606,"description":613,"link":614},"The intelligent orchestration platform for DevSecOps",{"text":615,"config":616},"Explore our Platform",{"href":617,"dataGaName":608,"dataGaLocation":588},"/platform/",{"title":619,"description":620,"link":621},"GitLab Duo Agent Platform","Agentic AI for the entire software lifecycle",{"text":622,"config":623},"Meet GitLab Duo",{"href":624,"dataGaName":625,"dataGaLocation":588},"/gitlab-duo-agent-platform/","gitlab duo agent platform",{"title":627,"description":628,"link":629},"Why GitLab","See the top reasons enterprises choose GitLab",{"text":630,"config":631},"Learn more",{"href":632,"dataGaName":633,"dataGaLocation":588},"/why-gitlab/","why gitlab",{"text":635,"left":569,"config":636,"menu":638},"Product",{"dataNavLevelOne":637},"solutions",{"type":639,"link":640,"columns":644,"feature":713},"lists",{"text":641,"config":642},"View all Solutions",{"href":643,"dataGaName":637,"dataGaLocation":588},"/solutions/",[645,669,692],{"title":646,"description":647,"link":648,"items":653},"Automation","CI/CD and automation to accelerate deployment",{"config":649},{"icon":650,"href":651,"dataGaName":652,"dataGaLocation":588},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[654,658,661,665],{"text":655,"config":656},"CI/CD",{"href":657,"dataGaLocation":588,"dataGaName":655},"/solutions/continuous-integration/",{"text":619,"config":659},{"href":624,"dataGaLocation":588,"dataGaName":660},"gitlab duo agent platform - product menu",{"text":662,"config":663},"Source Code Management",{"href":664,"dataGaLocation":588,"dataGaName":662},"/solutions/source-code-management/",{"text":666,"config":667},"Automated Software Delivery",{"href":651,"dataGaLocation":588,"dataGaName":668},"Automated software delivery",{"title":670,"description":671,"link":672,"items":677},"Security","Deliver code faster without compromising security",{"config":673},{"href":674,"dataGaName":675,"dataGaLocation":588,"icon":676},"/solutions/application-security-testing/","security and compliance","ShieldCheckLight",[678,682,687],{"text":679,"config":680},"Application Security Testing",{"href":674,"dataGaName":681,"dataGaLocation":588},"Application security testing",{"text":683,"config":684},"Software Supply Chain Security",{"href":685,"dataGaLocation":588,"dataGaName":686},"/solutions/supply-chain/","Software supply chain security",{"text":688,"config":689},"Software Compliance",{"href":690,"dataGaName":691,"dataGaLocation":588},"/solutions/software-compliance/","software compliance",{"title":693,"link":694,"items":699},"Measurement",{"config":695},{"icon":696,"href":697,"dataGaName":698,"dataGaLocation":588},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[700,704,708],{"text":701,"config":702},"Visibility & Measurement",{"href":697,"dataGaLocation":588,"dataGaName":703},"Visibility and Measurement",{"text":705,"config":706},"Value Stream Management",{"href":707,"dataGaLocation":588,"dataGaName":705},"/solutions/value-stream-management/",{"text":709,"config":710},"Analytics & Insights",{"href":711,"dataGaLocation":588,"dataGaName":712},"/solutions/analytics-and-insights/","Analytics and insights",{"title":714,"type":639,"items":715},"GitLab for",[716,722,728],{"text":717,"config":718},"Enterprise",{"icon":719,"href":720,"dataGaLocation":588,"dataGaName":721},"Building","/enterprise/","enterprise",{"text":723,"config":724},"Small Business",{"icon":725,"href":726,"dataGaLocation":588,"dataGaName":727},"Work","/small-business/","small business",{"text":729,"config":730},"Public Sector",{"icon":731,"href":732,"dataGaLocation":588,"dataGaName":733},"Organization","/solutions/public-sector/","public sector",{"text":735,"config":736},"Pricing",{"href":737,"dataGaName":738,"dataGaLocation":588,"dataNavLevelOne":738},"/pricing/","pricing",{"text":740,"config":741,"menu":743},"Resources",{"dataNavLevelOne":742},"resources",{"type":639,"link":744,"columns":748,"feature":842},{"text":745,"config":746},"View all resources",{"href":747,"dataGaName":742,"dataGaLocation":588},"/resources/",[749,782,809],{"title":750,"items":751},"Getting started",[752,757,762,767,772,777],{"text":753,"config":754},"Install",{"href":755,"dataGaName":756,"dataGaLocation":588},"/install/","install",{"text":758,"config":759},"Quick start guides",{"href":760,"dataGaName":761,"dataGaLocation":588},"/get-started/","quick setup checklists",{"text":763,"config":764},"Learn",{"href":765,"dataGaLocation":588,"dataGaName":766},"https://university.gitlab.com/","learn",{"text":768,"config":769},"Product documentation",{"href":770,"dataGaName":771,"dataGaLocation":588},"https://docs.gitlab.com/","product documentation",{"text":773,"config":774},"Best practice videos",{"href":775,"dataGaName":776,"dataGaLocation":588},"/getting-started-videos/","best practice videos",{"text":778,"config":779},"Integrations",{"href":780,"dataGaName":781,"dataGaLocation":588},"/integrations/","integrations",{"title":783,"items":784},"Discover",[785,790,795,800,804],{"text":786,"config":787},"Customer success stories",{"href":788,"dataGaName":789,"dataGaLocation":588},"/customers/","customer success stories",{"text":791,"config":792},"Blog",{"href":793,"dataGaName":794,"dataGaLocation":588},"/blog/","blog",{"text":796,"config":797},"Demo Hub",{"href":798,"dataGaName":799,"dataGaLocation":588},"/demo-hub/","demo hub",{"text":801,"config":802},"The Source",{"href":803,"dataGaName":794,"dataGaLocation":588},"/the-source/",{"text":805,"config":806},"Remote",{"href":807,"dataGaName":808,"dataGaLocation":588},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"title":810,"items":811},"Connect",[812,817,822,827,832,837],{"text":813,"config":814},"GitLab Services",{"href":815,"dataGaName":816,"dataGaLocation":588},"/services/","services",{"text":818,"config":819},"Contribute",{"href":820,"dataGaName":821,"dataGaLocation":588},"https://contributors.gitlab.com","contribute",{"text":823,"config":824},"Community",{"href":825,"dataGaName":826,"dataGaLocation":588},"/community/","community",{"text":828,"config":829},"Forum",{"href":830,"dataGaName":831,"dataGaLocation":588},"https://forum.gitlab.com/","forum",{"text":833,"config":834},"Events",{"href":835,"dataGaName":836,"dataGaLocation":588},"/events/","events",{"text":838,"config":839},"Partners",{"href":840,"dataGaName":841,"dataGaLocation":588},"/partners/","partners",{"config":843,"title":846,"text":847,"link":848},{"background":844,"textColor":845},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1777322348/qpq8yrgn8knii57omj0c.png')","#000","What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":849,"config":850},"Read the latest",{"href":851,"dataGaName":852,"dataGaLocation":588},"/whats-new/","whats new",{"text":854,"config":855,"menu":857},"Company",{"dataNavLevelOne":856},"company",{"type":639,"columns":858},[859],{"items":860},[861,866,872,874,879,884,889,894,899,904],{"text":862,"config":863},"About",{"href":864,"dataGaName":865,"dataGaLocation":588},"/company/","about",{"text":867,"config":868,"footerGa":871},"Jobs",{"href":869,"dataGaName":870,"dataGaLocation":588},"/jobs/","jobs",{"dataGaName":870},{"text":833,"config":873},{"href":835,"dataGaName":836,"dataGaLocation":588},{"text":875,"config":876},"Leadership",{"href":877,"dataGaName":878,"dataGaLocation":588},"/company/team/e-group/","leadership",{"text":880,"config":881},"Handbook",{"href":882,"dataGaName":883,"dataGaLocation":588},"https://handbook.gitlab.com/","handbook",{"text":885,"config":886},"Investor relations",{"href":887,"dataGaName":888,"dataGaLocation":588},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":890,"config":891},"Trust Center",{"href":892,"dataGaName":893,"dataGaLocation":588},"/security/","trust center",{"text":895,"config":896},"AI Transparency Center",{"href":897,"dataGaName":898,"dataGaLocation":588},"/ai-transparency-center/","ai transparency center",{"text":900,"config":901},"Newsletter",{"href":902,"dataGaName":903,"dataGaLocation":588},"/company/contact/#contact-forms","newsletter",{"text":905,"config":906},"Press",{"href":907,"dataGaName":908,"dataGaLocation":588},"/press/","press",{"text":910,"config":911,"menu":912},"Contact us",{"dataNavLevelOne":856},{"type":639,"columns":913},[914],{"items":915},[916,921,926],{"text":917,"config":918},"Talk to sales",{"href":919,"dataGaName":920,"dataGaLocation":588},"/sales/","talk to sales",{"text":922,"config":923},"Support portal",{"href":924,"dataGaName":925,"dataGaLocation":588},"https://support.gitlab.com/hc/en-us","support portal",{"text":927,"config":928},"Customer portal",{"href":929,"dataGaName":930,"dataGaLocation":588},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":932,"login":933,"suggestions":940},"Close",{"text":934,"link":935},"To search repositories and projects, login to",{"text":936,"config":937},"gitlab.com",{"href":602,"dataGaName":938,"dataGaLocation":939},"search login","search",{"text":941,"default":942},"Suggestions",[943,945,949,951,955,959],{"text":619,"config":944},{"href":624,"dataGaName":619,"dataGaLocation":939},{"text":946,"config":947},"Code Suggestions (AI)",{"href":948,"dataGaName":946,"dataGaLocation":939},"/solutions/code-suggestions/",{"text":655,"config":950},{"href":657,"dataGaName":655,"dataGaLocation":939},{"text":952,"config":953},"GitLab on AWS",{"href":954,"dataGaName":952,"dataGaLocation":939},"/partners/technology-partners/aws/",{"text":956,"config":957},"GitLab on Google Cloud",{"href":958,"dataGaName":956,"dataGaLocation":939},"/partners/technology-partners/google-cloud-platform/",{"text":960,"config":961},"Why GitLab?",{"href":632,"dataGaName":960,"dataGaLocation":939},{"freeTrial":963,"mobileIcon":968,"desktopIcon":973,"secondaryButton":976},{"text":964,"config":965},"Start free trial",{"href":966,"dataGaName":593,"dataGaLocation":967},"https://gitlab.com/-/trials/new/","nav",{"altText":969,"config":970},"Gitlab Icon",{"src":971,"dataGaName":972,"dataGaLocation":967},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":969,"config":974},{"src":975,"dataGaName":972,"dataGaLocation":967},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":977,"config":978},"Get Started",{"href":979,"dataGaName":980,"dataGaLocation":967},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":982,"mobileIcon":986,"desktopIcon":988},{"text":983,"config":984},"Learn more about GitLab Duo",{"href":624,"dataGaName":985,"dataGaLocation":967},"gitlab duo",{"altText":969,"config":987},{"src":971,"dataGaName":972,"dataGaLocation":967},{"altText":969,"config":989},{"src":975,"dataGaName":972,"dataGaLocation":967},{"button":991,"mobileIcon":996,"desktopIcon":998},{"text":992,"config":993},"/switch",{"href":994,"dataGaName":995,"dataGaLocation":967},"#contact","switch",{"altText":969,"config":997},{"src":971,"dataGaName":972,"dataGaLocation":967},{"altText":969,"config":999},{"src":1000,"dataGaName":972,"dataGaLocation":967},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":1002,"mobileIcon":1007,"desktopIcon":1009},{"text":1003,"config":1004},"Back to pricing",{"href":737,"dataGaName":1005,"dataGaLocation":967,"icon":1006},"back to pricing","GoBack",{"altText":969,"config":1008},{"src":971,"dataGaName":972,"dataGaLocation":967},{"altText":969,"config":1010},{"src":975,"dataGaName":972,"dataGaLocation":967},{"title":1012,"titleMobile":1013,"button":1014,"config":1019},"Duo Agent Platform delivers 400% ROI, per new Forrester Consulting study.","400% ROI: Forrester TEI for GitLab Duo",{"text":630,"config":1015},{"href":1016,"dataGaName":1017,"dataGaLocation":1018},"https://about.gitlab.com/blog/gitlab-duo-agent-platform-delivers-400-percent-roi/","forrester-tei-dap-banner","global-banner",{"layout":1020,"disabled":566},"release",{"data":1022},{"text":1023,"source":1024,"edit":1030,"contribute":1035,"config":1040,"items":1045,"minimal":1255},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":1025,"config":1026},"View page source",{"href":1027,"dataGaName":1028,"dataGaLocation":1029},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":1031,"config":1032},"Edit this page",{"href":1033,"dataGaName":1034,"dataGaLocation":1029},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":1036,"config":1037},"Please contribute",{"href":1038,"dataGaName":1039,"dataGaLocation":1029},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":1041,"facebook":1042,"youtube":1043,"linkedin":1044},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[1046,1093,1147,1191,1223],{"title":735,"links":1047,"subMenu":1062},[1048,1052,1057],{"text":1049,"config":1050},"View plans",{"href":737,"dataGaName":1051,"dataGaLocation":1029},"view plans",{"text":1053,"config":1054},"Why Premium?",{"href":1055,"dataGaName":1056,"dataGaLocation":1029},"/pricing/premium/","why premium",{"text":1058,"config":1059},"Why Ultimate?",{"href":1060,"dataGaName":1061,"dataGaLocation":1029},"/pricing/ultimate/","why ultimate",[1063],{"title":1064,"links":1065},"Contact Us",[1066,1069,1071,1073,1078,1083,1088],{"text":1067,"config":1068},"Contact sales",{"href":919,"dataGaName":598,"dataGaLocation":1029},{"text":922,"config":1070},{"href":924,"dataGaName":925,"dataGaLocation":1029},{"text":927,"config":1072},{"href":929,"dataGaName":930,"dataGaLocation":1029},{"text":1074,"config":1075},"Status",{"href":1076,"dataGaName":1077,"dataGaLocation":1029},"https://status.gitlab.com/","status",{"text":1079,"config":1080},"Terms of use",{"href":1081,"dataGaName":1082,"dataGaLocation":1029},"/terms/","terms of use",{"text":1084,"config":1085},"Privacy statement",{"href":1086,"dataGaName":1087,"dataGaLocation":1029},"/privacy/","privacy statement",{"text":1089,"config":1090},"Cookie preferences",{"dataGaName":1091,"dataGaLocation":1029,"id":1092,"isOneTrustButton":569},"cookie preferences","ot-sdk-btn",{"title":635,"links":1094,"subMenu":1103},[1095,1099],{"text":1096,"config":1097},"DevSecOps platform",{"href":617,"dataGaName":1098,"dataGaLocation":1029},"devsecops platform",{"text":1100,"config":1101},"AI-Assisted Development",{"href":624,"dataGaName":1102,"dataGaLocation":1029},"ai-assisted development",[1104],{"title":1105,"links":1106},"Topics",[1107,1112,1117,1122,1127,1132,1137,1142],{"text":1108,"config":1109},"CICD",{"href":1110,"dataGaName":1111,"dataGaLocation":1029},"/topics/ci-cd/","cicd",{"text":1113,"config":1114},"GitOps",{"href":1115,"dataGaName":1116,"dataGaLocation":1029},"/topics/gitops/","gitops",{"text":1118,"config":1119},"DevOps",{"href":1120,"dataGaName":1121,"dataGaLocation":1029},"/topics/devops/","devops",{"text":1123,"config":1124},"Version Control",{"href":1125,"dataGaName":1126,"dataGaLocation":1029},"/topics/version-control/","version control",{"text":1128,"config":1129},"DevSecOps",{"href":1130,"dataGaName":1131,"dataGaLocation":1029},"/topics/devsecops/","devsecops",{"text":1133,"config":1134},"Cloud Native",{"href":1135,"dataGaName":1136,"dataGaLocation":1029},"/topics/cloud-native/","cloud native",{"text":1138,"config":1139},"AI for Coding",{"href":1140,"dataGaName":1141,"dataGaLocation":1029},"/topics/devops/ai-for-coding/","ai for coding",{"text":1143,"config":1144},"Agentic AI",{"href":1145,"dataGaName":1146,"dataGaLocation":1029},"/topics/agentic-ai/","agentic ai",{"title":1148,"links":1149},"Solutions",[1150,1152,1154,1159,1163,1166,1170,1173,1175,1178,1181,1186],{"text":679,"config":1151},{"href":674,"dataGaName":679,"dataGaLocation":1029},{"text":668,"config":1153},{"href":651,"dataGaName":652,"dataGaLocation":1029},{"text":1155,"config":1156},"Agile development",{"href":1157,"dataGaName":1158,"dataGaLocation":1029},"/solutions/agile-delivery/","agile delivery",{"text":1160,"config":1161},"SCM",{"href":664,"dataGaName":1162,"dataGaLocation":1029},"source code management",{"text":1108,"config":1164},{"href":657,"dataGaName":1165,"dataGaLocation":1029},"continuous integration & delivery",{"text":1167,"config":1168},"Value stream management",{"href":707,"dataGaName":1169,"dataGaLocation":1029},"value stream management",{"text":1113,"config":1171},{"href":1172,"dataGaName":1116,"dataGaLocation":1029},"/solutions/gitops/",{"text":717,"config":1174},{"href":720,"dataGaName":721,"dataGaLocation":1029},{"text":1176,"config":1177},"Small business",{"href":726,"dataGaName":727,"dataGaLocation":1029},{"text":1179,"config":1180},"Public sector",{"href":732,"dataGaName":733,"dataGaLocation":1029},{"text":1182,"config":1183},"Education",{"href":1184,"dataGaName":1185,"dataGaLocation":1029},"/solutions/education/","education",{"text":1187,"config":1188},"Financial services",{"href":1189,"dataGaName":1190,"dataGaLocation":1029},"/solutions/finance/","financial services",{"title":740,"links":1192},[1193,1195,1197,1199,1202,1204,1207,1209,1211,1213,1215,1217,1219,1221],{"text":753,"config":1194},{"href":755,"dataGaName":756,"dataGaLocation":1029},{"text":758,"config":1196},{"href":760,"dataGaName":761,"dataGaLocation":1029},{"text":763,"config":1198},{"href":765,"dataGaName":766,"dataGaLocation":1029},{"text":768,"config":1200},{"href":770,"dataGaName":1201,"dataGaLocation":1029},"docs",{"text":791,"config":1203},{"href":793,"dataGaName":794,"dataGaLocation":1029},{"text":1205,"config":1206},"What's new",{"href":851,"dataGaName":852,"dataGaLocation":1029},{"text":786,"config":1208},{"href":788,"dataGaName":789,"dataGaLocation":1029},{"text":805,"config":1210},{"href":807,"dataGaName":808,"dataGaLocation":1029},{"text":813,"config":1212},{"href":815,"dataGaName":816,"dataGaLocation":1029},{"text":818,"config":1214},{"href":820,"dataGaName":821,"dataGaLocation":1029},{"text":823,"config":1216},{"href":825,"dataGaName":826,"dataGaLocation":1029},{"text":828,"config":1218},{"href":830,"dataGaName":831,"dataGaLocation":1029},{"text":833,"config":1220},{"href":835,"dataGaName":836,"dataGaLocation":1029},{"text":838,"config":1222},{"href":840,"dataGaName":841,"dataGaLocation":1029},{"title":854,"links":1224},[1225,1227,1229,1231,1233,1235,1239,1244,1246,1248,1250],{"text":862,"config":1226},{"href":864,"dataGaName":856,"dataGaLocation":1029},{"text":867,"config":1228},{"href":869,"dataGaName":870,"dataGaLocation":1029},{"text":875,"config":1230},{"href":877,"dataGaName":878,"dataGaLocation":1029},{"text":880,"config":1232},{"href":882,"dataGaName":883,"dataGaLocation":1029},{"text":885,"config":1234},{"href":887,"dataGaName":888,"dataGaLocation":1029},{"text":1236,"config":1237},"Sustainability",{"href":1238,"dataGaName":1236,"dataGaLocation":1029},"/sustainability/",{"text":1240,"config":1241},"Diversity, inclusion and belonging (DIB)",{"href":1242,"dataGaName":1243,"dataGaLocation":1029},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":890,"config":1245},{"href":892,"dataGaName":893,"dataGaLocation":1029},{"text":900,"config":1247},{"href":902,"dataGaName":903,"dataGaLocation":1029},{"text":905,"config":1249},{"href":907,"dataGaName":908,"dataGaLocation":1029},{"text":1251,"config":1252},"Modern Slavery Transparency Statement",{"href":1253,"dataGaName":1254,"dataGaLocation":1029},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":1256},[1257,1260,1263],{"text":1258,"config":1259},"Terms",{"href":1081,"dataGaName":1082,"dataGaLocation":1029},{"text":1261,"config":1262},"Cookies",{"dataGaName":1091,"dataGaLocation":1029,"id":1092,"isOneTrustButton":569},{"text":1264,"config":1265},"Privacy",{"href":1086,"dataGaName":1087,"dataGaLocation":1029},[1267],{"id":1268,"title":7,"body":565,"config":1269,"content":1271,"description":565,"extension":1275,"meta":1276,"navigation":569,"path":1277,"seo":1278,"stem":1279,"__hash__":1280},"blogAuthors/en-us/blog/authors/patrick-steinhardt.yml",{"template":1270},"BlogAuthor",{"name":7,"config":1272},{"headshot":1273,"ctfId":1274},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749661952/Blog/Author%20Headshots/pks-gitlab-headshot.png","pksgitlab","yml",{},"/en-us/blog/authors/patrick-steinhardt",{},"en-us/blog/authors/patrick-steinhardt","SV9Yd_vW69UbvntDP-SEOV9NKT_VwUAj5nfftf2ElSw",[1282,1291,1299],{"title":1283,"description":1284,"heroImage":1285,"category":561,"date":1286,"authors":1287,"slug":1290,"externalUrl":565},"Confidential AI for GitLab Self-Hosted","Give developers AI coding agents in GitLab Duo without source code leaving a hardware-encrypted boundary — no GPUs needed.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1773866173/vte9qh8rriznvyclhkes.png","2026-08-06",[1288,1289],"Mathias Ewald","Martin Paloncy, Edgeless Systems","confidential-ai-for-gitlab-self-hosted",{"title":1292,"description":1293,"heroImage":1294,"category":561,"date":1295,"authors":1296,"slug":1298,"externalUrl":565},"Green DevOps: Why carbon measurement belongs in your CI/CD pipeline","CI/CD pipelines have a hidden carbon cost. Here's why measuring it matters, and how you can get started with Eco CI and Carmen in GitLab.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1765809212/noh0mdfn9o94ry9ykura.png","2026-07-09",[1297],"Lysanne Pinto","green-devops-carbon-measurement-cicd-pipeline",{"title":1300,"description":1301,"heroImage":1302,"category":561,"date":1303,"authors":1304,"slug":1306,"externalUrl":565},"How to build CI/CD observability at scale","This practical guide to GitLab pipeline analytics helps self-managed users gain operational insights using Prometheus and Grafana.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-04-28",[1305],"Paul Meresanu","how-to-build-ci-cd-observability-at-scale",{"promotions":1308},[1309,1323,1335,1347],{"id":1310,"categories":1311,"header":1313,"text":1314,"button":1315,"image":1320},"ai-modernization",[1312],"ai","Is AI achieving its promise at scale?","Quiz will take 5 minutes or less",{"text":1316,"config":1317},"Get your AI maturity score",{"href":1318,"dataGaName":1319,"dataGaLocation":794},"/assessments/ai-modernization-assessment/","modernization assessment",{"config":1321},{"src":1322},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/qix0m7kwnd8x2fh1zq49.png",{"id":1324,"categories":1325,"header":1327,"text":1314,"button":1328,"image":1332},"devops-modernization",[1326,1131],"product","Are you just managing tools or shipping innovation?",{"text":1329,"config":1330},"Get your DevOps maturity score",{"href":1331,"dataGaName":1319,"dataGaLocation":794},"/assessments/devops-modernization-assessment/",{"config":1333},{"src":1334},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138785/eg818fmakweyuznttgid.png",{"id":1336,"categories":1337,"header":1339,"text":1314,"button":1340,"image":1344},"security-modernization",[1338],"security","Are you trading speed for security?",{"text":1341,"config":1342},"Get your security maturity score",{"href":1343,"dataGaName":1319,"dataGaLocation":794},"/assessments/security-modernization-assessment/",{"config":1345},{"src":1346},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/p4pbqd9nnjejg5ds6mdk.png",{"id":1348,"paths":1349,"header":1352,"text":1353,"button":1354,"image":1359},"github-azure-migration",[1350,1351],"migration-from-azure-devops-to-gitlab","integrating-azure-devops-scm-and-gitlab","Is your team ready for GitHub's Azure move?","GitHub is already rebuilding around Azure. Find out what it means for you.",{"text":1355,"config":1356},"See how GitLab compares to GitHub",{"href":1357,"dataGaName":1358,"dataGaLocation":794},"/compare/gitlab-vs-github/github-azure-migration/","github azure migration",{"config":1360},{"src":1334},{"header":1362,"blurb":1363,"button":1364,"secondaryButton":1369},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":1365,"config":1366},"Get your free trial",{"href":1367,"dataGaName":593,"dataGaLocation":1368},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":1067,"config":1370},{"href":919,"dataGaName":598,"dataGaLocation":1368},1786803772783]