[{"data":1,"prerenderedAt":2245},["ShallowReactive",2],{"/blog/database-case-study-store-and-update-namespace-statistics":3,"navigation-en-us":1457,"banner-en-us":1885,"footer-en-us":1895,"blog-post-authors-en-us-Mayra Cabrera":2140,"blog-related-posts-en-us-database-case-study-store-and-update-namespace-statistics":2155,"blog-promotions-en-us":2181,"next-steps-en-us":2235},{"id":4,"title":5,"authors":6,"body":8,"category":1437,"date":1438,"description":1439,"extension":1440,"externalUrl":1441,"faq":1441,"featured":1442,"heroImage":1443,"meta":1444,"navigation":1445,"path":1446,"seo":1447,"slug":1451,"stem":1452,"tags":1453,"template":1455,"updatedDate":1441,"__hash__":1456},"blogPosts/en-us/blog/database-case-study-store-and-update-namespace-statistics.md","Store and update namespace statistics in a performant manner",[7],"Mayra Cabrera",{"type":9,"value":10,"toc":1421},"minimark",[11,15,25,30,40,43,46,79,82,103,112,119,123,128,137,463,466,475,478,493,497,505,957,960,964,979,982,986,989,1039,1042,1108,1111,1129,1133,1136,1219,1222,1238,1241,1263,1271,1275,1284,1299,1302,1306,1315,1319,1322,1329,1333,1339,1345,1351,1354,1360,1366,1370,1377,1380,1403,1417],[12,13,14],"p",{},"Managing storage space on large GitLab instances, such as GitLab.com, can be a challenge. At the moment, we only have a restriction on repository limits, but no restriction on most of the other items that can consume storage space: wiki, lfs objects, artifacts, and packages, to mention a few.",[12,16,17,18,24],{},"We want to facilitate a method for easily viewing the amount of storage consumed by a group and allow easy management on GitLab.com by setting ",[19,20,23],"a",{"href":21,"rel":22},"https://gitlab.com/groups/gitlab-org/-/epics/886",[],"storage and limits management for groups",". But to do that we need a way to track the statistics of a namespace, whether it is a Group or a User namespace.",[26,27,29],"h2",{"id":28},"proposal-to-track-the-statistics-of-a-namespace","Proposal to track the statistics of a namespace",[31,32,33,37],"ol",{},[34,35,36],"li",{},"Create a new ActiveRecord model to hold the namespaces' statistics in an aggregated form: Only for root namespaces.",[34,38,39],{},"Refresh the statistics in this model every time a project belonging to this namespace is changed.",[12,41,42],{},"The \"refresh\" part is the tricky one. Currently we don't have a pattern to update/refresh the namespace statistics every time a project belonging to the namespace is updated.",[12,44,45],{},"We refreshed projects statistics in the following way:",[31,47,48,56,68],{},[34,49,50,51,55],{},"We have a model called ",[52,53,54],"code",{},"ProjectStatistics",",",[34,57,58,59,61,62,67],{},"The records on ",[52,60,54],{}," are updated through a ",[19,63,66],{"href":64,"rel":65},"https://gitlab.com/gitlab-org/gitlab-ce/blob/v12.2.0.pre/app/models/project.rb#L90",[],"callback"," every time the project is saved.",[34,69,70,71,78],{},"The summary of those statistics per namespace is then retrieved by ",[19,72,75],{"href":73,"rel":74},"https://gitlab.com/gitlab-org/gitlab-ce/blob/v12.2.0.pre/app/models/namespace.rb#L70",[],[52,76,77],{},"Namespaces#with_statistics"," scope.",[12,80,81],{},"Analyzing this query we noticed that:",[83,84,85,96],"ul",{},[34,86,87,88,91,92,95],{},"It takes up to ",[52,89,90],{},"1.2"," seconds for namespaces with over ",[52,93,94],{},"15 000"," projects.",[34,97,98,99,102],{},"Any attempt to run ",[52,100,101],{},"EXPLAIN ANALYZE"," results in query timeouts (15 seconds) when using our internal tooling.",[12,104,105,106,111],{},"Additionally, the callback to update the project statistics doesn't scale. It is currently one of the most ",[19,107,110],{"href":108,"rel":109},"https://gitlab.com/gitlab-org/gitlab-ce/issues/62488",[],"frequently run and expensive database queries"," on GitLab.com. We can't add one more query to it as\nit will increase the transaction's length.",[12,113,114,115,118],{},"Because of these reasons, we can't apply the same pattern to store\nand update the namespaces' statistics, as the ",[52,116,117],{},"namespaces"," table is one\nof the largest tables on GitLab.com. Therefore, we have to find a performant and\nalternative method.",[26,120,122],{"id":121},"our-attempts","Our Attempts",[124,125,127],"h3",{"id":126},"attempt-a-postgresql-materialized-view","Attempt A: PostgreSQL materialized view",[12,129,130,131,136],{},"Update the ActiveRecord model with a refresh strategy based on project routes and a ",[19,132,135],{"href":133,"rel":134},"https://www.postgresql.org/docs/9.6/rules-materializedviews.html",[],"materialized view",":",[138,139,144],"pre",{"className":140,"code":141,"language":142,"meta":143,"style":143},"language-sql shiki shiki-themes github-light","SELECT split_part(\"rs\".path, '/', 1) as root_path,\n        COALESCE(SUM(ps.storage_size), 0) AS storage_size,\n        COALESCE(SUM(ps.repository_size), 0) AS repository_size,\n        COALESCE(SUM(ps.wiki_size), 0) AS wiki_size,\n        COALESCE(SUM(ps.lfs_objects_size), 0) AS lfs_objects_size,\n        COALESCE(SUM(ps.build_artifacts_size), 0) AS build_artifacts_size,\n        COALESCE(SUM(ps.packages_size), 0) AS packages_size\nFROM \"projects\"\n    INNER JOIN routes rs ON rs.source_id = projects.id AND rs.source_type = 'Project'\n    INNER JOIN project_statistics ps ON ps.project_id  = projects.id\nGROUP BY root_path\n","sql","",[52,145,146,190,226,255,284,313,342,371,380,426,454],{"__ignoreMap":143},[147,148,151,155,159,163,166,169,172,175,177,181,184,187],"span",{"class":149,"line":150},"line",1,[147,152,154],{"class":153},"sD7c4","SELECT",[147,156,158],{"class":157},"sgsFI"," split_part(",[147,160,162],{"class":161},"sYBdl","\"rs\"",[147,164,165],{"class":157},".",[147,167,168],{"class":153},"path",[147,170,171],{"class":157},", ",[147,173,174],{"class":161},"'/'",[147,176,171],{"class":157},[147,178,180],{"class":179},"sYu0t","1",[147,182,183],{"class":157},") ",[147,185,186],{"class":153},"as",[147,188,189],{"class":157}," root_path,\n",[147,191,193,196,199,202,204,207,209,212,215,218,220,223],{"class":149,"line":192},2,[147,194,195],{"class":179},"        COALESCE",[147,197,198],{"class":157},"(",[147,200,201],{"class":179},"SUM",[147,203,198],{"class":157},[147,205,206],{"class":179},"ps",[147,208,165],{"class":157},[147,210,211],{"class":179},"storage_size",[147,213,214],{"class":157},"), ",[147,216,217],{"class":179},"0",[147,219,183],{"class":157},[147,221,222],{"class":153},"AS",[147,224,225],{"class":157}," storage_size,\n",[147,227,229,231,233,235,237,239,241,244,246,248,250,252],{"class":149,"line":228},3,[147,230,195],{"class":179},[147,232,198],{"class":157},[147,234,201],{"class":179},[147,236,198],{"class":157},[147,238,206],{"class":179},[147,240,165],{"class":157},[147,242,243],{"class":179},"repository_size",[147,245,214],{"class":157},[147,247,217],{"class":179},[147,249,183],{"class":157},[147,251,222],{"class":153},[147,253,254],{"class":157}," repository_size,\n",[147,256,258,260,262,264,266,268,270,273,275,277,279,281],{"class":149,"line":257},4,[147,259,195],{"class":179},[147,261,198],{"class":157},[147,263,201],{"class":179},[147,265,198],{"class":157},[147,267,206],{"class":179},[147,269,165],{"class":157},[147,271,272],{"class":179},"wiki_size",[147,274,214],{"class":157},[147,276,217],{"class":179},[147,278,183],{"class":157},[147,280,222],{"class":153},[147,282,283],{"class":157}," wiki_size,\n",[147,285,287,289,291,293,295,297,299,302,304,306,308,310],{"class":149,"line":286},5,[147,288,195],{"class":179},[147,290,198],{"class":157},[147,292,201],{"class":179},[147,294,198],{"class":157},[147,296,206],{"class":179},[147,298,165],{"class":157},[147,300,301],{"class":179},"lfs_objects_size",[147,303,214],{"class":157},[147,305,217],{"class":179},[147,307,183],{"class":157},[147,309,222],{"class":153},[147,311,312],{"class":157}," lfs_objects_size,\n",[147,314,316,318,320,322,324,326,328,331,333,335,337,339],{"class":149,"line":315},6,[147,317,195],{"class":179},[147,319,198],{"class":157},[147,321,201],{"class":179},[147,323,198],{"class":157},[147,325,206],{"class":179},[147,327,165],{"class":157},[147,329,330],{"class":179},"build_artifacts_size",[147,332,214],{"class":157},[147,334,217],{"class":179},[147,336,183],{"class":157},[147,338,222],{"class":153},[147,340,341],{"class":157}," build_artifacts_size,\n",[147,343,345,347,349,351,353,355,357,360,362,364,366,368],{"class":149,"line":344},7,[147,346,195],{"class":179},[147,348,198],{"class":157},[147,350,201],{"class":179},[147,352,198],{"class":157},[147,354,206],{"class":179},[147,356,165],{"class":157},[147,358,359],{"class":179},"packages_size",[147,361,214],{"class":157},[147,363,217],{"class":179},[147,365,183],{"class":157},[147,367,222],{"class":153},[147,369,370],{"class":157}," packages_size\n",[147,372,374,377],{"class":149,"line":373},8,[147,375,376],{"class":153},"FROM",[147,378,379],{"class":161}," \"projects\"\n",[147,381,383,386,389,392,395,397,400,403,406,408,411,414,416,418,421,423],{"class":149,"line":382},9,[147,384,385],{"class":153},"    INNER JOIN",[147,387,388],{"class":157}," routes rs ",[147,390,391],{"class":153},"ON",[147,393,394],{"class":179}," rs",[147,396,165],{"class":157},[147,398,399],{"class":179},"source_id",[147,401,402],{"class":153}," =",[147,404,405],{"class":179}," projects",[147,407,165],{"class":157},[147,409,410],{"class":179},"id",[147,412,413],{"class":153}," AND",[147,415,394],{"class":179},[147,417,165],{"class":157},[147,419,420],{"class":179},"source_type",[147,422,402],{"class":153},[147,424,425],{"class":161}," 'Project'\n",[147,427,429,431,434,436,439,441,444,447,449,451],{"class":149,"line":428},10,[147,430,385],{"class":153},[147,432,433],{"class":157}," project_statistics ps ",[147,435,391],{"class":153},[147,437,438],{"class":179}," ps",[147,440,165],{"class":157},[147,442,443],{"class":179},"project_id",[147,445,446],{"class":153},"  =",[147,448,405],{"class":179},[147,450,165],{"class":157},[147,452,453],{"class":179},"id\n",[147,455,457,460],{"class":149,"line":456},11,[147,458,459],{"class":153},"GROUP BY",[147,461,462],{"class":157}," root_path\n",[12,464,465],{},"We could then execute the query with:",[138,467,469],{"className":140,"code":468,"language":142,"meta":143,"style":143},"REFRESH MATERIALIZED VIEW root_namespace_storage_statistics;\n",[52,470,471],{"__ignoreMap":143},[147,472,473],{"class":149,"line":150},[147,474,468],{"class":157},[12,476,477],{},"While this implied a single query update, it has some downsides:",[83,479,480,483,490],{},[34,481,482],{},"The query itself would not be fast, as it would need to update all the statistics every time it runs. Execution time of this query will increase as the number of namespaces and projects grow.",[34,484,485,486,165],{},"Materialized views syntax varies from PostgreSQL and MySQL. At the time this feature was worked on, ",[19,487,489],{"href":488},"/blog/removing-mysql-support/","GitLab still supported MySQL, which it now no longer supports.",[34,491,492],{},"Rails does not have native support for materialized views. We'd need to use a specialized gem to take care of the management of the database views, which implies additional work.",[124,494,496],{"id":495},"attempt-b-an-update-through-a-cte","Attempt B: An update through a CTE",[12,498,499,500,165],{},"Update the ActiveRecord model with a refresh strategy through a ",[19,501,504],{"href":502,"rel":503},"https://www.postgresql.org/docs/9.1/queries-with.html",[],"Common Table Expression",[138,506,508],{"className":140,"code":507,"language":142,"meta":143,"style":143},"WITH refresh AS (\n  SELECT split_part(\"rs\".path, '/', 1) as root_path,\n        COALESCE(SUM(ps.storage_size), 0) AS storage_size,\n        COALESCE(SUM(ps.repository_size), 0) AS repository_size,\n        COALESCE(SUM(ps.wiki_size), 0) AS wiki_size,\n        COALESCE(SUM(ps.lfs_objects_size), 0) AS lfs_objects_size,\n        COALESCE(SUM(ps.build_artifacts_size), 0) AS build_artifacts_size,\n        COALESCE(SUM(ps.packages_size), 0) AS packages_size\n  FROM \"projects\"\n        INNER JOIN routes rs ON rs.source_id = projects.id AND rs.source_type = 'Project'\n        INNER JOIN project_statistics ps ON ps.project_id  = projects.id\n  GROUP BY root_path)\nUPDATE namespace_storage_statistics\nSET storage_size = refresh.storage_size,\n    repository_size = refresh.repository_size,\n    wiki_size = refresh.wiki_size,\n    lfs_objects_size = refresh.lfs_objects_size,\n    build_artifacts_size = refresh.build_artifacts_size,\n    packages_size  = refresh.packages_size\nFROM refresh\n    INNER JOIN routes rs ON rs.path = refresh.root_path AND rs.source_type = 'Namespace'\nWHERE namespace_storage_statistics.namespace_id = rs.source_id\n",[52,509,510,523,550,576,602,628,654,680,706,713,748,770,779,788,810,826,842,858,874,889,897,934],{"__ignoreMap":143},[147,511,512,515,518,520],{"class":149,"line":150},[147,513,514],{"class":153},"WITH",[147,516,517],{"class":157}," refresh ",[147,519,222],{"class":153},[147,521,522],{"class":157}," (\n",[147,524,525,528,530,532,534,536,538,540,542,544,546,548],{"class":149,"line":192},[147,526,527],{"class":153},"  SELECT",[147,529,158],{"class":157},[147,531,162],{"class":161},[147,533,165],{"class":157},[147,535,168],{"class":153},[147,537,171],{"class":157},[147,539,174],{"class":161},[147,541,171],{"class":157},[147,543,180],{"class":179},[147,545,183],{"class":157},[147,547,186],{"class":153},[147,549,189],{"class":157},[147,551,552,554,556,558,560,562,564,566,568,570,572,574],{"class":149,"line":228},[147,553,195],{"class":179},[147,555,198],{"class":157},[147,557,201],{"class":179},[147,559,198],{"class":157},[147,561,206],{"class":179},[147,563,165],{"class":157},[147,565,211],{"class":179},[147,567,214],{"class":157},[147,569,217],{"class":179},[147,571,183],{"class":157},[147,573,222],{"class":153},[147,575,225],{"class":157},[147,577,578,580,582,584,586,588,590,592,594,596,598,600],{"class":149,"line":257},[147,579,195],{"class":179},[147,581,198],{"class":157},[147,583,201],{"class":179},[147,585,198],{"class":157},[147,587,206],{"class":179},[147,589,165],{"class":157},[147,591,243],{"class":179},[147,593,214],{"class":157},[147,595,217],{"class":179},[147,597,183],{"class":157},[147,599,222],{"class":153},[147,601,254],{"class":157},[147,603,604,606,608,610,612,614,616,618,620,622,624,626],{"class":149,"line":286},[147,605,195],{"class":179},[147,607,198],{"class":157},[147,609,201],{"class":179},[147,611,198],{"class":157},[147,613,206],{"class":179},[147,615,165],{"class":157},[147,617,272],{"class":179},[147,619,214],{"class":157},[147,621,217],{"class":179},[147,623,183],{"class":157},[147,625,222],{"class":153},[147,627,283],{"class":157},[147,629,630,632,634,636,638,640,642,644,646,648,650,652],{"class":149,"line":315},[147,631,195],{"class":179},[147,633,198],{"class":157},[147,635,201],{"class":179},[147,637,198],{"class":157},[147,639,206],{"class":179},[147,641,165],{"class":157},[147,643,301],{"class":179},[147,645,214],{"class":157},[147,647,217],{"class":179},[147,649,183],{"class":157},[147,651,222],{"class":153},[147,653,312],{"class":157},[147,655,656,658,660,662,664,666,668,670,672,674,676,678],{"class":149,"line":344},[147,657,195],{"class":179},[147,659,198],{"class":157},[147,661,201],{"class":179},[147,663,198],{"class":157},[147,665,206],{"class":179},[147,667,165],{"class":157},[147,669,330],{"class":179},[147,671,214],{"class":157},[147,673,217],{"class":179},[147,675,183],{"class":157},[147,677,222],{"class":153},[147,679,341],{"class":157},[147,681,682,684,686,688,690,692,694,696,698,700,702,704],{"class":149,"line":373},[147,683,195],{"class":179},[147,685,198],{"class":157},[147,687,201],{"class":179},[147,689,198],{"class":157},[147,691,206],{"class":179},[147,693,165],{"class":157},[147,695,359],{"class":179},[147,697,214],{"class":157},[147,699,217],{"class":179},[147,701,183],{"class":157},[147,703,222],{"class":153},[147,705,370],{"class":157},[147,707,708,711],{"class":149,"line":382},[147,709,710],{"class":153},"  FROM",[147,712,379],{"class":161},[147,714,715,718,720,722,724,726,728,730,732,734,736,738,740,742,744,746],{"class":149,"line":428},[147,716,717],{"class":153},"        INNER JOIN",[147,719,388],{"class":157},[147,721,391],{"class":153},[147,723,394],{"class":179},[147,725,165],{"class":157},[147,727,399],{"class":179},[147,729,402],{"class":153},[147,731,405],{"class":179},[147,733,165],{"class":157},[147,735,410],{"class":179},[147,737,413],{"class":153},[147,739,394],{"class":179},[147,741,165],{"class":157},[147,743,420],{"class":179},[147,745,402],{"class":153},[147,747,425],{"class":161},[147,749,750,752,754,756,758,760,762,764,766,768],{"class":149,"line":456},[147,751,717],{"class":153},[147,753,433],{"class":157},[147,755,391],{"class":153},[147,757,438],{"class":179},[147,759,165],{"class":157},[147,761,443],{"class":179},[147,763,446],{"class":153},[147,765,405],{"class":179},[147,767,165],{"class":157},[147,769,453],{"class":179},[147,771,773,776],{"class":149,"line":772},12,[147,774,775],{"class":153},"  GROUP BY",[147,777,778],{"class":157}," root_path)\n",[147,780,782,785],{"class":149,"line":781},13,[147,783,784],{"class":153},"UPDATE",[147,786,787],{"class":157}," namespace_storage_statistics\n",[147,789,791,794,797,800,803,805,807],{"class":149,"line":790},14,[147,792,793],{"class":153},"SET",[147,795,796],{"class":157}," storage_size ",[147,798,799],{"class":153},"=",[147,801,802],{"class":179}," refresh",[147,804,165],{"class":157},[147,806,211],{"class":179},[147,808,809],{"class":157},",\n",[147,811,813,816,818,820,822,824],{"class":149,"line":812},15,[147,814,815],{"class":157},"    repository_size ",[147,817,799],{"class":153},[147,819,802],{"class":179},[147,821,165],{"class":157},[147,823,243],{"class":179},[147,825,809],{"class":157},[147,827,829,832,834,836,838,840],{"class":149,"line":828},16,[147,830,831],{"class":157},"    wiki_size ",[147,833,799],{"class":153},[147,835,802],{"class":179},[147,837,165],{"class":157},[147,839,272],{"class":179},[147,841,809],{"class":157},[147,843,845,848,850,852,854,856],{"class":149,"line":844},17,[147,846,847],{"class":157},"    lfs_objects_size ",[147,849,799],{"class":153},[147,851,802],{"class":179},[147,853,165],{"class":157},[147,855,301],{"class":179},[147,857,809],{"class":157},[147,859,861,864,866,868,870,872],{"class":149,"line":860},18,[147,862,863],{"class":157},"    build_artifacts_size ",[147,865,799],{"class":153},[147,867,802],{"class":179},[147,869,165],{"class":157},[147,871,330],{"class":179},[147,873,809],{"class":157},[147,875,877,880,882,884,886],{"class":149,"line":876},19,[147,878,879],{"class":157},"    packages_size  ",[147,881,799],{"class":153},[147,883,802],{"class":179},[147,885,165],{"class":157},[147,887,888],{"class":179},"packages_size\n",[147,890,892,894],{"class":149,"line":891},20,[147,893,376],{"class":153},[147,895,896],{"class":157}," refresh\n",[147,898,900,902,904,906,908,910,912,914,916,918,921,923,925,927,929,931],{"class":149,"line":899},21,[147,901,385],{"class":153},[147,903,388],{"class":157},[147,905,391],{"class":153},[147,907,394],{"class":179},[147,909,165],{"class":157},[147,911,168],{"class":179},[147,913,402],{"class":153},[147,915,802],{"class":179},[147,917,165],{"class":157},[147,919,920],{"class":179},"root_path",[147,922,413],{"class":153},[147,924,394],{"class":179},[147,926,165],{"class":157},[147,928,420],{"class":179},[147,930,402],{"class":153},[147,932,933],{"class":161}," 'Namespace'\n",[147,935,937,940,943,945,948,950,952,954],{"class":149,"line":936},22,[147,938,939],{"class":153},"WHERE",[147,941,942],{"class":179}," namespace_storage_statistics",[147,944,165],{"class":157},[147,946,947],{"class":179},"namespace_id",[147,949,402],{"class":153},[147,951,394],{"class":179},[147,953,165],{"class":157},[147,955,956],{"class":179},"source_id\n",[12,958,959],{},"Unlike Attempt A, a CTE will be limited to the namespace we care about instead of operating on all namespaces. The downside of it,\nis that earlier versions of MySQL do not support Common Table Expressions.",[124,961,963],{"id":962},"attempt-c-get-rid-of-the-model-and-store-the-statistics-on-redis","Attempt C: Get rid of the model and store the statistics on Redis",[12,965,966,967,972,973,978],{},"We could get rid of the model that stores the statistics in aggregated form and instead use a Redis Set.\nThis would be the ",[19,968,971],{"href":969,"rel":970},"https://handbook.gitlab.com/handbook/values/#boring-solutions",[],"boring solution"," and the fastest one\nto implement, as GitLab already includes ",[19,974,977],{"href":975,"rel":976},"https://docs.gitlab.com/development/architecture/#redis",[],"Redis"," as part of its Architecture.",[12,980,981],{},"The downside of this approach is that Redis does not provide the same persistence/consistency guarantees as PostgreSQL,\nand the namespace statistics are information we can't afford to lose in a case of a Redis failure. Also, searching for\ninformation like the largest namespaces per repository size will be easier to do in PostgreSQL than in Redis.",[124,983,985],{"id":984},"attempt-d-tag-the-root-namespace-and-its-child-namespaces","Attempt D: Tag the root namespace and its child namespaces",[12,987,988],{},"Directly relate the root namespace to its child namespaces, so\nwhenever a child namespace is created, it's also tagged with the\nroot namespace ID:",[990,991,992,1008],"table",{},[993,994,995],"thead",{},[996,997,998,1002,1005],"tr",{},[999,1000,410],"th",{"align":1001},"left",[999,1003,1004],{"align":1001},"root_id",[999,1006,1007],{"align":1001},"parent_id",[1009,1010,1011,1021,1030],"tbody",{},[996,1012,1013,1016,1018],{},[1014,1015,180],"td",{"align":1001},[1014,1017,180],{"align":1001},[1014,1019,1020],{"align":1001},"NULL",[996,1022,1023,1026,1028],{},[1014,1024,1025],{"align":1001},"2",[1014,1027,180],{"align":1001},[1014,1029,180],{"align":1001},[996,1031,1032,1035,1037],{},[1014,1033,1034],{"align":1001},"3",[1014,1036,180],{"align":1001},[1014,1038,1025],{"align":1001},[12,1040,1041],{},"To aggregate the statistics inside a namespace we'd execute something like:",[138,1043,1045],{"className":140,"code":1044,"language":142,"meta":143,"style":143},"SELECT COUNT(...)\nFROM projects\nWHERE namespace_id IN (\n  SELECT id\n  FROM namespaces\n  WHERE root_id = X\n)\n",[52,1046,1047,1057,1064,1076,1083,1090,1103],{"__ignoreMap":143},[147,1048,1049,1051,1054],{"class":149,"line":150},[147,1050,154],{"class":153},[147,1052,1053],{"class":179}," COUNT",[147,1055,1056],{"class":157},"(...)\n",[147,1058,1059,1061],{"class":149,"line":192},[147,1060,376],{"class":153},[147,1062,1063],{"class":157}," projects\n",[147,1065,1066,1068,1071,1074],{"class":149,"line":228},[147,1067,939],{"class":153},[147,1069,1070],{"class":157}," namespace_id ",[147,1072,1073],{"class":153},"IN",[147,1075,522],{"class":157},[147,1077,1078,1080],{"class":149,"line":257},[147,1079,527],{"class":153},[147,1081,1082],{"class":157}," id\n",[147,1084,1085,1087],{"class":149,"line":286},[147,1086,710],{"class":153},[147,1088,1089],{"class":157}," namespaces\n",[147,1091,1092,1095,1098,1100],{"class":149,"line":315},[147,1093,1094],{"class":153},"  WHERE",[147,1096,1097],{"class":157}," root_id ",[147,1099,799],{"class":153},[147,1101,1102],{"class":157}," X\n",[147,1104,1105],{"class":149,"line":344},[147,1106,1107],{"class":157},")\n",[12,1109,1110],{},"Even though this approach would make aggregating much easier, it has some major downsides:",[83,1112,1113,1126],{},[34,1114,1115,1116,1120,1121,165],{},"We'd have to migrate ",[1117,1118,1119],"strong",{},"all namespaces"," by adding and filling a new column. Because of the size of the table, dealing with the time/cost will not be great. The ",[19,1122,1125],{"href":1123,"rel":1124},"https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/29772#note_182201607",[],"background migration will take approximately 153 hours",[34,1127,1128],{},"The background migration has to be shipped one release before we want to start using the new data, delaying the functionality by another milestone.",[124,1130,1132],{"id":1131},"attempt-e-update-the-namespace-storage-statistics-asynchronously","Attempt E: Update the namespace storage statistics asynchronously",[12,1134,1135],{},"For this approach we continue using the incremental statistics updates we already have,\nbut we refresh them through Sidekiq jobs and in different SQL transactions:",[31,1137,1138,1150,1171,1194,1208],{},[34,1139,1140,1141,1144,1145,1147,1148,165],{},"Create a second table (",[52,1142,1143],{},"namespace_aggregation_schedules",") with two columns ",[52,1146,410],{}," and ",[52,1149,947],{},[34,1151,1152,1153,1155],{},"Whenever the statistics of a project changes, insert a row into ",[52,1154,1143],{},[83,1156,1157,1160],{},[34,1158,1159],{},"We don't insert a new row if there's already one related to the root namespace.",[34,1161,1162,1163,1170],{},"Keeping in mind the length of the transaction that involves ",[19,1164,1166,1167],{"href":108,"rel":1165},[],"updating ",[52,1168,1169],{},"project_statistics",", the insertion should be done in a different transaction and through a Sidekiq Job.",[34,1172,1173,1174],{},"After inserting the row, we schedule another worker to be executed async at two different moments:\n",[83,1175,1176,1183,1189],{},[34,1177,1178,1179,1182],{},"One enqueued for immediate execution and another one scheduled in ",[52,1180,1181],{},"1.5h"," hours.",[34,1184,1185,1186,1188],{},"We only schedule the jobs if we can obtain a ",[52,1187,1181],{}," lease on Redis on a key based on the root namespace ID.",[34,1190,1191,1192,165],{},"If we can't obtain the lease it indicates there's another aggregation already in progress or scheduled in no more than ",[52,1193,1181],{},[34,1195,1196,1197],{},"This worker will:\n",[83,1198,1199,1202],{},[34,1200,1201],{},"Update the root namespace storage statistics by querying all the namespaces through a service.",[34,1203,1204,1205,1207],{},"Delete the related ",[52,1206,1143],{}," after the update.",[34,1209,1210,1211,1213,1214],{},"Another Sidekiq job is also included to traverse any remaining rows on the ",[52,1212,1143],{}," table and schedule jobs for every pending row.\n",[83,1215,1216],{},[34,1217,1218],{},"This job is scheduled with cron to run every night (UTC).",[12,1220,1221],{},"This implementation has the following benefits:",[83,1223,1224,1229,1232,1235],{},[34,1225,1226,1227,165],{},"All the updates are done async, so we're not increasing the length of the transactions for ",[52,1228,1169],{},[34,1230,1231],{},"We're doing the update in a single SQL query.",[34,1233,1234],{},"It is compatible with PostgreSQL and MySQL.",[34,1236,1237],{},"No background migration is required.",[12,1239,1240],{},"The downsides of this approaches are:",[83,1242,1243,1255,1258],{},[34,1244,1245,1246,1249,1250,165],{},"Namespaces' statistics are updated up to ",[52,1247,1248],{},"1.5"," hours after the change is done, which means there's a brief window in time where the statistics are inaccurate. This is not a major problem because we're not currently ",[19,1251,1254],{"href":1252,"rel":1253},"https://gitlab.com/gitlab-org/gitlab-ce/issues/30421",[],"enforcing storage limits",[34,1256,1257],{},"From the implementation perspective, this approach is more complex than the migration approach (Attempt D).",[34,1259,1260,1262],{},[52,1261,1143],{}," table will see a high rate of inserts and deletes, which may require that we tune auto vacuuming for this table.",[12,1264,1265,1266,1270],{},"We went with ",[1267,1268,1269],"em",{},"Attempt E"," because updating the storage statistics asynchronously was the less problematic and\nperformant approach of aggregating the root namespaces.",[26,1272,1274],{"id":1273},"enabling-the-feature-on-gitlabcom","Enabling the feature on GitLab.com",[12,1276,1277,1278,1283],{},"Given this is a performance improvement, we have to be very careful introducing this change to GitLab.com: Which is why\nwe decided to release it under ",[19,1279,1282],{"href":1280,"rel":1281},"https://docs.gitlab.com/development/feature_flags/",[],"feature flag"," and roll it out gradually by:",[31,1285,1286,1289,1296],{},[34,1287,1288],{},"Enable it on our staging environment and measure the performance.",[34,1290,1291,1292,1295],{},"Enable it on GitLab.com on different periods for the ",[52,1293,1294],{},"gitlab-org"," group and measure the performance.",[34,1297,1298],{},"Enable it globally on GitLab.com on different periods and measure the performance.",[12,1300,1301],{},"Finally if no problem arises, we can be confident this change performs properly on GitLab.com and we can\nremove the feature flag.",[26,1303,1305],{"id":1304},"measuring-the-performance","Measuring the performance",[12,1307,1308,1309,1314],{},"To assess the execution of this approach, we monitored the ",[19,1310,1313],{"href":1311,"rel":1312},"https://dashboards.gitlab.com/d/9GOIu9Siz/sidekiq-stats?orgId=1",[],"Sidekiq dashboards"," on Kibana to ensure jobs were being executed flawlessly and without using too much memory or CPU. Particularly, we observed the \"Sidekiq queue size,\" \"Rate of running jobs,\" and \"Running jobs\" dashboards.",[124,1316,1318],{"id":1317},"on-staging","On staging",[12,1320,1321],{},"The feature was enabled globally on staging and the execution of the jobs was satisfactory. But there was barely any traffic to measure the impact of our changes:",[12,1323,1324],{},[1325,1326],"img",{"alt":1327,"src":1328},"Graph showing the queue size of the ScheduleAggregationWorker on staging","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398755/blog/Content%20Images/namespace_statistics/staging-1.png",[124,1330,1332],{"id":1331},"enabling-root-namespaces-on-gitlabcom","Enabling root namespaces on GitLab.com",[12,1334,1335,1336,1338],{},"Our results were different on GitLab.com. We first enabled it for the ",[52,1337,1294],{}," group and we quickly started to observe more traffic:",[12,1340,1341],{},[1325,1342],{"alt":1343,"src":1344},"Graph showing the queue size of the ScheduleAggregationWorker on GitLab.com","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398751/blog/Content%20Images/namespace_statistics/production-1.png",[12,1346,1347],{},[1325,1348],{"alt":1349,"src":1350},"Graph showing the running jobs of the ScheduleAggregationWorker on GitLab.com","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398751/blog/Content%20Images/namespace_statistics/production-2.png",[12,1352,1353],{},"Once we enabled the feature flag globally, the rate of running jobs increased considerably:",[12,1355,1356],{},[1325,1357],{"alt":1358,"src":1359},"Graph showing the rate running jobs of the ScheduleAggregationWorker on GitLab.com","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398751/blog/Content%20Images/namespace_statistics/production-3.png",[12,1361,1362],{},[1325,1363],{"alt":1364,"src":1365},"Graph showing the rate running jobs of the RootStatisticsWorker on GitLab.com","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398754/blog/Content%20Images/namespace_statistics/production-4.png",[26,1367,1369],{"id":1368},"root-namespaces-on-gitlabcom-today","Root namespaces on GitLab.com today",[12,1371,1372,1373,1376],{},"We currently have nearly ",[52,1374,1375],{},"400 000"," statistics stored for root namespaces on GitLab.com, which are updated at a high pace.\nBeing able to efficiently fetch those statistics allows one to easily track the top biggest repositories and/or namespaces on an instance\nand to start paving the way to enforce storage limits for groups on GitLab.com.",[12,1378,1379],{},"Learn more about this use case by reading:",[83,1381,1382,1389,1396],{},[34,1383,1384],{},[19,1385,1388],{"href":1386,"rel":1387},"https://gitlab.com/gitlab-org/gitlab-ce/issues/62214",[],"The original issue",[34,1390,1391],{},[19,1392,1395],{"href":1393,"rel":1394},"https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28996",[],"Merge Request with the implementation",[34,1397,1398],{},[19,1399,1402],{"href":1400,"rel":1401},"https://gitlab.com/gitlab-org/gitlab-ce/issues/64092",[],"Details of the performance measured against staging and production (GitLab.com)",[12,1404,1405,1406,1411,1412,165],{},"Cover photo by ",[19,1407,1410],{"href":1408,"rel":1409},"https://unsplash.com/@bill_oxford?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText",[],"Bill Oxford"," on ",[19,1413,1416],{"href":1414,"rel":1415},"https://unsplash.com/search/photos/engineering?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText",[],"Unsplash",[1418,1419,1420],"style",{},"html pre.shiki code .sD7c4, html code.shiki .sD7c4{--shiki-default:#D73A49}html pre.shiki code .sgsFI, html code.shiki .sgsFI{--shiki-default:#24292E}html pre.shiki code .sYBdl, html code.shiki .sYBdl{--shiki-default:#032F62}html pre.shiki code .sYu0t, html code.shiki .sYu0t{--shiki-default:#005CC5}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":143,"searchDepth":192,"depth":192,"links":1422},[1423,1424,1431,1432,1436],{"id":28,"depth":192,"text":29},{"id":121,"depth":192,"text":122,"children":1425},[1426,1427,1428,1429,1430],{"id":126,"depth":228,"text":127},{"id":495,"depth":228,"text":496},{"id":962,"depth":228,"text":963},{"id":984,"depth":228,"text":985},{"id":1131,"depth":228,"text":1132},{"id":1273,"depth":192,"text":1274},{"id":1304,"depth":192,"text":1305,"children":1433},[1434,1435],{"id":1317,"depth":228,"text":1318},{"id":1331,"depth":228,"text":1332},{"id":1368,"depth":192,"text":1369},"engineering","2019-10-14","Explore all the different engineering approaches to store and update the namespace statistics in a performant manner.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749672677/Blog/Hero%20Images/metalgears_databasecasestudy.jpg",{},true,"/en-us/blog/database-case-study-store-and-update-namespace-statistics",{"title":5,"description":1439,"ogTitle":5,"ogDescription":1439,"noIndex":1442,"ogImage":1443,"ogUrl":1448,"ogSiteName":1449,"ogType":1450,"canonicalUrls":1448},"https://about.gitlab.com/blog/database-case-study-store-and-update-namespace-statistics","https://about.gitlab.com","article","database-case-study-store-and-update-namespace-statistics","en-us/blog/database-case-study-store-and-update-namespace-statistics",[1454],"inside GitLab","BlogPost","X1VPASNm1elWec5twySwXxzcP9Pa4nL-8jWc4v-CZGg",{"logo":1458,"freeTrial":1463,"sales":1468,"login":1473,"items":1478,"search":1805,"minimal":1836,"duo":1855,"switchNav":1864,"pricingDeployment":1875},{"config":1459},{"href":1460,"dataGaName":1461,"dataGaLocation":1462},"/","gitlab logo","header",{"text":1464,"config":1465},"Get free trial",{"href":1466,"dataGaName":1467,"dataGaLocation":1462},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":1469,"config":1470},"Request a demo",{"href":1471,"dataGaName":1472,"dataGaLocation":1462},"/sales/?contact-topic=request-demo","sales",{"text":1474,"config":1475},"Sign in",{"href":1476,"dataGaName":1477,"dataGaLocation":1462},"https://gitlab.com/users/sign_in/","sign in",[1479,1508,1608,1613,1727,1783],{"text":1480,"config":1481,"menu":1483},"Platform",{"dataNavLevelOne":1482},"platform",{"type":1484,"columns":1485},"cards",[1486,1492,1500],{"title":1480,"description":1487,"link":1488},"The intelligent orchestration platform for DevSecOps",{"text":1489,"config":1490},"Explore our Platform",{"href":1491,"dataGaName":1482,"dataGaLocation":1462},"/platform/",{"title":1493,"description":1494,"link":1495},"GitLab Duo Agent Platform","Agentic AI for the entire software lifecycle",{"text":1496,"config":1497},"Meet GitLab Duo",{"href":1498,"dataGaName":1499,"dataGaLocation":1462},"/gitlab-duo-agent-platform/","gitlab duo agent platform",{"title":1501,"description":1502,"link":1503},"Why GitLab","See the top reasons enterprises choose GitLab",{"text":1504,"config":1505},"Learn more",{"href":1506,"dataGaName":1507,"dataGaLocation":1462},"/why-gitlab/","why gitlab",{"text":1509,"left":1445,"config":1510,"menu":1512},"Product",{"dataNavLevelOne":1511},"solutions",{"type":1513,"link":1514,"columns":1518,"feature":1587},"lists",{"text":1515,"config":1516},"View all Solutions",{"href":1517,"dataGaName":1511,"dataGaLocation":1462},"/solutions/",[1519,1543,1566],{"title":1520,"description":1521,"link":1522,"items":1527},"Automation","CI/CD and automation to accelerate deployment",{"config":1523},{"icon":1524,"href":1525,"dataGaName":1526,"dataGaLocation":1462},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[1528,1532,1535,1539],{"text":1529,"config":1530},"CI/CD",{"href":1531,"dataGaLocation":1462,"dataGaName":1529},"/solutions/continuous-integration/",{"text":1493,"config":1533},{"href":1498,"dataGaLocation":1462,"dataGaName":1534},"gitlab duo agent platform - product menu",{"text":1536,"config":1537},"Source Code Management",{"href":1538,"dataGaLocation":1462,"dataGaName":1536},"/solutions/source-code-management/",{"text":1540,"config":1541},"Automated Software Delivery",{"href":1525,"dataGaLocation":1462,"dataGaName":1542},"Automated software delivery",{"title":1544,"description":1545,"link":1546,"items":1551},"Security","Deliver code faster without compromising security",{"config":1547},{"href":1548,"dataGaName":1549,"dataGaLocation":1462,"icon":1550},"/solutions/application-security-testing/","security and compliance","ShieldCheckLight",[1552,1556,1561],{"text":1553,"config":1554},"Application Security Testing",{"href":1548,"dataGaName":1555,"dataGaLocation":1462},"Application security testing",{"text":1557,"config":1558},"Software Supply Chain Security",{"href":1559,"dataGaLocation":1462,"dataGaName":1560},"/solutions/supply-chain/","Software supply chain security",{"text":1562,"config":1563},"Software Compliance",{"href":1564,"dataGaName":1565,"dataGaLocation":1462},"/solutions/software-compliance/","software compliance",{"title":1567,"link":1568,"items":1573},"Measurement",{"config":1569},{"icon":1570,"href":1571,"dataGaName":1572,"dataGaLocation":1462},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[1574,1578,1582],{"text":1575,"config":1576},"Visibility & Measurement",{"href":1571,"dataGaLocation":1462,"dataGaName":1577},"Visibility and Measurement",{"text":1579,"config":1580},"Value Stream Management",{"href":1581,"dataGaLocation":1462,"dataGaName":1579},"/solutions/value-stream-management/",{"text":1583,"config":1584},"Analytics & Insights",{"href":1585,"dataGaLocation":1462,"dataGaName":1586},"/solutions/analytics-and-insights/","Analytics and insights",{"title":1588,"type":1513,"items":1589},"GitLab for",[1590,1596,1602],{"text":1591,"config":1592},"Enterprise",{"icon":1593,"href":1594,"dataGaLocation":1462,"dataGaName":1595},"Building","/enterprise/","enterprise",{"text":1597,"config":1598},"Small Business",{"icon":1599,"href":1600,"dataGaLocation":1462,"dataGaName":1601},"Work","/small-business/","small business",{"text":1603,"config":1604},"Public Sector",{"icon":1605,"href":1606,"dataGaLocation":1462,"dataGaName":1607},"Organization","/solutions/public-sector/","public sector",{"text":1609,"config":1610},"Pricing",{"href":1611,"dataGaName":1612,"dataGaLocation":1462,"dataNavLevelOne":1612},"/pricing/","pricing",{"text":1614,"config":1615,"menu":1617},"Resources",{"dataNavLevelOne":1616},"resources",{"type":1513,"link":1618,"columns":1622,"feature":1716},{"text":1619,"config":1620},"View all resources",{"href":1621,"dataGaName":1616,"dataGaLocation":1462},"/resources/",[1623,1656,1683],{"title":1624,"items":1625},"Getting started",[1626,1631,1636,1641,1646,1651],{"text":1627,"config":1628},"Install",{"href":1629,"dataGaName":1630,"dataGaLocation":1462},"/install/","install",{"text":1632,"config":1633},"Quick start guides",{"href":1634,"dataGaName":1635,"dataGaLocation":1462},"/get-started/","quick setup checklists",{"text":1637,"config":1638},"Learn",{"href":1639,"dataGaLocation":1462,"dataGaName":1640},"https://university.gitlab.com/","learn",{"text":1642,"config":1643},"Product documentation",{"href":1644,"dataGaName":1645,"dataGaLocation":1462},"https://docs.gitlab.com/","product documentation",{"text":1647,"config":1648},"Best practice videos",{"href":1649,"dataGaName":1650,"dataGaLocation":1462},"/getting-started-videos/","best practice videos",{"text":1652,"config":1653},"Integrations",{"href":1654,"dataGaName":1655,"dataGaLocation":1462},"/integrations/","integrations",{"title":1657,"items":1658},"Discover",[1659,1664,1669,1674,1678],{"text":1660,"config":1661},"Customer success stories",{"href":1662,"dataGaName":1663,"dataGaLocation":1462},"/customers/","customer success stories",{"text":1665,"config":1666},"Blog",{"href":1667,"dataGaName":1668,"dataGaLocation":1462},"/blog/","blog",{"text":1670,"config":1671},"Demo Hub",{"href":1672,"dataGaName":1673,"dataGaLocation":1462},"/demo-hub/","demo hub",{"text":1675,"config":1676},"The Source",{"href":1677,"dataGaName":1668,"dataGaLocation":1462},"/the-source/",{"text":1679,"config":1680},"Remote",{"href":1681,"dataGaName":1682,"dataGaLocation":1462},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"title":1684,"items":1685},"Connect",[1686,1691,1696,1701,1706,1711],{"text":1687,"config":1688},"GitLab Services",{"href":1689,"dataGaName":1690,"dataGaLocation":1462},"/services/","services",{"text":1692,"config":1693},"Contribute",{"href":1694,"dataGaName":1695,"dataGaLocation":1462},"https://contributors.gitlab.com","contribute",{"text":1697,"config":1698},"Community",{"href":1699,"dataGaName":1700,"dataGaLocation":1462},"/community/","community",{"text":1702,"config":1703},"Forum",{"href":1704,"dataGaName":1705,"dataGaLocation":1462},"https://forum.gitlab.com/","forum",{"text":1707,"config":1708},"Events",{"href":1709,"dataGaName":1710,"dataGaLocation":1462},"/events/","events",{"text":1712,"config":1713},"Partners",{"href":1714,"dataGaName":1715,"dataGaLocation":1462},"/partners/","partners",{"config":1717,"title":1720,"text":1721,"link":1722},{"background":1718,"textColor":1719},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1777322348/qpq8yrgn8knii57omj0c.png')","#000","What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":1723,"config":1724},"Read the latest",{"href":1725,"dataGaName":1726,"dataGaLocation":1462},"/whats-new/","whats new",{"text":1728,"config":1729,"menu":1731},"Company",{"dataNavLevelOne":1730},"company",{"type":1513,"columns":1732},[1733],{"items":1734},[1735,1740,1746,1748,1753,1758,1763,1768,1773,1778],{"text":1736,"config":1737},"About",{"href":1738,"dataGaName":1739,"dataGaLocation":1462},"/company/","about",{"text":1741,"config":1742,"footerGa":1745},"Jobs",{"href":1743,"dataGaName":1744,"dataGaLocation":1462},"/jobs/","jobs",{"dataGaName":1744},{"text":1707,"config":1747},{"href":1709,"dataGaName":1710,"dataGaLocation":1462},{"text":1749,"config":1750},"Leadership",{"href":1751,"dataGaName":1752,"dataGaLocation":1462},"/company/team/e-group/","leadership",{"text":1754,"config":1755},"Handbook",{"href":1756,"dataGaName":1757,"dataGaLocation":1462},"https://handbook.gitlab.com/","handbook",{"text":1759,"config":1760},"Investor relations",{"href":1761,"dataGaName":1762,"dataGaLocation":1462},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":1764,"config":1765},"Trust Center",{"href":1766,"dataGaName":1767,"dataGaLocation":1462},"/security/","trust center",{"text":1769,"config":1770},"AI Transparency Center",{"href":1771,"dataGaName":1772,"dataGaLocation":1462},"/ai-transparency-center/","ai transparency center",{"text":1774,"config":1775},"Newsletter",{"href":1776,"dataGaName":1777,"dataGaLocation":1462},"/company/contact/#contact-forms","newsletter",{"text":1779,"config":1780},"Press",{"href":1781,"dataGaName":1782,"dataGaLocation":1462},"/press/","press",{"text":1784,"config":1785,"menu":1786},"Contact us",{"dataNavLevelOne":1730},{"type":1513,"columns":1787},[1788],{"items":1789},[1790,1795,1800],{"text":1791,"config":1792},"Talk to sales",{"href":1793,"dataGaName":1794,"dataGaLocation":1462},"/sales/","talk to sales",{"text":1796,"config":1797},"Support portal",{"href":1798,"dataGaName":1799,"dataGaLocation":1462},"https://support.gitlab.com/hc/en-us","support portal",{"text":1801,"config":1802},"Customer portal",{"href":1803,"dataGaName":1804,"dataGaLocation":1462},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":1806,"login":1807,"suggestions":1814},"Close",{"text":1808,"link":1809},"To search repositories and projects, login to",{"text":1810,"config":1811},"gitlab.com",{"href":1476,"dataGaName":1812,"dataGaLocation":1813},"search login","search",{"text":1815,"default":1816},"Suggestions",[1817,1819,1823,1825,1829,1833],{"text":1493,"config":1818},{"href":1498,"dataGaName":1493,"dataGaLocation":1813},{"text":1820,"config":1821},"Code Suggestions (AI)",{"href":1822,"dataGaName":1820,"dataGaLocation":1813},"/solutions/code-suggestions/",{"text":1529,"config":1824},{"href":1531,"dataGaName":1529,"dataGaLocation":1813},{"text":1826,"config":1827},"GitLab on AWS",{"href":1828,"dataGaName":1826,"dataGaLocation":1813},"/partners/technology-partners/aws/",{"text":1830,"config":1831},"GitLab on Google Cloud",{"href":1832,"dataGaName":1830,"dataGaLocation":1813},"/partners/technology-partners/google-cloud-platform/",{"text":1834,"config":1835},"Why GitLab?",{"href":1506,"dataGaName":1834,"dataGaLocation":1813},{"freeTrial":1837,"mobileIcon":1842,"desktopIcon":1847,"secondaryButton":1850},{"text":1838,"config":1839},"Start free trial",{"href":1840,"dataGaName":1467,"dataGaLocation":1841},"https://gitlab.com/-/trials/new/","nav",{"altText":1843,"config":1844},"Gitlab Icon",{"src":1845,"dataGaName":1846,"dataGaLocation":1841},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":1843,"config":1848},{"src":1849,"dataGaName":1846,"dataGaLocation":1841},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":1851,"config":1852},"Get Started",{"href":1853,"dataGaName":1854,"dataGaLocation":1841},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":1856,"mobileIcon":1860,"desktopIcon":1862},{"text":1857,"config":1858},"Learn more about GitLab Duo",{"href":1498,"dataGaName":1859,"dataGaLocation":1841},"gitlab duo",{"altText":1843,"config":1861},{"src":1845,"dataGaName":1846,"dataGaLocation":1841},{"altText":1843,"config":1863},{"src":1849,"dataGaName":1846,"dataGaLocation":1841},{"button":1865,"mobileIcon":1870,"desktopIcon":1872},{"text":1866,"config":1867},"/switch",{"href":1868,"dataGaName":1869,"dataGaLocation":1841},"#contact","switch",{"altText":1843,"config":1871},{"src":1845,"dataGaName":1846,"dataGaLocation":1841},{"altText":1843,"config":1873},{"src":1874,"dataGaName":1846,"dataGaLocation":1841},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":1876,"mobileIcon":1881,"desktopIcon":1883},{"text":1877,"config":1878},"Back to pricing",{"href":1611,"dataGaName":1879,"dataGaLocation":1841,"icon":1880},"back to pricing","GoBack",{"altText":1843,"config":1882},{"src":1845,"dataGaName":1846,"dataGaLocation":1841},{"altText":1843,"config":1884},{"src":1849,"dataGaName":1846,"dataGaLocation":1841},{"title":1886,"titleMobile":1887,"button":1888,"config":1893},"Duo Agent Platform delivers 400% ROI, per new Forrester Consulting study.","400% ROI: Forrester TEI for GitLab Duo",{"text":1504,"config":1889},{"href":1890,"dataGaName":1891,"dataGaLocation":1892},"https://about.gitlab.com/blog/gitlab-duo-agent-platform-delivers-400-percent-roi/","forrester-tei-dap-banner","global-banner",{"layout":1894,"disabled":1442},"release",{"data":1896},{"text":1897,"source":1898,"edit":1904,"contribute":1909,"config":1914,"items":1919,"minimal":2129},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":1899,"config":1900},"View page source",{"href":1901,"dataGaName":1902,"dataGaLocation":1903},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":1905,"config":1906},"Edit this page",{"href":1907,"dataGaName":1908,"dataGaLocation":1903},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":1910,"config":1911},"Please contribute",{"href":1912,"dataGaName":1913,"dataGaLocation":1903},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":1915,"facebook":1916,"youtube":1917,"linkedin":1918},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[1920,1967,2021,2065,2097],{"title":1609,"links":1921,"subMenu":1936},[1922,1926,1931],{"text":1923,"config":1924},"View plans",{"href":1611,"dataGaName":1925,"dataGaLocation":1903},"view plans",{"text":1927,"config":1928},"Why Premium?",{"href":1929,"dataGaName":1930,"dataGaLocation":1903},"/pricing/premium/","why premium",{"text":1932,"config":1933},"Why Ultimate?",{"href":1934,"dataGaName":1935,"dataGaLocation":1903},"/pricing/ultimate/","why ultimate",[1937],{"title":1938,"links":1939},"Contact Us",[1940,1943,1945,1947,1952,1957,1962],{"text":1941,"config":1942},"Contact sales",{"href":1793,"dataGaName":1472,"dataGaLocation":1903},{"text":1796,"config":1944},{"href":1798,"dataGaName":1799,"dataGaLocation":1903},{"text":1801,"config":1946},{"href":1803,"dataGaName":1804,"dataGaLocation":1903},{"text":1948,"config":1949},"Status",{"href":1950,"dataGaName":1951,"dataGaLocation":1903},"https://status.gitlab.com/","status",{"text":1953,"config":1954},"Terms of use",{"href":1955,"dataGaName":1956,"dataGaLocation":1903},"/terms/","terms of use",{"text":1958,"config":1959},"Privacy statement",{"href":1960,"dataGaName":1961,"dataGaLocation":1903},"/privacy/","privacy statement",{"text":1963,"config":1964},"Cookie preferences",{"dataGaName":1965,"dataGaLocation":1903,"id":1966,"isOneTrustButton":1445},"cookie preferences","ot-sdk-btn",{"title":1509,"links":1968,"subMenu":1977},[1969,1973],{"text":1970,"config":1971},"DevSecOps platform",{"href":1491,"dataGaName":1972,"dataGaLocation":1903},"devsecops platform",{"text":1974,"config":1975},"AI-Assisted Development",{"href":1498,"dataGaName":1976,"dataGaLocation":1903},"ai-assisted development",[1978],{"title":1979,"links":1980},"Topics",[1981,1986,1991,1996,2001,2006,2011,2016],{"text":1982,"config":1983},"CICD",{"href":1984,"dataGaName":1985,"dataGaLocation":1903},"/topics/ci-cd/","cicd",{"text":1987,"config":1988},"GitOps",{"href":1989,"dataGaName":1990,"dataGaLocation":1903},"/topics/gitops/","gitops",{"text":1992,"config":1993},"DevOps",{"href":1994,"dataGaName":1995,"dataGaLocation":1903},"/topics/devops/","devops",{"text":1997,"config":1998},"Version Control",{"href":1999,"dataGaName":2000,"dataGaLocation":1903},"/topics/version-control/","version control",{"text":2002,"config":2003},"DevSecOps",{"href":2004,"dataGaName":2005,"dataGaLocation":1903},"/topics/devsecops/","devsecops",{"text":2007,"config":2008},"Cloud Native",{"href":2009,"dataGaName":2010,"dataGaLocation":1903},"/topics/cloud-native/","cloud native",{"text":2012,"config":2013},"AI for Coding",{"href":2014,"dataGaName":2015,"dataGaLocation":1903},"/topics/devops/ai-for-coding/","ai for coding",{"text":2017,"config":2018},"Agentic AI",{"href":2019,"dataGaName":2020,"dataGaLocation":1903},"/topics/agentic-ai/","agentic ai",{"title":2022,"links":2023},"Solutions",[2024,2026,2028,2033,2037,2040,2044,2047,2049,2052,2055,2060],{"text":1553,"config":2025},{"href":1548,"dataGaName":1553,"dataGaLocation":1903},{"text":1542,"config":2027},{"href":1525,"dataGaName":1526,"dataGaLocation":1903},{"text":2029,"config":2030},"Agile development",{"href":2031,"dataGaName":2032,"dataGaLocation":1903},"/solutions/agile-delivery/","agile delivery",{"text":2034,"config":2035},"SCM",{"href":1538,"dataGaName":2036,"dataGaLocation":1903},"source code management",{"text":1982,"config":2038},{"href":1531,"dataGaName":2039,"dataGaLocation":1903},"continuous integration & delivery",{"text":2041,"config":2042},"Value stream management",{"href":1581,"dataGaName":2043,"dataGaLocation":1903},"value stream management",{"text":1987,"config":2045},{"href":2046,"dataGaName":1990,"dataGaLocation":1903},"/solutions/gitops/",{"text":1591,"config":2048},{"href":1594,"dataGaName":1595,"dataGaLocation":1903},{"text":2050,"config":2051},"Small business",{"href":1600,"dataGaName":1601,"dataGaLocation":1903},{"text":2053,"config":2054},"Public sector",{"href":1606,"dataGaName":1607,"dataGaLocation":1903},{"text":2056,"config":2057},"Education",{"href":2058,"dataGaName":2059,"dataGaLocation":1903},"/solutions/education/","education",{"text":2061,"config":2062},"Financial services",{"href":2063,"dataGaName":2064,"dataGaLocation":1903},"/solutions/finance/","financial services",{"title":1614,"links":2066},[2067,2069,2071,2073,2076,2078,2081,2083,2085,2087,2089,2091,2093,2095],{"text":1627,"config":2068},{"href":1629,"dataGaName":1630,"dataGaLocation":1903},{"text":1632,"config":2070},{"href":1634,"dataGaName":1635,"dataGaLocation":1903},{"text":1637,"config":2072},{"href":1639,"dataGaName":1640,"dataGaLocation":1903},{"text":1642,"config":2074},{"href":1644,"dataGaName":2075,"dataGaLocation":1903},"docs",{"text":1665,"config":2077},{"href":1667,"dataGaName":1668,"dataGaLocation":1903},{"text":2079,"config":2080},"What's new",{"href":1725,"dataGaName":1726,"dataGaLocation":1903},{"text":1660,"config":2082},{"href":1662,"dataGaName":1663,"dataGaLocation":1903},{"text":1679,"config":2084},{"href":1681,"dataGaName":1682,"dataGaLocation":1903},{"text":1687,"config":2086},{"href":1689,"dataGaName":1690,"dataGaLocation":1903},{"text":1692,"config":2088},{"href":1694,"dataGaName":1695,"dataGaLocation":1903},{"text":1697,"config":2090},{"href":1699,"dataGaName":1700,"dataGaLocation":1903},{"text":1702,"config":2092},{"href":1704,"dataGaName":1705,"dataGaLocation":1903},{"text":1707,"config":2094},{"href":1709,"dataGaName":1710,"dataGaLocation":1903},{"text":1712,"config":2096},{"href":1714,"dataGaName":1715,"dataGaLocation":1903},{"title":1728,"links":2098},[2099,2101,2103,2105,2107,2109,2113,2118,2120,2122,2124],{"text":1736,"config":2100},{"href":1738,"dataGaName":1730,"dataGaLocation":1903},{"text":1741,"config":2102},{"href":1743,"dataGaName":1744,"dataGaLocation":1903},{"text":1749,"config":2104},{"href":1751,"dataGaName":1752,"dataGaLocation":1903},{"text":1754,"config":2106},{"href":1756,"dataGaName":1757,"dataGaLocation":1903},{"text":1759,"config":2108},{"href":1761,"dataGaName":1762,"dataGaLocation":1903},{"text":2110,"config":2111},"Sustainability",{"href":2112,"dataGaName":2110,"dataGaLocation":1903},"/sustainability/",{"text":2114,"config":2115},"Diversity, inclusion and belonging (DIB)",{"href":2116,"dataGaName":2117,"dataGaLocation":1903},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":1764,"config":2119},{"href":1766,"dataGaName":1767,"dataGaLocation":1903},{"text":1774,"config":2121},{"href":1776,"dataGaName":1777,"dataGaLocation":1903},{"text":1779,"config":2123},{"href":1781,"dataGaName":1782,"dataGaLocation":1903},{"text":2125,"config":2126},"Modern Slavery Transparency Statement",{"href":2127,"dataGaName":2128,"dataGaLocation":1903},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":2130},[2131,2134,2137],{"text":2132,"config":2133},"Terms",{"href":1955,"dataGaName":1956,"dataGaLocation":1903},{"text":2135,"config":2136},"Cookies",{"dataGaName":1965,"dataGaLocation":1903,"id":1966,"isOneTrustButton":1445},{"text":2138,"config":2139},"Privacy",{"href":1960,"dataGaName":1961,"dataGaLocation":1903},[2141],{"id":2142,"title":7,"body":1441,"config":2143,"content":2145,"description":1441,"extension":2149,"meta":2150,"navigation":1445,"path":2151,"seo":2152,"stem":2153,"__hash__":2154},"blogAuthors/en-us/blog/authors/mayra-cabrera.yml",{"template":2144},"BlogAuthor",{"name":7,"config":2146},{"headshot":2147,"ctfId":2148},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749663224/Blog/Author%20Headshots/mayra-cabrera-headshot.jpg","mayracabrera","yml",{},"/en-us/blog/authors/mayra-cabrera",{},"en-us/blog/authors/mayra-cabrera","GCqp-lemVuilgAOb64pels-pdypKfj3yN8pWtqP96o8",[2156,2165,2173],{"title":2157,"description":2158,"heroImage":2159,"category":1437,"date":2160,"authors":2161,"slug":2164,"externalUrl":1441},"Confidential AI for GitLab Self-Hosted","Give developers AI coding agents in GitLab Duo without source code leaving a hardware-encrypted boundary — no GPUs needed.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1773866173/vte9qh8rriznvyclhkes.png","2026-08-06",[2162,2163],"Mathias Ewald","Martin Paloncy, Edgeless Systems","confidential-ai-for-gitlab-self-hosted",{"title":2166,"description":2167,"heroImage":2168,"category":1437,"date":2169,"authors":2170,"slug":2172,"externalUrl":1441},"Green DevOps: Why carbon measurement belongs in your CI/CD pipeline","CI/CD pipelines have a hidden carbon cost. Here's why measuring it matters, and how you can get started with Eco CI and Carmen in GitLab.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1765809212/noh0mdfn9o94ry9ykura.png","2026-07-09",[2171],"Lysanne Pinto","green-devops-carbon-measurement-cicd-pipeline",{"title":2174,"description":2175,"heroImage":2176,"category":1437,"date":2177,"authors":2178,"slug":2180,"externalUrl":1441},"How to build CI/CD observability at scale","This practical guide to GitLab pipeline analytics helps self-managed users gain operational insights using Prometheus and Grafana.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-04-28",[2179],"Paul Meresanu","how-to-build-ci-cd-observability-at-scale",{"promotions":2182},[2183,2197,2209,2221],{"id":2184,"categories":2185,"header":2187,"text":2188,"button":2189,"image":2194},"ai-modernization",[2186],"ai","Is AI achieving its promise at scale?","Quiz will take 5 minutes or less",{"text":2190,"config":2191},"Get your AI maturity score",{"href":2192,"dataGaName":2193,"dataGaLocation":1668},"/assessments/ai-modernization-assessment/","modernization assessment",{"config":2195},{"src":2196},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/qix0m7kwnd8x2fh1zq49.png",{"id":2198,"categories":2199,"header":2201,"text":2188,"button":2202,"image":2206},"devops-modernization",[2200,2005],"product","Are you just managing tools or shipping innovation?",{"text":2203,"config":2204},"Get your DevOps maturity score",{"href":2205,"dataGaName":2193,"dataGaLocation":1668},"/assessments/devops-modernization-assessment/",{"config":2207},{"src":2208},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138785/eg818fmakweyuznttgid.png",{"id":2210,"categories":2211,"header":2213,"text":2188,"button":2214,"image":2218},"security-modernization",[2212],"security","Are you trading speed for security?",{"text":2215,"config":2216},"Get your security maturity score",{"href":2217,"dataGaName":2193,"dataGaLocation":1668},"/assessments/security-modernization-assessment/",{"config":2219},{"src":2220},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/p4pbqd9nnjejg5ds6mdk.png",{"id":2222,"paths":2223,"header":2226,"text":2227,"button":2228,"image":2233},"github-azure-migration",[2224,2225],"migration-from-azure-devops-to-gitlab","integrating-azure-devops-scm-and-gitlab","Is your team ready for GitHub's Azure move?","GitHub is already rebuilding around Azure. Find out what it means for you.",{"text":2229,"config":2230},"See how GitLab compares to GitHub",{"href":2231,"dataGaName":2232,"dataGaLocation":1668},"/compare/gitlab-vs-github/github-azure-migration/","github azure migration",{"config":2234},{"src":2208},{"header":2236,"blurb":2237,"button":2238,"secondaryButton":2243},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":2239,"config":2240},"Get your free trial",{"href":2241,"dataGaName":1467,"dataGaLocation":2242},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":1941,"config":2244},{"href":1793,"dataGaName":1472,"dataGaLocation":2242},1786803746337]