[{"data":1,"prerenderedAt":1163},["ShallowReactive",2],{"/blog/machine-learning-on-the-gitlab-devops-platform":3,"navigation-en-us":377,"banner-en-us":804,"footer-en-us":814,"blog-post-authors-en-us-William Arias":1058,"blog-related-posts-en-us-machine-learning-on-the-gitlab-devops-platform":1073,"blog-promotions-en-us":1099,"next-steps-en-us":1153},{"id":4,"title":5,"authors":6,"body":8,"category":354,"date":355,"description":356,"extension":357,"externalUrl":358,"faq":358,"featured":359,"heroImage":360,"meta":361,"navigation":362,"path":363,"seo":364,"slug":368,"stem":369,"tags":370,"template":375,"updatedDate":358,"__hash__":376},"blogPosts/en-us/blog/machine-learning-on-the-gitlab-devops-platform.md","How Comet can streamline machine learning on The GitLab DevOps Platform",[7],"William Arias",{"type":9,"value":10,"toc":340},"minimark",[11,15,24,27,30,44,49,52,55,58,61,65,68,71,74,78,93,100,107,112,115,118,123,128,131,137,140,144,149,157,162,165,170,175,180,183,186,191,194,199,201,204,213,216,219,224,227,231,234,249,259,265,270,273,278,281,286,290,293,304,307,310,313,326,333],[12,13,14],"p",{},"Building machine learning-powered applications comes with numerous challenges. When we talk about these challenges, there is a tendency to overly focus on problems related to the quality of a model’s predictions—things like data drift, changes in model architectures, or inference latency.",[12,16,17,18,23],{},"While these are all problems worthy of deep consideration, an often overlooked challenge in ",[19,20,22],"a",{"href":21},"/topics/devops/the-role-of-ai-in-devops/","ML development"," is the process of integrating a model into an existing software application.",[12,25,26],{},"If you’re tasked with adding an ML feature to a product, you will almost certainly run into an existing codebase that must play nicely with your model. This is, to put it mildly, not an easy task.",[12,28,29],{},"ML is a highly iterative discipline. Teams often make many changes to their codebase and pipelines in the process of developing a model. Coupling an ML codebase to an application’s dependencies, unit tests, and CI/CD pipelines will significantly reduce the velocity with which ML teams can deliver on a solution, since each change would require running these downstream dependencies before a merge can be approved.",[12,31,32,33,38,39,43],{},"In this post, we’re going to demonstrate how you can use ",[19,34,37],{"href":35,"rel":36},"https://www.comet.ml/site/",[],"Comet"," with ",[19,40,42],{"href":41},"/solutions/devops-platform/","GitLab’s DevOps platform"," to streamline the workflow for your ML and software engineering teams, allowing them to collaborate without getting in each other's way.",[45,46,48],"h2",{"id":47},"the-challenge-for-ml-teams-working-with-application-teams","The challenge for ML teams working with application teams",[12,50,51],{},"Let’s say your team is working on improving a feature engineering pipeline. You will likely have to test many combinations of features with some baseline model for the task to see which combinations make an impact on model performance.\nIt is hard to know beforehand which features might be significant, so having to run multiple experiments is inevitable. If your ML code is a part of your application codebase, this would mean having to run your application’s CI/CD pipeline for every feature combination you might be trying.",[12,53,54],{},"This will certainly frustrate your Engineering and DevOps teams, since you would be unnecessarily tying up system resources, given that software engineering teams do not need to run their pipelines with the same frequency as ML teams do.",[12,56,57],{},"The other issue is that despite having to run numerous experiments, only a single set of outputs from these experiments will make it to your production application. Therefore, the rest of the assets produced through these experiments are not relevant to your application code.",[12,59,60],{},"Keeping these two codebases separated will make life a lot easier for everyone – but it also introduces the problem of syncing the latest model between two codebases.",[45,62,64],{"id":63},"use-the-gitlab-devops-platform-and-comet-for-your-model-development-process","Use The GitLab DevOps Platform and Comet for your model development process",[12,66,67],{},"With The GitLab DevOps platform and Comet, we can keep the workflows between ML and engineering teams separated, while enabling cross-team collaboration by preserving the visibility and auditability of the entire model development process across teams.",[12,69,70],{},"We will use two separate projects to demonstrate this process. One project will contain our application code for a handwritten digit recognizer, while the other will contain all the code relevant to training and evaluating our model.",[12,72,73],{},"We will adopt a process where discussions, code reviews, and model performance metrics get automatically published and tracked within The GitLab DevOps Platform, increasing the velocity and opportunity for collaboration between data scientists and software engineers for machine learning workflows.",[45,75,77],{"id":76},"project-setup","Project setup",[12,79,80,81,86,87,92],{},"Our project consists of two projects: ",[19,82,85],{"href":83,"rel":84},"https://gitlab.com/tech-marketing/devops-platform/comet-model-trainer",[],"comet-model-trainer"," and ",[19,88,91],{"href":89,"rel":90},"https://gitlab.com/tech-marketing/devops-platform/canara-review-apps-testing",[],"ml-ui",".",[12,94,95],{},[96,97],"img",{"alt":98,"src":99},"Alt text for your image","https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398089/blog/Content%20Images/cometmodeltrainer.png",[12,101,102,103,106],{},"The ",[104,105,85],"strong",{}," repository contains scripts to train and evaluate a model on the MNIST dataset. We have set up The GitLab DevOps Platform in a way that runs the training and evaluation Pipeline whenever a new merge request is opened with the necessary changes.",[12,108,102,109,111],{},[104,110,91],{}," repository contains the necessary code to build the frontend of our ML application.",[12,113,114],{},"Since the code is integrated with Comet, your ML team can easily track the source code, hyperparameters, metrics, and other details related to the development of the model.",[12,116,117],{},"Once the training and evaluation steps are completed, we can use Comet to fetch summary metrics from the project as well as metrics from the Candidate model and display them within the merge request; This will allow the ML team to easily review the changes to the model.",[12,119,120],{},[96,121],{"alt":98,"src":122},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398024/blog/Content%20Images/buildmodelgraph.png",[12,124,125],{},[96,126],{"alt":98,"src":127},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782399060/blog/Content%20Images/summarymetrics.png",[12,129,130],{},"In our case, the average accuracy of the models in the project is 97%. Our Candidate model achieved an accuracy of 99%, so it looks like it is a good fit to promote to production. The metrics displayed here are completely configurable and can be changed as necessary.",[12,132,133,134],{},"When the merge request is approved, the deployment pipeline is triggered and the model is pushed to Comet’s Model Registry. The Model Registry versions each model and links it back to the Comet Experiment that produced it.\n",[96,135],{"alt":98,"src":136},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782397907/blog/Content%20Images/OpenComet_SparkVideo.gif",[12,138,139],{},"Once the model is pushed to the Model Registry, it is available to the application code. When the application team wishes to deploy this new version of the model to their app, they simply have to trigger their specific deployment pipeline.",[45,141,143],{"id":142},"running-the-pipeline","Running the pipeline",[145,146,148],"h3",{"id":147},"pipeline-outline","Pipeline outline",[12,150,151,152,156],{},"We will run the process outlined below every time a team member creates a merge request to change code in the ",[153,154,155],"code",{},"build-neural-network","script:",[12,158,159],{},[96,160],{"alt":98,"src":161},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398716/blog/Content%20Images/modelapprove.png",[12,163,164],{},"Now, let’s take a look at the yaml config used to define our CI/CD pipelines depicted in the previous diagram:",[12,166,167],{},[96,168],{"alt":98,"src":169},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782399330/blog/Content%20Images/workflowsbranch.png",[12,171,172],{},[96,173],{"alt":98,"src":174},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398977/blog/Content%20Images/script.png",[12,176,177],{},[96,178],{"alt":98,"src":179},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398914/blog/Content%20Images/registermodel.png",[12,181,182],{},"Let's break down the CI/CD pipeline by describing the gitlab-ci.yml file so you can use it and customize it to your needs.",[12,184,185],{},"We start by instructing our GitLab runners to utilize Python:3.8 to run the jobs specified in the pipeline:",[12,187,188],{},[153,189,190],{},"Image: python:3.8",[12,192,193],{},"Then, we define the job where we want to build and train the neural network:",[12,195,196],{},[153,197,198],{},"Build-neural-network",[145,200,198],{"id":155},[12,202,203],{},"In this step, we start by creating a folder where we will store the artifacts generated by this job, install dependencies using the requirements.txt file, and finally  execute the corresponding Python script that will be in charge of training the neural network. The training runs in the GitLab runner using the Python image defined above, along with its dependencies.",[12,205,206,207,209,210],{},"Once the ",[153,208,155],{}," job has finalized successfully, we move to the next job: ",[153,211,212],{},"write-report-mr",[12,214,215],{},"Here, we use another image created by DVC that will allow us to publish a report right in the merge request opened by the contributor who changed code in the neural network script. In this way, we’ve brought software development workflows to the development of ML applications. With the report provided by this job, code and model review can be executed within the merge request view, enabling teams to collaborate not only around the code but also the model performance.",[12,217,218],{},"From the merge request page, we get access to loss curves and other relevant performance metrics from the model we are training, along with a link to the Comet Experiment UI, where richer details are provided to evaluate the model performance. These details include interactive charts for model metrics, the model hyperparameters, and Confusion Matrices of the test set performance, to name a few.",[12,220,221],{},[96,222],{"alt":98,"src":223},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398646/blog/Content%20Images/manualDeploy_SparkVideo.gif",[12,225,226],{},"When the team is done with the code and model review,  the merge request gets approved, and the script that generated the model is merged into the main codebase, along with its respective commit and the CI pipeline associated to it. This takes us to the next job:",[145,228,230],{"id":229},"register-model","Register-model",[12,232,233],{},"This job uses an integration between GitLab and Comet to upload the reviewed and accepted version of the model to the Comet Model Registry. If you recall, the Model Registry is where models intended for production can be logged and versioned. In order to run the commands that will register the model, we need to set up these variables:",[235,236,237,241],"ul",{},[238,239,240],"li",{},"COMET_WORKSPACE",[238,242,243,244,92],{},"COMET_PROJECT_NAME\nIn order to do that, follow the steps described ",[19,245,248],{"href":246,"rel":247},"https://docs.gitlab.com/ci/variables/#add-a-cicd-variable-to-an-instance",[],"here",[12,250,251,252,254,255,258],{},"It is worth noting that the ",[153,253,229],{}," job only runs when the merge request gets reviewed and approved, and this behavior is obtained by setting ",[153,256,257],{},"only: main"," at the end of the job.",[12,260,261,262],{},"Finally, we decide to let a team member have final control of the deployment so therefore we define a manual job:\n",[153,263,264],{},"Deploy-ml-ui",[12,266,267],{},[96,268],{"alt":98,"src":269},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398173/blog/Content%20Images/deployuiml.png",[12,271,272],{},"When triggered, this job will import the model from Comet’s Model Registry and automatically create the necessary containers to build the user interface and deploy to a Kubernetes cluster.",[12,274,275],{},[96,276],{"alt":98,"src":277},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398203/blog/Content%20Images/downstream.png",[12,279,280],{},"This job triggers a downstream pipeline, which means that the UI for this MNIST application resides in a different project. This keeps the codebase for the UI and model training separated but integrated and connected at the moment of deploying the model to a production environment.",[12,282,283],{},[96,284],{"alt":98,"src":285},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1782398751/blog/Content%20Images/multipipeline_SparkVideo.gif",[45,287,289],{"id":288},"key-takeaways","Key takeaways",[12,291,292],{},"In this post, we addressed some of the challenges faced by ML and software teams when it comes to collaborating on delivering ML-powered applications. Some of these challenges include:",[235,294,295,298,301],{},[238,296,297],{},"The discrepancy in the frequency with which each of these teams need to iterate on their codebases and CI/CD pipelines.",[238,299,300],{},"The fact that only a single set of experiment assets from an ML experimentation pipeline is relevant to the application.",[238,302,303],{},"The challenge of syncing a model or other experiment assets across independent codebases.",[12,305,306],{},"Using The GitLab DevOps Platform and Comet, we can start bridging the gap between ML and software engineering teams over the course of a project.",[12,308,309],{},"By having model performance metrics adopted into software development workflows like the one we saw in the issue and merge request, we can keep track of the code changes, discussions, experiments, and models created in the process. All the operations executed by the team are recorded, can be audited, are end-to end-traceable, and (most importantly) reproducible.",[12,311,312],{},"Watch a demo of this process:",[314,315,318,319],"figure",{"className":316},[317],"video_container","\n  ",[320,321,325],"iframe",{"src":322,"frameBorder":323,"allowFullScreen":324},"https://www.youtube.com/embed/W_DsNl5aAVk","0","true"," ",[12,327,328,332],{},[329,330,331],"em",{},"About Comet:","\nComet is an MLOps Platform that is designed to help data scientists and teams build better models faster! Comet provides tooling to Track, Explain, Manage, and Monitor your models in a single place!",[12,334,335,336,339],{},"Learn more about Comet ",[19,337,248],{"href":35,"rel":338},[]," and get started for free!",{"title":341,"searchDepth":342,"depth":342,"links":343},"",2,[344,345,346,347,353],{"id":47,"depth":342,"text":48},{"id":63,"depth":342,"text":64},{"id":76,"depth":342,"text":77},{"id":142,"depth":342,"text":143,"children":348},[349,351,352],{"id":147,"depth":350,"text":148},3,{"id":155,"depth":350,"text":198},{"id":229,"depth":350,"text":230},{"id":288,"depth":342,"text":289},"engineering","2021-11-08","Here's a step-by-step look at how to bring ML into software development using Comet on GitLab's DevOps Platform.","md",null,false,"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749669991/Blog/Hero%20Images/ways-to-encourage-collaboration.jpg",{},true,"/en-us/blog/machine-learning-on-the-gitlab-devops-platform",{"title":5,"description":356,"ogTitle":5,"ogDescription":356,"noIndex":359,"ogImage":360,"ogUrl":365,"ogSiteName":366,"ogType":367,"canonicalUrls":365},"https://about.gitlab.com/blog/machine-learning-on-the-gitlab-devops-platform","https://about.gitlab.com","article","machine-learning-on-the-gitlab-devops-platform","en-us/blog/machine-learning-on-the-gitlab-devops-platform",[371,372,373,374],"DevOps","demo","integrations","AI","BlogPost","uX1Nj011LzXTg0SPdSsB7YkymhP5QrEFvNne9Q23YM0",{"logo":378,"freeTrial":383,"sales":388,"login":393,"items":398,"search":724,"minimal":755,"duo":774,"switchNav":783,"pricingDeployment":794},{"config":379},{"href":380,"dataGaName":381,"dataGaLocation":382},"/","gitlab logo","header",{"text":384,"config":385},"Get free trial",{"href":386,"dataGaName":387,"dataGaLocation":382},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":389,"config":390},"Request a demo",{"href":391,"dataGaName":392,"dataGaLocation":382},"/sales/?contact-topic=request-demo","sales",{"text":394,"config":395},"Sign in",{"href":396,"dataGaName":397,"dataGaLocation":382},"https://gitlab.com/users/sign_in/","sign in",[399,428,528,533,646,702],{"text":400,"config":401,"menu":403},"Platform",{"dataNavLevelOne":402},"platform",{"type":404,"columns":405},"cards",[406,412,420],{"title":400,"description":407,"link":408},"The intelligent orchestration platform for DevSecOps",{"text":409,"config":410},"Explore our Platform",{"href":411,"dataGaName":402,"dataGaLocation":382},"/platform/",{"title":413,"description":414,"link":415},"GitLab Duo Agent Platform","Agentic AI for the entire software lifecycle",{"text":416,"config":417},"Meet GitLab Duo",{"href":418,"dataGaName":419,"dataGaLocation":382},"/gitlab-duo-agent-platform/","gitlab duo agent platform",{"title":421,"description":422,"link":423},"Why GitLab","See the top reasons enterprises choose GitLab",{"text":424,"config":425},"Learn more",{"href":426,"dataGaName":427,"dataGaLocation":382},"/why-gitlab/","why gitlab",{"text":429,"left":362,"config":430,"menu":432},"Product",{"dataNavLevelOne":431},"solutions",{"type":433,"link":434,"columns":438,"feature":507},"lists",{"text":435,"config":436},"View all Solutions",{"href":437,"dataGaName":431,"dataGaLocation":382},"/solutions/",[439,463,486],{"title":440,"description":441,"link":442,"items":447},"Automation","CI/CD and automation to accelerate deployment",{"config":443},{"icon":444,"href":445,"dataGaName":446,"dataGaLocation":382},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[448,452,455,459],{"text":449,"config":450},"CI/CD",{"href":451,"dataGaLocation":382,"dataGaName":449},"/solutions/continuous-integration/",{"text":413,"config":453},{"href":418,"dataGaLocation":382,"dataGaName":454},"gitlab duo agent platform - product menu",{"text":456,"config":457},"Source Code Management",{"href":458,"dataGaLocation":382,"dataGaName":456},"/solutions/source-code-management/",{"text":460,"config":461},"Automated Software Delivery",{"href":445,"dataGaLocation":382,"dataGaName":462},"Automated software delivery",{"title":464,"description":465,"link":466,"items":471},"Security","Deliver code faster without compromising security",{"config":467},{"href":468,"dataGaName":469,"dataGaLocation":382,"icon":470},"/solutions/application-security-testing/","security and compliance","ShieldCheckLight",[472,476,481],{"text":473,"config":474},"Application Security Testing",{"href":468,"dataGaName":475,"dataGaLocation":382},"Application security testing",{"text":477,"config":478},"Software Supply Chain Security",{"href":479,"dataGaLocation":382,"dataGaName":480},"/solutions/supply-chain/","Software supply chain security",{"text":482,"config":483},"Software Compliance",{"href":484,"dataGaName":485,"dataGaLocation":382},"/solutions/software-compliance/","software compliance",{"title":487,"link":488,"items":493},"Measurement",{"config":489},{"icon":490,"href":491,"dataGaName":492,"dataGaLocation":382},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[494,498,502],{"text":495,"config":496},"Visibility & Measurement",{"href":491,"dataGaLocation":382,"dataGaName":497},"Visibility and Measurement",{"text":499,"config":500},"Value Stream Management",{"href":501,"dataGaLocation":382,"dataGaName":499},"/solutions/value-stream-management/",{"text":503,"config":504},"Analytics & Insights",{"href":505,"dataGaLocation":382,"dataGaName":506},"/solutions/analytics-and-insights/","Analytics and insights",{"title":508,"type":433,"items":509},"GitLab for",[510,516,522],{"text":511,"config":512},"Enterprise",{"icon":513,"href":514,"dataGaLocation":382,"dataGaName":515},"Building","/enterprise/","enterprise",{"text":517,"config":518},"Small Business",{"icon":519,"href":520,"dataGaLocation":382,"dataGaName":521},"Work","/small-business/","small business",{"text":523,"config":524},"Public Sector",{"icon":525,"href":526,"dataGaLocation":382,"dataGaName":527},"Organization","/solutions/public-sector/","public sector",{"text":529,"config":530},"Pricing",{"href":531,"dataGaName":532,"dataGaLocation":382,"dataNavLevelOne":532},"/pricing/","pricing",{"text":534,"config":535,"menu":537},"Resources",{"dataNavLevelOne":536},"resources",{"type":433,"link":538,"columns":542,"feature":635},{"text":539,"config":540},"View all resources",{"href":541,"dataGaName":536,"dataGaLocation":382},"/resources/",[543,575,602],{"title":544,"items":545},"Getting started",[546,551,556,561,566,571],{"text":547,"config":548},"Install",{"href":549,"dataGaName":550,"dataGaLocation":382},"/install/","install",{"text":552,"config":553},"Quick start guides",{"href":554,"dataGaName":555,"dataGaLocation":382},"/get-started/","quick setup checklists",{"text":557,"config":558},"Learn",{"href":559,"dataGaLocation":382,"dataGaName":560},"https://university.gitlab.com/","learn",{"text":562,"config":563},"Product documentation",{"href":564,"dataGaName":565,"dataGaLocation":382},"https://docs.gitlab.com/","product documentation",{"text":567,"config":568},"Best practice videos",{"href":569,"dataGaName":570,"dataGaLocation":382},"/getting-started-videos/","best practice videos",{"text":572,"config":573},"Integrations",{"href":574,"dataGaName":373,"dataGaLocation":382},"/integrations/",{"title":576,"items":577},"Discover",[578,583,588,593,597],{"text":579,"config":580},"Customer success stories",{"href":581,"dataGaName":582,"dataGaLocation":382},"/customers/","customer success stories",{"text":584,"config":585},"Blog",{"href":586,"dataGaName":587,"dataGaLocation":382},"/blog/","blog",{"text":589,"config":590},"Demo Hub",{"href":591,"dataGaName":592,"dataGaLocation":382},"/demo-hub/","demo hub",{"text":594,"config":595},"The Source",{"href":596,"dataGaName":587,"dataGaLocation":382},"/the-source/",{"text":598,"config":599},"Remote",{"href":600,"dataGaName":601,"dataGaLocation":382},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"title":603,"items":604},"Connect",[605,610,615,620,625,630],{"text":606,"config":607},"GitLab Services",{"href":608,"dataGaName":609,"dataGaLocation":382},"/services/","services",{"text":611,"config":612},"Contribute",{"href":613,"dataGaName":614,"dataGaLocation":382},"https://contributors.gitlab.com","contribute",{"text":616,"config":617},"Community",{"href":618,"dataGaName":619,"dataGaLocation":382},"/community/","community",{"text":621,"config":622},"Forum",{"href":623,"dataGaName":624,"dataGaLocation":382},"https://forum.gitlab.com/","forum",{"text":626,"config":627},"Events",{"href":628,"dataGaName":629,"dataGaLocation":382},"/events/","events",{"text":631,"config":632},"Partners",{"href":633,"dataGaName":634,"dataGaLocation":382},"/partners/","partners",{"config":636,"title":639,"text":640,"link":641},{"background":637,"textColor":638},"url('https://res.cloudinary.com/about-gitlab-com/image/upload/v1777322348/qpq8yrgn8knii57omj0c.png')","#000","What’s new in GitLab","Stay updated with our latest features and improvements.",{"text":642,"config":643},"Read the latest",{"href":644,"dataGaName":645,"dataGaLocation":382},"/whats-new/","whats new",{"text":647,"config":648,"menu":650},"Company",{"dataNavLevelOne":649},"company",{"type":433,"columns":651},[652],{"items":653},[654,659,665,667,672,677,682,687,692,697],{"text":655,"config":656},"About",{"href":657,"dataGaName":658,"dataGaLocation":382},"/company/","about",{"text":660,"config":661,"footerGa":664},"Jobs",{"href":662,"dataGaName":663,"dataGaLocation":382},"/jobs/","jobs",{"dataGaName":663},{"text":626,"config":666},{"href":628,"dataGaName":629,"dataGaLocation":382},{"text":668,"config":669},"Leadership",{"href":670,"dataGaName":671,"dataGaLocation":382},"/company/team/e-group/","leadership",{"text":673,"config":674},"Handbook",{"href":675,"dataGaName":676,"dataGaLocation":382},"https://handbook.gitlab.com/","handbook",{"text":678,"config":679},"Investor relations",{"href":680,"dataGaName":681,"dataGaLocation":382},"https://ir.gitlab.com/overview/default.aspx","investor relations",{"text":683,"config":684},"Trust Center",{"href":685,"dataGaName":686,"dataGaLocation":382},"/security/","trust center",{"text":688,"config":689},"AI Transparency Center",{"href":690,"dataGaName":691,"dataGaLocation":382},"/ai-transparency-center/","ai transparency center",{"text":693,"config":694},"Newsletter",{"href":695,"dataGaName":696,"dataGaLocation":382},"/company/contact/#contact-forms","newsletter",{"text":698,"config":699},"Press",{"href":700,"dataGaName":701,"dataGaLocation":382},"/press/","press",{"text":703,"config":704,"menu":705},"Contact us",{"dataNavLevelOne":649},{"type":433,"columns":706},[707],{"items":708},[709,714,719],{"text":710,"config":711},"Talk to sales",{"href":712,"dataGaName":713,"dataGaLocation":382},"/sales/","talk to sales",{"text":715,"config":716},"Support portal",{"href":717,"dataGaName":718,"dataGaLocation":382},"https://support.gitlab.com/hc/en-us","support portal",{"text":720,"config":721},"Customer portal",{"href":722,"dataGaName":723,"dataGaLocation":382},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":725,"login":726,"suggestions":733},"Close",{"text":727,"link":728},"To search repositories and projects, login to",{"text":729,"config":730},"gitlab.com",{"href":396,"dataGaName":731,"dataGaLocation":732},"search login","search",{"text":734,"default":735},"Suggestions",[736,738,742,744,748,752],{"text":413,"config":737},{"href":418,"dataGaName":413,"dataGaLocation":732},{"text":739,"config":740},"Code Suggestions (AI)",{"href":741,"dataGaName":739,"dataGaLocation":732},"/solutions/code-suggestions/",{"text":449,"config":743},{"href":451,"dataGaName":449,"dataGaLocation":732},{"text":745,"config":746},"GitLab on AWS",{"href":747,"dataGaName":745,"dataGaLocation":732},"/partners/technology-partners/aws/",{"text":749,"config":750},"GitLab on Google Cloud",{"href":751,"dataGaName":749,"dataGaLocation":732},"/partners/technology-partners/google-cloud-platform/",{"text":753,"config":754},"Why GitLab?",{"href":426,"dataGaName":753,"dataGaLocation":732},{"freeTrial":756,"mobileIcon":761,"desktopIcon":766,"secondaryButton":769},{"text":757,"config":758},"Start free trial",{"href":759,"dataGaName":387,"dataGaLocation":760},"https://gitlab.com/-/trials/new/","nav",{"altText":762,"config":763},"Gitlab Icon",{"src":764,"dataGaName":765,"dataGaLocation":760},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203874/jypbw1jx72aexsoohd7x.svg","gitlab icon",{"altText":762,"config":767},{"src":768,"dataGaName":765,"dataGaLocation":760},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1758203875/gs4c8p8opsgvflgkswz9.svg",{"text":770,"config":771},"Get Started",{"href":772,"dataGaName":773,"dataGaLocation":760},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/get-started/","get started",{"freeTrial":775,"mobileIcon":779,"desktopIcon":781},{"text":776,"config":777},"Learn more about GitLab Duo",{"href":418,"dataGaName":778,"dataGaLocation":760},"gitlab duo",{"altText":762,"config":780},{"src":764,"dataGaName":765,"dataGaLocation":760},{"altText":762,"config":782},{"src":768,"dataGaName":765,"dataGaLocation":760},{"button":784,"mobileIcon":789,"desktopIcon":791},{"text":785,"config":786},"/switch",{"href":787,"dataGaName":788,"dataGaLocation":760},"#contact","switch",{"altText":762,"config":790},{"src":764,"dataGaName":765,"dataGaLocation":760},{"altText":762,"config":792},{"src":793,"dataGaName":765,"dataGaLocation":760},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1773335277/ohhpiuoxoldryzrnhfrh.png",{"freeTrial":795,"mobileIcon":800,"desktopIcon":802},{"text":796,"config":797},"Back to pricing",{"href":531,"dataGaName":798,"dataGaLocation":760,"icon":799},"back to pricing","GoBack",{"altText":762,"config":801},{"src":764,"dataGaName":765,"dataGaLocation":760},{"altText":762,"config":803},{"src":768,"dataGaName":765,"dataGaLocation":760},{"title":805,"titleMobile":806,"button":807,"config":812},"Duo Agent Platform delivers 400% ROI, per new Forrester Consulting study.","400% ROI: Forrester TEI for GitLab Duo",{"text":424,"config":808},{"href":809,"dataGaName":810,"dataGaLocation":811},"https://about.gitlab.com/blog/gitlab-duo-agent-platform-delivers-400-percent-roi/","forrester-tei-dap-banner","global-banner",{"layout":813,"disabled":359},"release",{"data":815},{"text":816,"source":817,"edit":823,"contribute":828,"config":833,"items":838,"minimal":1047},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":818,"config":819},"View page source",{"href":820,"dataGaName":821,"dataGaLocation":822},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":824,"config":825},"Edit this page",{"href":826,"dataGaName":827,"dataGaLocation":822},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":829,"config":830},"Please contribute",{"href":831,"dataGaName":832,"dataGaLocation":822},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":834,"facebook":835,"youtube":836,"linkedin":837},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[839,886,939,983,1015],{"title":529,"links":840,"subMenu":855},[841,845,850],{"text":842,"config":843},"View plans",{"href":531,"dataGaName":844,"dataGaLocation":822},"view plans",{"text":846,"config":847},"Why Premium?",{"href":848,"dataGaName":849,"dataGaLocation":822},"/pricing/premium/","why premium",{"text":851,"config":852},"Why Ultimate?",{"href":853,"dataGaName":854,"dataGaLocation":822},"/pricing/ultimate/","why ultimate",[856],{"title":857,"links":858},"Contact Us",[859,862,864,866,871,876,881],{"text":860,"config":861},"Contact sales",{"href":712,"dataGaName":392,"dataGaLocation":822},{"text":715,"config":863},{"href":717,"dataGaName":718,"dataGaLocation":822},{"text":720,"config":865},{"href":722,"dataGaName":723,"dataGaLocation":822},{"text":867,"config":868},"Status",{"href":869,"dataGaName":870,"dataGaLocation":822},"https://status.gitlab.com/","status",{"text":872,"config":873},"Terms of use",{"href":874,"dataGaName":875,"dataGaLocation":822},"/terms/","terms of use",{"text":877,"config":878},"Privacy statement",{"href":879,"dataGaName":880,"dataGaLocation":822},"/privacy/","privacy statement",{"text":882,"config":883},"Cookie preferences",{"dataGaName":884,"dataGaLocation":822,"id":885,"isOneTrustButton":362},"cookie preferences","ot-sdk-btn",{"title":429,"links":887,"subMenu":896},[888,892],{"text":889,"config":890},"DevSecOps platform",{"href":411,"dataGaName":891,"dataGaLocation":822},"devsecops platform",{"text":893,"config":894},"AI-Assisted Development",{"href":418,"dataGaName":895,"dataGaLocation":822},"ai-assisted development",[897],{"title":898,"links":899},"Topics",[900,905,910,914,919,924,929,934],{"text":901,"config":902},"CICD",{"href":903,"dataGaName":904,"dataGaLocation":822},"/topics/ci-cd/","cicd",{"text":906,"config":907},"GitOps",{"href":908,"dataGaName":909,"dataGaLocation":822},"/topics/gitops/","gitops",{"text":371,"config":911},{"href":912,"dataGaName":913,"dataGaLocation":822},"/topics/devops/","devops",{"text":915,"config":916},"Version Control",{"href":917,"dataGaName":918,"dataGaLocation":822},"/topics/version-control/","version control",{"text":920,"config":921},"DevSecOps",{"href":922,"dataGaName":923,"dataGaLocation":822},"/topics/devsecops/","devsecops",{"text":925,"config":926},"Cloud Native",{"href":927,"dataGaName":928,"dataGaLocation":822},"/topics/cloud-native/","cloud native",{"text":930,"config":931},"AI for Coding",{"href":932,"dataGaName":933,"dataGaLocation":822},"/topics/devops/ai-for-coding/","ai for coding",{"text":935,"config":936},"Agentic AI",{"href":937,"dataGaName":938,"dataGaLocation":822},"/topics/agentic-ai/","agentic ai",{"title":940,"links":941},"Solutions",[942,944,946,951,955,958,962,965,967,970,973,978],{"text":473,"config":943},{"href":468,"dataGaName":473,"dataGaLocation":822},{"text":462,"config":945},{"href":445,"dataGaName":446,"dataGaLocation":822},{"text":947,"config":948},"Agile development",{"href":949,"dataGaName":950,"dataGaLocation":822},"/solutions/agile-delivery/","agile delivery",{"text":952,"config":953},"SCM",{"href":458,"dataGaName":954,"dataGaLocation":822},"source code management",{"text":901,"config":956},{"href":451,"dataGaName":957,"dataGaLocation":822},"continuous integration & delivery",{"text":959,"config":960},"Value stream management",{"href":501,"dataGaName":961,"dataGaLocation":822},"value stream management",{"text":906,"config":963},{"href":964,"dataGaName":909,"dataGaLocation":822},"/solutions/gitops/",{"text":511,"config":966},{"href":514,"dataGaName":515,"dataGaLocation":822},{"text":968,"config":969},"Small business",{"href":520,"dataGaName":521,"dataGaLocation":822},{"text":971,"config":972},"Public sector",{"href":526,"dataGaName":527,"dataGaLocation":822},{"text":974,"config":975},"Education",{"href":976,"dataGaName":977,"dataGaLocation":822},"/solutions/education/","education",{"text":979,"config":980},"Financial services",{"href":981,"dataGaName":982,"dataGaLocation":822},"/solutions/finance/","financial services",{"title":534,"links":984},[985,987,989,991,994,996,999,1001,1003,1005,1007,1009,1011,1013],{"text":547,"config":986},{"href":549,"dataGaName":550,"dataGaLocation":822},{"text":552,"config":988},{"href":554,"dataGaName":555,"dataGaLocation":822},{"text":557,"config":990},{"href":559,"dataGaName":560,"dataGaLocation":822},{"text":562,"config":992},{"href":564,"dataGaName":993,"dataGaLocation":822},"docs",{"text":584,"config":995},{"href":586,"dataGaName":587,"dataGaLocation":822},{"text":997,"config":998},"What's new",{"href":644,"dataGaName":645,"dataGaLocation":822},{"text":579,"config":1000},{"href":581,"dataGaName":582,"dataGaLocation":822},{"text":598,"config":1002},{"href":600,"dataGaName":601,"dataGaLocation":822},{"text":606,"config":1004},{"href":608,"dataGaName":609,"dataGaLocation":822},{"text":611,"config":1006},{"href":613,"dataGaName":614,"dataGaLocation":822},{"text":616,"config":1008},{"href":618,"dataGaName":619,"dataGaLocation":822},{"text":621,"config":1010},{"href":623,"dataGaName":624,"dataGaLocation":822},{"text":626,"config":1012},{"href":628,"dataGaName":629,"dataGaLocation":822},{"text":631,"config":1014},{"href":633,"dataGaName":634,"dataGaLocation":822},{"title":647,"links":1016},[1017,1019,1021,1023,1025,1027,1031,1036,1038,1040,1042],{"text":655,"config":1018},{"href":657,"dataGaName":649,"dataGaLocation":822},{"text":660,"config":1020},{"href":662,"dataGaName":663,"dataGaLocation":822},{"text":668,"config":1022},{"href":670,"dataGaName":671,"dataGaLocation":822},{"text":673,"config":1024},{"href":675,"dataGaName":676,"dataGaLocation":822},{"text":678,"config":1026},{"href":680,"dataGaName":681,"dataGaLocation":822},{"text":1028,"config":1029},"Sustainability",{"href":1030,"dataGaName":1028,"dataGaLocation":822},"/sustainability/",{"text":1032,"config":1033},"Diversity, inclusion and belonging (DIB)",{"href":1034,"dataGaName":1035,"dataGaLocation":822},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":683,"config":1037},{"href":685,"dataGaName":686,"dataGaLocation":822},{"text":693,"config":1039},{"href":695,"dataGaName":696,"dataGaLocation":822},{"text":698,"config":1041},{"href":700,"dataGaName":701,"dataGaLocation":822},{"text":1043,"config":1044},"Modern Slavery Transparency Statement",{"href":1045,"dataGaName":1046,"dataGaLocation":822},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"items":1048},[1049,1052,1055],{"text":1050,"config":1051},"Terms",{"href":874,"dataGaName":875,"dataGaLocation":822},{"text":1053,"config":1054},"Cookies",{"dataGaName":884,"dataGaLocation":822,"id":885,"isOneTrustButton":362},{"text":1056,"config":1057},"Privacy",{"href":879,"dataGaName":880,"dataGaLocation":822},[1059],{"id":1060,"title":7,"body":358,"config":1061,"content":1063,"description":358,"extension":1067,"meta":1068,"navigation":362,"path":1069,"seo":1070,"stem":1071,"__hash__":1072},"blogAuthors/en-us/blog/authors/william-arias.yml",{"template":1062},"BlogAuthor",{"name":7,"config":1064},{"headshot":1065,"ctfId":1066},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749667549/Blog/Author%20Headshots/warias-headshot.jpg","warias","yml",{},"/en-us/blog/authors/william-arias",{},"en-us/blog/authors/william-arias","1h59SugLZ7hePm0SChSE5WG0Z3uurYxGrujcXoxs4tA",[1074,1083,1091],{"title":1075,"description":1076,"heroImage":1077,"category":354,"date":1078,"authors":1079,"slug":1082,"externalUrl":358},"Confidential AI for GitLab Self-Hosted","Give developers AI coding agents in GitLab Duo without source code leaving a hardware-encrypted boundary — no GPUs needed.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1773866173/vte9qh8rriznvyclhkes.png","2026-08-06",[1080,1081],"Mathias Ewald","Martin Paloncy, Edgeless Systems","confidential-ai-for-gitlab-self-hosted",{"title":1084,"description":1085,"heroImage":1086,"category":354,"date":1087,"authors":1088,"slug":1090,"externalUrl":358},"Green DevOps: Why carbon measurement belongs in your CI/CD pipeline","CI/CD pipelines have a hidden carbon cost. Here's why measuring it matters, and how you can get started with Eco CI and Carmen in GitLab.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1765809212/noh0mdfn9o94ry9ykura.png","2026-07-09",[1089],"Lysanne Pinto","green-devops-carbon-measurement-cicd-pipeline",{"title":1092,"description":1093,"heroImage":1094,"category":354,"date":1095,"authors":1096,"slug":1098,"externalUrl":358},"How to build CI/CD observability at scale","This practical guide to GitLab pipeline analytics helps self-managed users gain operational insights using Prometheus and Grafana.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1774465167/n5hlvrsrheadeccyr1oz.png","2026-04-28",[1097],"Paul Meresanu","how-to-build-ci-cd-observability-at-scale",{"promotions":1100},[1101,1115,1127,1139],{"id":1102,"categories":1103,"header":1105,"text":1106,"button":1107,"image":1112},"ai-modernization",[1104],"ai","Is AI achieving its promise at scale?","Quiz will take 5 minutes or less",{"text":1108,"config":1109},"Get your AI maturity score",{"href":1110,"dataGaName":1111,"dataGaLocation":587},"/assessments/ai-modernization-assessment/","modernization assessment",{"config":1113},{"src":1114},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/qix0m7kwnd8x2fh1zq49.png",{"id":1116,"categories":1117,"header":1119,"text":1106,"button":1120,"image":1124},"devops-modernization",[1118,923],"product","Are you just managing tools or shipping innovation?",{"text":1121,"config":1122},"Get your DevOps maturity score",{"href":1123,"dataGaName":1111,"dataGaLocation":587},"/assessments/devops-modernization-assessment/",{"config":1125},{"src":1126},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138785/eg818fmakweyuznttgid.png",{"id":1128,"categories":1129,"header":1131,"text":1106,"button":1132,"image":1136},"security-modernization",[1130],"security","Are you trading speed for security?",{"text":1133,"config":1134},"Get your security maturity score",{"href":1135,"dataGaName":1111,"dataGaLocation":587},"/assessments/security-modernization-assessment/",{"config":1137},{"src":1138},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1772138786/p4pbqd9nnjejg5ds6mdk.png",{"id":1140,"paths":1141,"header":1144,"text":1145,"button":1146,"image":1151},"github-azure-migration",[1142,1143],"migration-from-azure-devops-to-gitlab","integrating-azure-devops-scm-and-gitlab","Is your team ready for GitHub's Azure move?","GitHub is already rebuilding around Azure. Find out what it means for you.",{"text":1147,"config":1148},"See how GitLab compares to GitHub",{"href":1149,"dataGaName":1150,"dataGaLocation":587},"/compare/gitlab-vs-github/github-azure-migration/","github azure migration",{"config":1152},{"src":1126},{"header":1154,"blurb":1155,"button":1156,"secondaryButton":1161},"Start building faster today","See what your team can do with the intelligent orchestration platform for DevSecOps.\n",{"text":1157,"config":1158},"Get your free trial",{"href":1159,"dataGaName":387,"dataGaLocation":1160},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":860,"config":1162},{"href":712,"dataGaName":392,"dataGaLocation":1160},1786803772855]