Gitlab-CI: How do I run a job on a specific server?
I set up Gitlab and Gitlab-CI on a k8s cluster in AWS. I have jobs that use a lot of resources. I want to run these jobs on specific instances in AWS. How can this be done?
1 answer
-
answered 2019-07-15 10:14
Nicolas Pepinster
Kubernetes configuration
You need to add a node selector which enable you to assign pods on specific nodes
kubectl label nodes <node-name> gitlab=true
Gitlab Runner configuration
- Specify a tag associated to the runner. In your case, uncheck the Run untagged jobs option.
- Specify a node selector using the keyword
node_selector
:
[runners.kubernetes.node_selector] gitlab = "true"
Check a more complete example of
config.toml
on gitlab website.Gitlab CI configuration
Refer the tag of your runner in your
.gitlab-ci.yml
job: tags: - big_server
See also questions close to this topic
-
Gitlab Large Artifact Upload Fails - Timeout Awaiting Reponse Headers
I’ve been trying to upload ~2GB of artifacts from a pipeline in our self-hosted Gitlab instance, but am getting this error:
ERROR: Uploading artifacts to coordinator… error error=couldn’t execute POST against https://git.onerain.com/api/v4/jobs/25628/artifacts?artifact_format=zip&artifact_type=archive&expire_in=30+days: Post https://git.onerain.com/api/v4/jobs/25628/artifacts?artifact_format=zip&artifact_type=archive&expire_in=30+days: net/http: timeout awaiting response headers id=25628 token=t8k_zdxV WARNING: Retrying… error=invalid argument ERROR: Uploading artifacts to coordinator… error error=couldn’t execute POST against https://git.onerain.com/api/v4/jobs/25628/artifacts?artifact_format=zip&artifact_type=archive&expire_in=30+days: Post https://git.onerain.com/api/v4/jobs/25628/artifacts?artifact_format=zip&artifact_type=archive&expire_in=30+days: net/http: timeout awaiting response headers id=25628 token=t8k_zdxV WARNING: Retrying… error=invalid argument ERROR: Uploading artifacts to coordinator… error error=couldn’t execute POST against https://git.onerain.com/api/v4/jobs/25628/artifacts?artifact_format=zip&artifact_type=archive&expire_in=30+days: Post https://git.onerain.com/api/v4/jobs/25628/artifacts?artifact_format=zip&artifact_type=archive&expire_in=30+days: net/http: timeout awaiting response headers id=25628 token=t8k_zdxV FATAL: invalid argument ERROR: Job failed: exit code 1
Looking in /var/log/gitlab/gitlab-workhorse/current I see:
{“correlation_id”:“aRQJ7UtGjb”,“error”:“handleFileUploads: extract files from multipart: persisting multipart file: unexpected EOF”,“level”:“error”,“method”:“POST”,“msg”:“error”,“time”:“2019-12-04T23:34:56Z”,“uri”:"/api/v4/jobs/25628/artifacts?artifact_format=zip\u0026artifact_type=archive\u0026expire_in=30+days"}
I’ve seen that some other folks have had this problem, and I’ve tried many of the suggestions:
- increasing unicorn[‘worker_timeout’]
- Made sure artifact maximum size is large enough for instance/group/project
- Made sure max attachment size for instance large enough
- increase gitlab_workhorse[‘api_ci_long_polling_duration’] = “15m0s” and gitlab_workhorse[‘proxy_headers_timeout’] = “15m0s”
- increased nginx[‘proxy_read_timeout’] = 3600 and nginx[‘proxy_connect_timeout’] = 3600
- set nginx[‘client_max_body_size’] = 0
I’m on gitlab ce 12.5.2 and gitlab-runner 12.5.0. Our server has plenty of room in the artifacts directory. I’m about out of ideas. Anybody else have any?
Thanks!
-
External GitLab CE (source) Docker registry via Sonatype Nexus
I need help with setting up an integration of GitLab (12.4.3) with external docker registry.
This docker registry runs in Sonatype Nexus (3.18.1-01). I am quite dumbfounded as to how to achieve this integration. The docs mention a token based authentication. Unfortunately via Nexus I am not aware how to check what type of authentication is setup (for the docker registry) and I don’t see an option for the token based one.
Both Nexus and GitLab successfully use LDAP so I am guessing this could be somehow exposed as a common realm.
Has anyone anyhow integrated Nexus docker registry into GitLab? Is it even possible? My google-fu has failed me terribly.
-
Error loading key "(stdin)": invalid format Gitlab CI
This is piece of my yaml file about ssh for gitlab ci:
eval $(ssh-agent -s) echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null mkdir -p ~/.ssh chmod 700 ~/.ssh '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
When I attempted to set SSH_PRIVATE_KEY variable to protected state, it occurs error:
Error loading key "(stdin)": invalid format
and I don't know why it's okay when I set SSH_PRIVATE_KEY variable to unprotected state
-
Problems accessing Nextcloud webdav with gitlab runner
Using pythons webdav.client I connect to a Nextcloud.
import webdav.client as wc def webdav_con(webdav_hostname, webdav_usr, webdav_password): options = { 'webdav_hostname': webdav_hostname, 'webdav_login': webdav_usr, 'webdav_password': webdav_password, 'webdav_root': "/remote.php/webdav" } return client = wc.Client(options) cloud_client = webdav_con(hostname_webdav_nextcloud, usr_nextcloud, pw_nextcloud)
I then list the files in my path and process them afterwards.
files_in_cloud = cloud_client.list("path_in_cloud")
So far so good - there is no problem when executing the script on my local maschine. The problem arises, when I try to run the script on our gitlab server.
webdav.exceptions.RemoteResourceNotFound: Remote resource: path_in_cloud not found
Even when I just want to list the files in the root directory I get an error. It seems like I have problems with the authentication which is strange since I use the same credentials/the same url and root as on my local maschine.
webdav.exceptions.ResponseErrorCode: Request to my_cloud_url failed with code 401 and message: b'<?xml version="1.0" encoding="utf-8"?>\n<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">\n <s:exception>Sabre\\DAV\\Exception\\NotAuthenticated</s:exception>\n <s:message>No \'Authorization: Basic\' header found. Either the client didn\'t send one, or the server is misconfigured, No \'Authorization: Bearer\' header found. Either the client didn\'t send one, or the server is mis-configured</s:message>\n</d:error>\n'
Any help is very much appreciated!
-
How to install native dependency when using GitLab Auto DevOps?
My project is a Ruby application that is tested and deployed with GitLab Auto DevOps.
During the test and in production, the application needs to have OpenJDK 11 and Graphviz installed.
Currently, I solved this by specifying my own
Dockerfile
. It works, but I need to duplicate a lot of work that Auto DevOps could handle for me.I already extend the Auto DevOps configuration by having this in my
.gitlab-ci.yml
:include: - template: Auto-DevOps.gitlab-ci.yml
I suspect that it’s just a matter of adding a few lines to hook into the Auto DevOps process. However, Auto DevOps is (deliberately?) a bit like a black box. Therefore it’s hard for me to understand how the Docker container for running the tests and production is built exactly.
-
GitLab on Docker - How to set the external URL for GitLab
I'm running GitLab on my machine by using the docker image. On this machine I have several other services. So the machine Port 80 is already taken by something else. So I did this port mapping 9088:80 ... now I face the problem that Gitlab still thinks for external requests it runs on port 80 and also the runner stuck because:
So it uses http://psmonster/ instead of http://psmonster:9088/
Checking for jobs... received job=39 repo_url=http://psmonster/edi-tools/xsltestsuite.git runner=_z9oqvay WARNING: Job failed: exit status 1 duration=155.353588ms job=39 project=9 runner=_z9oqvay WARNING: Failed to process runner builds=0 error=exit status 1 executor=shell runner=_z9oqvay
Does somebody know how to handle this?
-
How to pull new docker images and restart docker containers after building docker images on gitlab?
There is an
asp.net core api
project, with sources ingitlab
. Createdgitlab
ci/cd pipeline
to builddocker
image and put the image intogitlab
docker
registry (thanks to https://medium.com/faun/building-a-docker-image-with-gitlab-ci-and-net-core-8f59681a86c4).How to update
docker containers
on my production system after putting the image togitlab
docker registry
?*by update I mean:
docker-compose down && docker pull && docker-compose up