jjb to scap deploy
Open, LowPublic
Actions

Assigned To

None

Authored By

	hashar
	Mar 9 2016, 3:39 PM

Description

integration/config.git has the configuration layout for Zuul. It is currently deployed used a fabric recipe at the root of the repo which requires shell access on the CI machine gallium.wikimedia.org

We should migrate to scap3 deploy so more people can do deployment.

Doc: https://doc.wikimedia.org/mw-tools-scap/scap3/quickstart/setup.html

Has to create a scap/scap.cfg and figure out the ssh access https://doc.wikimedia.org/mw-tools-scap/scap3/ssh-access.html#ssh-access

JJB would have to run with jenkins-jobs update --delete-old in order to delete unmaintained jobs (was T91410 , marked as dupe)

Details

	Subject	Repo	Branch	Lines +/-
	(WIP) Zuul deployment with scap? (WIP)	integration/config	master	+30 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T225730 Reduce runtime of MW shared gate Jenkins jobs to 5 min
Declined	None	T248531 Abort a Zuul pipeline when one job completed with failures (change zuul scheduler's failure check from areAllJobsComplete to didAnyJobFail)
Open	None	T129357 Migrate deployment of integration/config for zuul / jjb to scap deploy
Resolved	None	T134001 scap to reload a service instead of restart

Event Timeline

hashar created this task.Mar 9 2016, 3:39 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 9 2016, 3:39 PM

Paladox subscribed.Mar 9 2016, 3:39 PM

greg awarded a token.Mar 9 2016, 4:06 PM

hashar triaged this task as Low priority.Mar 10 2016, 2:12 PM

hashar moved this task from Untriaged to Backlog on the Continuous-Integration-Infrastructure board.

So people will be able to deploy without having access to gallium? That seems non-ideal because they won't be able to look at zuul logs then. Or even reload the zuul service?

So people will be able to deploy without having access to gallium?

Yes that is the idea. Would let more people deploy changes if we got to grant CR+2 to more people. But the real reason is solely to adopt the generic deployment tool.

That seems non-ideal because they won't be able to look at zuul logs then.

I havent though about that one. Looks like we would need a task to send Zuul logs to a Logstash. Python logging can emit to syslog, or there might be a logstash plugin.

Or even reload the zuul service?

scap3 is able to handle restart/reload of service and even has rollback capability.

Change 286207 had a related patch set uploaded (by Hashar):
(WIP) Zuul deployment with scap? (WIP)

https://gerrit.wikimedia.org/r/286207

gerritbot added a project: Patch-For-Review.Apr 29 2016, 7:43 PM

hashar mentioned this in T134001: scap to reload a service instead of restart.Apr 29 2016, 8:06 PM

hashar created subtask T134001: scap to reload a service instead of restart.

Change 286207 abandoned by Hashar:
(WIP) Zuul deployment with scap? (WIP)

https://gerrit.wikimedia.org/r/286207

• demon closed subtask T134001: scap to reload a service instead of restart as Resolved.Mar 9 2017, 12:06 AM

rMSCAbede7516f18f3c72482b0f7407fe62b96412de33 adds reload support. The scap.cfg would have service_reload: True which would trigger a service zuul reload.

greg added a parent task: T111559: Unify deployment of integration/config.git changes using the official Wikimedia deployment system.May 20 2017, 12:19 PM

Will stick to deb packages for now. There is only a couple host that relies on it and we only upgrade it once in a while.

Reopening, wrong task / I am confused.

hashar added projects: Technical-Debt, Release-Engineering-Team.Dec 21 2018, 10:06 PM

greg edited projects, added Release-Engineering-Team (Backlog); removed Release-Engineering-Team.Jan 3 2019, 7:16 PM

hashar renamed this task from Migrate Zuul deployment of integration/config to scap deploy to Migrate deployment of integration/config for zuul to scap deploy.Feb 7 2019, 9:39 AM

hashar edited projects, added Scap (Scap3-Adoption-Phase2); removed Scap (Scap3-Adoption-Phase1).

• Phabricator_maintenance edited projects, added Release-Engineering-Team-TODO; removed Release-Engineering-Team (Backlog).Jun 12 2019, 11:53 PM

• Phabricator_maintenance moved this task from Should be empty (use Release-Engineering-Team) to Later / Need volunteer on the Release-Engineering-Team-TODO board.Jun 12 2019, 11:56 PM

greg added a project: Release-Engineering-Team.Jun 21 2019, 10:35 PM

greg edited projects, added Release-Engineering-Team (CI & Testing services); removed Release-Engineering-Team.Jun 24 2019, 7:51 PM

• mmodell subscribed.Jan 28 2020, 10:11 PM

Legoktm mentioned this in T236689: Upgrade integration/config to use Fabric 2.x / python3.Feb 4 2020, 4:38 AM

Jdforrester-WMF added a parent task: T248531: Abort a Zuul pipeline when one job completed with failures (change zuul scheduler's failure check from areAllJobsComplete to didAnyJobFail).Apr 3 2020, 4:36 PM

There is no bandwidth to use scap to deploy integration/config changes. For now we use a Fabric script at the root of integration/config which ssh to the machine and do the commands. That is good enough for now.

In T129357#6043000, @hashar wrote:

There is no bandwidth to use scap to deploy integration/config changes. For now we use a Fabric script at the root of integration/config which ssh to the machine and do the commands. That is good enough for now.

But long-term it's still something that should be done IMO. As I discovered in T236689: Upgrade integration/config to use Fabric 2.x / python3, fabric upstream is a mess and I don't think it's something we want to rely on forever. Sticking with our current fabric 1.x version isn't an option because Python 2 will go away too.

In T129357#6043000, @hashar wrote:

There is no bandwidth to use scap to deploy integration/config changes. For now we use a Fabric script at the root of integration/config which ssh to the machine and do the commands. That is good enough for now.

Can we do neither of these options please, avoid the complexity of scap and also not have people manually ssh to contint*? Puppet would reload/restart the zuul service when config changes and could also git pull from the repo. As long as it is limited who can merge changes that would be ok and simplify everything. Having users manually SSH to prod servers to restart stuff is really not expected as normal anymore in 2020.

hashar renamed this task from Migrate deployment of integration/config for zuul to scap deploy to Migrate deployment of integration/config for zuul / jjb to scap deploy.Jun 17 2020, 2:05 PM

hashar removed a project: Scap (Scap3-Adoption-Phase2).

thcipriani removed a project: Release-Engineering-Team (CI & Testing services).Apr 20 2021, 1:09 AM

thcipriani edited projects, added Release-Engineering-Team (thcipriani-workboard-fiddling); removed Release-Engineering-Team-TODO.Apr 20 2021, 3:43 AM

thcipriani moved this task from thcipriani-workboard-fiddling to Seen (ARCHIVE) on the Release-Engineering-Team board.Apr 20 2021, 4:05 AM

thcipriani edited projects, added Release-Engineering-Team; removed Release-Engineering-Team (thcipriani-workboard-fiddling).

thcipriani edited projects, added Release-Engineering-Team (Seen); removed Release-Engineering-Team.Apr 20 2021, 3:24 PM

hashar merged a task: T91410: Automatically delete old jobs not (or no longer) managed by JJB.May 21 2021, 2:00 PM

hashar added subscribers: Krinkle, Stashbot, JanZerebecki.

hashar updated the task description. (Show Details)May 21 2021, 2:03 PM

hashar raised the priority of this task from Lowest to Low.May 27 2021, 11:01 AM

hashar removed a parent task: T111559: Unify deployment of integration/config.git changes using the official Wikimedia deployment system.

hashar mentioned this in T111559: Unify deployment of integration/config.git changes using the official Wikimedia deployment system.