Page MenuHomePhabricator

netopsComponent
ActivePublic

Members (12)

Watchers (8)

Details

Description

This is a project for Wikimedia network's operation tasks, i.e. everything relating to routers, switches etc.

Parent project: SRE

Recent Activity

Today

cmooney added a subtask for T268802: Manage frack switches with Netbox: Unknown Object (Task).
Fri, Nov 15, 3:47 PM · Infrastructure-Foundations, SRE, netops, fundraising-tech-ops
cmooney removed a subtask for T377996: Manange fundraising network elements from Netbox: Unknown Object (Task).
Fri, Nov 15, 3:46 PM · netops, Infrastructure-Foundations, SRE
cmooney closed Restricted Task, a subtask of T377996: Manange fundraising network elements from Netbox, as Resolved.
Fri, Nov 15, 3:46 PM · netops, Infrastructure-Foundations, SRE
RobH added a comment to T379778: Decom prod infra side of the ulsfo-office link.
"Created by: cbissell Cross connect has been disconnected, per our policy it will be removed after 48 hours"
Fri, Nov 15, 2:34 PM · DC-Ops, ops-ulsfo, Infrastructure-Foundations, netops, SRE, procurement
Maintenance_bot removed a project from T379778: Decom prod infra side of the ulsfo-office link: Patch-For-Review.
Fri, Nov 15, 1:30 PM · DC-Ops, ops-ulsfo, Infrastructure-Foundations, netops, SRE, procurement
ayounsi closed T335028: Put Dell SONiC switches in production as Declined.

Because of the various limitations listed in {T342673} (plus the ones from pygnmi) we're not going to proceed any further on Dell SONiC, focusing on {T371088} now.

Fri, Nov 15, 1:04 PM · SRE, netops, Infrastructure-Foundations
ayounsi closed T320638: Add Dell switches support to Homer/Cookbooks, a subtask of T335028: Put Dell SONiC switches in production, as Declined.
Fri, Nov 15, 1:03 PM · SRE, netops, Infrastructure-Foundations
ayounsi closed T320638: Add Dell switches support to Homer/Cookbooks as Declined.

Because of the various limitations listed in T340045: Package pyGNMI and dictdiffer to be used by cookbooks we're not going to proceed any further on Dell SONiC, focusing on {T371088} now.

Fri, Nov 15, 1:03 PM · Patch-For-Review, SRE, Infrastructure-Foundations, netops
ayounsi closed T340045: Package pyGNMI and dictdiffer to be used by cookbooks, a subtask of T320638: Add Dell switches support to Homer/Cookbooks, as Declined.
Fri, Nov 15, 1:02 PM · Patch-For-Review, SRE, Infrastructure-Foundations, netops
ayounsi closed T340045: Package pyGNMI and dictdiffer to be used by cookbooks, a subtask of T338028: Users management on SONiC, as Declined.
Fri, Nov 15, 1:02 PM · SRE, Infrastructure-Foundations, netops
ayounsi closed T344325: gNMI module in Spicerack, a subtask of T320638: Add Dell switches support to Homer/Cookbooks, as Declined.
Fri, Nov 15, 1:00 PM · Patch-For-Review, SRE, Infrastructure-Foundations, netops
ayounsi closed Restricted Task, a subtask of T320638: Add Dell switches support to Homer/Cookbooks, as Declined.
Fri, Nov 15, 12:57 PM · Patch-For-Review, SRE, Infrastructure-Foundations, netops
aborrero closed T245495: CloudVPS: IPv6 in codfw1dev as Resolved.

I think we can consider IPv6 to be fully working on codfw1dev.

Fri, Nov 15, 12:53 PM · Cloud-VPS, User-aborrero, cloud-services-team, Infrastructure-Foundations, SRE, netops
aborrero updated the task description for T245495: CloudVPS: IPv6 in codfw1dev.
Fri, Nov 15, 12:52 PM · Cloud-VPS, User-aborrero, cloud-services-team, Infrastructure-Foundations, SRE, netops
aborrero closed T376462: dns: integrate PTR support for 2a02:ec80:a100::/48, a subtask of T374712: netbox: create IPv6 entries for Cloud VPS, as Resolved.
Fri, Nov 15, 12:51 PM · cloud-services-team, Cloud-VPS, User-aborrero, Infrastructure-Foundations, SRE, netops
aborrero closed T376462: dns: integrate PTR support for 2a02:ec80:a100::/48 as Resolved.
Fri, Nov 15, 12:51 PM · cloud-services-team, Cloud-VPS, User-aborrero, Infrastructure-Foundations, SRE, netops
ayounsi moved T364092: Upgrade core routers to Junos 23.4R2 from Backlog to This quarter on the netops board.
Fri, Nov 15, 12:50 PM · netops, Infrastructure-Foundations, SRE
aborrero closed T374715: openstack: work out IPv6 and designate integration as Resolved.

this was done by means of T378192: openstack: wmf sink: extend it to support IPv6

Fri, Nov 15, 12:48 PM · User-aborrero, Cloud-VPS, cloud-services-team, Infrastructure-Foundations, SRE, netops
aborrero closed T374715: openstack: work out IPv6 and designate integration, a subtask of T245495: CloudVPS: IPv6 in codfw1dev, as Resolved.
Fri, Nov 15, 12:47 PM · Cloud-VPS, User-aborrero, cloud-services-team, Infrastructure-Foundations, SRE, netops
aborrero closed T374715: openstack: work out IPv6 and designate integration, a subtask of T376462: dns: integrate PTR support for 2a02:ec80:a100::/48, as Resolved.
Fri, Nov 15, 12:47 PM · cloud-services-team, Cloud-VPS, User-aborrero, Infrastructure-Foundations, SRE, netops
aborrero closed T377740: neutron: clarify why DNS extension is not enabled, a subtask of T374715: openstack: work out IPv6 and designate integration, as Resolved.
Fri, Nov 15, 12:47 PM · User-aborrero, Cloud-VPS, cloud-services-team, Infrastructure-Foundations, SRE, netops
aborrero updated the task description for T245495: CloudVPS: IPv6 in codfw1dev.
Fri, Nov 15, 12:46 PM · Cloud-VPS, User-aborrero, cloud-services-team, Infrastructure-Foundations, SRE, netops
gerritbot added a comment to T379778: Decom prod infra side of the ulsfo-office link.

Change #1090897 merged by jenkins-bot:

[operations/homer/public@master] Remove office policy-options

https://gerrit.wikimedia.org/r/1090897

Fri, Nov 15, 12:38 PM · DC-Ops, ops-ulsfo, Infrastructure-Foundations, netops, SRE, procurement

Yesterday

RobH added a project to T379778: Decom prod infra side of the ulsfo-office link: ops-ulsfo.
Thu, Nov 14, 8:57 PM · DC-Ops, ops-ulsfo, Infrastructure-Foundations, netops, SRE, procurement
RobH updated the task description for T379778: Decom prod infra side of the ulsfo-office link.
Thu, Nov 14, 8:53 PM · DC-Ops, ops-ulsfo, Infrastructure-Foundations, netops, SRE, procurement
Papaul updated the task description for T364092: Upgrade core routers to Junos 23.4R2.
Thu, Nov 14, 7:39 PM · netops, Infrastructure-Foundations, SRE
Stashbot added a comment to T364092: Upgrade core routers to Junos 23.4R2.

Mentioned in SAL (#wikimedia-operations) [2024-11-14T19:37:29Z] <sukhe@cumin1002> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: pool site eqiad [reason: junos upgrade done, T364092]

Thu, Nov 14, 7:37 PM · netops, Infrastructure-Foundations, SRE
Stashbot added a comment to T364092: Upgrade core routers to Junos 23.4R2.

Mentioned in SAL (#wikimedia-operations) [2024-11-14T19:37:25Z] <sukhe@cumin1002> START - Cookbook sre.dns.admin DNS admin: pool site eqiad [reason: junos upgrade done, T364092]

Thu, Nov 14, 7:37 PM · netops, Infrastructure-Foundations, SRE
Papaul updated the task description for T364092: Upgrade core routers to Junos 23.4R2.
Thu, Nov 14, 5:48 PM · netops, Infrastructure-Foundations, SRE
Papaul closed T372781: cr1-eqiad: disk failure as Resolved.

This is done, re0 is now the master. Closing this task

re0.cr1-eqiad> show chassis routing-engine 
Routing Engine status:
  Slot 0:
    Current state                  Master
    Election priority              Master (default)
    Temperature                 45 degrees C / 113 degrees F
    CPU temperature             57 degrees C / 134 degrees F
    DRAM                      49096 MB (49152 MB installed)
    Memory utilization          17 percent
    5 sec CPU utilization:
Thu, Nov 14, 5:47 PM · SRE, Infrastructure-Foundations, ops-eqiad, DC-Ops, netops
Maintenance_bot removed a project from T377381: Frack eqiad network upgrade: design, installation and configuration: Patch-For-Review.
Thu, Nov 14, 4:33 PM · DC-Ops, ops-eqiad, fundraising-tech-ops, netops, Infrastructure-Foundations, SRE
gerritbot added a comment to T377381: Frack eqiad network upgrade: design, installation and configuration.

Change #1091182 merged by Cathal Mooney:

[operations/puppet@production] Remove old fasw-c-eqiad from monitoring

https://gerrit.wikimedia.org/r/1091182

Thu, Nov 14, 4:10 PM · DC-Ops, ops-eqiad, fundraising-tech-ops, netops, Infrastructure-Foundations, SRE
Stashbot added a comment to T364092: Upgrade core routers to Junos 23.4R2.

Mentioned in SAL (#wikimedia-operations) [2024-11-14T15:36:21Z] <sukhe@cumin1002> END (PASS) - Cookbook sre.dns.admin (exit_code=0) DNS admin: depool site eqiad [reason: junos upgrade, T364092]

Thu, Nov 14, 3:37 PM · netops, Infrastructure-Foundations, SRE
Stashbot added a comment to T364092: Upgrade core routers to Junos 23.4R2.

Mentioned in SAL (#wikimedia-operations) [2024-11-14T15:36:05Z] <sukhe@cumin1002> START - Cookbook sre.dns.admin DNS admin: depool site eqiad [reason: junos upgrade, T364092]

Thu, Nov 14, 3:37 PM · netops, Infrastructure-Foundations, SRE
gerritbot added a comment to T377381: Frack eqiad network upgrade: design, installation and configuration.

Change #1091228 abandoned by Cathal Mooney:

[operations/puppet@production] Remove old fr-tech switch stack from rancid backups

Reason:

forgot to remove device from common.yaml too

https://gerrit.wikimedia.org/r/1091228

Thu, Nov 14, 2:52 PM · DC-Ops, ops-eqiad, fundraising-tech-ops, netops, Infrastructure-Foundations, SRE
akosiaris updated the task description for T379790: Reimage one of the wikikube-worker1240 to wikikube-worker1304 node in eqiad as a replacement for wikikube-ctrl1001.
Thu, Nov 14, 2:52 PM · Kubernetes, netops, Infrastructure-Foundations, serviceops
akosiaris added a comment to T379790: Reimage one of the wikikube-worker1240 to wikikube-worker1304 node in eqiad as a replacement for wikikube-ctrl1001.

Cool, thanks. In that case, I randomly picked wikikube-worker1290 and let's take it from there.

Thu, Nov 14, 2:51 PM · Kubernetes, netops, Infrastructure-Foundations, serviceops
RobH updated the task description for T379778: Decom prod infra side of the ulsfo-office link.
Thu, Nov 14, 1:29 PM · ops-ulsfo, DC-Ops, netops, Infrastructure-Foundations, procurement, SRE
gerritbot added a comment to T377381: Frack eqiad network upgrade: design, installation and configuration.

Change #1091228 had a related patch set uploaded (by Cathal Mooney; author: Cathal Mooney):

[operations/puppet@production] Remove old fr-tech switch stack from rancid backups

https://gerrit.wikimedia.org/r/1091228

Thu, Nov 14, 12:53 PM · DC-Ops, ops-eqiad, fundraising-tech-ops, netops, Infrastructure-Foundations, SRE
cmooney updated subscribers of T371435: Q1:eqiad:frack network upgrade tracking task.

I'd hand this over to either John or Valerie as ops-eqiad for them to remove any devices and cables.

Thu, Nov 14, 12:03 PM · SRE, fundraising-tech-ops, netops, ops-eqiad, Infrastructure-Foundations, DC-Ops
gerritbot added a project to T379907: Netbox: librenms report errors: Patch-For-Review.
Thu, Nov 14, 12:00 PM · Patch-For-Review, Infrastructure-Foundations, netops, netbox
gerritbot added a comment to T379907: Netbox: librenms report errors.

Change #1091212 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/software/netbox-extras@master] LibreNMS report: various fixes

https://gerrit.wikimedia.org/r/1091212

Thu, Nov 14, 12:00 PM · Patch-For-Review, Infrastructure-Foundations, netops, netbox
ayounsi created T379907: Netbox: librenms report errors.
Thu, Nov 14, 12:00 PM · Patch-For-Review, Infrastructure-Foundations, netops, netbox
gerritbot added a project to T377381: Frack eqiad network upgrade: design, installation and configuration: Patch-For-Review.
Thu, Nov 14, 8:46 AM · DC-Ops, ops-eqiad, fundraising-tech-ops, netops, Infrastructure-Foundations, SRE
gerritbot added a comment to T377381: Frack eqiad network upgrade: design, installation and configuration.

Change #1091182 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/puppet@production] Replace fasw-c-eqiad with new fasw2

https://gerrit.wikimedia.org/r/1091182

Thu, Nov 14, 8:46 AM · DC-Ops, ops-eqiad, fundraising-tech-ops, netops, Infrastructure-Foundations, SRE
ayounsi updated the task description for T379778: Decom prod infra side of the ulsfo-office link.
Thu, Nov 14, 7:36 AM · ops-ulsfo, DC-Ops, Infrastructure-Foundations, netops, SRE, procurement
gerritbot added a comment to T379778: Decom prod infra side of the ulsfo-office link.

Change #1091130 merged by Ayounsi:

[operations/dns@master] Remove office interco include

https://gerrit.wikimedia.org/r/1091130

Thu, Nov 14, 7:28 AM · ops-ulsfo, DC-Ops, Infrastructure-Foundations, netops, SRE, procurement
gerritbot added a comment to T379778: Decom prod infra side of the ulsfo-office link.

Change #1091130 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[operations/dns@master] Remove office interco include

https://gerrit.wikimedia.org/r/1091130

Thu, Nov 14, 7:17 AM · ops-ulsfo, DC-Ops, Infrastructure-Foundations, netops, SRE, procurement
Stashbot added a comment to T379778: Decom prod infra side of the ulsfo-office link.

Mentioned in SAL (#wikimedia-operations) [2024-11-14T07:06:24Z] <XioNoX> delete office interco IP/prefixes/vlan in ulsfo - T379778

Thu, Nov 14, 7:06 AM · ops-ulsfo, DC-Ops, netops, Infrastructure-Foundations, procurement, SRE

Wed, Nov 13

cmooney added a comment to T379790: Reimage one of the wikikube-worker1240 to wikikube-worker1304 node in eqiad as a replacement for wikikube-ctrl1001.

Polling Netbox to find what switch each of those are connected to it appears none of them in rows A-D are connected to a 10G capable switch.

Wed, Nov 13, 6:41 PM · Kubernetes, netops, serviceops, Infrastructure-Foundations