Mentioned in SAL (#wikimedia-operations) [2020-05-18T11:52:42Z] <marostegui> Install 10.1.43-2 on db1122 and db1109 - T251981

package updated on db1122 and db1109.

Procedure:

Maintenance day:

• Silence all hosts in s2 and s8
• Set read only on s2 and s8:

dbctl --scope eqiad section s2 ro "Maintenance on s2 and s8 T251981" && dbctl --scope eqiad section s8 ro "Maintenance on s2 and s8 T251981"  && dbctl config commit -m "Set s2 and s8 as read-only for maintenance T251981"

• Confirm read only on the wikis
• Set read-only on the masters on mysql: db1122 db1109
• Drop T248086_wb_terms table on s8 master
• Restart mysql on both hosts db1122 db1109
• Confirm the slaves are connected
• Run puppet
• Remove read only:

dbctl --scope eqiad section s2 rw  && dbctl --scope eqiad section s8 rw && dbctl config commit -m "Set s2 and s8 as read-only=off for maintenance T251981"

• Confirm writes can go through
• Run mysql_upgrade db1122 db1109
• Close task

Mentioned in SAL (#wikimedia-operations) [2020-05-19T05:00:44Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Set s2 and s8 as read-only for maintenance T251981', diff saved to https://phabricator.wikimedia.org/P11226 and previous config saved to /var/cache/conftool/dbconfig/20200519-050043-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2020-05-19T05:03:47Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Set s2 and s8 as read-only=off for maintenance T251981', diff saved to https://phabricator.wikimedia.org/P11227 and previous config saved to /var/cache/conftool/dbconfig/20200519-050346-marostegui.json