June 2021 - Cloud-admin-feed - lists.wikimedia.org

** RECOVERY alert - cloudcontrol1005/Check systemd state is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: Check systemd state Host: cloudcontrol1005 Address: 208.80.154.85 State: OK Date/Time: Tue Jun 15 17:01:44 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state Acknowledged by : Additional Info: OK - running: The system is fully operational

2 years, 11 months

1
0
0 0

** RECOVERY alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: OK Date/Time: Tue Jun 15 14:29:54 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP OK: HTTP/1.1 200 OK - 158 bytes in 0.007 second response time

2 years, 11 months

1
0
0 0

** PROBLEM alert - checker.tools.wmflabs.org/toolschecker: check mtime mod from tools cron job is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: toolschecker: check mtime mod from tools cron job Host: checker.tools.wmflabs.org Address: checker.tools.wmflabs.org State: CRITICAL Date/Time: Tue Jun 15 14:19:12 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolschecker Acknowledged by : Additional Info: HTTP CRITICAL: HTTP/1.1 503 SERVICE UNAVAILABLE - string OK not found on http://checker.tools.wmflabs.org:80/cron - 177 bytes in 0.010 second response time

2 years, 11 months

1
0
0 0

** ACKNOWLEDGEMENT alert - labstore1004/NFS Share Volume Space /srv/tools is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: ACKNOWLEDGEMENT Service: NFS Share Volume Space /srv/tools Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Tue Jun 15 00:00:39 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_stora… https://grafana.wikimedia.org/d/50z0i4XWz/tools-overall-nfs-storage-utiliza… Acknowledged by Bstorm: Working on this via T284964 Additional Info: DISK CRITICAL - free space: /srv/tools 1263734 MB (15% inode=79%):

2 years, 11 months

1
0
0 0

** PROBLEM alert - labstore1004/NFS Share Volume Space /srv/tools is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: NFS Share Volume Space /srv/tools Host: labstore1004 Address: 10.64.37.19 State: CRITICAL Date/Time: Mon Jun 14 23:45:10 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_stora… https://grafana.wikimedia.org/d/50z0i4XWz/tools-overall-nfs-storage-utiliza… Acknowledged by : Additional Info: DISK CRITICAL - free space: /srv/tools 1263734 MB (15% inode=79%):

2 years, 11 months

1
0
0 0

** ACKNOWLEDGEMENT alert - cloudcontrol1005/Check unit status of backup_glance_images is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: ACKNOWLEDGEMENT Service: Check unit status of backup_glance_images Host: cloudcontrol1005 Address: 208.80.154.85 State: CRITICAL Date/Time: Mon Jun 14 18:04:24 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by andrew bogott: this seems to flap Additional Info: CRITICAL: Status of the systemd unit backup_glance_images

2 years, 11 months

1
0
0 0

** ACKNOWLEDGEMENT alert - cloudcontrol1005/Check systemd state is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: ACKNOWLEDGEMENT Service: Check systemd state Host: cloudcontrol1005 Address: 208.80.154.85 State: CRITICAL Date/Time: Mon Jun 14 18:04:24 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state Acknowledged by andrew bogott: this seems to flap Additional Info: CRITICAL - degraded: The following units failed: backup_glance_images.service

2 years, 11 months

1
0
0 0

** PROBLEM alert - cloudcontrol1005/Check unit status of backup_glance_images is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: Check unit status of backup_glance_images Host: cloudcontrol1005 Address: 208.80.154.85 State: CRITICAL Date/Time: Mon Jun 14 17:21:02 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_time… Acknowledged by : Additional Info: CRITICAL: Status of the systemd unit backup_glance_images

2 years, 11 months

1
0
0 0

** PROBLEM alert - cloudcontrol1005/Check systemd state is CRITICAL **

by nagios＠alert1001.wikimedia.org

Notification Type: PROBLEM Service: Check systemd state Host: cloudcontrol1005 Address: 208.80.154.85 State: CRITICAL Date/Time: Mon Jun 14 17:17:52 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state Acknowledged by : Additional Info: CRITICAL - degraded: The following units failed: backup_glance_images.service

2 years, 11 months

1
0
0 0

** RECOVERY alert - cloudcontrol1003/Check for VMs leaked by the nova-fullstack test is OK **

by nagios＠alert1001.wikimedia.org

Notification Type: RECOVERY Service: Check for VMs leaked by the nova-fullstack test Host: cloudcontrol1003 Address: 208.80.154.23 State: OK Date/Time: Thu Jun 10 16:11:47 UTC 2021 Notes URLs: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Check_f… Acknowledged by : Additional Info: 1 instances in the admin-monitoring project

2 years, 11 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

Cloud-admin-feed June 2021