Gmail down?

This topic has 10 replies, 3 voices, and was last updated 1 year, 7 months ago by Michael Pietroforte.

Viewing 6 reply threads

Author
Posts
- Mon, Dec 14 2020 at 4:28 am #1559528
  Michael Pietroforte
  Keymaster
  Member Points: 35,424
  Rank: 4
  It seems Gmail is completely down. Is anyone able to login?
- Mon, Dec 14 2020 at 4:38 am #1559531
  Michael Pietroforte
  Keymaster
  Member Points: 35,424
  Rank: 4
  Gmail just came back. Scary. All services were red (see attached below) for 10 minutes or so.
  Attachments:
  You must be logged in to view attached files.
- Mon, Dec 14 2020 at 4:42 am #1559533
  Paolo Maffezzoli
  Participant
  Member Points: 70,121
  Rank: 4
  I have an account down, another one is still running. But there is a global problem random about Google services down.
- Mon, Dec 14 2020 at 4:44 am #1559534
  Paolo Maffezzoli
  Participant
  Member Points: 70,121
  Rank: 4
  More info available on Google Dashboard: https://www.google.com/appsstatus
  - Mon, Dec 14 2020 at 4:49 am #1559535
    Michael Pietroforte
    Keymaster
    Member Points: 35,424
    Rank: 4
    My accounts all came back now. But the dashboard is still red. Amazing that this happens to Google.
    Scary when you try to login and Google claims your account does not exist.
- Mon, Dec 14 2020 at 7:57 am #1559537
  Steven
  Participant
  Member Points: 1,114
  Rank: 3
  It happened to Microsoft and ISP worldwide not long ago as well.
  I believe the expectation are unrealistic regarding availability.
  - Mon, Dec 14 2020 at 8:05 am #1559538
    Michael Pietroforte
    Keymaster
    Member Points: 35,424
    Rank: 4
    Yeah, you never get 100% availability. However, there are big differences. Microsoft is not really known for providing reliable software and services. Google is a bit better, I’d say. And in the 10 years 4sysops is running on AWS, I experienced 0 downtime.
- Tue, Dec 15 2020 at 11:43 am #1559578
  Paolo Maffezzoli
  Participant
  Member Points: 70,121
  Rank: 4
  Someone in Google has changed something in the quota management…
  Outage caused by a bug in storage quota system
  According to a tweet and a Google status report, the outage was caused by the company’s automated quota management system reducing the amount of storage available to Google’s authentication system.
  “Today, at 3.47AM PT Google experienced an authentication system outage for approximately 45 minutes due to an internal storage quota issue. This was resolved at 4:32AM PT, and all services are now restored,” Google stated in a tweet from their Google Cloud account.
  Link : Google outage caused by critical system running out of storage
  - Tue, Dec 15 2020 at 11:50 am #1559579
    Michael Pietroforte
    Keymaster
    Member Points: 35,424
    Rank: 4
    That explains why Google claimed that my account does not exist. A large company like Google should have a monitoring system that notices when a storage system runs low on space.
    - Wed, Dec 16 2020 at 5:52 am #1559606
      Steven
      Participant
      Member Points: 1,114
      Rank: 3
      I believe Google has such a set up for their own system.
- Wed, Dec 16 2020 at 6:11 am #1559608
  Michael Pietroforte
  Keymaster
  Member Points: 35,424
  Rank: 4
  Google worships automation. My guess is that the Google engineers were so convinced of their automated quota management system that they neglected monitoring. I mean it is easy to confuse quota automation with storage space monitoring. In the latter case you need humans who receive alerts when unusual storage consumptions occur. Google tries to get rid of human workers as much as possible. It is hard to explain how an human admin could have missed that such an important system will run out of storage.
Author
Posts