Google Cloud SQL Experiencing Issues
Anyone have any inside contacts at Google who have more information than their useless status page?
In particular, I have a high availability instance (HA!), with a bonus read replica. The main server and the failover server seem to be malfunctioning. The read replica appears to be fine, and I can access that to pull data.
Having been down this road perhaps a half-dozen or a dozen times, I take the same attitude towards exposing business processes to Google Cloud as I do to exposing myself to COVID19:
* Only when strictly necessary (there are no viable alternatives to Google login or YouTube for many purposes, and you need good search rankings)
* When absolutely necessary, use appropriate protection (don't rely on Google search for business, have fail overs for videos and auth, and do periodic Google Takeouts of my data)
If not for Google Takeout, I'd do a strict quarantine. It's kind of like the N95 mask for doing business with Google.
Note that this is on the B2B end. For personal use, I use many (free) Google services. It works great for things like maps, finding interesting videos on Youtube, using Android on my phone, or even sharing family photos. I am, apparently, a satisfied Google product (you know the old saying: if you're not a paying customer, you're a ....).
Same
This has been going on for over two hours with no update from Google aside from the single sentence "We are experiencing an issue with Cloud SQL" in the status dashboard.
status.cloud.google.com now shows all green (the note about Cloud SQL has disappeared), but the errors continue, and my Cloud SQL console won't load.
I can confirm this is also my status. Thanks for the reassurance that it's not just my instance.
Was green and now it's back to orange
The errors have become less frequent since about 21:00 UTC. The most recent errors I've seen were at 22:45. The update of my instance has finished. As of now (23:03) there is still no new word on the incident page: https://status.cloud.google.com/incident/cloud-sql/21001
the new incident link is: https://status.cloud.google.com/incident/cloud-sql/21002
Thank you! I've been polling the original incident page all day. Grrr...
One of my Cloud SQL instances disappeared from my dashboard on and off. Of course, that makes me nice and nervous.
My App Engine logs show Cloud SQL connection errors since 17:31 UTC. I use 2nd generation MySQL 5.6. My SQL console shows that the instance is being updated since 18:39 (more than an hour ago). Perhaps that's part of the attempted fix.
unrelated but is there a reason why so many servers still seem to be on mysql 5.6? it's an absolutely ancient version, yet I keep running into it
My instances have stayed up but I am having problems with API calls. If your instance is down, I wonder if it's isolated to CloudSQL Proxy connections. Seems like this could be an issue at the API level.
status.cloud.google.com now shows an actual incident thread (not just a note). The only new info is that it affects the us-central1 region, but it says we can expect an update by 21:00 UTC.
Here's a direct link to the incident page: https://status.cloud.google.com/incident/cloud-sql/21001
Thanks everyone for providing real-time information. It's unnerving when an entire SQL server disappears from your control panel:-). I'd rather not have to restore everything from backups, if it's all the same!
Still having issues and disconnects
Could it be related to CVE-2021-3156 the 10 years old vulnerability in sudo?
Any information on how it would be linked?
Here you go: https://news.ycombinator.com/item?id=24216009
Category 502 hurricane spotted over Mountain View. Expected to reach category 503 before moderating down to category 200 later this evening.