“This was an especially difficult outage in that it involved a combination of several factors,” David Baszucki, Roblox founder and CEO, said in a blog post.
“A core system in our infrastructure became overwhelmed, prompted by a subtle bug in our backend service communications while under heavy load. This was not due to any peak in external traffic or any particular experience. Rather the failure was caused by the growth in the number of servers in our datacenters. The result was that most services at Roblox were unable to effectively communicate and deploy.
“Due to the difficulty in diagnosing the actual bug, recovery took longer than any of us would have liked. Upon successfully identifying this root cause, we were able to resolve the issue through performance tuning, re-configuration, and scaling back of some load. We were able to fully restore service as of this afternoon.”
Baszucki said Roblox will publish a post-mortem with more details once the company has completed its analysis, along with actions it will take to prevent this from happening again.
Roblox will also implement a policy to make its creator community “economically whole” as a result of this outage. Expect more details on what that means soon.
“To the best of our knowledge there has been no loss of player persistence data, and your Roblox experience should now be fully back to normal,” Baszucki said.
ORIGINAL STORY 30/10/21: Roblox is down.
The enormously popular and controversial game creation platform went offline yesterday - and it remains offline this morning.
“We know you’re having issues using Roblox right now,” read a message Tweeted by the official Roblox account yesterday afternoon.
“We’re sorry and working hard to get things back to normal.”
Then, late last night, an update:
“Still making progress on today’s outage. We’ll continue to keep you updated. Once again, we apologise for the delay.
“We know that this outage was not related to any specific experiences or partnerships on the platform.”
That last line about the outage not being related “to any specific experiences or partnerships on the platform” is in reference to a theory that Roblox was knocked offline because of Chipotle.
The restaurant announced it would give away free burritos in real life to the first 30,000 players who visited its in-game store at certain times of day.
Chipotle created the Boorito Maze, which opened on the 28th, inside Roblox. To get a free burrito, you had to complete the maze and visit the virtual cashier at specific times each day until tomorrow, 31st October. The theory is this promo proved so popular it crashed the platform.
But in its tweet, Roblox clearly denies this is the case. But without any further information from the company, speculation is running rampant.
And now, parents of children who play Roblox are up in arms. I’ve seen multiple posts on various parents’ Facebook groups complaining about their kids complaining about Roblox being down.
“Roblox is down. I repeat Roblox is down,” said one exasperated mum on the Happy Mum Happy Baby Facebook group last night.
“I have an all-boy household. I’ll need a whiskey later.”
“Honestly, my son was freaking out because it wouldn’t let me log on,” said another mum. “It was like the end of the world.”
“All my girls have done is kick off about it not working,” said another.
“I feel your pain,” replied a parent. “My boys are absolutely broken.”
And finally: “My six-year-old ran in waking her brother up in fury it wasn’t on.”
Roblox is enormously popular with children, but it has hit the headlines for the wrong reasons recently.
In August, a video investigation from People Make Games accused Roblox Corporation of “exploiting” young game developers.
The video, below, from People Make Games’ Quintin Smith, reveals Roblox’s shady exchange system, which, it claims, makes it incredibly hard for developers to make money from the platform.
The video also runs through how Roblox, which has 200 million monthly users, encourages young game developers to work incredibly hard with unreasonable expectations of success.
Roblox revolves around kids making games for other kids to play. Its developers can monetise their games, and are paid in a virtual currency called Robux.
Roblox has said it’s increasing the amount of money paid out to its developers, and highlighted the various support programs available to help its developers succeed on the platform.
There are now calls from some for Roblox to reimburse any users who had ads running on the platform from Thursday.
“The amount of money developers have lost from this downtime is bad,” said Twitter user @KonekoKittenYT. “All memes aside they’re the ones that get hit the most w/ these downtimes since they rely on the platform being up to make income.”
