This is a fun post to break from the norm of SRE work
It’s Halloween tomorrow and you know what that means… 😱
I’ve thought of 3 spooky activities your team can have scary fun planning
Chaos Engineering Carnival
Sounds like a tame event at first…
… but it will give chaos its true meaning to others.
HR will wonder where all the engineers are…”What’s happened to them all?”
“What’s that tent outside with gremlins all abound?” 😨
Execs will sweat bullets at the mere sound of “chaos”. 🥵
We’ll make merry by deliberately injecting chaos into our systems.
Why? Because we’re all about menace resilience.
We’ll throw in some network failures.
We’ll drop some databases. And so much more fun that will have the boss go 🥵
But of course, we will uncover vulnerabilities and strengthen the system’s ability to handle unexpected issues in the process.
Build Self-Healing Systems
Just like that shapeshifting T-1000 android in Terminator 2.
Keep shooting at the computer, but it keeps on healing itself.
Noice! 🤠
Until it won’t stop running computations even when you press the power button!
The AWS bill! The horror! 😱
So why not build your own self-healing algorithms?
They can detect and automatically recover from common failures.
Build that prototype for your very own T-1000 computer, why don’t you?
When the service hits high latency or errors, your liquid polymer self-healing system could analyze the issue and initiate corrective actions.
You can make it all happen like automatically restarting containers or rerouting traffic.
Magic… or is it AI?
AI-Powered Anomaly Detection
Smart robot’s keen eye,
Spots strange things as they pass by,
Computer gremlins sigh.
The work needed to make anomaly detection is itself… spooky 👻
You must go to the Badlands to find a willing ML engineer
The ML folk tell me stories, but what do they mean?
“Your friend always wears a blue shirt 
”
🤔 I ponder that I don’t have a friend who always wears a blue shirt
“but then one day they show up in a wifebeater with a sash 
”
The ML engineer then tells me, “That’s kind of strange, right?”
Yes, wifebeaters with a sash are definitely in the strange territory.
“My handmade AI friend notices these things.”
“It watches and learns from what people usually do. If most people usually wear blue shirts, it knows that. Now, if someone does something really different, like wearing a wifebeater, the robot says, “Hey, that’s strange!” It’s like magic, but it’s not magic; it’s just smart computer stuff.”
So one day it will know that I’ve skipped leg day?! 😮😱🙀
That my friend is true horror.
- #17 Lessons from SRE’s Wild West Days (with Rick Boone) - November 29, 2023
- #16 Journey to Cloud Infrastructure Leadership (with Sreejith Chelanchery) - November 29, 2023
- #15 Growing Reliability Engineering Across 5+ Companies (with Nash Seshan) - November 14, 2023
