In 2016, two buddies booked a room at a Gallup, New Mexico, motel on their long road trip to off-roading trails in California. In the middle of the night, a stranger unlocked their room’s door and walked in. They don’t say much about the confrontation that ensued, but they make an interesting (and terrifying) discovery in the morning: Their room’s keycard could unlock every room on the property.
The motel had compromised their security by giving everyone escalated privileges to take a specific action they shouldn’t have been allowed to. In response, the motel chain promised they had already resolved what they classified as a “malfunction.”
What are shadow admins?
Shadow admins are non-administrator users with unknown or untraceable privileges, granting them administrator rights to the identities, software, and environments your organization uses. These privileges can include direct access to data these users shouldn’t be able to see, the ability to modify their peers’ settings or passwords, or the capacity to add or integrate shadow IT to sensitive environments without proper oversight.
If you ask the folks at Silverfort, who have for years been leading efforts to educate admins and engineers about shadow admins, their definition is even more complicated: “Additionally, any user who can take over and control a Shadow Admin of any level, would also be considered a shadow admin.” You can quickly imagine spiraling and cyclical scenarios where many of your users are shadow admins to some dangerous degree.
Unfortunately, most conversations about shadow admins have been limited to Active Directory (AD), a service that helps administrators understand which users and machines can perform which actions. AD is absolutely a valid concern, as it’s so common for businesses with heavy on-premises infrastructure, but that focus heavily downplays the breadth of possible risk.
Every identity provider (IdP), like Okta and OAuth, is inherently vulnerable to shadow admins. The same goes for SaaS apps, both approved and shadowy, that your users rely on to accomplish their daily tasks. Of unique concern for companies that have heavily invested in the cloud: the famously complex world of identity and access management (IAM) rules for cloud providers like GCP, Azure, and AWS, which have more than 5,000 permissions apiece.
Five common causes of shadow admins
According to The Identity Underground Report, 4 percent of accounts at small-to-midsize businesses are shadow admins. When you reach the enterprise level, that number falls to 1.2 percent. That might not sound like a broad risk to your organization, particularly in light of more visible threats like ransomware, but they can quickly emerge from some unexpected sources, both intentional and mistaken.
- Non-administrative users might request temporary privileges to take a specific action, then not follow through with IT/DevOps to have said privileges removed after realizing they can work more efficiently (i.e. without having to ask and wait for help).
- Real administrators often create these accounts intentionally as a hidden alternative to going through official channels.
- Complex actions, such as on-premises-to-cloud migrations or adopting new SaaS tools, can inadvertently create many shadow admins without any visibility or warning.
- Malicious actors who want to lurk around your systems, operating completely in stealth, as they collect more information to create a more coordinated attack, such as taking over a valid administrator account to install malware or exfiltrate data.
- Administrators and regular users who, after leaving the company, are not fully cleaned up after, leaving privileged accounts.
The hidden consequences of shadow admins
The core problem with shadow admins is that they are an unknown unknown—a risk stemming from a root cause so granular that you probably hadn’t given it serious thought before. The same goes for the potential impacts on your organization, which involve data loss and downtime through means you might not have thought about before.
Privilege escalation
If an attacker accesses a shadow admin, particularly one from a rarely-used or inactive user, they can use the existing privileges to gain access to new data or commandeer a true administrator for complete ownership of your SaaS environment.
Colin Estep, a senior security researcher at Netskope, related to Dark Reading how readily shadow admins could leverage the complexity of cloud IAM to get unfettered access to your cloud environments: “If you don’t have your identity solution figured out, then you could expose sensitive data, you could have resources be abused or deleted. … [An attacker] could go in and change logging, they could go in and create resources, they could delete things, access all the data, add users for themselves.
“They could probably do everything except delete the entire environment.”
Human error
If one person unknowingly has privileges to your entire customer database, and makes a simple mistake, you could end up with an empty table. If you’re not prepared, you’ll have very few paths to remediation.
Not convinced of the menacing threat of human error? Our recent deep-dive explainer covers all the confusion, complacency, and (over)confidence at the root of data loss caused by human error.
Some, like Rupesh Kumar at Lepide Software, argue that data loss caused by human error and shadow admins is still vastly underreported because it’s uniquely embarrassing and, in the case of publicly traded companies, often massively destructive to their financial picture.
Migrations
When migrating to the cloud for the first time, an on-premises deployment is often recreated in the new environment, migrating along all the mistakes and misconfigurations of the past. The new deployment isn’t inherently less secure, but once you’re operating in the cloud, your data is far more exposed to external attackers, and you have far less control than you did when the environment operated from within the office.
A similar scenario happens when migrating between two competing SaaS solutions. The often manual process of mapping one system’s privileges to another creates scenarios where a read-only switch becomes read-and-write, creating unwanted instances where anyone has unchecked access to take dramatic action, like deleting a database.
Mergers and acquisitions
These activities don’t just involve the transfer of people, things, and intellectual property. The two companies must also marry their environments and data. The larger organization often performs a “lift and shift,” which is essentially a complex copy-paste operation whereby infrastructure and data is moved to existing servers.
Shadow admins, particularly those involving inactive user accounts, are a unique risk. Attackers will often target companies in the middle of M&A activity, knowing these companies’ IT and DevOps folks are already overwhelmed with the sheer complexity of the lift and shift. If they can gain access to the acquired company, they’ve essentially written themselves a master key to the newly merged dataset.
‘Shadow bursts’
The Silverfort team defines shadow bursts as a single misconfiguration that spirals out of control, creating many new shadow admins. While most bursts create around 70 shadow admins, they’ve also seen 1,000 new attack vectors stemming from a single mistake.
Even more frightening—30% of organizations they surveyed experienced at least one shadow burst within the last 30 days. With those numbers, you can quickly extrapolate how quickly shadow admins propagate under the radar in SaaS environments where you share a massive part of the responsibility for data security.
How to identify and mitigate shadow admins and spiraling permissions
The only conclusion to this shadowy conversation is that you simply can’t afford to let these ultra-privileged accounts go unsighted and untreated any longer. Unfortunately, that is easier said than done.
Carolyn Crandall, Chief Security Advocate at Attivo Networks, writes, “Typically, there is no straightforward way of finding these delegated administrator accounts except to conduct an exhaustive audit, meaning they can pose a threat that is often not fully quantified. If one can’t see a problem and gauge its extent, how can one prepare for it?”
Implement least privilege
Within SaaS environments, users should have only the minimum access and privileges necessary to perform their specific job function.
The principle of least privilege curbs “privilege creep” through the central management of identity, credentials, and access information. It’s not a panacea, and won’t eliminate your army of shadow admins right away, but it does minimize the risk of new instances from popping up through the education and enforcement of broad cybersecurity policy.
Encourage staff to pitch in
Your employees can be a rich source of shadow admin clues if you don’t punish them for past mistakes, either intentional or inadvertent. Encourage them to report instances where they realize they have more than the least privilege for their job.
On the opposite side, you’ll want to proactively examine account privileges for anomalies ahead of situations you know will frustrate or confuse your peers. Could one of them delete swaths of data in retaliation for a layoff? Could they extract intellectual property from your secure systems to share with a competitor after learning their program has been canceled? These situations often blossom from painful to catastrophic for everyone involved.
Enforce comprehensive SaaS backups
When we discussed privilege escalation via shadow admins, we illustrated a scenario in which a user accidentally deletes an entire database because they are unaware of their admin-level privileges. Not all data loss caused by shadow admins will be this catastrophic, but no matter the root cause or result, your remediation is the same: Look to your backups.
You do have backups for your SaaS data, right?
Even if your SaaS backup data contains shadow admins, you can still isolate that data in a siloed environment, perform the mitigation work required, and then bring your fixed data back into production. Hopefully the backup solution you choose for SaaS data also comes with automated restorations, too—no one wants to spend cycles managing CSV or JSON files you hope will upload and restore your instance to its last good state.
Separate duties among peers
If you have multiple IT administrators or DevOps engineers responsible for maintaining your SaaS apps and environments, consider separating their duties—and privileges—whenever possible. By encouraging administrators to monitor one another’s actions, they’ll collectively be in a better position to identify existing shadow admins and stop new ones from being created during shadow bursts.
Split personal and administrative accounts
Let’s say only one team in your organization uses a particular SaaS app. This team includes a VP-level employee, who needs administrative access to the SaaS to ensure their employees can work effectively. Instead of giving full admin rights to the account they use day-to-day, consider giving them two accounts, leaving the more privileged one only for agreed-upon and documented actions.
Splitting accounts and their intended purposes allows you to place different monitoring levels around each. While their regular user account is locked down and low-risk, you can keep a closer eye on the administrator-level alternative for anomalies.
Build privilege graphs to visualize shadow admin networks
When asked to picture your organization’s network of users, admins, and privileges, you might think of an intricate spider’s web. In reality, it might be more like Gordian Knot.
When exploring GCP permissions, Estep used graph data to help folks answer a deceptively complex question: “Do you know what all your users can do?” His solution gathers data via the GCP API and then maps users, the GCP environment structure, service accounts, and their associated privileges to help IT administrators and DevOps engineers more easily understand which operate as shadow admins.
Silverfort adopted a similar solution by representing every account as a node and every permission as an edge between them.
By understanding this knot of privileges and users, they help folks understand which edges (being privileges) would disconnect the most shadow admins. In their active testing with customers, they discovered that by using this method, they could eliminate 70% of shadow admins with only three permission revocations.
Build honeypots… for fun?
You can also take more proactive approaches to dealing with those who might seek out and abuse your shadow admins: a “honeypot” trap.
Crandall writes, “Forward-looking organizations could also take advantage of the fact that shadow admins are attractive to adversaries by using fake accounts to detect and redirect them to decoys. Deception and concealment technologies can hide and deny access to accounts with privileges, such as domain or shadow admin accounts.”
By deploying decoy shadow admins in your SaaS environments, you can set up targeted alerts when an attacker attempts to gain control or even sequester them in a fake environment of valueless dummy data.
Bringing light into the SaaS shadows
At this point, you might wish you didn’t know about shadow admins—sometimes, there’s comfort in letting a threat stay an unknown unknown. But that’s like knowing about that New Mexico motel’s master key “malfunction,” booking a room, and not throwing the deadbolt on before you tuck yourself into bed for the night.
With Rewind’s daily backup and on-demand restoration platform, you get safety far better than a deadbolt: an alternative booking at a far nicer hotel that honors your security and gives you complete confidence in your share of SaaS data responsibility. Rewind can’t can’t stop your team from creating shadow admins in your SaaS environments, but it will quickly become your first step in preventing the damage these often-unseen accounts wreak on your mission-critical SaaS apps.
We’re also preparing the general availability of Rewind for Okta, which includes full and item-level restorations of your essential identity and access data. With Rewind, you can take aggressive approaches to eliminating shadow admins without worrying about the availability of your IdP, and take comfort in knowing you’re only a single click away from complete restoration.
If you’d like to know more about Okta backups and move yourself to the head of the line for release, start by joining the waitlist.
